Data Analysis: Do high emissions predict reporting non-compliance?

[colton-lapp]

Description

This pull request creates:

1. A jupyter notebook file with data analysis findings
2. A blog post describing the findings in the jupyter notebook file

The blog post investigates a question raised in #114 - does poor performance correlate with non-reporting? The short answer is no, I didn't find that pattern in the data.

The data analysis in the Jupter notebook consists of the following steps:

1. Create some basic data viz showing variables of interest and compliance trends over time
2. Create lagged variables of emissions last year and the emission trends from 2 years ago to 1 year ago
3. Create graphs comparing mean/median GHG intensity last year and GHG trend from 2 years ago to 1 year ago vs reporting compliance, showing basically no difference
4. Run a regression with a single control variable (square footage) to confirm there is no statistically significant relationship
5. Run some robustness checks by dropping outliers and dropping covid and repeating steps 3-4; still no significant results

These findings are then summarized in a new blog post.

A couple other notes:

• I used the graphing package plotly to make html graphs that are interactive, and embedded those in my blog. Because the html graphs allow you to over over individual data points and display info, they take up a decent amount of space (between 1-20 mb). This also makes the Jupyter notebook file larger, but I tried to cut down the size by making some of the plots static image files
• I don't know anything about javascript or html so I relied on Gen AI to do a lot of the coding for embedding and rendering interactive html graphs and fetching regression results from a JSON file. This could probably use some serious attention
• I've introduced some new dependencies for data visualization and am not sure how this is managed in the project

This is my first time creating a PR for a public repo and for this project specifically so happy to restructure any work or accept any feedback! I'm expecting some heavy feedback on files committed (i.e. new packages used in requirements.txt, python virtual environment, directory structure).

Fixes #114

Testing Instructions

I would recommend pulling, running docker-compose up and looking at my blog. Additionally, check out the Jupyter notebook to verify that I'm analyzing the correct variables and don't have data analysis mistakes, etc. To see the interactive html graphs in the notebook, you have to view the Jupyter file in NBViewer as it won't render in Github

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[netlify]

✅ Deploy Preview for radiant-cucurucho-d09bae ready!

Name	Link
🔨 Latest commit	4065b79
🔍 Latest deploy log	https://app.netlify.com/sites/radiant-cucurucho-d09bae/deploys/6790771436d07d0008f2e3da
😎 Deploy Preview	https://deploy-preview-147--radiant-cucurucho-d09bae.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[colton-lapp]

I've reduced the file size of the graphs down to a cumulative 2MB. I did this by dropping some of the data displayed on hover. We could reduce the file size even more by not displaying every single observation in the scatterplots and only displaying a handful of of the buildings that have standard emissions (it's hard to tell them apart anyways). We could also just convert the images to static PNG files. Let me know what you think is best.

[vkoves]

@colton-lapp - I meant to comment when I pushed up my fixes - I've added some date stamps to the blog posts and reodered it so yours comes first (since it's newer). I'm fine with that file size, but it looks like there's some responsiveness issues with the graphs - if you can fix those, I'm good with it, but otherwise we could move to images. Here's an example:

Desktop (shows scrollbar)	Mobile (cut-off)

Also is there a way to note dependencies for you Jupyter notebook? I tried running it locally but had to manually install dependencies like plotly, which aren't in our requirements.txt. Maybe you can add some instructions at the start of your notebook and maybe a requirements.txt file? I don't know what's typical there