Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Docker for Development #88

Merged
merged 35 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
1a9759d
Set up Dockerfile, docker-compose.ym, and .dockerignore
gaylem Apr 1, 2024
5856b68
Add comment
gaylem Apr 1, 2024
0b1823f
Merge branch 'main' into feature/docker
gaylem Apr 1, 2024
1c5323c
Update comments
gaylem Apr 3, 2024
523251b
Merge branch 'main' into feature/docker
gaylem Apr 3, 2024
d94e30c
Update Dockerfile
gaylem Apr 3, 2024
f56fe03
Update Dockerfile
gaylem Apr 3, 2024
5cc112a
Remove volumes from docker-compose.yml
gaylem Apr 6, 2024
921feed
Update README
gaylem Apr 6, 2024
e1ef311
Update README.md
gaylem Apr 6, 2024
ba49a65
Merge branch 'main' into feature/docker
gaylem Apr 6, 2024
1f08ca9
Update README.md
gaylem Apr 6, 2024
b510b39
Add bash shell instructions to README
gaylem Apr 7, 2024
aa54be1
Merge branch 'feature/docker' of https://github.com/gaylem/electrify-…
gaylem Apr 7, 2024
b693676
Update README.md
gaylem Apr 7, 2024
92f9372
Update README.md
gaylem Apr 7, 2024
f4ac584
Remove MacOS known error since Docker should resolve this
gaylem Apr 7, 2024
2dcf7cf
Update README.md
gaylem Apr 7, 2024
957285d
Update README.md
gaylem Apr 7, 2024
7cf7771
Update README.md
gaylem Apr 7, 2024
c4d7be4
Update README.md
gaylem Apr 7, 2024
f345324
Update README.md
gaylem Apr 7, 2024
7925159
Update README.md
gaylem Apr 7, 2024
5a5e4ec
Change COPY . ./ to COPY . ./app
gaylem Apr 10, 2024
ff5c79d
Create docker-entrypoint.sh
gaylem Apr 10, 2024
6b5c2b1
Add entrypoint
gaylem Apr 10, 2024
8f68f86
Revert Dockerfile
gaylem Apr 11, 2024
8a910fa
Revert docker-compose.yml
gaylem Apr 11, 2024
6530463
Delete docker-entrypoint.sh
gaylem Apr 11, 2024
a34b08d
Update README
gaylem Apr 11, 2024
eb96599
Cleaned up Dockerfile to reduce number of layers and image size
gaylem Apr 11, 2024
d355918
Add volumes back in so that changes display on browser
gaylem Apr 11, 2024
1b5e322
Add command to docker-compose.yml to enable hot reloading
gaylem Apr 11, 2024
d7bf7d7
Update README
gaylem Apr 11, 2024
e069adc
Add newline to EOF
vkoves Apr 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node_modules
33 changes: 33 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
FROM python:3.9

# Add the NodeSource PPA
RUN echo 'Package: nodejs\nPin: origin deb.nodesource.com\nPin-Priority: 600' > /etc/apt/preferences.d/nodesource \
&& curl -fsSL https://deb.nodesource.com/setup_16.x | bash - \
&& apt-get update \
&& apt-get install -y nodejs

# Install yarn
RUN npm install -g yarn

# Set working directory
WORKDIR /app

# Copy requirements and dependency files to the working directory
COPY requirements.txt .
COPY package.json .
COPY yarn.lock .

# Copy the rest of the application code
COPY . .
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why . .? Is that different than just ./?

Copy link
Owner

@vkoves vkoves Apr 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nevermind it's the arguments to copy, an input to output.


# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Install dependencies
RUN yarn install

# Expose the port that the app runs on
EXPOSE 8080

# Start the app in development mode
CMD ["yarn", "develop"]
116 changes: 53 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ Sources:

GraphQL requires data key names to have no spaces or special characters, so there's a raw data file (only filtered by GHG emissions > 1,000 tons and year = 2020) and a cleaned file that just hast he headers renamed for GraphQL.

## Tools

[python](https://www.python.org/) and [pandas](https://pandas.pydata.org/)
for data processing

Leaflet and Leaflet Google mutant https://www.npmjs.com/package/leaflet.gridlayer.googlemutant

## General To-Do List

- [x] Pick a framework - statically built VueJSS, maybe [VitePress](https://vitepress.dev/guide/getting-started)
Expand All @@ -36,112 +43,90 @@ GraphQL requires data key names to have no spaces or special characters, so ther
- [ ] Create ward page that shows data by ward (needs new data source)
- [ ] Figure out a way to rank buildings by opportunity for improvement (perhaps higher than avg. in category, uses a lot of natural gas?)

## Environment setup

## Development
### **1. Set up Docker**

### Front-End Setup
Docker is the recommended approach to quickly getting started with local development. Docker helps create a version of the Electrify Chicago website on your computer so you can test out your code before submitting a pull request.

Make sure you **are running Node v16** and have [Yarn](https://yarnpkg.com/) installed, the
`cd` into the project directory (after cloning it) and run:
- The recommended installation method for your operating system can be found [here](https://docs.docker.com/install/).
- [Get started with Docker](https://docs.docker.com/get-started/)

```bash
yarn install
```
### **2. Start Docker**

### Running The Front-End
> [!IMPORTANT]
> Please make sure the `Docker Desktop` application is **running on your computer** before you run the bash commands below.

Run `yarn develop` to start a local dev server at `http://localhost:8080`

Happy coding 🎉🙌

### Run Front-End Linting

To run linting with auto-fix, run:
This command starts server locally. To start it, `cd` into the project directory in your terminal then run the following command:

```bash
yarn lint-fix
docker-compose up
```

## Deploys

This site deploys automatically via Netlify by running `gridsome build`.
Running the above command will result in the following output in your terminal

<details>
<summary><strong>Click here</strong> to see an example terminal output</summary>
<img width="662" alt="Screenshot 2024-04-05 at 7 23 04 PM" src="https://github.com/gaylem/electrify-chicago/assets/76500899/ad25d216-d58d-48f1-9f7c-16473db50537">
</details>

## Tools
When you see the above output, it means the site is now running and now you can browse to http://localhost:8080

[python](https://www.python.org/) and [pandas](https://pandas.pydata.org/)
for data processing
### **3. Stop Docker**

Leaflet and Leaflet Google mutant https://www.npmjs.com/package/leaflet.gridlayer.googlemutant
- To stop and completely remove the server (i.e. the running Docker container), run `docker-compose down`
- To stop the server, but not destroy it (often sufficient for day-to-day work), run `docker-compose stop`
- Bring the same server back up later with `docker-compose up`

## Open Bash Shell

## Data Processing
> [!IMPORTANT]
> To run any of the commands below, you'll need to do the following:
> 1. Open a new terminal and `cd` into the root project directory after spinning up your Docker container
> 2. Open up a bash shell inside the Docker container with the following command:

### Python Setup (For Data Processing & tests)

This project's Python data pipeline requires:

- pip
- python 3.9
```bash
docker-compose exec electrify-chicago bash
```
### Run Front-End Linting

To install our Python dependencies, from the root of the project, run:
To run linting with auto-fix, run the following command inside the Docker bash shell:

```bash
pip install --no-cache-dir -r requirements.txt
yarn lint-fix
```

### Run Data Processing

If you update the raw data CSVs or the data scripts that post-process them (like if you are adding
a new statistical analysis), you need to re-run the data processing. Make sure to follow the "Python
Setup" steps first.
1. If you update the raw data CSVs or the data scripts that post-process them (like if you are adding
a new statistical analysis), you need to re-run the data processing.

To then process a new CSV file (at `src/data/source/ChicagoEnergyBenchmarking.csv`), from the project
directory run:
2. To then process a new CSV file (at `src/data/source/ChicagoEnergyBenchmarking.csv`), you need to run the following command inside the Docker bash shell:

```bash
bash run_all.sh
```

### Run Data Processing Tests

Make sure test data is created/replaced before running tests by running the following script from
the main project directory (it will overwrite the existing test data file if it exists):
1. Make sure test data is created/replaced before running tests by running the following script from
the Docker bash shell (it will overwrite the existing test data file if it exists):

```bash
bash create_test_data.sh
```

To run all tests simply in the project directory run:
2. To run all tests in the project directory, enter the following command inside the Docker bash shell:

```bash
pytest
python -m pytest
```

This assumes that `pytest` has been installed, see setup.

Run the following command for individual unit test suite (where XXX is something like
`test_clean_all_years`):
3. Run the following command for individual unit test suite (where YOUR_FILE_NAME is something like
`test_clean_all_years`) in the Docker bash shell:

```bash
python3 -m pytest test/data/scripts/unit/XXX.py
```


## Known Development Issues

#### macOS libvips Error

If you encounter an error on macOS such as `sharp Prebuilt libvips 8.10.5 binaries are not yet available for darwin-arm64v8`, you'll need to install these dependencies separately. Install the [Brew package manager](https://brew.sh/), then run the following commands:

```
brew install --build-from-source gcc
xcode-select install
brew install vips
python -m pytest tests/data/scripts/unit/YOUR_FILE_NAME.py
```
=======
**Important!** When you update the data, make sure to update the `LatestDataYear` in
`globals.vue`, as well as the filter year in all page queries.

## Managing The Data

Expand Down Expand Up @@ -189,3 +174,8 @@ const BuildingOwnerIds = [

**Note:** You'll have to restart your `yarn develop` after step 3 to see changes, since
`gridsome.server.js` just runs once.

## Deploys

This site deploys automatically via Netlify by running `gridsome build`.

14 changes: 14 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
version: '3.8'

services:
electrify-chicago:
image: electrify-chicago:latest
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
volumes:
- .:/app
- /app/node_modules
command: yarn develop
Loading
Loading