Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add build and publish workflow #1102

Merged
merged 28 commits into from
Jan 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
c629e82
add build and publish workflow
thetoolsmith Jan 3, 2025
b8e23af
test pulling from ghcr
thetoolsmith Jan 3, 2025
aa8d8e9
test pulling from ghcr
thetoolsmith Jan 3, 2025
27eb681
fix logs export when pulling images
thetoolsmith Jan 3, 2025
b2cccfc
build and publish test frontend image, update test to optionally pull
thetoolsmith Jan 3, 2025
8fe5656
missing char
thetoolsmith Jan 3, 2025
66c556b
publish test build
thetoolsmith Jan 3, 2025
81eed96
publish test build
thetoolsmith Jan 3, 2025
6e3311a
publish test build
thetoolsmith Jan 3, 2025
e2942d4
missing brace
thetoolsmith Jan 3, 2025
0a62285
try compose using released images
thetoolsmith Jan 3, 2025
50b153e
toggle releases vs local for running compose
thetoolsmith Jan 3, 2025
c5c9953
testing new workflow
thetoolsmith Jan 3, 2025
b4bc6c1
pull most updated images
thetoolsmith Jan 6, 2025
53c1561
tag name reset
thetoolsmith Jan 6, 2025
2eda669
toggle to use local build images for tests
thetoolsmith Jan 6, 2025
44a4b15
add platform flag for mail-api build
thetoolsmith Jan 6, 2025
c7760bd
Revert "add platform flag for mail-api build"
thetoolsmith Jan 6, 2025
ee3944a
switch back to using published images for test workflow
thetoolsmith Jan 7, 2025
871dc86
dump pg conf
thetoolsmith Jan 8, 2025
4aec02f
reset to standard api image names
thetoolsmith Jan 14, 2025
50fd1e2
no need to use build specific for pr_test image
thetoolsmith Jan 14, 2025
426663b
add pr number to the test image
thetoolsmith Jan 14, 2025
301e6a8
move success tag and push to build workflow
thetoolsmith Jan 14, 2025
8e3666e
need to send stderr to logs
thetoolsmith Jan 14, 2025
19c2195
remove duplicate workflow call
thetoolsmith Jan 14, 2025
d0fd7d2
compound or expressions not working as expected
thetoolsmith Jan 14, 2025
626907c
remove test conf dump
thetoolsmith Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: BuildImage

on:
workflow_call:
push:
branches:
- main

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: 'Checkout GitHub Action'
id: checkout
uses: actions/checkout@v4

# not tagging with version yet, adding this in the event we start versioning
- name: Get Version
id: version
if: github.ref == 'refs/heads/main'
run: |
echo "TAG_VERSION=$(cat package.json | jq -r .version)" >> $GITHUB_ENV

- name: 'Build Release Image'
id: buildrelease
if: github.ref == 'refs/heads/main'
run: |
docker build -t ghcr.io/cfpb/regtech/sbl/sbl-frontend:latest -f Dockerfile .

- name: 'Build Test Image'
id: buildtest
if: github.ref != 'refs/heads/main'
run: |
docker build -t ghcr.io/cfpb/regtech/sbl/sbl-frontend:pr_test_${{ github.event.number }} -f Dockerfile .

- name: 'Login to GitHub Container Registry'
id: login
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{github.actor}}
password: ${{secrets.GITHUB_TOKEN}}

- name: 'Publish Image'
id: publish
if: steps.login.conclusion == 'success'
run: |
docker push ghcr.io/cfpb/regtech/sbl/sbl-frontend --all-tags
14 changes: 11 additions & 3 deletions .github/workflows/pull_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,23 @@ jobs:
uses: ./.github/workflows/lint.yml
needs: call-pre

call-build:
name: Build
if: ${{ always() }}
uses: ./.github/workflows/build.yml
needs: call-lint

call-test:
name: Test
if: ${{ always() }}
uses: ./.github/workflows/test.yml
needs: call-lint

needs: call-build
with:
use_release_images: true

call-codeql:
name: CodeQL
if: ${{ always() }}
uses: ./.github/workflows/codeql.yml
needs: call-lint


14 changes: 11 additions & 3 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,23 @@ jobs:
uses: ./.github/workflows/lint.yml
needs: call-pre

call-build:
name: Build
if: ${{ always() }}
uses: ./.github/workflows/build.yml
needs: call-lint

call-test:
name: Test
if: ${{ always() }}
uses: ./.github/workflows/test.yml
needs: call-lint

needs: call-build
with:
use_release_images: true

call-codeql:
name: CodeQL
if: ${{ always() }}
uses: ./.github/workflows/codeql.yml
needs: call-lint


88 changes: 81 additions & 7 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,12 @@ name: Test

on:
workflow_call:

inputs:
use_release_images:
description: 'Optionally pull images from GHCR instead of building'
default: false
required: false
type: boolean
jobs:
react:
name: React
Expand Down Expand Up @@ -41,60 +46,120 @@ jobs:
uses: actions/checkout@v4
with:
path: 'sbl-frontend'

- name: Checkout Cleanup API
uses: actions/checkout@v4
if: inputs.use_release_images != true
with:
repository: 'cfpb/regtech-cleanup-api'
path: 'regtech-cleanup-api'

- name: Checkout Mail API
uses: actions/checkout@v4
if: inputs.use_release_images != true
with:
repository: 'cfpb/regtech-mail-api'
path: 'regtech-mail-api'

- name: Checkout User Fi
uses: actions/checkout@v4
if: inputs.use_release_images != true
with:
repository: 'cfpb/regtech-user-fi-management'
path: 'regtech-user-fi-management'

- name: Checkout Filing API
uses: actions/checkout@v4
if: inputs.use_release_images != true
with:
repository: 'cfpb/sbl-filing-api'
path: 'sbl-filing-api'

- name: Checkout SBL Project
uses: actions/checkout@v4
with:
repository: 'cfpb/sbl-project'
path: 'sbl-project'
ref: test/docker-compose-using-released-images

# Set the environment
- name: Set Environment
uses: ./sbl-frontend/.github/actions/setvars
with:
varFilePath: ./sbl-frontend/.github/variables/.env

# Build images
# Build images
- name: Build Frontend
if: inputs.use_release_images != true
run: |
cd sbl-frontend
docker build -t sbl-project-sbl-frontend:latest .

- name: Build Cleanup API
if: inputs.use_release_images != true
run: |
cd regtech-cleanup-api
docker build -t sbl-project-cleanup:latest .

- name: Build Mail API
if: inputs.use_release_images != true
run: |
cd regtech-mail-api
docker build -t sbl-project-mail-api:latest .

- name: Build User Fi API
if: inputs.use_release_images != true
run: |
cd regtech-user-fi-management
docker build -t sbl-project-user-fi:latest .

- name: Build Filing API
if: inputs.use_release_images != true
run: |
cd sbl-filing-api
docker build -t sbl-project-filing:latest .

# Pull images and tag as if we were building local
- name: 'Login to GitHub Container Registry'
id: login
uses: docker/login-action@v3
if: inputs.use_release_images == true
with:
registry: ghcr.io
username: ${{github.actor}}
password: ${{secrets.GITHUB_TOKEN}}

# Pull Frontend image just built and published
- name: Pull Frontend
if: inputs.use_release_images == true && steps.login.conclusion == 'success'
run: |
docker pull ghcr.io/cfpb/regtech/sbl/sbl-frontend:pr_test_${{ github.event.number }}
docker tag ghcr.io/cfpb/regtech/sbl/sbl-frontend:pr_test_${{ github.event.number }} sbl-project-sbl-frontend:latest

- name: Pull Cleanup API
if: inputs.use_release_images == true && steps.login.conclusion == 'success'
run: |
docker pull ghcr.io/cfpb/regtech/sbl/regtech-cleanup-api:latest
docker tag ghcr.io/cfpb/regtech/sbl/regtech-cleanup-api:latest sbl-project-cleanup:latest

- name: Pull Mail API
if: inputs.use_release_images == true && steps.login.conclusion == 'success'
run: |
docker pull ghcr.io/cfpb/regtech/sbl/regtech-mail-api:latest
docker tag ghcr.io/cfpb/regtech/sbl/regtech-mail-api:latest sbl-project-mail-api:latest

- name: Pull User Fi API
if: inputs.use_release_images == true && steps.login.conclusion == 'success'
run: |
docker pull ghcr.io/cfpb/regtech/sbl/regtech-user-fi-management:latest
docker tag ghcr.io/cfpb/regtech/sbl/regtech-user-fi-management:latest sbl-project-user-fi:latest

- name: Pull Filing API
if: inputs.use_release_images == true && steps.login.conclusion == 'success'
run: |
docker pull ghcr.io/cfpb/regtech/sbl/sbl-filing-api:latest
docker tag ghcr.io/cfpb/regtech/sbl/sbl-filing-api:latest sbl-project-filing:latest

# Setup node stuff
- name: Setup Node 20
uses: actions/setup-node@v4
Expand All @@ -105,23 +170,31 @@ jobs:

# Standup stack
- name: Standup Stack
if: inputs.use_release_images != true
run: |
cd sbl-project
docker compose --profile="backend" --profile="frontend" up -d --remove-orphans --build


# Standup releases stack
- name: Standup Releases Stack
if: inputs.use_release_images == true
run: |
cd sbl-project
docker compose --profile="backend" --profile="frontend" up -d --remove-orphans

- name: Check running containers
run: |
ls -alh /tmp/filing_uploads
docker ps

# Setup for test
- name: Install Local Yarn Dependencies
run: |
run: |
cd sbl-frontend
yarn
yarn playwright install --with-deps

- name: Seed Database
- name: Seed Database
run: |
cd sbl-project/dev_setup/mock_data/
bash create_institutions.sh
Expand All @@ -138,9 +211,10 @@ jobs:
cat ./.env.example.private >> ./.env
yarn playwright test --workers 4

# Store artifact test results
# Store artifact test results
- name: Export docker logs
if: '!cancelled()'
continue-on-error: true
run: |
mkdir -p docker-logs
docker logs sbl-project-sbl-frontend-1 &> ./docker-logs/frontend.log
Expand All @@ -151,6 +225,7 @@ jobs:
docker logs sbl-project-keycloak-1 &> ./docker-logs/keycloak.log
docker logs sbl-project-mailpit-1 &> ./docker-logs/mailpit.log
docker logs sbl-project-pg-1 &> ./docker-logs/postgres.log
ls -alt docker-logs

- name: Archive Test Results
uses: actions/upload-artifact@v4
Expand All @@ -167,4 +242,3 @@ jobs:
name: docker-logs
path: |
docker-logs

Loading