Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add staging of python libraries #84

Merged
merged 10 commits into from
Jul 24, 2024

Conversation

bradarm
Copy link
Contributor

@bradarm bradarm commented Jul 24, 2024

Adds functionality to stage python wheels into coresvc-registry's PyPi server as defined via a requirements.txt file. If this file is empty staging will be skipped.

@bradarm bradarm requested a review from a team as a code owner July 24, 2024 20:54
@bradarm
Copy link
Contributor Author

bradarm commented Jul 24, 2024

Verification

root [ / ]# /data/scripts/stage_python_packages.sh 
Downloading python packages to /data/staging
Collecting contourpy==1.0.6
  Using cached contourpy-1.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (296 kB)
Collecting cycler==0.11.0
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools==4.38.0
  Using cached fonttools-4.38.0-py3-none-any.whl (965 kB)
Collecting kiwisolver==1.4.4
  Using cached kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
Collecting matplotlib==3.6.2
  Using cached matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.8 MB)
Collecting numpy==1.23.5
  Using cached numpy-1.23.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting packaging==21.3
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting pandas==1.5.1
  Using cached pandas-1.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)
Collecting pillow==9.3.0
  Using cached Pillow-9.3.0-cp39-cp39-manylinux_2_28_x86_64.whl (3.3 MB)
Collecting pyparsing==3.0.9
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting python-dateutil==2.8.2
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz==2022.6
  Using cached pytz-2022.6-py2.py3-none-any.whl (498 kB)
Collecting scipy==1.9.3
  Using cached scipy-1.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (33.8 MB)
Collecting seaborn==0.10.1
  Using cached seaborn-0.10.1-py3-none-any.whl (215 kB)
Collecting six==1.16.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Saved /data/staging/contourpy-1.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved /data/staging/cycler-0.11.0-py3-none-any.whl
Saved /data/staging/fonttools-4.38.0-py3-none-any.whl
Saved /data/staging/kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Saved /data/staging/matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved /data/staging/numpy-1.23.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved /data/staging/packaging-21.3-py3-none-any.whl
Saved /data/staging/pandas-1.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved /data/staging/Pillow-9.3.0-cp39-cp39-manylinux_2_28_x86_64.whl
Saved /data/staging/pyparsing-3.0.9-py3-none-any.whl
Saved /data/staging/python_dateutil-2.8.2-py2.py3-none-any.whl
Saved /data/staging/pytz-2022.6-py2.py3-none-any.whl
Saved /data/staging/scipy-1.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Saved /data/staging/seaborn-0.10.1-py3-none-any.whl
Saved /data/staging/six-1.16.0-py2.py3-none-any.whl
Successfully downloaded contourpy cycler fonttools kiwisolver matplotlib numpy packaging pandas pillow pyparsing python-dateutil pytz scipy seaborn six

[notice] A new release of pip is available: 23.0.1 -> 24.1.2
[notice] To update, run: pip install --upgrade pip
Uploading Pillow-9.3.0-cp39-cp39-manylinux_2_28_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading Pillow-9.3.0-cp39-cp39-manylinux_2_28_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB • 00:00 • 336.4 MB/s
Uploading contourpy-1.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading contourpy-1.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 304.9/304.9 kB • 00:00 • 291.9 MB/s
Uploading cycler-0.11.0-py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading cycler-0.11.0-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 kB • 00:00 • ?
Uploading fonttools-4.38.0-py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading fonttools-4.38.0-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB • 00:00 • 270.2 MB/s
Uploading kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading kiwisolver-1.4.4-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB • 00:00 • 318.1 MB/s
Uploading matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading matplotlib-3.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB • 00:00 • 231.3 MB/s
Uploading numpy-1.23.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading numpy-1.23.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.1/17.1 MB • 00:00 • 282.2 MB/s
Uploading packaging-21.3-py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading packaging-21.3-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 kB • 00:00 • 108.7 MB/s
Uploading pandas-1.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading pandas-1.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.2/12.2 MB • 00:00 • 341.6 MB/s
Uploading pyparsing-3.0.9-py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading pyparsing-3.0.9-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.5/106.5 kB • 00:00 • 277.8 MB/s
Uploading python_dateutil-2.8.2-py2.py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading python_dateutil-2.8.2-py2.py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 259.8/259.8 kB • 00:00 • 325.5 MB/s
Uploading pytz-2022.6-py2.py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading pytz-2022.6-py2.py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 523.9/523.9 kB • 00:00 • 199.5 MB/s
Uploading scipy-1.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading scipy-1.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 33.8/33.8 MB • 00:00 • 337.6 MB/s
Uploading seaborn-0.10.1-py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading seaborn-0.10.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 221.3/221.3 kB • 00:00 • 201.3 MB/s
Uploading six-1.16.0-py2.py3-none-any.whl to https://localhost:8080
Uploading distributions to https://localhost:8080
Uploading six-1.16.0-py2.py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.9/15.9 kB • 00:00 • ?

@bigtallcampbell bigtallcampbell merged commit 0f22dbf into main Jul 24, 2024
9 checks passed
@bigtallcampbell bigtallcampbell deleted the add-staging-of-python-libraries branch July 24, 2024 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants