Skip to content

Commit

Permalink
Add versioned dev_requirements for py 3.8 3.9 3.10
Browse files Browse the repository at this point in the history
Different version of python have different dev requirements. Previously,
`dev_requirements.txt` was generated for python3.8 and assumed to work
for other versions of python.

That was more or less a safe assumption, except is now causing problems.
Specifically, gtsam is not available on python 3.10, but we'd like to
add it as a dev requirement for python 3.8 and 3.9.

`symforce_requirements_test.py` cannot be run until symforce has been
installed, but one might reasonably want to `pip install -r
dev_requirements.txt` before installing symforce (to use the pinned
build depencies during installation).

To make this circular dependency less painful for the user, I'm adding
- `dev_requirements_38.txt`
- `dev_requirements_39.txt`
- `dev_requirements_310.txt`
so that the correct dev requirements can be installed before symforce
installation for any supported python version.

Modified `test/symforce_requirements_test.py` to only test/update the
`dev_requirements_XY.txt` of the python version being used.

Since the public CI is already running this test, it will guarentee that
none of these files will go out of date.

Needed to amend both ci.yml and docs.yml to use the correct
`dev_requirements_XY.txt`.

Also needed to update the `README.md` accordingly.
  • Loading branch information
bradley-solliday-skydio committed Sep 22, 2022
1 parent 559e521 commit 853fc03
Show file tree
Hide file tree
Showing 7 changed files with 761 additions and 11 deletions.
8 changes: 3 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
libeigen3-dev
- name: Install python dependencies
run: pip install -r dev_requirements.txt
run: pip install -r dev_requirements_$(${{ matrix.python }} -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")').txt

- name: Run cmake build
run: |
Expand All @@ -87,14 +87,12 @@ jobs:
cmake --build build -j $(nproc)
# - lcmtypes need to be available for tests
# - Exact contents of dev_requirements.txt depend on python version. Need to update file to
# match current python version to avoid failure of corresponding gen test. symforce needs
# to be on the PYTHONPATH to run gen test in this manner.
# TODO(brad): Check that it is still necessary for $(pwd) to be on the PYTHONPATH
# - symforce needs to be on the PYTHONPATH to run gen test in this manner.
- name: Run tests
run: |
pip install build/lcmtypes/python2.7
export PYTHONPATH=$PYTHONPATH:$(pwd)
${{ matrix.python }} test/symforce_requirements_test.py --update
ctest --test-dir build -j $(nproc) || true
ctest --test-dir build -j $(nproc) --rerun-failed --output-on-failure
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
pandoc
- name: Install python dependencies
run: pip install -r dev_requirements.txt
run: pip install -r dev_requirements_$(python3 -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")').txt

- name: Run cmake build
run: |
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -565,7 +565,7 @@ pip install -e .

This will build the C++ components of SymForce, but you won't be able to run `pip install -e .` repeatedly if you need to rebuild C++ code. If you're changing C++ code and rebuilding, you should build with CMake directly as described <a href="#build-with-cmake">below</a>.

`pip install .` will not install pinned versions of SymForce's dependencies, it'll install any compatible versions. It also won't install all packages required to run all of the SymForce tests and build all of the targets (e.g. building the docs or running the linters). If you want all packages required for that, you should `pip install .[dev]` instead (or one of the other groups of extra requirements in our `setup.py`). If you additionally want pinned versions of our dependencies, which are the exact versions guaranteed by CI to pass all of our tests, you can install them from `pip install -r dev_requirements.txt`.
`pip install .` will not install pinned versions of SymForce's dependencies, it'll install any compatible versions. It also won't install all packages required to run all of the SymForce tests and build all of the targets (e.g. building the docs or running the linters). If you want all packages required for that, you should `pip install .[dev]` instead (or one of the other groups of extra requirements in our `setup.py`). If you additionally want pinned versions of our dependencies, which are the exact versions guaranteed by CI to pass all of our tests, you can install them from `pip install -r dev_requirements_XY.txt` (where `X` and `Y` are the major and minor versions of your python; e.g. `dev_requirements_38.txt` for python3.8).

## Build with CMake

Expand All @@ -574,8 +574,9 @@ SymForce into your Python environment, so you'll need to add it to your PYTHONPA

Install python requirements:
```bash
pip install -r dev_requirements.txt
pip install -r dev_requirements_38.txt
```
(or `dev_requirements_39.txt`/`dev_requirements_310.txt` for python3.9 and python3.10, respectively).

Build SymForce (requires C++14 or later):
```bash
Expand Down
Loading

0 comments on commit 853fc03

Please sign in to comment.