Install text-generation-server from poetry.lock export
#2786
Conversation
Installing from an exported `requirements.txt` like file generated out of the `poetry.lock` makes sense to be able to reuse the generated lock with the pinned dependencies and avoid dependency issues or conflicts when building outdated Dockerfiles in case any dependency is loose and runs into conflicts i.e. any potential conflict is always solved before the release
Manually install the plugin to avoid issues with future `poetry` versions as of the warning messages recommend: ``` WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Warning: poetry-plugin-export will not be installed by default in a future version of Poetry. In order to avoid a breaking change and make your automation forward-compatible, please install poetry-plugin-export explicitly. See https://python-poetry.org/docs/plugins/#using-plugins for details on how to install a plugin. To disable this warning run 'poetry config warnings.export false'. ```
| @@ -233,7 +250,8 @@ COPY server/Makefile server/Makefile | |||
| RUN cd server && \ | |||
| make gen-server && \ | |||
| pip install -r requirements_cuda.txt && \ | |||
| pip install ".[attention, bnb, accelerate, compressed-tensors, marlin, moe, quantize, peft, outlines]" --no-cache-dir && \ | |||
| pip install -r requirements_poetry.txt --no-cache-dir && \ | |||
There was a problem hiding this comment.
What's the issue again ? Why are we not using poetry directly ?
Also maybe we should take the opportunity to switch to uv ? https://docs.astral.sh/uv/pip/compile/#locking-requirements I have no idea, but it seems like the poetry lock isn't locking tightly enough, maybe ?
There was a problem hiding this comment.
Sorry for the delay on this PR, and happy new year BTW! So the issue is that since we're not using poetry as the pip installer the locked dependencies are not used leading to potential issues when partners as e.g. Google build the containers ahead in time, as those dependencies are installed from the pyproject.toml rather than the lock.
I know this is not an issue impacting directly TGI per se, but for robustness and reproducibility we should try to lock those and install those from a lock. The installation from the lock can either be via poetry (the current approach), or switching to uv instead and installing from the uv.lock.
Maybe we should start an internal conversation and decide what's best, happy to help whatever the solution / proposal is!
What does this PR do?
This PR installs the
text-generation-serverPython requirements from an exportedrequirements.txt-like file generated out of thepoetry.lock, to be able to reuse the generated lock with the pinned dependencies and avoid dependency issues or conflicts when building outdated Dockerfiles in case any dependency is loose, as any potential conflict is always solved before the release and the lock is always working.Before submitting
Who can review?
cc @OlivierDehaene