Resolving dependencies in 2.0 takes forever #9956
Comments
ace-han
added
kind/bug
|
poetry 1.8.5 also takes a long time to resolve this one for me. most likely all that has happened is that the cache format has changed in the new release, so poetry 2.0 cannot use your poetry 1.8.5 cache. In that case the second and subsequent runs will be fast again. |
@dimbleby I was setting up a brand new docker env (no poetry installed before), so the situation you mentioned is not my case. |
|
well then I do not reproduce: poetry 1.8.5 also takes a long time to resolve this one for me (with an empty cache). I see no difference in the new release. |
|
either way the usual problem in such cases is awscli / boto3 / botocore - Amazon like to release thousands of versions of the last of these, which can cause resolvers to have to do a lot of backtracking. Probably if you raise the lower bounds on Amazon-ecosystem dependencies that will reduce your exposure to this. |
|
Take it back, this actually is a regression - though one complicated by the problem being slow to solve even without the change. Here is a simplified repro: [tool.poetry]
name = "foo"
version = "1.0.0"
description = ""
authors = []
keywords = []
license = "MIT"
readme = "README.md"
[tool.poetry.dependencies]
python = ">=3.8,<3.11"
opencv-python = {version = "^4.6.0.66", optional = true}
ale-py = { version = "0.8.1", optional = true }
[tool.poetry.extras]
extra01 = ["ale-py"]
extra02 = ["ale-py"]
extra03 = ["ale-py"]
extra04 = ["ale-py"]
extra05 = ["ale-py"]
extra06 = ["ale-py"]
extra07 = ["ale-py"]
extra08 = ["ale-py"]
extra09 = ["ale-py"]
extra10 = ["ale-py"]
extra11 = ["ale-py"]
extra12 = ["ale-py"]
extra13 = ["ale-py"]
extra14 = ["ale-py"]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"and we are getting stuck in the new code that wants to maintain markers during resolution so that it can write them to the lock file #9427. Because: the markers that we calculate are long and unwieldy, and converting them to and from CNF and DNF is exponentially expensive |
|
Can confirm there seems to be some regression going on. When generating the lock file from the following Poetry Installation Method: pipx
|
|
You may try #10018. As you can see in the PR description it does not fix the regression, it only mitigates it. The performance might be enough for simpler cases. However, it is also possible that it makes no relevant difference for your use case... |
|
After investigating the issue a little further, it seems like the performance regression can be fixed (at least for the simplified repro) if we improve extra handling in _merge_single_markers, i.e. support |
|
@radoering thank you for this update. |
Description
Install cleanrl 's dependencies via
poetry installtakes forever to resolve dependenciesWorkarounds
using
poetry==1.8.5works finePoetry Installation Method
install.python-poetry.org
Operating System
Ubuntu 22.04.2 LTS
Poetry Version
2.0.0
Poetry Configuration
Python Sysconfig
sysconfig.log
Example pyproject.toml
Poetry Runtime Logs
Super large. Can not upload. generated via `poetry install -vvv | tee /tmp/p.log` root@dsw-498793-6c5c78668-ps57p:/tmp# ll -trh 总计 132M drwxr-xr-x 1 root root 4.0K Jan 6 09:21 ../ drwx------ 2 root root 4.0K Jan 6 13:57 tmpfbxagxq5_kernels/ -rw-r--r-- 1 root root 0 Jan 6 13:57 wow_what_a_coincidence -rw------- 1 root root 0 Jan 6 13:57 stdout-stderr---supervisor-oi3so2gc.log drwx------ 33 root root 12K Jan 6 13:57 _MEIZhlMUp/ drwx------ 3 root root 4.0K Jan 6 13:57 tmp9omelb31/ -rw------- 1 root root 30K Jan 6 15:05 stdout-stdout---supervisor-zagcr3vz.log srwxr-xr-x 1 root root 0 Jan 6 15:05 vscode-ipc-8702339b-3baa-4dfe-8dfd-26f2b4d1d897.sock= srwxr-xr-x 1 root root 0 Jan 6 15:05 vscode-git-8511e23733.sock= drwxrwxrwx 5 root root 4.0K Jan 6 15:33 ./ -rw-r--r-- 1 root root 132M Jan 6 15:34 p.logThe text was updated successfully, but these errors were encountered: