Compatibility Issue with pydantic and annotated-types in pyiceberg 0.8.1

[pawansanz]

Apache Iceberg version

0.8.1 (latest release)

Please describe the bug 🐞

I am encountering an error when using pyiceberg 0.8.1 with pydantic 2.10.3 and annotated-types 0.7.0 in a Python virtual environment. The error message is:

File "\site-packages\pydantic\types.py", line 898, in conlist
return Annotated[List[item_type], annotated_types.Len(min_length or 0, max_length)]
TypeError: Len() takes no arguments

Environment:
OS: Windows 10
Python Version: 3.10
pyiceberg Version: 0.8.1
pydantic Version: 2.10.3
annotated-types Version: 0.7.0

Additional Notes:

I have ensured the latest versions of pydantic and annotated-types are installed, and the error still persists.
Cleaning the environment and reinstalling dependencies did not resolve the issue.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[djouallah]

got this error too

ImportError: cannot import name 'RootModel' from 'pydantic' (/home/trusted-service-user/jupyter-env/python3.11/lib/python3.11/site-packages/pydantic/__init__.cpython-311-x86_64-linux-gnu.so)

[Fokko]

@djouallah do you know which version of Pydantic you're using? You can easily check it using:

Python 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydantic 
>>> pydantic.__version__
'2.10.3'

[djouallah]

@djouallah do you know which version of Pydantic you're using? You can easily check it using:

Python 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydantic 
>>> pydantic.__version__
'2.10.3'

sorry, my notebook did not properly restarted, everything is fine

[pawansanz]

can anyone help with original issue that was reported regarding "Apache Iceberg version"

[kevinjqliu]

Hi @pawansanz, I've also encountered this issue before. In my case, it was due to the different library versions on my laptop.
A way to debug is to ensure that the python,poetry, and pydantic all refer to the same python virtual environment.

Could you try recreating the python virtual environment and see if the issue persists?