--cmake-args option kinda breaks with python 3.12.7

[TarikViehmann]

Issue

When using colcon on fedora 40 with python 3.12.7 (python 3.12.6 works fine, as expected), argument parsing gets messed up, when passing cmake-args via colcon:

$ colcon build --cmake-args -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF --sym --packages-select my_package
usage: colcon [-h] [--log-base LOG_BASE] [--log-level LOG_LEVEL]
              {build,extension-points,extensions,graph,info,list,metadata,test,test-result,version-check} ...
colcon: error: unrecognized arguments: -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF

I scanned the python issue tracker and since argparse is widely used and no relevant issues to it where filed since the release of 3.12.7 (and it's packaging in fedora on 2024-10-02 12:59:21), I would presume that the issue might be on colcon's side. If you think otherwise I can also report the Issue elsewhere.

Steps to Reproduce

Use fedora 40, notably:

$ dnf install python3-3.12.7-1.fc40.x86_64 \
                      python3-colcon-ros-0.5.0-1.fc40.noarch \
                      python3-colcon-common-extensions-0.3.0-8.fc40.noarch                   

$ dnf list python3-colcon-*
python3-colcon-argcomplete.noarch                   0.3.3-18.fc40  
python3-colcon-bash.noarch                          0.5.0-1.fc40   
python3-colcon-cd.noarch                            0.2.1-1.fc40   
python3-colcon-cmake.noarch                         0.2.28-1.fc40  
python3-colcon-common-extensions.noarch             0.3.0-8.fc40   
python3-colcon-core.noarch                          0.18.1-1.fc40  
python3-colcon-defaults.noarch                      0.2.8-5.fc40   
python3-colcon-devtools.noarch                      0.3.0-1.fc40   
python3-colcon-installed-package-information.noarch 0.2.1-1.fc40   
python3-colcon-library-path.noarch                  0.2.1-20.fc40  
python3-colcon-metadata.noarch                      0.2.5-12.fc40  
python3-colcon-notification.noarch                  0.3.0-1.fc40   
python3-colcon-output.noarch                        0.2.13-5.fc40  
python3-colcon-override-check.noarch                0.0.1-3.fc40   
python3-colcon-package-information.noarch           0.4.0-1.fc40   
python3-colcon-package-selection.noarch             0.2.10-12.fc40 
python3-colcon-parallel-executor.noarch             0.3.0-1.fc40   
python3-colcon-pkg-config.noarch                    0.1.0-19.fc40  
python3-colcon-powershell.noarch                    0.4.0-1.fc40   
python3-colcon-python-setup-py.noarch               0.2.9-1.fc40   
python3-colcon-recursive-crawl.noarch               0.2.3-1.fc40   
python3-colcon-ros.noarch                           0.5.0-1.fc40   
python3-colcon-test-result.noarch                   0.3.8-15.fc40  
python3-colcon-zsh.noarch                           0.5.0-1.fc40

Then try to pass one or multiple cmake args via colcon as described in the beginning:

$ colcon build --cmake-args -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF --sym --packages-select my_package

Expected Result

The cmake arguments are handed over to my package.

Actual Result

An unrecognized argument error is thrown.

usage: colcon [-h] [--log-base LOG_BASE] [--log-level LOG_LEVEL]
              {build,extension-points,extensions,graph,info,list,metadata,test,test-result,version-check} ...
colcon: error: unrecognized arguments: -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF

Workaround

Either downgrading to an earlier python version (i tested 3.12.6-1) or wrapping each individual cmake arg with parenthesis, including a leading space in each, will circumvent the issue:

$ colcon build --cmake-args " -DCMAKE_VERBOSE_MAKEFILE=ON" " -DCMAKE_BUILD_TYPE=Debug" " -DBUILD_TESTING=OFF" --sym --packages-select my_package
Finished <<< my_package [45.8s]
Starting >>> my_package
[Processing: my_package]
--- stderr: my_package
Summary: 1 package finished [46.1s]

I presume that this is due to several changes made in argparse between 3.12.6 and 3.12.7 python/cpython@v3.12.6...v3.12.7, but I don't know whether this is intended behavior (from python side) and should be addressed in colcon or it is a bug introduced by the python update.

[cottsay]

Hi there, thanks for the report. This issue is fixed in colcon-core 0.18.2 by #673.

That version of colcon-core is currently in testing for Fedora 40: https://bodhi.fedoraproject.org/updates/FEDORA-2024-b18f4426e4

You should be able to install it early with dnf update python3-colcon-core --enablerepo=updates-testing.

[TarikViehmann]

awesome, tested it, works!