Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Christmas 2024 rebuild #5

Merged
merged 56 commits into from
Jan 3, 2025
Merged

Christmas 2024 rebuild #5

merged 56 commits into from
Jan 3, 2025

Conversation

oursland
Copy link
Contributor

No description provided.

@oursland oursland changed the title CI: osx_arm64 CI: osx and osx_arm64 Sep 15, 2024
@oursland oursland mentioned this pull request Sep 15, 2024
Open
1 task
@Tobias-Fischer
Copy link
Contributor

Are you happy to take a look at migrating the PR CI from boa to pixi/rattler-build?

@oursland
Copy link
Contributor Author

@Tobias-Fischer #8 migrates the PR CI to pixi and rattler-build.

@oursland oursland mentioned this pull request Sep 15, 2024
Closed
oursland added a commit to oursland/ros-jazzy that referenced this pull request Sep 20, 2024
@oursland
WIP: rclpy interaction crashes.
8466bcd 
Current processes:

rebuild:
  reset ; rm -rf output/bld && pixi run generate-recipes && time pixi run -v rattler-build build --recipe recipes/ros-jazzy-rosidl-generator-py --recipe recipes/ros-jazzy-rclpy --recipe recipes/ros-jazzy-rcl-interfaces -m conda_build_config.yaml -c robostack-jazzy -c conda-forge --keep-build

test:
  mamba create -n ros-jazzy -c conda-forge -c file:///Users/jso/code/ros-jazzy/output python=3.11\* ros-jazzy-rclpy
  mamba run --live-stream -n ros-jazzy lldb python -- -c 'import rclpy; rclpy.init(); node = rclpy.create_node("test"); rclpy.spin(node)'

  Then run with the 'r' command:

The second line will run a simple python script that exhibits the crash.  The log looks like this:

  (lldb) target create "python"
  Current executable set to '/Users/jso/code/FreeCAD/FreeCAD/.conda/ros-jazzy/bin/python' (arm64).
  (lldb) settings set -- target.run-args  "-c" "import rclpy; rclpy.init(); node = rclpy.create_node(\"test\"); rclpy.spin(node)"
  (lldb) r
  Process 78922 launched: '/Users/jso/code/FreeCAD/FreeCAD/.conda/ros-jazzy/bin/python' (arm64)
  Process 78922 stopped
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x60)
      frame #0: 0x00000001027d6314 libpython3.11.dylib`set_attribute_error_context + 60
  libpython3.11.dylib`set_attribute_error_context:
  ->  0x1027d6314 <+60>: ldr    x0, [x8, #0x60]
      0x1027d6318 <+64>: bl     0x1028a0b0c    ; PyErr_GivenExceptionMatches
      0x1027d631c <+68>: cbz    w0, 0x1027d6384 ; <+172>
      0x1027d6320 <+72>: ldr    x8, [x21, #0x358]
  Target 0: (python) stopped.
  (lldb) bt
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x60)
    * frame #0: 0x00000001027d6314 libpython3.11.dylib`set_attribute_error_context + 60
      frame #1: 0x00000001027d68f0 libpython3.11.dylib`PyObject_GetAttr + 192
      frame #2: 0x00000001027d67e0 libpython3.11.dylib`PyObject_GetAttrString + 108
      frame #3: 0x0000000101a87b04 libbuiltin_interfaces__rosidl_generator_py.dylib`builtin_interfaces__msg__time__convert_from_py + 60
      frame #4: 0x000000010219c0a8 librcl_interfaces__rosidl_generator_py.dylib`rcl_interfaces__msg__parameter_event__convert_from_py + 356
      frame RoboStack#5: 0x0000000100e48250 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol2190 + 360
      frame RoboStack#6: 0x0000000100e2b844 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol1795 + 76
      frame RoboStack#7: 0x0000000100e2c1a0 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol1802 + 208
      frame RoboStack#8: 0x0000000100dc17c0 _rclpy_pybind11.cpython-311-darwin.so`___lldb_unnamed_symbol708 + 4556
      frame RoboStack#9: 0x00000001000b7f4c python`cfunction_call + 124
      frame RoboStack#10: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame RoboStack#11: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame RoboStack#12: 0x00000001001674c8 python`_PyEval_Vector + 184
      frame RoboStack#13: 0x00000001000608b8 python`_PyObject_FastCallDictTstate + 156
      frame RoboStack#14: 0x00000001000617f0 python`_PyObject_Call_Prepend + 176
      frame RoboStack#15: 0x00000001000dc888 python`slot_tp_init + 196
      frame RoboStack#16: 0x00000001000d4de4 python`type_call + 464
      frame #17: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame #18: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame #19: 0x00000001001674c8 python`_PyEval_Vector + 184
      frame #20: 0x00000001000608b8 python`_PyObject_FastCallDictTstate + 156
      frame #21: 0x00000001000617f0 python`_PyObject_Call_Prepend + 176
      frame #22: 0x00000001000dc888 python`slot_tp_init + 196
      frame #23: 0x00000001000d4de4 python`type_call + 464
      frame #24: 0x00000001000606f0 python`_PyObject_MakeTpCall + 332
      frame #25: 0x0000000100162e3c python`_PyEval_EvalFrameDefault + 46484
      frame #26: 0x00000001001568f4 python`PyEval_EvalCode + 220
      frame #27: 0x00000001001bc970 python`run_mod + 144
      frame #28: 0x00000001001c04a4 python`PyRun_SimpleStringFlags + 272
      frame #29: 0x00000001001e1b3c python`Py_RunMain + 1396
      frame #30: 0x00000001001e3050 python`pymain_main + 1252
      frame #31: 0x0000000100003398 python`main + 56
      frame #32: 0x00000001921f0274 dyld`start + 2840
@oursland oursland mentioned this pull request Sep 21, 2024
Closed
oursland added 20 commits October 8, 2024 08:57
@oursland
Remove ros-humble packages.
afae6d6
@oursland
Remove homebrew from $PATH.
d83ebcc
@oursland
Run pixi with verbosity.
ce05747
@oursland
Use the rattler-build branch of vinca.
2a4135c
@oursland
Set protobuf version to match conda-forge dependencies.
056800d
@oursland
Add build and run dependencies.
76010fc
@oursland
Build with Python 3.11.*
057c804
@oursland
Add ros-jazzy-rviz-ogre-vendor.patch.
047acb5
@oursland
Add ros-jazzy-python-qt-binding.patch
11889f0
@oursland
Add ros-jazzy-qt-gui-cpp.patch
7acc473
@oursland
Add ros-jazzy-rosidl-generator-py.patch
b171664
@oursland
Add ros-jazzy-ament-cmake-vendor-package.patch
b82cb10
@oursland
Add ros-jazzy-gz-ogre-next-vendor.patch
cfc968a
@oursland
Add ros-jazzy-ros-gz-image.patch
efb2e9a
@oursland
Add ros-jazzy-rosbag2-py.osx.patch
9acb6b5
@oursland
Add ros-jazzy-rosidl-generator-py.osx.patch.
a22c989
@oursland
Rename ros-jazzy-rclpy.patch to ros-jazzy-rclpy.osx.patch
adbe79b
@oursland
Build ros-jazzy-desktop, with some exceptions.
eef3d3a
@oursland
Update pixi.lock
fac10b2
@oursland
testpr: use rattler-build.
f916d0e
@oursland
Copy link
Contributor Author

@Tobias-Fischer How can I assist in this? Is there a priority that you're working on?

@Tobias-Fischer
Copy link
Contributor

Thanks a lot @oursland! I was able to build locally osx-arm64 and linux-aarch64 (via Multipass on Mac).

  • It seems like upstream has just bumped some versions, so some patches do not apply cleanly anymore - if you see any and want to fix them up, that'd be great.
  • If you have a linux-64 machine and want to try a local build, that'd be cool.
  • If you can think of any meaningful tests to add, please go ahead
  • A big headache is still that I don't understand why the windows CI fails - if you want to take a look at that, that'd be very helpful, too (see https://discord.gg/kKV8ZxyzY4 discord - #robotics channel).

@Tobias-Fischer
Copy link
Contributor

For Linux we need https://github.com/traversaro/ros-humble/blob/rebuild-christmas-2024-linux-64-rattler-build/patch/ros-humble-python-qt-binding.patch

@oursland
Copy link
Contributor Author

oursland commented Jan 1, 2025

I have been attempting local builds on macOS M2, but keep getting this error that I do not see in CI:

Error:
  × Failed to resolve dependencies: failed to unclobber clobbered files
  ├─▶ failed to unclobber clobbered files
  ├─▶ failed to rename include/X11/Xlib.h__clobber-from-xorg-libx11 to include/X11/Xlib.h
  ╰─▶ failed to rename file from /Users/jso/code/ros-jazzy/output/bld/rattler-build_ros-jazzy-theora-image-transport_1735719765/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include/X11/Xlib.h__clobber-from-xorg-libx11 to /Users/jso/code/ros-
      jazzy/output/bld/rattler-build_ros-jazzy-theora-image-transport_1735719765/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include/X11/Xlib.h: No such file or directory (os error 2)

Was this something you encountered and resolved, @Tobias-Fischer ?

@Tobias-Fischer
Copy link
Contributor

See prefix-dev/rattler-build#1288 - for me it’s typically sufficient to just rerun ..

@Tobias-Fischer
Copy link
Contributor

The good news is that all Linux and macOS runs succeeded - I’m tempted to merge here without the Windows builds and tackle them later seeing that there are still some problems with them in the humble PR.

What do you think @traversaro @oursland @wolfv ?

@traversaro
Copy link
Member

No problem on not having windows builds as we never had them for jazzy if I am not wrong.

@oursland
Copy link
Contributor Author

oursland commented Jan 1, 2025

I'm most definitely in favor of having builds for the platforms that are successful. For my personal use, linux-64, linux-aarch64, and macOS are all high priority.

@Tobias-Fischer
Copy link
Contributor

Ok - In this case, the PR is ready for review :)

@oursland
Copy link
Contributor Author

oursland commented Jan 2, 2025

See prefix-dev/rattler-build#1288 - for me it’s typically sufficient to just rerun ..

@Tobias-Fischer I can confirm that after many re-runs, I was able to succeed in rebuilding locally.

@Tobias-Fischer Tobias-Fischer merged commit 2507b86 into RoboStack:main Jan 3, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@traversaro traversaro traversaro left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@oursland @Tobias-Fischer @traversaro @wolfv