Examples realsense_mpc.py don't work

[adrien-cardinale]

Example stop after start the simulaiton with the python error:
Traceback (most recent call last):

  File "/home/c05pc13/curobo/examples/isaac_sim/realsense_mpc.py", line 475, in <module>

    articulation_controller.apply_action(art_action)

  File "/home/c05pc13/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.isaac.core/omni/isaac/core/controllers/articulation_controller.py", line 64, in apply_action

    if joint_positions[0][i] is None or np.isnan(

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

cuRobo installation mode : isaac sim python
python version: 3.10.14
Isaac Sim version (if using): 4.2.0

[from1tomax]

I met the same question, although I can run the other examples successfully. When I run omni_python mpc_example.py , it could open the isaac sim and everything is loaded successfully, after I press start, the simulation crash and complains:

[18.243s] [ext: omni.kit.asset_converter-2.1.21] startup
[18.264s] [ext: omni.kit.tool.asset_importer-2.5.5] startup
[18.332s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[18.334s] [ext: omni.kit.browser.core-2.3.11] startup
[18.338s] [ext: omni.kit.browser.folder.core-1.9.13] startup
[18.341s] [ext: omni.isaac.asset_browser-0.5.1] startup
Traceback (most recent call last):
  File "/home/peishi/devel/curobo/examples/isaac_sim/mpc_example.py", line 377, in <module>
    main()
  File "/home/peishi/devel/curobo/examples/isaac_sim/mpc_example.py", line 368, in main
    articulation_controller.apply_action(art_action)
  File "/home/peishi/.local/share/ov/pkg/isaac-sim-4.2.0/exts/omni.isaac.core/omni/isaac/core/controllers/articulation_controller.py", line 64, in apply_action
    if joint_positions[0][i] is None or np.isnan(
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
2024-10-10 22:24:32 [32,367ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2024-10-10 22:24:32 [32,368ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2024-10-10 22:24:32 [32,469ms] [Warning] [carb] Recursive unloadAllPlugins() detected!

cuRobo installation mode : isaac sim python
python version: 3.10.12
Isaac Sim version (if using): 4.2.0
nvidia driver: 560.35.03
CUDA Version: 12.6
CUDA toolkit: 11.8

[from1tomax]

I have fixed it by adding to .any() in file articulation_controller.py

def apply_action(self, control_actions: ArticulationAction) -> None:
        """[summary]

        Args:
            control_actions (ArticulationAction): actions to be applied for next physics step.
            indices (Optional[Union[list, np.ndarray]], optional): degree of freedom indices to apply actions to.
                                                                   Defaults to all degrees of freedom.

        Raises:
            Exception: [description]
        """
        applied_actions = self.get_applied_action()
        joint_positions = control_actions.joint_positions
        if control_actions.joint_indices is None:
            joint_indices = self._articulation_view._backend_utils.resolve_indices(
                control_actions.joint_indices, applied_actions.joint_positions.shape[0], self._articulation_view._device
            )
        else:
            joint_indices = control_actions.joint_indices

        if control_actions.joint_positions is not None:
            joint_positions = self._articulation_view._backend_utils.convert(
                control_actions.joint_positions, device=self._articulation_view._device
            )
            joint_positions = self._articulation_view._backend_utils.expand_dims(joint_positions, 0)
            for i in range(control_actions.get_length()):
                if joint_positions[0][i] is None or np.isnan(
                    self._articulation_view._backend_utils.to_numpy(joint_positions[0][i]).all()#add .any()
                ):
                    joint_positions[0][i] = applied_actions.joint_positions[joint_indices[i]]
        joint_velocities = control_actions.joint_velocities
        if control_actions.joint_velocities is not None:
            joint_velocities = self._articulation_view._backend_utils.convert(
                control_actions.joint_velocities, device=self._articulation_view._device
            )
            joint_velocities = self._articulation_view._backend_utils.expand_dims(joint_velocities, 0)
            for i in range(control_actions.get_length()):
                if joint_velocities[0][i] is None or np.isnan(joint_velocities[0][i]).all():#add.any()
                    joint_velocities[0][i] = applied_actions.joint_velocities[joint_indices[i]]
        joint_efforts = control_actions.joint_efforts
        if control_actions.joint_efforts is not None:
            joint_efforts = self._articulation_view._backend_utils.convert(
                control_actions.joint_efforts, device=self._articulation_view._device
            )
            joint_efforts = self._articulation_view._backend_utils.expand_dims(joint_efforts, 0)
            for i in range(control_actions.get_length()):
                if joint_efforts[0][i] is None or np.isnan(joint_efforts[0][i]).all():#add.any()
                    joint_efforts[0][i] = 0
        self._articulation_view.apply_action(
            ArticulationActions(
                joint_positions=joint_positions,
                joint_velocities=joint_velocities,
                joint_efforts=joint_efforts,
                joint_indices=control_actions.joint_indices,
            )
        )
        return

[benoit-robotics]

The issue might be that you are using Isaac sim 4.2.0 instead of the recommended version that is 4.0.0