Inverse Dynamics / Capability of Hand Model

[JudyYe]

Dear team,

Thank you for your great efforts!
I am using your suite to perform inverse dynamics, i.e. given a pose trajectory, compute muscle activation/excitation. However, I found that the myohand is able to replicate some trajectory while struggling to follow some others.

To simplify the problem, I have provided an ipynb example to inverse a "holding pose" trajectory for muscle activation.

min_example.zip

Just to summarize the gist of the example, the goal is to find $a$, such that $$a^* = \arg\min_{a} |FD(a, q, \dot q) - \hat{\tau}| \quad s.t. \quad 0 \preceq a \preceq 1 $$, where $FK(a, q, \dot q)$ is the HILL-type muscle force function from the document and your tutorial, and the desired $\tau$ is from inherent mujoco inverse dynamics, as suggested by your tutorial.

When I rollout the torques $\tau$, it can perfectly replicate the motion. (hand does not move since the motion is holding still.)

torque.mp4

But if optimizing $a$ with box constraints $[0, 1]$ cannot reconstruct the desired torques $\tau$.

01.mp4

Yet, if we relax the box constraints to $[-100, 100]$, then the torque can be reconstructed.

100.mp4

Finally, we notice that whether the pose is feasible to hold depends on the pose itself.

index	able to hold still?	results (left: target, right: activation)
0	N	https://github.com/user-attachments/assets/d3409853-d44e-457a-9a37-b360de6f1892
1	N	https://github.com/user-attachments/assets/876af1c2-6871-4015-8fcc-162e7ef4371d
2	Y	https://github.com/user-attachments/assets/ba63d7e9-8095-46f4-9ea1-58d81bda52fd
3	Y	https://github.com/user-attachments/assets/69e35118-7cd8-4589-91bc-bef4a5cac3fe
4	N	https://github.com/user-attachments/assets/77bef539-70c2-446a-9632-6313d546f374

I would appreciate your insight into why this could happen, if this suggests some capacity limit of myohand model. Thank you!

[andreh1111]

Hi @JudyYe,
I tested your trajectories with the ID code from tutorials and it works, so I'll start saying that your poses are feasible to hold.

I looked at your code, and the first thing that comes to mind as to why it doesn't work is that you are calculating a priori the target qfrc sequence, and then later you estimate act to get the qfrc that you are going to apply. However, in doing so you are not compensating for possible deviations within the sequence, thus leading to divergence. So basically at each time step you are assuming that the previous target position was achieved and you estimate the new act starting from that but, if the actual position is not the intended one, your trajectory diverges. It's actually true that for some poses this divergence is more likely than for others.

Try to do both qfrc and act estimations within the same iteration step, and let's see if this fixes your issue.

[JudyYe]

Hi Andrea,

Thank you for your prompt reply.

What you say is correct -- My code implements open-loop control while the tutorial is closed-loop. To be more precise, I was checking if the torque required, ie $\hat \tau$, to achieve the holding motion is feasible. The closed-loop control may avoid the problem by allowing (maybe unnoticeable) deviation from target pose trajectory.

Let us call everything related to target pose / torque with a hat.

$$ r_{OPEN} = | FD(a, \hat{q}, \hat{\dot{q}}) - \hat \tau | $$

$$ r_{CLOSE} = | FD(a, q, {\dot q}) - \tau | $$

I indeed started with your nice tutorial (thanks for writing it!). Your tutorial uses $r_{CLOSE}$ with a scaled down qfrc -- I noticed that the qfrc and velocity are scaled down by 100 and 5 times respectively. The tutorial says that the scaled down version helps to stabilize. It's true for my holding pose -- without scaling leads to oscillating motion while scaled down version works fine.

	video	activation - time
scale qfrc (vel) = 100 (5) (same in tutorial)	https://github.com/user-attachments/assets/2ced49c0-cc0c-4bac-8273-2f66cdd3d654
scale qfrc (vel) = 1 (1)	https://github.com/user-attachments/assets/0067679e-4de3-4ee0-bcd7-a218c17000b4

While the trick is fine to include for the tutorial motion, it first makes me wonder why the scale is necessary. Furthermore, when I try with more complicated motion, even scaling down by 100 times leads to very noisy oscillating motion too.

Therefore, I was hypothesizing that the un-scaled qfrc may be infeasible to achieve. and the feedback covers the problem by deviating from the target pose a bit. So instead of computing with the feedback loop, I implement an open loop process.

It may be too strict to ask for the rolling out the reconstructed torque $rollout(FD(a*, q, \dot q))$ to follow the trajectory $\dot q$, but I actually found a consistent residual error in $r_{OPEN}$ in $\tau$ space given different seeds.

This suggest that these $\tau$ are infeasible under $q$, $\dot q$. The $\tau$, however, should be rather easy to achieve as they are computed from maintaining a static pose.

[andreh1111]

Consider that the qfrc you get from mj_inverse depends on the acceleration you set, and in some cases (such as this one) setting the acceleration as in this code (and in the tutorial) is not the most effective method. One methodology I often employ is to use a PD controller to get the acceleration to provide to mj_inverse:
$qacc = (qpos_{target} - qpos) * kp - qvel * kv$
I tested your code with kp=kv=100 and it works for both positions 1 and 4.

This is mainly to say that it is not straightforward to use the qfrc obtained from mj_inverse as a "golden reference" since it depends on something we set first.

PS: in your code you are calculating qfrc using the estimated act and then setting it with qfrc_applied. Note, however, that the muscles are still applying a passive qfrc even though you are not setting act, and you included the passive component in the calculation of the qfrc you are going to apply too, thus doubling its contribution. This is not strictly related to my previous point, it's just something I noticed and wanted to make you aware of.