Converting animations between skeletons

I have written one more piece about the skeleton merge about half year ago - you can read it here. What follows is the a much better version of the original algoritm.

The problem

Say you have some good animations (there are many free BVH sites in web) and want to use these for your own models. Unfortunately - unless you made your models using exactly the same skeleton layout, the bones do not match and copying animations is not trivial problem.

There are many ways, how skeletons may differ:

• Bone names do not match. This is easy to overcome - either build mapping or rename bones.
• Rest pose orientations use different systems (like XYZ euler angles vs. ZXY euler angles vs. quaternions). This is also minor problem, as normally you want to convert everything to quaternions or matrices anyways.
• Bone lengths do not match. Thus copying animation directly can make movement unnatural. 
• Rest pose bone positions (directions) do not match. Thus animations can not be immediately copied between skeletons, but have to be transformed to absolute space (object space) before.
• Skeleton topologies do not match. This is often a showstopper, because if you do not even have similar bones, there is no way to copy animations. 

Example of different common skeleton topologies

The topology mismatch is the hardest problem to overcome. In the example above, if you, say have animation for the right skeleton, the 4 parts of backbone should move the 2 backbone parts and pelvis of the left skeleton. While in theory it should be possible to solve this problem analytically, it is not easy - and has to be done by hand for each combination of skeletons.
Instead of that, using IK solvers gives us easier way to map those skeletons. It is still not universal - the mapping algorithms have to be adjusted to object itself - in current case a humanoid. But normally this is not big problem, because in 99% cases humanoid is precisely what you want to animate.

In any case you need good IK solver, that can do at least 6-7 bones long chains, adjust both positions and rotations and use both absolute positions and bone directions for targets. Also, the solver has to support comparing up to 4 target points or directions simultaneously. I am currently using simplex method in Khayyam (the code is in helpers/ik.cpp), but there are other options as well.
Also the skeleton should have bone constraints defined - otherwise you may discover that elbows or knees will be bent to completely wrong direction.

We start implementing the algorithm by a small observation. Namely - while there are different ways to construct skeletons, the joints have to be still placed to precise spots - otherwise the movement of joints will not be natural. Thus we can use the joints as anchor spots for the algorithm.

The body

The first and hardest part is moving the main bodies of skeletons to the same position. First - the root bones may be arranged differently. Second - root bones may be joined with body either below or above pelvis. Third - there may be different number of backbone elements. Fourth - there may or may not be extra bones, like hips or collar bones.
To solve this problem, we can do the following:

• Set 4 IK target points to the corner joints of body - both hips and both shoulders.
• Scale the source skeleton so, that the distances between hips and shoulders are approximately the same, as in target skeleton.
• Find the topmost common parents of both hips and shoulders Ah and As.
• Find the topmost common parent of Ah and As B.
• Build rotating IK chain of bones from  B to Ah and from B to As, not including B.
• Add both the rotations and translations of the rootmost bone R to chain. 

The schematics of body IK solver (green: translate + rotate, red: rotate, blue: not moving)

After solving this IK chain, the corner joins of body will be approximately aligned to the original. Normally you do not achieve perfect alignment, because the width of shoulders and hips is different in skeletons - but this is not big problem, because the direction of hip-hip and shoulder-shoulder lines should be aligned.
It is important to do this alignment in one step - otherwise the alignment of hips may involve extensive rotations and translations of root bone R - and you cannot later align shoulders at all.

Limbs 

To align limbs, there are two strategies - depending on what we want to achieve.

• We can align the endpoints - if we want to model some specific action - like taking some item from table. In thus case the hand will be in correct place - but because the lengths of arms and forearm probably differ, elbow will be off.
• Or we can align the directions of bones - for example if modeling hand movement during walking - in that case both elbow and wrist will be off, but the overall look of hand will be more similar.

At the first case it is just ordinary single-target IK solver. Or if you want to keep the elbow/knee direction more similar to original, there should be additional directional target at midpoint.
At the second case, the IK chain has to be solved with two directional targets (shoulder-elbow + elbow-wrist or hip-knee + knee-ankle) simultaneously. Otherwise the topmost bone may turn it to such direction, that 1 DOF joint at midpoint (elbow or knee) cannot move other bone to the right direction.

Palm

There are probably different strategies for adjusting palms. In khayyam I did the following:

• Set 4 directional targets from wrist to the topmost joints of fingers
• Add the 3 DOF rotations of palm to IK chain
• If forearm includes extra rotational bone (like at the skeleton at left), include the axial rotation of it to chain 

By solving this chain, the positions of finger joints will be approximately aligned to the original.

Fingers

Fingers can be aligned in the same way as limbs - but here we need three targets instead of two. Also, as the fingertips are normally the end sites, the last target does not coincide with any joint, but has to be placed about 2-3 cm from the last joint along the bone direction.

Head

Head should be placed by adjusting three positional targets - eyes and skulltop. This can become tricky, because many skeletons may not include eye positions at all.

Sara chan in skeleton merge editor (blue: figure skeleton, red: animation skeleton)

At the above picture you can see the results of adjustment of Sara-chan skeleton to Illusion Characolle animated skeleton. Although the skeleton topologies and bone lengths differ, the pose is pretty close to the original.

At moment the skeleton merge works only in CVS version of Khayyam, but expect it in the next release.