What's the most reliable approach for getting a person's orientation given their bbox, 3d skeleton, 2d skeleton and homography projection plane of the floor?

[u/DiddlyDinq]

As per the title, I've trying to find a way of calculating orientation of a person. I assumed it would be fairly straight forward of doing a cross product of the shoulder to shoulder vector and the spine vectors but it seems to be unreliable. I cant use motion vectors for this use case either as it needs to support people walking backwards or sideways. Any ideas?

I'm only focused on their yaw value rather than full 3d orientation

[u/AbsorbedEpilepsy5]

shoulder cross product is kinda janky if the pose estimation is ever noisy, which it always is in the wild

have you tried projecting the hip and shoulder midpoints down to the floor plane and just taking the vector between them, smooth it over a few frames with a simple moving average to kill the jitter

[u/_d0s_]

Do you try to fit a plane through all keypoints?

[u/Nice-Bodybuilder-862]

KAMA(key point aware mesh articulation) + solid angles gets you what you need. Lemme know if you need example repo, I got one.

[u/soylentgraham]

if your shoulder data is unreliable, what data IS reliable?

how would you do it with a human?

do you have left/right hips?