HuMoR

HuMoR: 3D Human Motion Model for Robust Pose Estimation

원래 내 생각보다 더 대단하고 영향력이 더 있고, 앞으로 더 참고할만한 논문인 것 같음.

During training, HuMoR learns a conditional Variational AutoEncoder that estimates different transition distributions per previous timestep’s state x (includes joint location, velocity, etc… ) - See “Section 3 Latent Variable Dynamics Model”

 

In addition to the encoder of VAE that encodes the transition latent between state x_(t-1) and x_t and that decodes the change in states, HuMoR has a conditional prior encoder that learns the transition probability given the condition x_(t-1). This enables the evaluation of the likelihood of the transition latent z 

 

During test-time optimization of SMPL, the above conditional prior encoder can be used to evaluate the likelihood of the transition latent z predicted by the VAE encoder given consecutive states x_(t-1) and x_t. - See “3.3. Incorporating learned human motion priors” of SLAHMR

 

Pros.

1. Maybe we can use this motion prior for HumanPlus’ HumanShadowingTransformer as one of the rewards, if we can retarget the human state to the robot state 

 

Cons.

1. Statice camera and plain ground assumption