|
Accurately estimating the 3D pose and shape is an essential step towards understanding
animal behavior, and can potentially benefit many downstream applications, such as wildlife
conservation. However, research in this area is held back by the lack of a comprehensive
and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we
propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation.
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations
of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
All annotations were labeled and checked manually in a multi-stage process to ensure highest
quality results.Based on the Animal3D dataset, we benchmark representative shape and pose
estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to
real transfer from synthetically generated images, and (3) fine-tuning human pose and shape
estimation models. Our experimental results demonstrate that predicting the 3D shape and
pose of animals across species remains a very challenging task, despite significant advances
in human pose estimation. Our results further demonstrate that synthetic pre-training is a
viable strategy to boost the model performance. Overall, Animal3D opens new directions for
facilitating future research in animal 3D pose and shape estimation, and is publicly available.
|
