Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
ICCV 2023
Jiacong Xu
Yi Zhang
Jiawei Peng
Wufei Ma
Artur Jesslen
Pengliang Ji
Qixin Hu
Jiehua Zhang
Qihao Liu
Jiahao Wang
Wei Ji
Chen Wang
Xiaoding Yuan
Prakhar Kaushik
Guofeng Zhang
Jie Liu
Yushan Xie
Yawen Cui
Alan Yuille
Adam Kortylewski



Accurately estimating the 3D pose and shape is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. However, research in this area is held back by the lack of a comprehensive and diverse dataset with high-quality 3D pose and shape annotations. In this paper, we propose Animal3D, the first comprehensive dataset for mammal animal 3D pose and shape estimation. Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model. All annotations were labeled and checked manually in a multi-stage process to ensure highest quality results.Based on the Animal3D dataset, we benchmark representative shape and pose estimation models at: (1) supervised learning from only the Animal3D data, (2) synthetic to real transfer from synthetically generated images, and (3) fine-tuning human pose and shape estimation models. Our experimental results demonstrate that predicting the 3D shape and pose of animals across species remains a very challenging task, despite significant advances in human pose estimation. Our results further demonstrate that synthetic pre-training is a viable strategy to boost the model performance. Overall, Animal3D opens new directions for facilitating future research in animal 3D pose and shape estimation, and is publicly available.


The entire dataset can be download from google drive: Animal3D
Note that contains the original segmentation masks while consists of the masks rendered by our 3D annotations. Our SMAL and keypoint annotations are stored in train.json and test.json.

Supplementary Material

Animal3D contains a total of 3379 of images, which are classified into 40 clasees according to the official ImageNet labels. We further categorize all animal classes into five families (dog, tiger, cow, horse, hippo) that summarize objects with a similar overall shape.


Adam Kortylewski acknowledges support via his Emmy Noether Research Group funded by the German Science Foundation (DFG) under Grant No. 468670075. Alan Yuille acknowledges support from Army Research Laboratory award W911NF2320008 and Office of Naval Research N00014-21-1-2812. The template code of web can be found here.