...

CHI3D

631 multi-view sequences; 2,524 interaction contact events; 728,664 highly accurate ground truth 3d skeletons, GHUM & SMPLX human pose and shape parameters;

More Info Download

...

FlickrCI3D

Interaction Contact Signatures: 11,770 images; 14,866 contact events; 138,213 selected contact regions; 81,233 facet-level surface correspondences;
Interaction Contact Classification: 90,167 pairs of people;

More Info Download

Abstract

Understanding 3d human interactions is fundamental for fine grained scene analysis and behavioural modeling. However, most of the existing models focus on analyzing a single person in isolation, and those who process several people focus largely on resolving multi-person data association, rather than inferring interactions. This may lead to incorrect, lifeless 3d estimates, that miss the subtle human contact aspects–the essence of the event–and are of little use for detailed behavioral understanding. This paper addresses such issues and makes several contributions: (1) we introduce models for interaction signature estimation (ISP) encompassing contact detection, segmentation, and 3d contact signature prediction; (2) we show how such components can be leveraged in order to produce augmented losses that ensure contact consistency during 3d reconstruction; (3) we construct several large datasets for learning and evaluating 3d contact prediction and reconstruction methods; specifically, we introduce CHI3D, a lab-based accurate 3d motion capture dataset with 631 sequences containing 2,525 contact events, 728,664 ground truth 3d poses, as well as FlickrCI3D, a dataset of 11,216 images, with 14,081 processed pairs of people, and 81,233 facet-level surface correspondences within 138,213 selected contact regions. Finally, (4) we present models and baselines to illustrate how contact estimation supports meaningful 3d reconstruction where essential interactions are captured. Models and data are made available for research purposes at http://vision.imar.ro/ci3d.

Paper

...

Video

Citation

@InProceedings{Fieraru_2020_CVPR,
author = {Fieraru, Mihai and Zanfir, Mihai and Oneata, Elisabeta and Popa, Alin-Ionut and Olaru, Vlad and Sminchisescu, Cristian},
title = {Three-Dimensional Reconstruction of Human Interactions},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

 

Related Datasets

...

FlickrSC3D

Self-Contact Signatures: 3,415 images of 3,969 self-contact events; 25, 297 facet-level surface correspondences;
Self-Contact Classification:24,312 people;

More Info

...

HumanSC3D

1032 multi-view sequences; 4,128 self-contact events; 1,246,487 highly accurate ground truth 3d skeletons, GHUM & SMPLX human pose and shape parameters;

More Info

...

Fit3D Dataset

611 multi-view sequences; minimum 5 annotated repetitions per sequence; 2,964,236 highly accurate ground truth 3d skeletons, GHUM & SMPLX human pose and shape parameters;

More Info