Expressive Personalized 3D Face Models from 3D Face Scans

Typ: Fortschritt-Berichte VDI
Erscheinungsdatum: 18.08.2020
Reihe: 10
Band Nummer: 868
Autor: Stella Graßhof, M.Sc.
Ort: Kopenhagen
ISBN: 978-3-18-386810-0
ISSN: 0178-9627
Erscheinungsjahr: 2020
Anzahl Seiten: 216
Anzahl Abbildungen: 57
Anzahl Tabellen: 6
Produktart: Buch (paperback DINA5)

Produktbeschreibung

In this work, different methods are presented to create 3D face models from databases of 3D face scans. The challenge in this endeavour is to balance the limited training data with the high demands of various applications.
The 3D scans stem from various persons showing different expressions, with varying number of points per 3D scan and different numbers of scans per person. This data of posed facial expressions revealed substructures, which are utilised to improve the proposed model. In the process of creating and using the models, for each specifc application objective quality criteria are carefully designed tailored to the task to quantify the quality.
In total four face models built from three databases are compared based on: 3D face synthesis, 3D approximation, person and expression transfer, and 3D reconstruction from 2D.

Contents

Abbreviations and Nomenclature XII
1 Introduction 1

1.1 The Difficulty of Quality Assessment . . . . . . . . . . . . . . 2
1.2 Face Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Data Preprocessing and Alignment . . . . . . . . . . . . . . . 8
1.4 Summary of Contributions . . . . . . . . . . . . . . . . . . . . 10
1.5 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Fundamentals 15
2.1 Camera Models . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Orthographic Camera Model . . . . . . . . . . . . . . 15
2.1.2 Weak-Perspective Camera Model . . . . . . . . . . . . 15
2.1.3 Projective Camera Model . . . . . . . . . . . . . . . . 16
2.2 Estimation of Camera Parameters . . . . . . . . . . . . . . . 17
2.3 Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 Principal Component Analysis . . . . . . . . . . . . . 20
2.3.2 Whitening . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.3 Correlation vs. Dependence . . . . . . . . . . . . . . . 22
2.3.4 Independent Component Analysis . . . . . . . . . . . 23
2.3.5 Projection Pursuit . . . . . . . . . . . . . . . . . . . . 26
2.4 Tensor Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.2 High-Order Singular Value Decomposition . . . . . . . 30
2.5 Numerical Optimization . . . . . . . . . . . . . . . . . . . . . 31
2.5.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5.2 Line-Search based Methods . . . . . . . . . . . . . . . 33
2.6 Generalized Canonical Time Warping . . . . . . . . . . . . . 37
3 Face Databases 40
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Selected Databases . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.1 BU3DFE . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.2 BU4DFE . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.3 Bosphorus . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.4 Facewarehouse . . . . . . . . . . . . . . . . . . . . . . 54
3.2.5 MMI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2.6 ADFES . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4 From 3D Face Scans to Aligned Faces 59
4.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1.1 Rigid Global Alignment . . . . . . . . . . . . . . . . . 60
4.1.2 Detection of Outliers . . . . . . . . . . . . . . . . . . . 60
4.1.3 Removing Points outside of the Face Region . . . . . . 61
4.2 Spatial Alignment by nonrigid Registration . . . . . . . . . . 66
4.2.1 Correspondence between Point Sets . . . . . . . . . . 66
4.2.2 Nonrigid 3D Registration . . . . . . . . . . . . . . . . 68
4.2.3 Quantifying Quality . . . . . . . . . . . . . . . . . . . 78
4.2.4 Experiments and Evaluation . . . . . . . . . . . . . . 84
4.3 Temporal Alignment . . . . . . . . . . . . . . . . . . . . . . . 100
4.3.1 Quantifying Expression Intensity . . . . . . . . . . . . 101
4.3.2 Alignment of Expression Intensities . . . . . . . . . . . 107
4.3.3 Applications for Proposed Expression Intensities . . . 110
5 Face Models 114
5.1 Surrey’s 3D Morphable Face Model . . . . . . . . . . . . . . . 114
5.2 Sela’s Neural Network for detailed 3D Face Reconstruction . 115
5.3 Proposed Tensor Face Models . . . . . . . . . . . . . . . . . . 116
5.3.1 The Expression Space and the Apathy Mode . . . . . 117
5.3.2 Model 1: Basic Model . . . . . . . . . . . . . . . . . . 124
5.3.3 Model 2: Subspace-aware Parameterization . . . . . . 128
5.3.4 Model 3: Projection Pursuit in Expression Space . . . 132
5.3.5 Model 4: Four-Way Model including Expression Strength134
5.3.6 Overview of Presented Tensor Face Models . . . . . . 141
5.4 Quality of Face Models . . . . . . . . . . . . . . . . . . . . . . 142
6 Experiments 146
6.1 Facial Animation by Improved Synthesis Using Apathy . . . . 146

6.2 3D Approximation, Person and Expression Transfer . . . . . 149
6.2.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 153
6.3 Dense 3D Reconstruction from sparse 2D . . . . . . . . . . . 156
6.3.1 3D Reconstruction With Ground Truth . . . . . . . . 156
6.3.2 3D Reconstruction Without Ground Truth . . . . . . 170
6.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 170
7 Summary and Conclusions 174
7.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Appendix 179
A 3D Rotations and Computing Optimal Angles . . . . . . . . . 179
B Normal Vector of 3D Points . . . . . . . . . . . . . . . . . . . 182
C Parameterization of Lines along Principal Axis . . . . . . . . 182
D Apathy Estimation – How to Find the Closest Point . . . . . 183
E Examples of Dense 3D Reconstruction of Bosphorus Database 185
Literature 189
Index 199

Keywords: Stella Graßhof, TNT, Universität Hannover, Informatik, Kommunikation, Fortschritt-Berichte VDI, Reihe 10, Band 868, 3D-Gesichts-Scan, nicht-rigide Registrierung, Korrespondenzschätzung, Intensität von Gesichtsausdrücken, Tensor, statistische Modelle, Transfer von Gesichtsausdrücken, 3D-Rekonstruktion, 3D face scans, nonrigid registration, correspondence estimation, expression intensity, tensor, factorization, statistical models. expression transfer, 3D reconstruction

76,00 € inkl. MwSt.
VDI-Mitgliedspreis:*
68,40 € inkl. MwSt.

* Der VDI-Mitgliedsrabatt gilt nur für Privatpersonen

Contents

Abbreviations and Nomenclature XII
1 Introduction 1

1.1 The Difficulty of Quality Assessment . . . . . . . . . . . . . . 2
1.2 Face Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Data Preprocessing and Alignment . . . . . . . . . . . . . . . 8
1.4 Summary of Contributions . . . . . . . . . . . . . . . . . . . . 10
1.5 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Fundamentals 15
2.1 Camera Models . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1.1 Orthographic Camera Model . . . . . . . . . . . . . . 15
2.1.2 Weak-Perspective Camera Model . . . . . . . . . . . . 15
2.1.3 Projective Camera Model . . . . . . . . . . . . . . . . 16
2.2 Estimation of Camera Parameters . . . . . . . . . . . . . . . 17
2.3 Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 Principal Component Analysis . . . . . . . . . . . . . 20
2.3.2 Whitening . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.3 Correlation vs. Dependence . . . . . . . . . . . . . . . 22
2.3.4 Independent Component Analysis . . . . . . . . . . . 23
2.3.5 Projection Pursuit . . . . . . . . . . . . . . . . . . . . 26
2.4 Tensor Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.2 High-Order Singular Value Decomposition . . . . . . . 30
2.5 Numerical Optimization . . . . . . . . . . . . . . . . . . . . . 31
2.5.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5.2 Line-Search based Methods . . . . . . . . . . . . . . . 33
2.6 Generalized Canonical Time Warping . . . . . . . . . . . . . 37
3 Face Databases 40
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Selected Databases . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.1 BU3DFE . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.2 BU4DFE . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.3 Bosphorus . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.4 Facewarehouse . . . . . . . . . . . . . . . . . . . . . . 54
3.2.5 MMI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2.6 ADFES . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4 From 3D Face Scans to Aligned Faces 59
4.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.1.1 Rigid Global Alignment . . . . . . . . . . . . . . . . . 60
4.1.2 Detection of Outliers . . . . . . . . . . . . . . . . . . . 60
4.1.3 Removing Points outside of the Face Region . . . . . . 61
4.2 Spatial Alignment by nonrigid Registration . . . . . . . . . . 66
4.2.1 Correspondence between Point Sets . . . . . . . . . . 66
4.2.2 Nonrigid 3D Registration . . . . . . . . . . . . . . . . 68
4.2.3 Quantifying Quality . . . . . . . . . . . . . . . . . . . 78
4.2.4 Experiments and Evaluation . . . . . . . . . . . . . . 84
4.3 Temporal Alignment . . . . . . . . . . . . . . . . . . . . . . . 100
4.3.1 Quantifying Expression Intensity . . . . . . . . . . . . 101
4.3.2 Alignment of Expression Intensities . . . . . . . . . . . 107
4.3.3 Applications for Proposed Expression Intensities . . . 110
5 Face Models 114
5.1 Surrey’s 3D Morphable Face Model . . . . . . . . . . . . . . . 114
5.2 Sela’s Neural Network for detailed 3D Face Reconstruction . 115
5.3 Proposed Tensor Face Models . . . . . . . . . . . . . . . . . . 116
5.3.1 The Expression Space and the Apathy Mode . . . . . 117
5.3.2 Model 1: Basic Model . . . . . . . . . . . . . . . . . . 124
5.3.3 Model 2: Subspace-aware Parameterization . . . . . . 128
5.3.4 Model 3: Projection Pursuit in Expression Space . . . 132
5.3.5 Model 4: Four-Way Model including Expression Strength134
5.3.6 Overview of Presented Tensor Face Models . . . . . . 141
5.4 Quality of Face Models . . . . . . . . . . . . . . . . . . . . . . 142
6 Experiments 146
6.1 Facial Animation by Improved Synthesis Using Apathy . . . . 146

6.2 3D Approximation, Person and Expression Transfer . . . . . 149
6.2.1 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 153
6.3 Dense 3D Reconstruction from sparse 2D . . . . . . . . . . . 156
6.3.1 3D Reconstruction With Ground Truth . . . . . . . . 156
6.3.2 3D Reconstruction Without Ground Truth . . . . . . 170
6.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 170
7 Summary and Conclusions 174
7.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Appendix 179
A 3D Rotations and Computing Optimal Angles . . . . . . . . . 179
B Normal Vector of 3D Points . . . . . . . . . . . . . . . . . . . 182
C Parameterization of Lines along Principal Axis . . . . . . . . 182
D Apathy Estimation – How to Find the Closest Point . . . . . 183
E Examples of Dense 3D Reconstruction of Bosphorus Database 185
Literature 189
Index 199

", "genre": "Informationstechnik", "isbn": "978-3-18-386810-0", "name": "Expressive Personalized 3D Face Models from 3D Face Scans", "numberOfPages": "216", "publisher": { "@type": "Organization", "name": "VDI Verlag GmbH", "url" : "https://www.vdi-verlag.de" }, "author" : { "@type" : "Person", "name" : "Stella Graßhof, M.Sc." }, "offers": { "@type": "Offer", "availability": "http://schema.org/InStock", "price": "76,00", "priceCurrency": "EUR" } }