1 THE EXTENDED COHN-KANADE DATASET(CK+):A COMPLETE DATASET FOR ACTION UNIT AND EMOTIONSPECIFIED EXPRESSION Author：Patrick Lucey, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar Conference on Computer Vision and Pattern Recognition 2010 Speaker：Liu, Yi-Hsien 2 Outline • Introduction • The CK+ Dataset • Emotion Labels • Baseline System • Experiments • Conclusion 3 Introduction • In 2000, the Cohn-Kanade (CK) database was released • Automatically detecting facial expressions has become an increasingly important research area 4 Introduction(Cont.) • The CK database contains 486 sequences across 97 subjects. • Each of the sequences contains images from onset (neutral frame) to peak expression (last frame). • The peak frame was reliably FACS(Facial Action Coding System ) coded for facial action units (AUs). 5 Introduction(Cont.) • Facial Action Coding System (FACS) is a system to taxonomize human facial movements by their appearance on the face 6 Introduction(Cont.) • While AU codes are well validated, emotion labels are not • The lack of a common performance metric against which to evaluate new algorithms • Standard protocols for common databases have not emerged 7 The CK+ Dataset • Participants were 18 to 50 years of age, 69% female, 81% Euro-American, 13% Afro-American, and 6% other groups • Image sequences for frontal views and 30-degree views were digitized into either 640x490 or 640x480 pixel arrays with 8- bit gray-scale or 24-bit color values. 8 The CK+ Dataset(Cont.) • For the CK+ distribution, they have augmented the dataset further to include 593 sequences from 123 subjects (an additional 107 (22%) sequences and 26 (27%) subjects). • For the 593 posed sequences, full FACS coding of peak frames is provided. 9 Emotion Labels • They included all image data from the pool of 593 sequences that had a nominal emotion label based on the subject’s impression of each of the 7 basic emotion categories: Anger, Contempt, Disgust, Fear, Happy, Sadness and Surprise. 10 Emotion Labels(Cont.) 1. Compared the FACS codes with the Emotion Prediction Table from the FACS 2. After the first pass, a more loose comparison was performed 3. The third step involved perceptual judgment of whether or not the expression resembled the target emotion category. 11 Emotion Labels(Cont.) 12 Emotion Labels(Cont.) • As a result of this multistep selection process, 327 of the 593 sequences were found to meet criteria for one of seven discrete emotions. 13 Baseline System 14 Baseline System(Cont.) • Active Appearance Models (AAMs) • The shape s of an AAM is described by a 2D triangulated mesh. • In particular, the coordinates of the mesh vertices define the shape s = [x1; y1; x2; y2; …. ; xn; yn] 15 Baseline System(Cont.) • SPTS：The similarity normalized shape, refers to the 68 vertex points for both the x- and y- coordinates, resulting in a raw 136 dimensional feature vector • CAPP： The canonical normalized appearance, refers to where all the shape variation has been normalized with respect to the base shape 16 Baseline System(Cont.) • SVMs(Support Vector Machines) attempt to find the hyper plane that maximizes the margin between positive and negative observations for a specified class. 17 Experiments • Emotion detection. • To maximize the amount of training and testing data, they believe the use of a leave-one-subject-out cross- validation configuration should be used. 18 Experiments(Cont.) • SPTS 19 Experiments(Cont.) • CAPP 20 Experiments(Cont.) • SPTS+CAPP 21 Conclusion • In this paper, they try to address those three issues by presenting the Extended Cohn-Kanade (CK+) database • Added another 107 sequences as well as another 26 subjects. • The peak expression for each sequence is fully FACS coded and emotion labels have been revised and validated 22 Conclusion(Cont.) • Propose the use of a leave-one-out subject cross- validation strategy for evaluating performance • Present baseline results on this using our Active Appearance Model (AAM)/support vector machine (SVM) system.