Multimodal Affective Computing : Technologies and Applications in Learning Environments 9783031325427, 9783031325410

Table of contents :
1 Affective Computing
1.​1 Introduction
1.​2 Theories of Emotions, Sentiments, and Affect
1.​3 Theories of Personality and Learning
1.​3.​1 Main Personality Theories
1.​3.​2 The Effect of Personality on Learning
1.​4 Cognitive Processing and Learning-Oriented Emotions
1.​5 Emotions, Sentiment, Personality, and the Machine
1.​6 Discussion
References
2 Machine Learning and Pattern Recognition in Affective Computing
2.​1 Introduction
2.​2 Input Data in Affective Computing
2.​3 Machine Learning Variants and Models
2.​3.​1 Supervised Learning
2.​3.​2 Unsupervised Learning
2.​3.​3 Other Learning Variants
2.​4 Dimensionality Reduction
2.​5 Deep Learning
2.​5.​1 Neural Networks
2.​5.​2 Convolutional Neural Networks
2.​5.​3 Sequential Models
2.​5.​4 Other DL-Based Models
2.​6 Discussion
References
3 Affective Learning Environments
3.​1 Introduction
3.​2 The Dynamics of Teaching and Learning
3.​3 Theoretical Models for the Role of Affect in Learning
3.​4 Design of Affective Learning Environments
3.​5 Discussion
References
Part II Sentiment Analysis for Learning Environments
4 Building Resources for Sentiment Detection
4.​1 Introduction
4.​2 Experimental Setup Design
4.​3 Data Mining System Design and Implementation
4.​4 Data Mining Challenges
4.​5 Discussion
References
5 Methods for Data Representation
5.​1 Introduction
5.​2 Tokenization
5.​3 Parsing
5.​4 Stemming and Lemmatization
5.​5 Word Embeddings
5.​6 Discussion
References
6 Designing and Testing the Classification Models
6.​1 Introduction
6.​2 Lexicon-Based Sentiment Analysis
6.​3 Multilayer Perceptron
6.​4 Convolutional Neural Networks
6.​5 Long Short-Term Memory Neural Networks
6.​6 Evaluation Protocols
6.​7 Discussion
References
7 Model Integration to a Learning System
7.​1 Introduction
7.​2 Building Resources
7.​3 Dataset Focused on the Programming Language Domain
7.​4 Creation of a Dictionary of Emotions Focused on Learning (SentiDICC)
7.​5 Model Selection Process
7.​6 Evaluation Metrics
7.​7 Model Training and Validation
7.​8 Affective Learning Environment
7.​8.​1 Model Implementation
7.​8.​2 Affective Tutoring Agent
7.​9 Discussion
References
Part III Multimodal Recognition of Learning-Oriented Emotions
8 Building Resources for Emotion Detection
8.​1 Introduction
8.​2 Experimental Setup Design
8.​2.​1 Selecting Data Modalities
8.​2.​2 Labeling Process
8.​2.​3 Work Environment
8.​2.​4 Emotion Elicitation
8.​3 Discussion
References
9 Methods for Data Representation
9.​1 Introduction
9.​2 Image-Based Data Representation for Facial Expressions
9.​3 Spectrogram-Based Data Representation for Speech
9.​4 Signal-Based Data Representation for Physiological Data
9.​5 Practical Considerations for Choosing Data Representation Methods
9.​6 Discussion
References
10 Multimodal Recognition Systems
10.​1 Introduction
10.​2 Data Fusion Techniques
10.​3 Convolutional Neural Networks in Multimodal Emotion Recognition
10.​4 Long Short-Term Memory in Multimodal Emotion Recognition
10.​5 Evaluation Protocols
10.​6 Discussion
References
11 Multimodal Emotion Recognition in Learning Environments
11.​1 Introduction
11.​2 Enhancing the Student Motivation, Engagement, and Cognitive Processing
11.​3 Dataset Creation
11.​3.​1 Labeling Process
11.​3.​2 Fusing Different Datasets
11.​4 Defining DL Architectures
11.​4.​1 Convolutional Neural Networks
11.​4.​2 Long Short-Term Memory
11.​5 Evaluation Protocols
11.​6 Model Deployment
11.​6.​1 Data Pipelines
11.​6.​2 Model Interpretation
11.​7 Affective Tutoring Agent
11.​8 Discussion
References
Part IV Automatic Personality Recognition
12 Building Resources for Personality Recognition
12.​1 Introduction
12.​2 Data Structure Design
12.​3 Personality Data Annotation
12.​4 Applications for Data Collection
12.​5 Discussion
References
13 Methods for Data Representation
13.​1 Introduction
13.​2 Speech Data Representation
13.​3 Text Data Representation
13.​4 Facial Expressions Data Representation
13.​5 Physiological Signals Data Representation
13.​6 Differences Between Emotion and Personality Data Representation
13.​7 Discussion
References
14 Personality Recognition Models
14.​1 Introduction
14.​2 Unimodal Architectures
14.​3 Multimodal Architectures
14.​4 Discussion
References
15 Multimodal Personality Recognition for Affective Computing
15.​1 Introduction
15.​2 Design of a Data Structure
15.​2.​1 Collecting a Dataset for APP
15.​2.​2 Creating a Dataset for APR
15.​2.​3 Apparent Personality Perception (APP) Versus Automatic Personality Recognition (APR)
15.​3 An Application for Data Collection
15.​3.​1 Architectural Model of the Platform
15.​4 Data Recollection Process
15.​5 Adapting a Dataset to a Working Environment
15.​5.​1 Image Preprocessing
15.​5.​2 Sound Preprocessing
15.​6 Personality Recognition Model Design
15.​6.​1 Image-Based Models
15.​6.​2 Sound-Based Models
15.​6.​3 Multimodal Models
15.​7 Laboratory Tests
15.​8 Models as a Service
15.​9 Personality Recognition in Education
15.​10 Discussion
References
Glossary