A Deep Learning-Based Approach for Adaptive Virtual Learning with Human Facial Emotion Detection

Abstract

Classroom learning has become difficult since COVID-19 began. Students and educators have had to adapt to virtual learning by using the available tools and technologies. However, a virtual classroom does not simulate the same experience as a real, in-person classroom.

In this setting, teachers can immediately receive feedback on the students’ understanding of content by analyzing their facial expressions. By doing so, they can take immediate action to create a more effective learning experience. For example, teachers can individually help students that express an emotion of confusion by reiterating the concept privately and in greater detail. This style of teaching allows educators to ensure every student is receiving the appropriate amount of support and guidance. With online learning, this method of adaptive teaching is compromised. In a virtual class with video conferencing software such as Zoom, it is not practical for a teacher to be able to constantly check each students’ webcam while also teaching and managing technical difficulties. 

Utilizing classification models in deep learning, an advanced subfield of machine learning based on neural networks, offers  a novel approach to potentially working towards solving this issue. These models are trained with large datasets to mimic human behavior and achieve Artificial Intelligence (AI). The software simulates an effective virtual learning environment by using these methods to detect students’ emotions from facial expressions and providing educators this real time feedback. In this study, it was discovered that a convolutional neural network classification model produced results with the highest accuracy of 55.0%.