An experimental design was developed to investigate the real-time influences of an examination experience to assess the emotional realities students experience in higher education settings and tasks. This design is the result of a cross-disciplinary (e.g., educational psychology, biology, physiology, engineering) and multi-modal (e.g., salivary markers, surveys, electrodermal sensor) approach.
Over the past ten years, research into students' emotions in educational environments has increased. Although researchers have called for more studies that rely on objective measures of emotional experience, limitations on utilizing multi-modal data sources exist. Studies of emotion and emotional regulation in classrooms traditionally rely on survey instruments, experience-sampling, artifacts, interviews, or observational procedures. These methods, while valuable, are mainly dependent on participant or observer subjectivity and is limited in its authentic measurement of students' real-time performance to a classroom activity or task. The latter, in particular, poses a stumbling block to many scholars seeking to objectively measure emotions and other related measures in the classroom, in real-time.
The purpose of this work is to present a protocol to experimentally study students' real-time responses to exam experiences during an authentic assessment situation. For this, a team of educational psychologists, engineers, and engineering education researchers designed an experimental protocol that retained the limits required for accurate physiological sensor measurement, best-practices of salivary collection, and an authentic testing environment. In particular, existing studies that rely on physiological sensors are conducted in experimental environments that are disconnected from educational settings (e.g., Trier Stress Test), detached in time (e.g., before or after a task), or introduce analysis error (e.g., use of sensors in environments where students are likely to move). This limits our understanding of students' real-time responses to classroom activities and tasks. Furthermore, recent research has called for more considerations to be covered around issues of recruitment, replicability, validity, setups, data cleaning, preliminary analysis, and particular circumstances (e.g., adding a variable in the experimental design) in academic emotions research that relies on multi-modal approaches.
Psychologists have long understood the importance of humans' emotions in elucidating their behaviors1. Within the study of education, Academic Achievement Emotions (AEE) has become the focus of emotion research2. Researchers that use AAE argue that the situational contexts students find themselves in are important to consider when examining students' emotions. Students may experience test-related, class-related, or learning-related emotions that involve multi-component processes, including affective, physiological, motivational, and cognitive components. AEE is expressed in two forms: valence (positive/negative) and activation (focused/unfocused energy). Positive activating emotions, such as enjoyment, may increase reflective processes like metacognition, whereas positive deactivating emotions such as pride may result in low levels of cognitive processing. Negative activating emotions such as anger and anxiety may spark engagement, whereas negative deactivating emotions such as hopelessness may dampen motivation3,4,5. Academic emotions contribute to how we learn, perceive, decide, respond, and problem-solve2. To regulate academic emotions, an individual must possess self-efficacy (SE)6,7,8, which is their confidence in their ability to employ control over their motivation, behavior, and social environment6. Self-efficacy and academic emotions are interrelated, where lower self-efficacy is tied to negative deactivating emotions (e.g., anxiety, anger, boredom) and higher self-efficacy is tied to positive activating emotions (e.g., happiness, hope, excitement)6,7,8. SE is also believed to be strongly tied to performance6,7,8.
Research that has examined classroom emotions have relied on self-reports, observations, interviews, and artifacts (e.g., exams, projects)9,10. Although these methods provide rich contextual information about students' classroom experiences, they have significant limitations. For example, interviews, observations, and self-reports rely on individuals' introspections10. Other methods have sought to examine academic emotions more proximally than prior researchers, such as those based on experience-sampling approaches where researchers ask students to report on their emotions during the school day11. Although this research allows us to report students' emotions more accurately, this work relies on self-report methods and does not allow for real-time reporting as students have to pause their work on the exam to address the experience survey.
Recently, researchers have begun to address concerns about self-report measures through the use of biological or physiological measures of emotion9, that combined with other instruments or techniques such as surveys, observations, or interviews, consists of a multi-modal form of data collection for educational and psychological research12. For example, biological techniques, including salivary biomarkers, are being used to understand the role biological processes have on cognition, emotion, learning, and performance13,14,15. For cognitive processes, androgens (e.g., testosterone) have been linked to different spatial recognition patterns in adults and children16,17 whereas hypothalamic-pituitary-adrenocortical hormones (e.g., cortisol) and adrenergic hormones (e.g., salivary α-amylase or sAA) are linked to stress responsiveness amongst individuals18,19,20.
Electrodermal activity (EDA) represents a physiological measure of the activation of the autonomic nervous system (ANS) and is linked to increased activation of the system, cognitive load, or intense emotional responses21,22,23. In examination activities, EDA is affected by physical mobility21,22, bodily and ambient temperatures24,25,26,27, and verbalization of thoughts28, as well as sensitivity and degree of connectivity of the analog-digital electrodes to the skin29.
Although these can be limitations to using EDA, this technique can still provide valuable insight into what happens during near-real-time examinations and can serve as a promising tool to explore AEE and by extent, self-efficacy. As a result, an accurate picture of students' AEE can be obtained through a combination of survey methods, to determine the valence of emotion, and physiological and biological data, to measure the activation of that emotion. This paper builds upon a previous publication on examination activities30 and expands the scope of that work to include multi-modal approaches (using experience-sampling surveys, EDA sensors, and salivary biomarkers) in an examination scenario. It is essential to mention that the protocol described below allows for multiple participant data to be collected at the same time within a single experimental setting.
Although physiological measures have been used in many authentic learning contexts, it is critical to design a study environment that is mindful of the limits of the current technology. Our design balances the need for an authentic testing environment and accommodates the technology. Comfortably limiting participant movement, reducing unintended interruptions, and timestamping participants' testing responses are all critical steps within the protocol.
The space and expense of the electrode…
The authors have nothing to disclose.
This material is based upon work supported in part by the National Science Foundation (NSF) No. EED-1661100 as well as an NSF GRFP grant given to Darcie Christensen (No. 120214). Any opinions, findings, and conclusions or recommendations expressed in this material do not necessarily reflect those of NSF or USU. We want to thank Sheree Benson for her kind discussions and recommendations for our statistical analysis.
Author contributions in this paper are as follows: Villanueva (research design, data collection and analysis, writing, editing); Husman (research design, data collection, writing, editing); Christensen (data collection and analysis, writing, editing); Youmans (data collection and analysis, writing, and editing); Khan (data collection and analysis, writing, editing); Vicioso (data collection and analysis, editing); Lampkins (data collection and editing); Graham (data collection and editing)
1.1 cu ft medical freezer | Compact Compliance | # bci2801863 | They can use any freezer as long as it can go below -20 degrees Celsius; these can be used to store salivary samples for longer periods of time (~4 months) before running salivary assays. |
Camping Cooler | Amazon | (any size/type) | Can be used to store salivary samples during data collection |
E4 sensor | Empatica Inc | E4 Wristband Rev2 | You can use any EDA sensor or company as long as it records EDA and accelerometry |
EDA Explorer | https://eda-explorer.media.mit.edu/ | (open-source) | Can be used to identify potential sources of noise that are not necessarily due to movement |
Laptops | Dell | Latitude 3480 | They can use any desktop or laptop |
Ledalab | http://www.ledalab.de/ | (open-source) | Can be used to separate tonic and phasic EDA signals after following filtration steps |
MATLAB | https://www.mathworks.com/products/matlab.html | (version varies according to updates) | To be used for Ledalab, EDA Explorer, and to create customized time-stamping programs. |
Salivary Alpha Amylase Enzymatic Kit | Salimetrics | # 1-1902 | For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing |
Salivary Cortisol ELISA Kit | Salimetrics | # 1-3002 | For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing |
Testing Divider (Privacy Shields) | Amazon | #60005 | They can use any brand of testing shield as long as they cover the workspace |
Web Camera | Amazon | Logitech c920 | They can use any web camera as long as it is HD and 1080p or greater |