Source: Laboratories of Gary Lewandowski, Dave Strohmetz, and Natalie Ciarocco—Monmouth University
One of the challenges in measuring an experimental variable is identifying the technique that will produce the more accurate measurement. The most common way to measure a dependent variable is self-report—asking the participant to describe his/her feelings, thoughts, or behaviors. Yet, people may not be honest. To truly know something about a participant, it may be necessary to see what they actually do in a situation.
This video uses a multi-group experiment to see if feeling close to others results in more favorable attitudes toward environmental consciousness measured both by self-report and behavioral observation.
Psychological studies often use higher sample sizes than studies in other sciences. A large number of participants helps to better ensure that the population under study is better represented, i.e., the margin of error accompanied by studying human behavior is sufficiently accounted for. In this video, we demonstrate this experiment using just one participant. However, as represented in the results, we used a total of 186 participants to reach the experiment’s conclusions.
This procedure was repeated 185 times so that the results reflect data from 186 total participants. This large number of participants helps to ensure that the results reflect accurate mean numbers of items recycled. If this research were conducted using just one or two participants, it is likely that the results would have been much different, and not reflective of the greater population. Numbers in Figure 1 reflect participants’ average responses for the recycling questions on the survey. Numbers in Figure 2 represent the number of items (out of 10 total) the participant properly put in the recycling bin.
To determine if there were differences between the three picture conditions, an analysis of variance (ANOVA) was performed. The results indicated that though there were significant differences for the self-report survey, such that those in the feeling-close-to-people condition said they cared about recycling more, there were no significant differences when it came to participants’ actual behavior.
Figure 1. Average recycling attitude score by condition. Means were calculated from survey questions 3, 5, and 7.
Figure 2. Average number of properly recycled items by condition. Means were calculated from researcher’s observations of items placed in the recycling bin.
This multi-group experiment shows how participants’ feelings measured via a survey may not match up exactly with how they behave. In this case, participants reported more favorable attitudes (especially in the feeling-close-to-people condition), but when it came time to actually recycle, participants did not engage in the behavior.
This study replicates and extends previous research on self-reported recycling attitudes vs. actual recycling by Huffman et al., which showed that actual recycling was not associated with recycling attitudes and weakly correlated with self-reported recycling (i.e., did they recycle in the past).1
Several papers have emphasized the importance of behavioral measures, including Baumeister et al. and Lewandowski and Strohmetz.2,3 In each case, the key take away was that what a person actually does is more revealing than what the same person claims they will do. Though more challenging, measuring behavior is more informative.
Psychologists study many aspects of behavior by asking participants to report their own feelings or by directly observing their performance on specific tasks.
While self-report is often used, the authenticity of reported responses is difficult to determine. Therefore, observing participants’ behaviors during specific situations provides more reliable data.
Here, psychologists compare two forms of measurement, self-report and behavioral observation, to see if participants’ feelings reported through a survey correlate with their actual behavior.
This video demonstrates how to design, conduct, analyze, and interpret an experiment that investigates whether feeling close to others results in more favorable attitudes toward recycling.
During this experiment, participants are randomly divided into one of three groups: Feeling Close to People, Feeling Close to Animals, or No Connection. In this case, feeling close to people or animals involves perceived connections such as emotional attachments toward other people or animals.
Participants are then asked to view slides in which five pictures of people, animals, or non-living household items are shown, according to their group assignment. They are all then given the same two surveys to complete.
The first survey involves self-reported measures regarding environmental practices for protecting the natural world, whereas the second survey serves as a distraction while the researcher creates a scene for participant observations.
In this scene, the researcher pretends to eat lunch out of the participant’s sight. The meal includes a variety of recyclable and non-recyclable items.
After participants finish the second survey, they are asked to help clear off the table where lunch was eaten while the researcher steps away.
The dependent variables, the values reported on the first self-report survey and the number of items that each participant recycled when clearing the lunch table, are compared.
Participants in the Feeling Close to People group are hypothesized to have more favorable attitudes towards recycling and would recycle more items compared to the other groups.
To begin the study, meet the participant in the laboratory. Provide them with informed consent and discuss the overall procedure of the experiment.
Randomly assign each participant to one of three experimental groups: Feeling Close to People, Feeling Close to Animals, or No Connection, to view five pictures of people, animals, or non-living household items, respectively.
To assure that participants pay attention to the images, have each one write down how close or connected they feel to each image on a scale of 1 to 10.
After the participant views the images, hand them two surveys and instruct them to take a minute to consider each question and indicate how they feel.
While the participant fills out the second survey, pretend to eat lunch out of their sight.
After the participant completes the last survey, ask them to help clear off the lunch table while you prepare for the next part of the study. Have them place garbage into the garbage bin and recyclable items in the recycling bin.
When the participant finishes, record how many items were correctly placed into the recycling bin.
To conclude the experiment, debrief the participant and explain why deception was necessary for the experiment.
To analyze the data, first calculate the participants’ recycling attitude score for each condition by averaging the ratings from items 3, 5, and 7 on Survey 1. Note that the other items on this survey, as well as the second survey with yes or no questions, serve as distractors. Graph the mean recycling attitude by condition.
Notice that those in the Feeling Close to People condition indicated that they cared more about recycling than controls.
To determine the observed behavioral score, average the number of items that were counted as being correctly recycled for each condition. Graph the means across conditions. Notice that no significant differences were observed between groups.
Notice that participants who reported favoring attitudes towards recycling did not actually engage in the behavior. In other words, participants’ feelings were not correlated with their observed behaviors.
Now that you are familiar with how to compare self-report surveys against actual observed behavior, you can apply this approach to other forms of research.
Several studies have emphasized the importance of measuring behavior over the use of self-reports to better understand emotion. The most thorough studies collect data from a set of questionnaires, as well as correlate the answers with results from a goal-oriented behavioral task.
In addition, individuals with an addiction, such as cigarette smoking, do not always provide reliable answers to self-report questions. Researchers must use alternative methods, such as fMRI or PET imaging, to better understand smoking behavior.
Finally, self-report questionnaires about sleep are only as informative as the individual’s memory or awareness of their sleep behavior. Therefore, using polysomnography to observe participants during sleep is especially useful when trying to understand abnormal occurrences.
You’ve just watched JoVE’s introduction to compare and contrast measurements of self-reporting and observed behavior. Now you should have a good understanding of how to design and conduct this type of study by administering surveys and directly monitoring behavior, as well as how to analyze and interpret the results.
Thanks for watching!