In psychology research, we talk a lot about how biases within a specific project create problems with validity, whether that be internal validity (issues with the actual design of a study) or external validity (issues in the ability of the study to be generalized to all populations). I am very interested in how biases within psychology studies lead to issues of external validity. Every day new pop culture articles are released to the hungry public, looking to stir up commotion on the most recent findings in psychology research.
—“New study says you can tell how smart your child will be based on the number of freckles on its nose!”—
These articles, which are all too popular with the general public, are often based on a small correlation or finding in a published study. Pop culture journalists latch onto these tiny findings. An issue with this is that many of these studies draw from a small, sometimes biased sample. For instance, consider the many psychology studies that occur on this campus. If you have taken an introductory psychology course here at the University of Michigan, or at any university, you may know that a lot of psychology studies use college students as their sample population. Why? Well, because it is easy to tell students that they must participate in studies voluntarily in order to receive points towards their grade.
So, when you have a study whose main sample population is college students, you run into some problems. College students are not like every other person on this planet: they are often put into a group that researchers call WEIRD. WEIRD stands for western, educated, industrialized, rich, and democratic. This is a common sampling bias in psychology research, as these are the individuals who are easy to find and keep as participants for studies.
Why does this cause a problem with external validity? Well, it keeps us from being able to generalize our research findings to any individual at any timepoint. If you only surveyed WEIRD participants, you can only generalize your findings to WEIRD individuals. So, if a study finds that number of nose freckles is correlated to intelligence, but the participants were all wealthy infants born on the same day in the same hospital, you can’t say that all other children will show the same pattern.
All in all, this is a problem that all researchers face each day, though it is more common amongst psychology researchers, who base most of their research on humans. In the research I am working on this summer, we utilize children and moms as our participants. Additionally, the study is longitudinal, as we must meet with the families at two different timepoints, which can be several months apart. This study requires that families who are involved must have reliable transportation, be okay with having researchers come into their homes, and keep communication with the research team. Not all families are equipped to participate in a study like this, and thus we end up with a biased sample population.
How do we work around this? Well, once results are available and meaningful, it is necessary to communicate your research findings in a way that conveys the issues with external validity. In order to do this, it is necessary to specify what population one’s results can be extended to, and which populations the results can’t be extended to. It is important to communicate your research this way in order to most truthfully inform your audiences. This is a key point of being a truthful researcher with integrity.