The difference between internal and external validity in research involves two essential aspects of evaluating the strength and generalizability of a study’s findings in terms of cause-and-effect relationships.
- Internal validity is the extent to which a study can confidently claim that the observed changes in the dependent variable are directly caused by the independent variable, rather than other confounding factors or variables. In other words, it assesses whether the research design and methodology effectively control for potential alternative explanations.
- External validity is the degree to which the findings of a study can be generalized and applied to other contexts, populations, or situations beyond the specific research setting. It evaluates the representativeness and transferability of the results.
The validity of a study heavily relies on the robustness of the experimental design, which should aim to maximize both internal and external validity. Additionally, to ensure the validity of the instruments or assessments used in the study, researchers must also consider measurement validity, which assesses whether the tools accurately measure the intended constructs.
| Internal Validity | External Validity |
| Focuses on the extent to which a study establishes a trustworthy cause-and-effect relationship between a treatment and an outcome. | Focuses on the extent to which the results of a study can be generalized to other settings, people, or situations. |
| Concerned with the study’s internal structure and controls. | Concerned with the study’s applicability outside the research setting. |
| Asks, “Are the changes in the dependent variable actually due to the independent variable?” | Asks, “Can the results of this study be applied to other populations or contexts?” |
Trade-off between internal validity and external validity
There is often a trade-off between internal validity vs external validity in research studies. Studies with high internal validity are often conducted in tightly controlled laboratory settings, which may limit their external validity. Conversely, studies with high external validity are often conducted in more natural settings, which may reduce their internal validity.
Trade-off example
A study on the effects of a new teaching method is conducted in a controlled classroom setting with a specific group of students. The study has high internal validity but may have limited external validity, as the results may not generalize to other classrooms or students.
Solution
To balance internal vs external validity, researchers can conduct studies in more diverse settings, use larger and more representative samples, and replicate studies in different contexts.
Threats to internal validity
Internal validity can be threatened by various factors that can lead to alternative explanations for the observed results. These threats should be considered and controlled for in research design.
Research example
A school principal wants to investigate the effectiveness of a new math curriculum in improving student performance. The principal sets up an experiment with two groups:
- a control group of students who continue with the old curriculum, and
- an experimental group of students who will use the new curriculum. The experiment will run for one academic year. All students take a standardized math test before the experiment (pre-test) and after the experiment (post-test).
| Threat | Explanation | Example |
| History | Unanticipated events change the conditions of the study and influence the outcome. | A highly effective math tutor starts working with some students during the study, improving their math performance. |
| Maturation | The passage of time influences the dependent variable (math performance). | As students grow older and develop their cognitive abilities during the year, their math performance may improve regardless of the curriculum. |
| Testing | The pre-test (used to establish a baseline) affects the results of the post-test. | Students become familiar with the standardized math test format during the pre-test, leading to improved scores in the post-test. |
| Participant selection | Participants in the control and experimental group differ substantially and can thus not be compared. | The principal assigns high-performing students to the experimental group and low-performing students to the control group, resulting in selection bias. |
| Attrition | Over the course of a (longer) study, participants may drop out. If the drop out is caused by the experimental treatment (as opposed to coincidence) it can threaten internal validity and cause attrition bias. | Students who struggle with the new curriculum transfer to other schools during the study. The average math performance in the experimental group improves, not because the curriculum is effective, but because the struggling students are not included in the post-test. |
| Regression to the mean | Extreme scores tend to be closer to the average on a second measurement. | Students who scored extremely low in the pre-test probably show greater improvement in math performance than students who scored average. |
| Instrumentation | There is a change in how the dependent variable is measured during the study. | The post-test math questions are inadvertently made easier than the pre-test questions, leading to an apparent improvement in math performance. |
| Social interaction | Interaction between participants from different groups influences the outcome. | Students in the experimental group discuss the new curriculum with their peers in the control group, influencing their learning and math performance. |
Threats to external validity
External validity can be threatened by factors that limit the generalizability of the study’s results to other populations, settings, or times. These threats should be considered when interpreting and applying research findings.
| Threat | Explanation | Example |
| Testing | The act of measuring participants affects their behavior, limiting generalizability. | Participants become more self-aware of their anxiety due to repeated assessments, which may not reflect real-world behavior. |
| Sampling bias | The sample is not representative of the target population. | The study only includes college students, limiting generalizability to other age groups. |
| Hawthorne effect | Participants behave differently because they know they are being observed. | Participants in the experimental group report lower anxiety levels to please the researchers. |