Common secondary data sources: U.S. Census, National Health Interview Survey, General Social Survey, Add Health, PISA, administrative school records.
Systematic Review and Meta-Analysis
A systematic review identifies, evaluates, and synthesizes all existing research on a question using a pre-specified protocol. A meta-analysis quantitatively combines effect sizes from multiple studies to produce a pooled estimate.
Systematic reviews and meta-analyses sit at the top of the evidence hierarchy for many fields because they synthesize the entire body of evidence rather than relying on any single study.
How to Choose a Research Method
The choice of method flows from the research question:
| If you want to... | Consider... |
|---|
| Establish that X causes Y | Randomized experiment |
| Measure the prevalence of X in a population | Survey with representative sample |
| Understand the experience of X | Qualitative interviews or focus groups |
| Observe behavior as it naturally occurs | Observation or ethnography |
| Synthesize all existing evidence on X | Systematic review or meta-analysis |
| Explore a new or poorly understood topic | Case study, ethnography, or grounded theory |
| Test a theory using large existing datasets | Secondary data analysis |
Practical constraints also matter: available time, funding, access to participants, and institutional requirements all shape the choice of method. A question that ideally calls for an RCT may need to be addressed with an observational study if random assignment is not feasible.
Validity and Reliability
Two concepts apply across all research methods:
Validity: Does the study measure what it claims to measure?
- Internal validity: Are the causal claims justified by the design?
- External validity: Do the findings generalize beyond the study sample or setting?
- Construct validity: Does the measure accurately capture the theoretical construct?
Reliability: Would the study produce the same results if repeated?
- Test-retest reliability: Consistency over time
- Inter-rater reliability: Consistency between different raters or coders
- Internal consistency: Items in a scale measuring the same construct
A measurement can be reliable without being valid (consistently measuring the wrong thing). A measurement cannot be valid if it is not reliable.
Frequently Asked Questions
What are the main types of research methods?
The main categories are quantitative (collecting numerical data to measure relationships and test hypotheses) and qualitative (collecting textual or observational data to understand experiences and meaning). Within quantitative research, methods include experiments, surveys, and secondary data analysis. Within qualitative research, methods include interviews, focus groups, observation, and case studies. Mixed methods research combines both approaches.
What is the difference between research methods and research methodology?
Research methods are the specific techniques used to collect and analyze data (e.g., survey, experiment, interview). Research methodology is the broader philosophical framework that guides the choice of methods, including assumptions about knowledge (epistemology) and what counts as valid evidence. In a thesis or dissertation, the methodology chapter explains both why you chose a particular method and what assumptions underlie that choice.
Which research method is best?
There is no single best research method. The best method is the one that best answers the research question given practical constraints. If you want to establish causation, a randomized experiment is best. If you want to understand lived experience, in-depth interviews are better. Most research questions can be addressed in multiple ways, and the choice involves trade-offs between internal validity, external validity, feasibility, and depth.
What is quantitative vs qualitative research?
Quantitative research collects numerical data and uses statistical analysis to measure relationships, compare groups, and test hypotheses. It typically uses large samples and aims to generalize findings to a population. Qualitative research collects non-numerical data (text, images, observations) and uses interpretive analysis to understand experiences, meanings, and processes. It typically uses small samples studied in depth. The two approaches answer different types of questions and are often used together.
What is a hypothesis in research methods?
A hypothesis is a testable prediction about the relationship between variables. In quantitative research, hypotheses are stated before data collection and specify the expected direction of the relationship (e.g., "students who receive tutoring will have higher grades than students who do not"). A null hypothesis states that there is no relationship. The researcher then collects data and uses statistical tests to determine whether the evidence is consistent with or against the hypothesis. Qualitative research typically does not state formal hypotheses, instead developing propositions inductively from the data.
For related research guidance, see primary vs secondary sources, how to write a literature review, and how to write a research paper introduction.
Streamlined AI