Understanding Research Designs and External Scientific Evidence
- Evidence-Based Practice
- How to Search for Evidence in Communication Sciences and Disorders
- Understanding Research Designs and External Scientific Evidence
- Bias, Appraisal Tools, and Levels of Evidence
- Statistics Refresher
External scientific evidence is a component of evidence-based practice (EBP) and refers to sources outside of everyday clinical practice. This page discusses
Primary Research
Primary research pertains to individual studies attempting to answer a specific research question using raw data collected by the researcher(s). In experimental studies, the investigator manipulates one or more variables to compare those that received the manipulated condition to those that did not. In qualitative and observational/non-experimental studies, although there is no experimental manipulation, they may involve a comparison group.
Primary research can be conducted
- retrospectively (i.e., researchers collect data on the study participants’ past) or
- prospectively (i.e., researchers follow study participants over time and collect data to capture change).
Here are some common types of study designs:
Experimental Study Designs
- Randomized controlled trial (RCT) – Participants are randomly assigned to either the control group or an experimental group. Researchers compare outcomes from each group to determine whether the intervention caused any change.
- Controlled trial – A study involving non-randomized groups (i.e., experimental, comparison/control), which helps determine the effects of the intervention.
- Single-subject designs – Also known as single-case experimental designs, this type of experimental design allows researchers to closely examine specific changes in each participant. Each participant serves as their own control (i.e., compared to themselves) and researchers measure the outcome or dependent variable repeatedly across phases (e.g., baseline phase, intervention phase, withdraw phase). There are many variations of a single-subject design study.
- Cross-over trial – This is a study in which participants first receive one type of treatment and then researchers switch them to a different type of treatment.
Observational/Non-Experimental Study Designs
- Cohort – A cohort is an observational design study, possibly including a control group, in which researchers follow participants over time to determine the factors leading to different outcomes. Cohort studies can be retrospective or prospective.
- Case–control – This retrospective, observational study identifies an outcome of interest and compares a sample of people with that outcome (case) and a sample of people without that outcome (control). This design enables researchers to determine possible differences of previous exposures, experiences, and risk factors—any of which could explain their different outcomes.
- Cross-sectional – This is a study of a single sample at one point in time to understand the relationships among variables in the sample.
- Case study – A case study is an uncontrolled, observational study of events and outcomes in a single case.
- Case series – A description of uncontrolled, non-experimental events and outcomes for a series of similar cases who receive the same intervention or have the same outcome.
The chart below can help you better understand the features of the study designs commonly seen in audiology and speech-language pathology research.
| Study Design Type | Experimental | Observational | Retrospective | Prospective | Random Assignment of Groups |
|---|---|---|---|---|---|
| Controlled Trial | Y | Y | S | ||
| Single-Subject | Y | Y | |||
| Cross-over | Y | Y | S | ||
| Cohort | Y | S | S | ||
| Case-Control | Y | Y | |||
| Cross-Sectional | Y | Y | |||
| Case Study/Series | Y | S | S |
Note: Y=yes; S=sometimes.
Secondary Research
Secondary research, also called synthesized research, combines the findings from primary research studies and provides conclusions about that body of evidence. Below are three common types of synthesized research, which are also found on the ASHA Evidence Maps:
Systematic Reviews
Systematic reviews use systematic methods to search for and compile a body of evidence to answer a research or clinical question about the efficacy/effectiveness of an assessment or treatment approach. Typically, studies included in a systematic review have met predetermined eligibility and quality criteria (e.g., studies must be experimental designs). The systematic review then provides qualitative conclusions based on the included studies.
Well-done systematic reviews offer greater transparency because they provide details about their inclusion/exclusion process. They also typically assess each study for its methodological quality and level of evidence. Using transparent methods reduces bias and increases the confidence of the findings and conclusions of the research. Systematic reviews can provide a synopsis of the state of the evidence about a given clinical topic.
Meta-Analyses
Meta-analyses use systematic and statistical methods to answer a research or clinical question about a specific assessment or treatment approach. Like systematic reviews, included primary studies must meet predetermined eligibility and quality criteria. The meta-analyses provide quantitative conclusions (e.g., pooled effect size, confidence interval) to determine the overall treatment effect or effect size across studies. The additional statistical measures can provide a better picture of the clinical significance.
Clinical Practice Guidelines
Clinical practice guidelines are systematically developed statements created by a group of subject matter experts to provide a comprehensive overview of a disorder, detail the benefits and harms of specific assessment and treatment approaches, and optimize delivery of services. Guidelines grade recommendations based on the quality and amount of available evidence and classify them as either of the following two types of recommendations:
- Evidence-based recommendations: A systematic review of the evidence informs the group of experts and their recommendations.
- Consensus-based recommendations: These recommendations are based on a summary of expert opinions.
Matching Your Clinical Question to a Type of Evidence
Your clinical question determines the study design (e.g., randomized controlled trials, single-subject design) needed to address your question and impacts your search for evidence. Systematic reviews and meta-analyses should also include the study designs with the highest likelihood of answering your clinical question.
Keep in mind that if you are looking for research on a newer treatment or assessment approach, you may only find early-stage research designs, such as case studies and case series. These designs may provide preliminary evidence but cannot demonstrate the efficacy of the newer approaches.
Quality Control
Once you find study designs appropriate for your clinical question, you need to determine the methodological quality of the primary or secondary studies. There are different methods and checklists to appraise the methodological quality of primary and synthesized research.
See below to find out which study design best addresses your clinical question:
For Screening/Diagnosis Questions
Assess accuracy in differentiating clients with or without a condition.
Example question:
Is an auditory brainstem response or an otoacoustic emissions screening more accurate in identifying newborns with hearing loss?
Preferred Study Design(s):
Prospective, blind comparison to reference standard
Other Study Design(s):
Cross-sectional
For Treatment/Service Delivery Questions
Determine the efficacy of an intervention.
Example question:
What is the most effective treatment to improve cognition in adults with traumatic brain injury?
Preferred Study Design(s):
Randomized controlled trial (RCT)
Other Study Design(s):
Controlled trial (non-randomized)
Single-subject/single-case experimental design
For Etiology Questions
Identify causes or risk factors of a condition.
Example question:
What are the risk factors for speech and language disorders?
Preferred Study Design(s):
Cohort
Other Study Design(s):
Case–contro
Case series
For Quality of Life/Perspective Questions
Obtain and assess clients’ opinions and experiences.
Example question:
How do parents feel about implementing parent-mediated interventions?
Preferred Study Design(s):
Qualitative studies (e.g., case study, case series)
Other Study Design(s):
Not Applicable
For Prognosis Questions
Predict client’s likelihood of outcomes over time due to factors other than intervention.
Example question:
What is the prognosis of a child with autism spectrum disorder?
Preferred Study Design(s):
Cohort
Other Study Design(s):
Case–control
Case series
For Cost Questions
Compare cost of treatments, tests, and other factors due to the disorder.
Example question:
What is the cost of care for individuals with dysphagia requiring a feeding tube compared to those requiring diet modification?
Preferred Study Design(s):
Economic analysis
Other Study Design(s):
Not Applicable
For Prevention Questions
Identify factors to reduce likelihood of a disorder.
Example question:
What are some strategies to prevent hearing loss?
Preferred Study Design(s):
Randomized control trial
Other Study Design(s):
Controlled trial (non-randomized)
Cohort
Case-control
