November 19, 2002 Feature

Make Research Methods an Integral Part of Your Clinical Practice

see also

There is great need and potential for the application of research to clinical practice. From our perspective, these applications fall into two general categories: the use of existing research findings and the implementation of research techniques to better serve individuals with communication disorders.

These areas of interface between research and practice are crucial to the ethical practice of speech-language pathology and audiology. Specifically, ASHA's Code of Ethics mandates that individuals are obligated "to achieve and maintain the highest level of professional competence" and to "evaluate the effectiveness of services rendered and products dispensed."

Attaining and sustaining high levels of professional competence obviously require that clinicians remain current in their knowledge of pertinent research. Issues related to obtaining, evaluating, and applying new research findings to practice are beyond the scope of this discussion. Suffice it to say that this enterprise can be quite challenging, even for a circumscribed area of expertise. Fortunately, significant strides are being made in the development of evidence-based practice guidelines (see ASHA's Compendium of EBP Guidelines and Systematic Reviews and the Academy of Neurologic Communication Disorders & Sciences Web site). The focus of this article will be on the utilization of research methods in clinical practice in order to assist in making objective decisions regarding treatment effects.

We believe that the use of research methods should be integral to clinical practice. Furthermore, certain single-case designs are particularly amenable to clinical application. One of the most important duties of clinicians is to determine the effectiveness of the treatment they provide. Ideally, clinicians should have extensive databases available to assist in making decisions about treatment. Unfortunately, although clinical researchers have been progressing in the provision of efficacy and effectiveness data for various treatments, clinicians must frequently draw upon their own resources to make such decisions for individual patients. (Note that we are using the term "patient" in a generic manner to denote any individual receiving professional services from a clinician.)

Making a judgment about whether or not a treatment will have positive effects is not a simple matter. Documenting changes in a patient does not equate with documenting treatment effects. That is, clinicians cannot infer that treatment has had an impact just because measurable changes have occurred over the time course of the treatment. There are generally many possible alternate explanations for the observed changes (e.g., spontaneous recovery, developmental changes) that must be accounted for in determining treatment effects. The following discussion will provide an overview of research methods that clinicians may use to facilitate decisions regarding treatment and to ensure accountability.

Operational Definitions

Operational definitions are a critical component of research that is equally important in clinic. In order to understand the relationship between the treatment being applied and the targeted behavior(s), both must be clearly defined.

Prior to beginning treatment, all behaviors that the clinician wishes to affect should be carefully specified, including how the behavior will be measured, who will be conducting the measurements, where the measurements will occur, and how frequently they will be made. For example, instead of identifying "word-finding" as the behavior to be improved, one could specify "verbal confrontation naming of high-frequency nouns depicted in line drawings" and/or "verbal production of correct information units during narrative and procedural discourse" (as measured by the primary clinician, twice weekly, in the clinic).

Throughout the course of treatment, the behaviors should be measured as originally detailed. Any change in measurement procedures (i.e., a different clinician conducts the measurements, cues are provided during the measurements) could result in changes in behavior and, thus, confound the effects of treatment.

Similarly, treatment should be defined as clearly as possible. All possible response contingencies should be considered in the treatment planning (e.g., What will happen if the patient's response is incorrect? vs. What will happen if the patient's response is partially correct?). Aspects of treatment that are particularly important to specify and are frequently uncontrolled in clinic include type and frequency of feedback, number of trials per session, stimuli used during treatment, and ordering or grouping of stimuli/trials. Any modifications to treatment that occur should be carefully documented.

Reliability

Reliability is absolutely essential to the conduct of both research and treatment. If treatment cannot be applied reliably within and across clinicians, then the utility of that treatment is highly questionable. Likewise, if similar measurements of the targeted behavior cannot be obtained within or across clinicians, then the effects of treatment cannot be determined.

The use of operational definitions helps to ensure adequate reliability. Reliability measures can be conducted online or through audio/visual recordings by another clinician (interobserver) or through recordings by the primary clinician (intraobserver). Frequent measurements of reliability are particularly helpful prior to or at the start of treatment. After adequate reliability has been established, periodic checks of reliability should be made throughout the course of treatment.

Learning, Types of Data, and Treatment Effects

As clinicians apply research principles to clinical practice, three interrelated issues must be addressed. First, what are the clinical questions to be answered? This issue is similar to the questions or hypotheses formulated by researchers. Second, what stage of learning is the clinician interested in addressing or documenting? The patient's stage of learning is often embedded in the clinical question that is addressed. Third, what type of data will aid the clinician in answering the clinical questions posed?

Although many models of learning have been proposed, in general, three stages are of interest to clinicians providing treatment—acquisition, generalization, and maintenance. The first, or acquisition stage, is also referred to as the establishment and evocation phases. During this stage, the clinician is determining how to help an individual do something that is not in the person's repertoire. For example, if a child is deleting final consonants, the clinician is providing cues and prompts to help the child produce final consonants. The acquisition stage involves the clinician manipulating antecedents and consequences of a treatment program with the intention of gearing the task slightly beyond the patient's ability and keeping the performance increasing with success.

The clinician provides feedback regarding performance accuracy and attention to task. Clinical questions posed during the acquisition phase may be: "Is the patient responding to the treatment program?"; "Should the consequences be changed?"; Should the antecedents be less supportive?"; "Is the patient ready for a more complex/advanced response?"

Quantitative treatment response data are used to address these types of clinical questions. Generally, treatment response data are collected each session through a sampling approach. For example, 10 consecutive responses by a client may be recorded by the clinician to determine if the antecedents or consequences should be adjusted for the client. The clinician would not record all responses within a session, as that would detract from the patient/clinician interaction. During the recording of treatment response data, the clinician typically provides feedback about the patient's responses as the clinician is "teaching."

Quantitative treatment response data do not provide information as to the status of generalization or independent performance by the patient. The results are "colored" by clinician's prompts, cues, and feedback. In order to assess independent performance, acquisition probe data also may be collected periodically. This would entail removal of all aspects of treatment (e.g., feedback, cueing) during measurement of the targeted behaviors. A clinician also may note qualitative data during the treatment process such as how much effort the patient is expending during the treatment session.

Generalization is the second stage of learning and indicates if the patient is producing a recently learned response in different contexts and situations, or producing new untrained responses related to recent learning. Generalization may occur either spontaneously or be programmed or included in the treatment program. Clinical questions posed during the generalization stage may include: "Is significant and important change occurring for the patient?" and "Is the patient beginning to acquire the target behavior independently?"

Generalization data are used to address these clinical decisions. Generalization data may be either response generalization data, which show that the effects of learning of a particular target behavior during treatment have spread to other related behaviors of the patient, or stimulus generalization data, which involve the target behavior occurring in the presence of novel antecedent stimuli, different settings, people, other pictures, and objects of targets. An example of response generalization would be when the patient begins to produce untreated sounds that are related to the treated sounds (e.g., when training of /s/ and /f/ results in improved production of /z/ and /v/). An example of stimulus generalization would be the production of trained vocabulary with persons not involved in the treatment process.

A particularly important aspect of stimulus generalization relates to the question, "Are patients using these behaviors in their natural environment?" The collection of data to answer this question often is most effectively conducted by someone other than the clinician. Qualitative reports from significant persons in the patient's natural environment are also helpful in determining the degree of independence demonstrated by the patient outside the treatment setting.

Generalization data describe a patient's "real learning." Generalization data are collected systematically but not necessarily every session. Usually, the clinician uses structured probes so that the generalization data can be obtained quickly and efficiently. The clinician must not provide feedback about a patient's performance when obtaining generalization data. The clinician needs to determine the patient's level of performance prior to beginning treatment (for clinical purposes, at least two points are needed to establish a stable baseline), during the treatment period, and following the termination of treatment on a particular target. Qualitative data regarding the patient's performance on the treatment target obtained from other persons are useful to support the quantitative generalization data and to help determine if important and significant changes have occurred.

Maintenance is the third stage of learning and indicates if the treatment targets/behaviors have been retained over a period of time after treatment of those behaviors has ended. The clinical questions posed for the maintenance stage may include "Is the patient ready to be dismissed from the treatment process?" or "Is the patient ready to attempt new targets?"

Quantitative generalization data are collected at specified lengths of time following the tentative withdrawal of treatment to assess if the patient has achieved maintenance. Maintenance data may suggest the need for additional (i.e., booster) treatment or the need to withdraw treatment slowly.

Another important clinical question to address is "To what degree is treatment responsible for the change in the patient?" To help answer this question, the clinician may obtain control data in addition to the generalization data on a specific treatment target described in the generalization stages section. To obtain control data, the clinician must select a patient's behavior that is not expected to change as a result of treatment. Control behaviors should not be related to the treatment goals/behaviors but should be similar in terms of development and/or complexity. Quantitative control data allow the clinician to monitor the effects of other factors that might influence a patient's performance such as maturation or spontaneous recovery.

Control data should be collected at the same time as the stimulus or response generalization data are obtained. The clinician then compares the performance on the generalization data to the control data. Hopefully, the patient's performance is higher on the generalization data than the control data, and the clinician can reasonably determine that the patient made more progress with treatment than would have occurred without treatment.

Variability, Stability, Specific Single-Case Designs, and Clinical Decisions

The concepts of variability and stability are essential for clinicians to interpret generalization and control data. The clinical utilization of specific single-case research designs aids the clinician in displaying information in a systematic manner to address some of the clinical questions and make better clinical decisions when providing treatment to individuals with communication disorders.

As described above, clinicians have been encouraged to collect generalization and control data to answer specific clinical questions. In order to interpret these data, clinicians must obtain repeated measures of the patient's function across time for each of these measures. These repeated measures assess variability in performance for the types of data or measures collected. This in turn provides the context or background for evaluating how a patient responds to treatment. Stability refers to the pattern that exists in the repeated measures. Prior to providing treatment, the clinician must continue the baseline phase until stability or pattern is clear. The trend and variability around that trend must be clear before initiating the treatment phase. Often, as few as two points are needed in clinical work, although longer baselines are required in research.

Two single-subject designs will be discussed that have a great deal of clinical utility to address specific clinical questions. These designs provide a way for the clinician to organize and visually inspect the generalization or control data that have been obtained in an ongoing manner. The designs are the alternating treatments design and the multiple baseline across behaviors design.

The alternating treatment design provides an opportunity to compare the effects of two or more treatments. A brief application of this design can help determine which potential treatment might promote better results for a given patient. With this design, the clinician selects two similar behaviors, each of which will be submitted to a different treatment. The behaviors should be of comparable difficulty for the patient but should not be expected to co-vary.

Repeated pretreatment measurement of both behaviors is recommended to ensure stability and equal difficulty. The clinician then applies treatment to each of the behaviors simultaneously in an alternating manner, while continuously measuring the target behaviors. For example, two word-finding treatments (Treatment A and Treatment B) could be compared by treating one list of words with Treatment A and one list of words with Treatment B. Performance on an untreated list of words also could be measured for additional control. The alternating treatments design is based on the assumption that behavior under two or more treatments will respond differently. If the two series of data points separate, differences in the treatments' conditions are inferred.

The multiple baseline design across behaviors allows the clinician to determine if significant change is occurring and the degree to which treatment is responsible for the change. The clinician identifies a number of behaviors to measure over time, and these different behavioral measures provide baselines against which changes can be evaluated.

A multiple baseline design across behaviors requires comparisons both within a single set of data points and between two or more sets of data points in order to interpret the results. For example, one of the behaviors represents the treatment target and the measures are generalization probes. The second and third behaviors selected may represent behaviors in need of treatment but not treated initially. These might serve as control behaviors.

With baselines established on the three behaviors, treatment is applied to one of the behaviors and produces a desired change, with no or minimal change in the other baselines. Treatment is then sequentially applied to the other remaining behaviors, hopefully with the desired outcome in performance. This design allows the clinician to answer the questions: "Does treatment work? "; "Is significant change occurring?"; and "To what extent is the change due to treatment?"

Informed Consent

Whenever human participants are involved in research, informed consent must be obtained from those participants. This involves having the research approved by an Institutional Review Board (IRB), assuring that the participant is fully informed about the procedures and risks involved with the research, and protecting the participant's rights.

Obtaining IRB approval is often a relatively lengthy process (e.g., frequently a 2–3 month period of time is required from submission to approval) that should be taken into account when planning a study. Local IRBs are typically available to review and approve proposals for researchers working in schools, universities, hospitals, or other large facilities. Independent or contract IRBs are available for researchers who are not affiliated with an institutional IRB (e.g., private practitioners). Local IRBs may or may not be willing to review research by individuals not affiliated with their institutions. The determination of when informed consent and IRB approval is necessary is not always clear and depends upon whether or not "research" is being conducted.

The application of research methods in clinical practice solely for the purpose of improving patient management would not typically be considered research. If, however, results of a clinical endeavor are to be published, informed consent must be obtained (this may sometimes occur retroactively). Informed consent is required prior to the initiation of an investigation when research methods are used with human participants for the purpose of gaining new knowledge (e.g., when a new treatment that is not the standard of care is being investigated). Informed consent also must be obtained when screening potential participants for research projects. The Office of Human Subjects Research is responsible for interpreting and overseeing regulations for protecting human subjects and is an excellent resource for information on this topic.

Implementation

Many of the questions that a clinician faces on a daily basis regarding patient care are similar to research hypotheses that can (and should) be addressed from a scientific point of view. Implementation of some of the strategies we have discussed may incur additional time and resources, but the alternative of providing treatment without objectively documenting its effects may be more costly. Use of these principles and methods usually results in better services for persons with communication disorders.

Julie Wambaugh, is an associate professor at the University of Utah and a researcher with the VA Salt Lake City Healthcare System. She teaches and conducts research in the area of neurogenic speech and language disorders. Her primary area of research is acquired apraxia of speech, with a specific focus on treatment. Contact her by e-mail at julie.wambaugh@health.utah.edu.

Barbara Bain, is a professor in the department of communication sciences and disorders and education of the deaf at Idaho State University in Pocatello. She teaches in the areas of clinical processes, research, and language disorders in children. Her research has focused on treatment efficacy/effectiveness, language disorders, dynamic assessment, and phonology disorders. Contact her by e-mail at bainbarb@isu.edu.

cite as: Wambaugh, J.  & Bain, B. (2002, November 19). Make Research Methods an Integral Part of Your Clinical Practice. The ASHA Leader.

References

Bain, B. A., & Dollaghan, C. A. (1991). Treatment efficacy forum: The notion of clinically significant change. Language, Speech, and Hearing Services in Schools , 22, 264–270.

Campbell, T. C., & Bain, B. A. (1991). Treatment efficacy forum. How long to treat: A multiple outcome approach. Language, Speech, and Hearing Services in Schools , 22, 271–276.

Holland, A. L. (1998). Some guidelines for bridging the research-practice gap in adult neurogenic communication disorders. Topics in Language Disorders , 18(2), 49– 57.

Wilcox, M. J., Hadley, P. A., & Bacon, C. K. (1998). Linking science and practice in management of childhood language disorders: Models and problem-solving strategies. Topics in Language Disorders , 18(2), 10–22.


  

Advertise With UsAdvertisement