Rationale for Formative Assessment Component in New Audiology Certification Standards

Judith A. Rassi
Vanderbilt University
(1999)

I'd like to begin my remarks by stating that formative assessment , which is conducted while an educational event or procedure is ongoing, yields information substantially different from that obtained in summative assessment , which is administered at the time of term completion (Rassi, 1998, p. 482). The purpose of formative assessment is to monitor students' acquisition of knowledge and skills during educational preparation, that is, during the time when students' basic and professional knowledge, skills, and attitudes are being 'formed.' Ongoing assessment helps students and their academic and clinical instructors to systematically track and document progress toward the attainment of learning outcomes. Assessment results can also serve as useful indicators for responsive and responsible educational planning.

The Standards Council's decision to include a formative assessment component in the new audiology certification standards was part of an overall determination to make the standards more effective in ensuring that entry-level clinicians (and, ultimately, all clinicians) have the appropriate skills and knowledge to be competent, independent practitioners. Our reasoning was guided, for the most part, by results of the Skills Validation Study. This study, a comprehensive job analysis commissioned by ASHA, was designed and administered by the Division of Applied Measurement Research of the Educational Testing Service (ETS) and was constructed on the basis of input from members of the Council's audiology subcommittee as well as a panel of audiology subject-matter experts.

Results of the study were published and made available to ASHA members (Tannenbaum & Rosenfeld, 1996). As stated in this publication, and as explained to the Standards Council by ETS, "a job analysis is the primary mechanism for establishing the job-relatedness of decisions concerning standards and curriculum redesign and professional certification" (p. i).

As you may recall, the findings of this study indicated a substantial discrepancy between the views of educators and the views of graduates, CFY supervisors, and employers on where information had been learned by students. Educators said that most information was learned by the students when they were in school. In contrast, the other groups in this study-that is, the graduates themselves, CFY supervisors, and other employers-said that much of the information, including core knowledge and skills, was not learned until after graduation, on the job. All groups agreed, however, that this information should be learned in school, before graduation. In other words, educators said it should and it is ; others said it should but it isn't.

In analyzing the study's results, those of us on the Standards Council audiology subcommittee at the time concluded that audiology graduates, CFY supervisors, and employers, who directly observe and experience the evidence of outcomes, surely must recognize when knowledge and skills are lacking. On the other hand, the educators who participated in this study had a different perspective and came to a different conclusion. If they considered only what was taught in their programs, then these educators may have been convinced that their students learned the core information that they needed to learn before graduation. This 'we-taught-them-everything-they-need-to-know' view is certainly reinforced by our old certification standards, the input-and-process standards, where specific courses and clinical practicum categories are emphasized. Whatever led the educators to respond as they did, they probably assumed that all of the following steps in the teaching-learning process had taken place:

1. That they taught their students the necessary knowledge and skills;
2. That what they taught their students was learned;
3. That what was learned by their students was retained; and
4. That what was taught, learned, and retained was then applied effectively by their students after graduation.

But these assumptions were not validated by the other groups, the groups positioned to see the actual outcomes of educational preparation. Without appropriate and timely assessment, however, only the first step of this sequence, the teaching step, can be known. In essence, then, the skills validation study had served as a measure of functional educational outcomes. The study told us that the outcomes of our educational programs' efforts fall short of what all of the respondent groups, including the educators, think the outcomes should be.

In light of these findings, the audiology subcommittee proceeded to draft new certification standards that would address our concerns and would also be more effective in making educational preparation consistent with contemporary clinical practice (ASHA, 1997b). Among the features of the new standards, here are the ones that led our subcommittee to include a formative assessment component, which is the focus of today's discussion:

1. As in clinical settings, there has been a shift of emphasis in educational programs, in other disciplines as well as our own, from input and process assessment to outcomes assessment. Health care reform, educational reform, and the seemingly endless problem of diminishing resources, have been demanding this change, as is evident in the increasing scrutiny by government officials, accreditation agencies, reform commissions, and the public (Rassi, 1998, p. 477). As you know, our academic accreditation standards have moved in this direction as well (ASHA, 1997a).
2. The new audiology core standards comprise specific knowledge-and-skill statements based on items from the job analysis study that had been validated as being important for entry-level audiologists to know and to be able to do.
3. These knowledge-and-skill statements are outcome based in that they require educators and students to be accountable for what has been learned.
4. The new standards require a summative assessment, which will continue to be administered in the form of a national comprehensive examination and whose design will correspond to the knowledge and skill areas in the new standards.
5. The new standards also require a series of formative assessments that measure students' acquisition of knowledge and mastery of skills throughout their course of study.

Why did we add formative assessments? We reasoned that the differences between what has been taught and what has been learned need to be discovered during a student's course of study when something can be done by educators and/or students to remedy the situation, not left until after graduation for others to discover. The most direct way to tackle this, it seemed to us, is for the graduate program to infuse into its curriculum a systematic way of conducting formative assessments, both clinical and academic, so that students' progress toward the acquisition of skills and knowledge can be monitored and evaluated. If designed to address the skills and knowledge required by the standards, formative assessments thereby become the actual tools for measuring outcomes along the way. These are functional outcomes, of course, if they provide information that is acted upon or changes behavior in accordance with program objectives (Rassi, 1998). The outcome-based nature of the new standards thus leads naturally to the need for formative assessments.

There are added benefits. Assessments, of course, can be used for a variety of purposes, going well beyond the examination of students' recall of circumscribed facts taught in topic-specific courses. Appropriate assessments conducted in the classroom, clinic, or laboratory can inform teaching as well as learning. Is teaching effective? Is learning taking place? Are we teaching what really matters? Are our students learning what really matters? Are we assessing what really matters? (Rassi, 1998).

If administered regularly and appropriately, assessments of classroom and laboratory learning, along with actual clinical performance, can facilitate the integration of a program's curriculum across these areas (Rassi & McElroy, 1992). Similarly, assessments of student performance in each of these areas can determine whether or not instruction is consistent and balanced across areas. Is information learned in the classroom being retained and applied in the clinic? Are skills learned in the clinic compatible with what is being taught in the classroom? Are students becoming independent decision-makers in the laboratory and clinic as they progress through the educational program? Asking appropriate questions such as these helps to form a system of checks and balances for different components of the program, while also revealing information about curriculum gaps and inconsistencies. In other words, with periodic checks and cross-checks of students' progress in all three areas, curriculum and course design changes can be made in a timely manner, and students' concerns can be addressed more readily.

If conducted at strategic junctures in a student's course of study, assessments also give instructors and students valuable, ongoing feedback that can help them to know when students are not moving along the continuum of learning and professional development as expected. Depending on university or departmental requirements, the timing and intervals of formative assessments may be varied in a number of ways. For example, an assessment or final examination of students' content recall at the end of each course is a time-honored tradition for many instructors. In light of the new standards, course instructors may or may not opt to continue doing this. Whatever they decide, an educational program may now want to supplement, if not replace, end-of-course examinations with formative assessments on material covered in a sequence of several related courses or several related clinical practicum placements or both. Because the new certification standards require longer courses of study, interval formative assessments should fit into the curriculum more easily. They may also be more necessary in a longer program because they can show progress, or lack thereof, before a student has invested a lot of time and money in this endeavor.

Finally, I should note that resources for helping us to expand our assessment repertoire are abundant. The higher-education literature is replete with information on the role of assessment in higher education (AAHE, 1992; Angelo, 1995; Angelo & Cross, 1993; Banta, et al., 1996). Classroom assessment techniques, or CATs, as they are called in the world of higher education (Angelo & Cross, 1993), can be used to assess anything and everything from students' critical thinking and problem solving abilities to their views about the effectiveness of a particular class session.

In conclusion, the audiology subcommittee chose to incorporate formative assessments into the new standards because of the validation study results, which led to the decision to write outcome-based standards, which led to the requirement for formative assessments.

References

American Association for Higher Education. (1992). Principles of good practice for assessing student learning. Washington, DC: Author.

American Speech-Language-Hearing Association. (1997a). Accreditation standards and guidelines. Standards for accreditation of graduate education programs in audiology and speech-language pathology. ASHA Leader, 2 (22), 7-8.

American Speech-Language-Hearing Association. (1997b). Audiology standards. Standards and implementations for the Certificate of Clinical Competence in Audiology. ASHA Leader, 2 (20), 7-8.

Angelo, T.A. (1995). Improving classroom assessment to improve learning. Guidelines from research and practice. Assessment Update, 7 (6) 1,2,12,13.

Angelo, T.A., & Cross, K.P. (1993). Classroom assessment techniques. A handbook for college teachers.  (2 ^nd ed.) San Francisco: Jossey-Bass Publishers.

Banta, T.W., Lund, J.P., Black, K.E., & Oblander, F.W. (1996). Assessment in practice.  San Francisco: Jossey-Bass Publishers.

Rassi, J.A. (1998). Outcomes measurement in universities. In C.M. Frattali (Ed.), Measuring outcomes in speech-language pathology  (pp. 477-502). New York: Thieme.

Rassi, J.A., & McElroy, M.D. (Eds.). (1992). The education of audiologists and speech-language pathologists.  Timonium, MD: York Press.

Shadden, B.B. (1999, July). Gifts from the Web: Tips on assessing student learning. In Newsletter of Special Interest Division 10: Issues in Higher Education, 3 (1), 3.

Tannenbaum, R.J., & Rosenfeld, M. (1996). The practice of audiology. A study of clinical activities and knowledge areas for the certified audiologist.  Rockville, MD: American Speech-Language-Hearing Association.