Guidelines for Developing Formative Assessment Plans for Implementation of New Standards for the Certificate of Clinical Competence

Council on Academic Accreditation in Audiology and Speech-Language Pathology (CAA)
Council For Clinical Certification (CFCC)
October 2001

Introduction

These guidelines are being provided by the Council on Academic Accreditation in Audiology and Speech-Language Pathology (CAA) and the Council for Clinical Certification in Audiology and Speech-Language Pathology (CFCC) in order to assist graduate programs in preparing their students to meet the Standards and Implementation for the Certificate of Clinical Competence in Speech-Language Pathology that go into effect in 2005 and the Standards and Implementation for the Certificate of Clinical Competence in Audiology that go into effect in 2007.

A new feature of the standards requires programs to conduct a series of formative assessments that measure students' acquisition of the identified knowledge and mastery of skills throughout their course of study in the program, i.e., during the time when students' basic and professional knowledge, skills, and attitudes are being "formed." Ongoing assessment helps students and their academic and clinical instructors to systematically track and document progress toward the attainment of learning outcomes (Rassi, 1999).

II. Definitions and Theoretical Concepts for Assessment Plans

These guidelines provide a conceptual and philosophical framework to assist you in developing your own Formative Assessment Plan and offer some suggestions for the process. For purposes of this process, the following definitions are provided:

Formative Assessment - ongoing measurement during educational preparation for the purpose of improving student learning. Formative assessment yields critical information for monitoring an individual's acquisition of knowledge and skills. Such assessment must evaluate critical thinking, decision-making, and problem-solving skills. Measures should include oral and written components, as well as demonstrations of clinical proficiency. Summative Assessment - comprehensive evaluation of learning outcomes at the culmination of educational preparation. Summative assessment yields critical information for determining an individual's achievement of knowledge and skills.

Concepts

Programs should be mindful of the following guidelines when developing their assessment plans:

• Programs should apply a "Teaching/Learning/Retention/Application" process to assess student progress (see Rationale for Formative Assessment Component in New Audiology Certification Standards by Judith A. Rassi).
  
• Students should be able to identify, analyze, synthesize, and apply knowledge and skills learned over time, on a cumulative basis.
  
• The assessment process should include a step for remediation, where warranted, until student achieves mastery of knowledge and/or skills.
  
• Assessment should incorporate academic and clinical education experiences and their integration.
  
• Assessment on a course-by-course basis is not the only mechanism to employ.
  
• All faculty, including academic and clinical, should play a part in assessing students.
  
• Assessments should be conducted in relationship to ASHA's Scope of Practice in Audiology, the Scope of Practice in Speech-Language Pathology, and the Code of Ethics, all available on ASHA's Web site.
• Consider the processes of formative and summative assessments in light of student competence, as defined by the certification standards.

Models

The CAA and CFCC do not intend to provide prescriptive models of formative assessment for programs; rather, programs will have the opportunity and freedom to develop their own formative assessment plans. To assist in this effort, an Ad Hoc Joint Committee on Academic Assessment (composed of representatives from ASHA, the Council of Academic Programs in Communications Sciences and Disorders, the American Academy of Audiology, the National Association of Preprofessional Programs, and ASHA's Special Interest Division 10: Issues in Higher Education) has been established to develop models of formative assessment and a systematic plan to educate faculty and academic program directors on how to use formative assessment tools to evaluate student competency. This group also expects to reach consensus on specific terminology related to formative assessment that may be adopted by ASHA's CAA and CFCC in communicating with the academic community. The CAA and CFCC will work collaboratively with this joint committee to provide consistent and helpful information and guidance to academic programs to ensure that entry-level clinicians have the appropriate skills and knowledge to be competent, independent practitioners. These activities should allow for effective integration of the knowledge and skill areas into the programs of graduate study to allow students to meet the new ASHA certification standards in 2005 (for speech-language pathology) and 2007 (for audiology).

II. Suggested Steps in Developing Assessment Plans

As you begin the process of developing your program's plan for formative assessment, you may wish to consider the following suggested steps:

1. Conduct meetings with academic and clinical faculty (including adjuncts) to discuss the need to develop a formative assessment plan.

2. Identify specific individuals to be involved in developing the assessment plan, with the identification of the individuals responsible for each component.

3. Inform university administrators of the project as appropriate (e.g., curriculum changes, required resources, etc.).

4. Determine the method/model for program review. Bear in mind that, for purposes of certification, course-specific semester credit hours and disorder-specific practicum clock hours will be replaced by outcomes or achievement of competencies (also see the guiding principles in Section III below and the enclosed comparison of the current and new standards). You may choose one of the models below or develop an alternative model or system that is appropriate for your own institution and program.

Two possible models may include: Identification of Learning Outcomes:

• Using the new certification standards and other resources, define "mastery" of the identified knowledge and skills for competence at the completion of the graduate program for individuals entering the profession.
• Determine how you will measure progress/successful achievement/acquisition of the knowledge and skills, including the sequence of assessments (how, what, and when to assess) to determine mastery or progress toward achievement.
• Identify a process for remediation where warranted, until student achieves mastery of knowledge and/or skills.
• Review the academic and clinical curricula to determine whether students will meet the minimum standards.

Review of Current Curriculum:

• Assess or match course content/syllabi with defined knowledge and skills in the certification standards; identify which knowledge and skills are not currently addressed in the existing program of study.
• Identify opportunities for students to demonstrate ability to analyze, synthesize, and evaluate information.
• Determine where/when in the existing program of study specific knowledge and skills are obtained, including all academic and clinical experiences, and where competency should be achieved; identify the knowledge and skill areas that are not currently addressed and incorporate into program.
• Identify measurement tool(s) and description of record keeping/ documentation and achievement of competence.
• Identify a process for remediation where warranted, until student achieves mastery of knowledge and/or skills

5. Develop a timeline for completing your assessment plan and obtaining university approval, if necessary (see Calendar in Section IV below). The program will be required to advise CAA of progress in developing the assessment plan as part of its Annual Report or Reaccreditation Application beginning July 2002. For programs submitting a Reaccreditation Application during academic year 2001-2002, an addendum to the application may be requested.

6. Conduct program review and develop ongoing systematic and coordinated plan of assessment as it applies to the new certification standards, including the following steps:

• Apply Principles (Section III) and use listed Resources (Section V) below in developing assessment plan.

• Involve academic and clinical faculty, adjunct faculty, and administration, as appropriate.

• Examine mission and intended outcomes of program.

• Define educational goals and objectives consistent with the program's purpose or mission.

• Develop procedures for evaluating the extent to which goals and objectives are achieved.

• Develop procedures for evaluating the extent to which competencies in the certification standards are achieved by students (determine short- and long-range assessment schedules).

• Develop and implement a plan for student remediation where necessary.

• Select assessment methods based on review of mission, goals, and objectives and competencies to demonstrate that graduates are prepared for employment.

• Use results of evaluations to inform teaching methods and content and to improve the effectiveness of the program.

7. Obtain internal approval of curriculum changes as necessary; make changes as needed to graduate catalog, Web site, or other publication vehicles.

III. Principles for Developing and Evaluating Assessment Plans

Key Components

At a minimum, assessment plans must include the following components. Specific questions are posed to facilitate the development of your plan.

Definitions - develop behaviorally defined levels of achievement for each knowledge and skill identified; if necessary, break down each knowledge and skill into intermediate steps

• Does the program cover each identified competency? How? Where?

Assessment Mechanism - describe the mechanisms and time schedule used to assess students' progress in reaching defined levels of achievement; obtain input from multiple sources (e.g., clinical supervisors, employers)

• How are the knowledge and skills assessed? When/how frequently? How is the evaluation used?
• What types of assessment will you use? (e.g., portfolio, goal setting, written, practical, and/or oral exams, etc.)
• Who should be involved in the assessment? How will you establish inter-rater reliability?

Feedback - inform students, faculty, staff, and off-campus supervisors of the required competencies and assessment procedures; provide frequent feedback to students about their progress so that each student knows (1) if he or she has achieved a level of achievement (intermediate or final), and, if not, (2) what steps the student must take to achieve the competency

• How will you provide feedback to students?
• What are your plans for remediation if student does not meet the competencies or achieve outcomes?

Record Keeping - develop tracking/record keeping systems to document students' progress toward each intermediate and final knowledge and skill

• What documentation of assessments will you maintain? For how long?

Evaluation - develop mechanisms to evaluate effectiveness of the formative assessment program and demonstrate how the data are used to enhance program effectiveness; seek input from multiple sources; analyze data collected, identify corrective measures, and revise assessment plan as required to strengthen program

• How is evaluation used to improve your program? Is the teaching effective? How are students learning?

Suggestions

In relation to the key components above, the CAA and CFCC believe that the development of effective assessment strategies will include:

• A review of institutional and program goals and objectives to develop a plan that addresses the program's targets for excellence, in addition to the minimum standards
• Development of specific systematic plan(s) of assessment
• Demonstration of the link between evaluation procedures (assessment methods) and all relevant goals and objectives
• Review of data to determine effectiveness of plan of assessment and program's success in facilitating student mastery
• Identification and implementation of remediation or corrective measures
• Demonstration of the cyclical nature of effective assessment
• Demonstration that the competencies identified in the certification standards and taught in the program have adequately prepared the graduate for employment

  IV. Calendar for Programs to Follow in Preparing to Meet New Certification Standards

  In order to prepare for implementation of the new Standards for the Certificate of Clinical Competence in Audiology (effective January 2007) and the Standards for the Certificate of Clinical Competence in Speech-Language Pathology (effective January 2005), the following calendar is provided to assist programs in this process.

October 1, 2001 - Begin/continue development of Formative Assessment Plan

July 2002 - June 2003 - Incorporate with program's regular CAA annual report due between July 2002 - June 2003, a progress report articulating Formative Assessment Plan and progress to date

August 2002 - July 2003 - Continue to make changes in program curriculum, obtain necessary university approvals, modify publications, etc.; respond to CAA concerns, if any

September 2003 - Implement Formative Assessment Plan for students entering speech-language pathology programs (to meet new certification standards beginning January 2005)
- Implement Formative Assessment Plan for students entering audiology programs (to meet new certification standards beginning January 2007)

V. Resources

The following print and electronic resources may be useful to you as you develop your plan for assessment.

Print Resources

Most of the following ASHA materials may be viewed on ASHA's Web site or are available for purchase.

• Standards and Implementation for the Certificate of Clinical Competence in Audiology (effective 2007)
• Standards and Implementation for the Certificate of Clinical Competence in Speech-Language Pathology [PDF] (effective 2005)
• CAA Standards for Accreditation of Graduate Education Programs in Audiology and Speech-Language Pathology
• Scope of Practice in Audiology
• Scope of Practice in Speech-Language Pathology
• Guide to Verifying Competencies in Speech-Language Pathology (ASHA Products Catalog #011928, manual and CD)
• Code of Ethics
• Rationale for Formative Assessment Component in New Audiology Certification Standards (presentation by Judith A. Rassi at 1999 ASHA Convention, San Francisco, CA)

Other references regarding educational assessment that you may find helpful are the following:

1. Angelo, T. A. & Cross, K. P. (1993). Classroom assessment techniques (2 ^nd ed.). San Francisco: Jossey-Bass.
2. Berger, R. (1991). Building a school culture of high standards: A teacher's perspective. In V. Perrone (Ed.), Expanding student assessment (pp. 32-39). Alexandria, VA: Association for Supervision and Curriculum Development.
3. Bloom, B. S., Madaus, G. F., & Hastings, J. T. (1981). Evaluation to improve learning. New York: Macmillan.
4. Childers, P. B. and Lowry, M. (1997). Engaging students through formative assessment in science. Clearing House, 71(2) , 97-102.
5. Chittenden, E. Authentic assessment, evaluation, and documentation of student performance. In V. Perrone (Ed.), Expanding student assessment (pp. 22-31). Alexandria, VA: Association for Supervision and Curriculum Development.
6. Hobson, E. H (Ed.). (1997). Forms and functions of formative assessment. Clearing House , 71(2).
7. Scriven, M. (1991). Beyond formative and summative evaluation. In M. W. McLaughlin and D. C. Phillips (Eds.), Evaluation and education: At quarter century (pp. 19-64). Chicago: National Society for the Study of Education.
8. McIntosh, M. E. (1997). Formative assessment in mathematics. Clearing House, 71(2 ), 92-96.
9. Murphy, S. M. (1997). Designing portfolio assessment programs to enhance learning. Clearing House, 71(2), 81-84.
10. Weasmer, J. & Woods, A. M. (1997). Teacher preparation: A revision process fostered by formative assessment. Clearing House, 71(2), 113-115.
11. Zessoules, R. & Gardner, H. (1991). Authentic assessment: Beyond the buzzword and into the classroom. In V. Perrone (Ed.), Expanding student assessment (pp. 47-71). Alexandria, VA: Association for Supervision and Curriculum Development.

Electronic Sources

Internet Web sites you may wish to consult for additional information regarding assessment and student learning are the following:

1. American Council on Education
2. Council of Graduate Schools