IV. Self Study Guidelines
What is Self-Study?
Self study is a procedure whereby an education program describes, evaluates, and subsequently improves the quality of its efforts. Self assessment involves the identification of a program's strengths and limitations and the delineation of the steps necessary to correct those limitations. Self study requires a commitment to change rather than to maintaining the status quo.
Through the self-study procedure, a program conducts a systematic and thorough examination of all its components in light of its stated mission. Such an evaluation allows a program to determine the success it is having in accomplishing its self-established goals and objectives through careful evaluation of input obtained from individuals and groups directly and indirectly involved with the program.
Self study is a process that should be ongoing. Active and continuous involvement in self study reflects a commitment to the concept of providing students with a quality educational experience.
How is Self-Study Accomplished?
There is no single way to complete the task of self-examination. It may take as long as 6 to 18 months to complete the initial phase of self assessment. At the end of that period, a working document should have been developed that facilitates further program review. This document should be an accurate statement of the status of the program at that time.
In conducting a self-study, many programs have established steering committees to coordinate the various tasks involved in this complex effort. Other programs merely assign a project director.
Regardless of how the study is coordinated, if it is to be successful, it must be systematic. For this reason a prospectus should be developed before initiation of the study. If properly developed, the prospectus guarantees that the self-study will be comprehensive.
The prospectus should specify the various components of the program to be assessed. It should indicate how data will be collected, what the review procedure is, how progress will be measured, how decisions will be reached, who will have the responsibility for each task, and what the general timelines will be.
A well-planned and successful self-study must involve collection of data from all who are associated with a program either directly or indirectly. Information must be obtained throughout each of the various stages of the study and should be obtained from a representative sample of the parent institution's administration, instructional staff, student body, and governing body-as well as the program's graduates, employers of program graduates, and other appropriate constituencies.
Self-study goes beyond mere data collection. Self-evaluation requires that a program evaluate the procedures it will use in attempting to alleviate limitations that the data reveal. Examination of problem solving procedures allows a program to reflect on its own efficiency and to determine if the methods in use are, in fact, the most expeditious available. Efficient problem solving obviously is critical if a program is to attain its objectives.
One of the most important components of self-study is the data collected from the follow-up evaluation of a program's graduates and employers of those graduates. The quality of any program is ultimately determined by whether it produces competent professionals. In most cases, assessment relates to providers of clinical services to the public, but it can include research, teaching, administrative, community, and professional leadership activities of graduates. Therefore, a program must continually assess the achievement of its students and the performance of its graduates in light of its objectives and correct the observed discrepancies through program change.
What is the Relation of Accreditation to Self-Study?
The CAA's interest in accreditation is based on its belief that all professional fields that provide services to the public have an obligation to ensure, insofar as possible, that services provided by its professionals are of high professional quality. One of the most effective means by which this obligation can be met is to establish appropriate standards of education and training for its future practitioners, as well as standards for continuing professional education, and to identify those education institutions that maintain adequate standards.
The CAA was established to accredit entry-level graduate education programs in audiology and/or speech-language pathology. The specific purposes of CAA's voluntary accreditation are to
- Promote the value of accreditation and continuous quality improvement to programs, professionals, and the public
- Formulate standards of accreditation that foster continuing improvement within education programs in audiology and speech-language pathology
- Provide a mechanism for peer-review in the formulation and implementation of standards of accreditation
- Evaluate programs to determine if they meet their established goals and missions
- Ensure that program outcomes are aligned with the expectations of the profession with regard to the knowledge and skills required for practice in speech-language pathology or audiology
- Respect program autonomy and encourage program development and innovation
- Maintain an accreditation process that is efficient, timely, and cost effective
- Work collaboratively with other agencies to avoid duplication of efforts in continuous quality improvement
- Maintain recognition by the U.S. Secretary of Education, Council for Higher Education Accreditation, and other organizations with oversight responsibility for accreditation
As is apparent, self-study and accreditation are linked, but they also are essentially parallel and different procedures. Self-study is, as the name implies, a self-generated evaluation of one's own program, whereas the accreditation process involves external peer review. This Accreditation Manual can provide the basic framework by which a program may engage in self-evaluation activities. Although the accreditation application can act as a general guide to self-study, the application for accreditation is not synonymous with self-study. Self-examination is the first step toward accreditation, but it must be developed as an ongoing mechanism that is operational even after accreditation is achieved. If a program is to remain viable, it must engage in continuous self-assessment and not depend on periodic external review to accomplish this end. The application for accreditation must grow out of self-study, not the reverse.
Although assessment of a program's readiness for external review is not the purpose of self-study, after such evaluation, a program should be able to determine if it is actually ready for peer review. When a program, following self study, deems itself unprepared for external review, it should not initiate the application for accreditation, but attempt to correct the deficiencies. Additional self-study should indicate if the problems have been solved and whether the program is then ready for external review. At any time during the accreditation process, the CAA chair or members of the CAA are available to provide ongoing guidance to programs. The Accreditation Office staff is also available to provide consultation on the accreditation process, interpretation of standards, and on procedural issues.
As previously noted, each program is responsible for developing and conducting its own self assessment. Although there is no single way to complete the task, all successful self studies involve singular procedures and obtain and evaluate information from relatively defined constituencies. As a result of the similarities in procedures, the CAA has developed a flow chart (see Figure 3 in the appendices) that defines the basic requirements of a successful self study. A program should add procedures to the chart as it develops a strategy for assessing its own quality.
Questions to be Addressed in Self-Study
The CAA has developed a series of questions that might be asked during a self-study and are offered below only as suggestions to facilitate a program's preliminary discussions of self-study in relation to its compliance with the Standards for Accreditation.
- What is the administrative structure of the program?
- What are the lines of authority? What is the allocation of responsibility?
- How stable is the administrative structure and program support? Is the administrative structure functional?
- What is the budget support for the program? What foreseeable changes may occur in budget support?
- What budget support is provided for salaries, equipment, maintenance, and library?
- How dependent is the program on soft money? Is administrative support adequate to continue a quality educational service?
- How does the administration assure instructional staff of opportunities for salary and rank increases?
- Does administration policy permit structuring of reasonable workload? If not, how can this be corrected?
- What is the program's status within the institution? Does the program have adequate communication with the administration of the institution? If not, how can this be improved?
- To what extent does the program's instructional staff have the responsibility for designing, approving, and evaluating the curriculum?
- Do all the instructional staff share in the decision-making activities of the program? If not, why not?
- Is the difference between disagreement and dissension recognized in the program? When dissension exists, how can this be converted to a constructive activity in program development?
- Does the institution's administration understand the unique goals of the program? If not, why not?
- Is the course of study described in terms of course content?
- Is sufficient course work provided to meet program objectives? For example, does the course work permit students to meet qualifications for ASHA's Certificate of Clinical Competence, state licensure, state and/or local department of education certification, and state and/or local department of health qualifications?
- Are courses offered frequently enough to permit a student sufficient opportunity to obtain qualifications as described in B.2 above?
- Is the course work sequenced to provide maximum educational growth?
- How does the program ensure that each student follows the appropriate curricular sequence?
- Does the curricular sequence move from courses on normal processes to classes on communication disorders? If not, why not?
- Are courses taught by faculty/instructional staff with appropriate academic and experiential background? If not, what steps are planned to correct this situation?
- Is the program adequately planned in terms of length, timing, progressive specialization, and availability of advanced courses and seminars? Is any future restructuring planned?
- Is the graduate program clearly identifiable and qualitatively different from the undergraduate sequence? If not, how soon can this separation be effected? When graduates are placed in undergraduate courses, what is the difference in the performance standard required?
- Does the curriculum reflect a commitment to currency in terms of changes in knowledge, legislation, and human resource needs?
- How adequate are the assessment procedures used to evaluate students? If inadequate, how can they be improved?
- Do grades accurately reflect a student's academic and clinical performance? If not, in what way can grading be improved?
- Do students have sufficient research experience so that they are able to develop a viable method of problem formulation and solution? If not, are additional research opportunities planned?
- When and where, in the program sequence, do students obtain adequate guidance in professional and scientific responsibility, as well as ethics?
- What is the mechanism for systematic review and updating of each course in the program?
- What do students value most in the curriculum? Least? What is the cause of this difference? How can "least valuable" be improved?
- Is the program sufficiently flexible to allow students an opportunity to maximize their own personal and professional growth? How can this flexibility be expanded?
- What course work is permitted and encouraged outside the immediate program to give students an opportunity to learn the viewpoints of those in related professions-for example, psychology, learning disabilities, deaf education?
- What procedures have been established to evaluate transfer credit? Does the mechanism adequately evaluate the competencies that the units attest to?
C. Clinical Practicum
- Is the clinical experience appropriately sequenced with the academic offering? Do students always understand the theoretical principles of a particular procedure before they are required to conduct the task? If not, how can this be ensured in the future? How soon can necessary changes be implemented? What learning experiences are provided that actually relate theory and practice?
- Is the clinical experience designed so that it follows substantial course work in general education and normal development as well as specific courses related to communication disorders?
- Are the clinical hours distributed over a reasonable period of time? If not, how can distribution be appropriately spread? When can this be accomplished?
- Is the student's clinical experience graduated and sequential? If not, what changes are planned and when?
- Does the program and its associated facilities have a sufficient client pool to provide the broad clinical experience that the profession requires? If not, what steps are being taken to remedy this situation? What types of experiences are not provided for students? What types of clients did students see last year? What types are planned for next year?
- Is the clinical program sequenced in a way that allows all of the students to obtain similar educational experience? Is there significant variation in the experiences provided students? Why?
- Does the program's off campus component provide students with an opportunity to participate in a variety of inter-professional activities? If not, how can this be improved?
- Are all of the field supervisors committed to educating students, or are some totally service oriented?
- How much contact with the field supervisor is needed to ensure that the instructional staff is aware of the progress the students are actually making in the placement? Is this amount of contactprovided?
- How is continuity of supervisory practices across the instructional staff promoted?
- How objective is assessment of students' clinical performance? Do assigned grades actually represent their performance? If not, why not?
- How is feedback provided to students regarding their clinical efforts? How frequently is such feedback provided?
- Is adequate and appropriate clinical supervision provided for all of the students? Is the staffing formula correct? If not, what improvements are planned?
- Are any of the instructional staff providing direct clinical service for student observation? If not, why not?
- How are supervisors (both on and off campus) made aware of requirements for supervision? How is compliance with these requirements ensured?
D. Faculty/Instructional Staff
- Are there sufficient experienced, trained personnel capable of effective teaching in all necessary areas of specialization of a comprehensive program? If not, what plans exist to remedy any gaps?
- To what extent are faculty/instructional staff teaching outside of their area of expertise?
- How is the teaching load balanced with various other professional responsibilities?
- If new instructional staff members could be hired, what would be the priorities in terms of selection of experience and education?
- How is research competence promoted and rewarded?
- Has the instructional staff remained sufficiently stable so that continuity exists in the program?
- Are the rank and tenure of the faculty/instructional staff different from those of other departments? If so, what steps can be taken to adjust for the discrepancy?
- What is the teaching load carried by other departments that have clinical programs? Does this vary significantly from that within the audiology and/or speech language pathology program? If so, why?
- What mechanism exists within the program to ensure that all faculty/instructional staff remain educationally current?
- Do the students have sufficient opportunity for contact with all the faculty/instructional staff? If not, how can this be remedied?
- Is the faculty/instructional staff sufficiently diverse so that students can be exposed to a variety of thought? If not, can steps be taken to provide diversity?
- Is the space available for classrooms, offices, observation areas, and research labs adequate? If not, is there a plan, including time lines, to obtain additional space?
- Are there architectural barriers that limit participation by persons with physical disabilities? If so, when will steps be taken to remove them?
- Is equipment adequate for all aspects of the program? If not, when will necessary equipment be purchased?
- What are the equipment priorities? How are these priorities developed?
- Are the standards for calibration and maintenance of equipment adequate? If not, what steps are planned to improve them?
- Are the library holdings adequate both within the professional disciplines of the program and related disciplines? Is the budget sufficient to ensure that holdings are current?
- Is there adequate support staff for the program-for example, secretaries, maintenance, technology assistance? If not, is additional staff budgeted?
F. Admission and Advisement
- What objective measures are used to select students? What is the relative value of each?
- What evidence exists to show that the criteria used for selecting students are related to success within the program?
- Are students admitted to the program who do not meet the selection criteria? What happens to these students? Are special students provided with any additional experiences to facilitate their successful completion of the program? Does additional support really help? Given experience with these students, under what circumstances should they continue to be admitted?
- Once a student is admitted to the program, how is his or her progress monitored?
- How adequate is the feedback provided to students regarding their performance? If not considered adequate, what improvements should be made, and how soon can they be put into effect?
- Is there a systematic advising program? If not, why not?
- Are students' academic and clinical records up to date? If not, why not?
- How is the acquisition of knowledge and skills tracked within the program?
G. Questions for Employers
- Can a graduate of the program complete speech, language, or hearing screening procedures appropriately? Are there procedures that the graduate cannot perform? If so, please specify.
- Does the program graduate demonstrate adequate knowledge of diagnostic techniques and instrumentation? Which techniques and/or instruments does the graduate handle with proficiency? With which techniques and/or instruments should the graduate be more proficient?
- Is the graduate able to establish an appropriate caseload? What, if any, problems does the graduate have in establishing a caseload? How would you suggest the training program be modified to correct for any problems that occur in establishing a caseload?
- Does the graduate apply current research findings to therapy regimens?
- Does the graduate work better in a one to one treatment situation or in a group situation? What accounts for this difference? Is it a problem? What do you believe the training program could do to ensure that the graduate works equally well in a one to one and in a group situation?
- Does the graduate establish appropriate long and short range goals for each client in the caseload? What problems, if any, does the graduate have in establishing these? What do you believe the education program could do to improve the graduate's ability to establish clinical objectives?
- Are there certain types of handicapping conditions the graduate handles extremely well? Poorly? What do you believe accounts for the difference? Is it a problem? What do you believe the educational program could do to equalize the graduate's ability to handle all types of problems?
- Are the reports that the graduate writes complete? What are their strengths, weaknesses? What do you believe the educational program could do to improve report writing?
- Does the graduate respond well to supervision? If not, what appears to be the primary source of difficulty? How could this be resolved?
- Does the graduate maintain positive relationships with clients and instructional staff? If not, what appears to be the primary source of difficulty? How could this be resolved?
H. Questions for Students
- In general, are the objectives of the program and of the courses in the curricular sequence clear? If not, what do you believe could be done to improve the situation?
- Have you found that, in general, there has been considerable agreement between the announced objectives of the courses and what was actually taught? If there have been major discrepancies, what, in your opinion, has been the cause?
- In general, have the reading assignments been relevant to class objectives? If not, what do you believe caused the discrepancy?
- Are the lectures given by the program's faculty/instructional staff well organized and designed to facilitate the understanding of the subject? If not, how do you believe they could be improved?
- In general, does the program's faculty/instructional staff challenge you? If not, what steps would you suggest modifying this situation?
- Has your interest in the professions been increased or decreased as a result of your interaction with the program's faculty/instructional staff? If decreased, why?
- Does the program's faculty/instructional staff attempt to relate course content to the total discipline? If not, how could this be improved?
- Does the program's faculty/instructional staff provide sufficient opportunity for you to apply concepts and to demonstrate understanding of the subject? If not, how could this be improved?
- In general, has the program's faculty/instructional staff genuinely been concerned about your progress and attempted to be actively helpful? If not, how do you believe this could be improved?
- Is the program's faculty/instructional staff readily available to you for consultation? If not, how could the program be modified to provide more student/faculty dialogue?
I. Questions for Alumni
- If you were starting school again, would you apply for admission to the program? If not, why not?
- Considering all aspects, were you completely satisfied with the program? If not, which aspects would you improve and how?
- Did you have sufficient opportunity to present problems, complaints, or suggestions to the instructional staff? If not, what hampered you in these efforts?
- Do you believe that your clinical practicum supervisors spent sufficient time observing and guiding your clinical practicum? If not, what do you believe accounted for their lack of availability?
- Do you believe the counseling that the program provided was adequate? If not, what would you suggest be done to improve this situation?
- Which academic/clinical areas do you feel most/least prepared in? What do you believe accounts for the difference? What steps would you recommend be taken to reduce this discrepancy?
- Generally, do you believe that most of what you learned was relevant to clinical work? If not, why not? What could be done to improve the curriculum so that it is more relevant?
- Which courses in the program have proven to be the most/least beneficial? Please list and explain why.
- Given the opportunity, what would you have deleted from your academic program? Why?
- Given the opportunity, what would you have added to your academic program? Why?
Table of Contents | Continue to Substantive Changes to CAA-Accredited Graduate Education Programs