What To Ask When Evaluating Any Procedure, Product, or Program

There is no single answer to any of the questions. Some considerations are presented that may assist in the decision-making process before using a procedure, purchasing a product, or attending an educational program.

Resource for clients and families: Heard About a New Product or Treatment?

• What are the stated uses of the procedure, product, or program?
• To which client/patient population does it apply? Is there documented evidence that it is valid for use with a specified population?
• Are outcomes clearly stated?
• Are there publications about this procedure, product, or program? Is the information published in a peer-reviewed professional journal? Is promotional material (e.g., brochures, training manuals, newsletters, popular press) the only published source of information?
• Is there peer-reviewed research that supports or contradicts the stated outcomes or benefits?
• What is the professional background of the developers of the procedure/product/program? If a company, how long has it been in existence? Have there been complaints to the Better Business Bureau, governmental consumer offices, or licensing boards about the company?
• Are there similar procedures, products, or programs currently available? How do they compare in performance and cost?
• Have you talked with others who have experience with this procedure, product, or program? What was their experience?
• Is it within my profession's Scope of Practice? Is it within my personal scope of practice (i.e., personal training, competence, experience) to use this procedure, product, or program?
• Is certification offered for completing a program? Is certification required to purchase or use the product or technique?
• Does ASHA have statements or guidelines on this topic?
• Based on the factors listed above, is the cost reasonable and justifiable?
• Additional Questions for Techniques/Procedures
• Additional Questions for Products
• Additional Questions for Programs
• Additional Questions for Mobile Devices and Computer Applications ("Apps")

What are the stated uses of the procedure, product, or program?

Uses should be clearly specified and information about reliability should be available. You need to know if you can count on the procedure to work in a consistent way with repeated use.

To which client/patient population does it apply? Is there documented evidence that it is valid for use with a specified population?

The procedure should be used primarily with populations where validity data exists. When applicable, standardization procedures should include the racial/ethnic and linguistic diversity of the client population with whom it will be used. When validity data is not available, the clinician must be able to justify use of the procedure and note limitations to generalization with non-validated populations.

Are outcomes clearly stated?

The expected outcomes should be available with supporting data to demonstrate benefit to the target population.

Are there publications about this procedure, product, or program? Is the information published in a peer-reviewed professional journal? Is promotional material (e.g., brochures, training manuals, newsletters, popular press) the only published source of information?

Peer-reviewed research in a professional journal has the most credibility. Claims made in promotional material should be substantiated by available data.

Is there peer-reviewed research that supports or contradicts the stated outcomes or benefits?

Peer-reviewed research will assist in determining the reliability and validity of a treatment procedure. The quality of the research design (e.g., use of control groups, clear specification of subject characteristics before and after treatment, replication) and level of evidence should be considered. The ideal is peer-reviewed research from a series of well-designed studies that have been replicated and that demonstrate benefits to the client.

Available Resources

• Evidence-based practice
• ASHA Evidence Maps
• ASHA's evidence-based systematic reviews
• ASHA Journals

What is the professional background of the developers of the procedure/product/program? If a company, how long has it been in existence? Have there been complaints to the Better Business Bureau, governmental consumer offices, or licensing boards about the company?

Although the credentials of the developers do not necessarily determine the value of a procedure, product, or program, the background and experience of the developer can give an indication of the knowledge and expertise related to the procedure, product, or program under consideration. Similarly, although a new company may be able to develop a high quality procedure, product, or program, a company in business for a long time may have more credibility if a sound reputation is known based on evidence from sources such as the Better Business Bureau or "word of mouth."

Are there similar procedures, products, or programs currently available? How do they compare in performance and cost?

Costs may be a consideration if other less costly procedures, products, or programs are available and if there is comparability in terms of documented benefits.

Have you talked with others who have experience with this procedure, product, or program? What was their experience?

Others with experience may be able to provide valuable insights into the actual use of the product from their perspective. You can use available networking tools, such as ASHA's member community, Facebook, LinkedIn, and Twitter. ASHA's blog, ASHAsphere, is also available.

Is it within my profession's Scope of Practice? Is it within my personal scope of practice (i.e., personal training, competence, experience) to use this procedure, product, or program?

It is mandatory for ASHA members to abide by ASHA's Code of Ethics and only do what they are trained and competent to do and only use procedures that benefit the client.

Is certification offered for completing a program? Is certification required to purchase or use the product or technique?

Some course providers offer certification for successfully completing a course and may require completion of the course and subsequent certification to purchase or use the product or technique. ASHA recognizes only one certification, the Certificate of Clinical Competence. The Code of Ethics requires that "individuals shall provide all services competently" (Principle I, Rule A). How one attains and maintains an appropriate level of competency is at the discretion of the individual clinician and her supervisor, director, or administration.

Does ASHA have statements or guidelines on this topic?

Related background information and policy documents may assist in making an informed decision.

Based on the factors listed above, is the cost reasonable and justifiable?

After considering the answers to these questions, do you think this procedure, product, or program is worth the time and money? Do you think it will benefit your patients?

Additional Questions for Techniques/Procedures

1. What are the potential risks/adverse consequences?
2. What are the potential benefits?
3. What is recommended as sufficient training to be considered a qualified user of the technique/procedure?

Additional Questions For Products

1. Is it FDA approved? Has the product or its application been changed from its original design/intent as approved by the Food and Drug Administration or Underwriter's Laboratory?
2. What training and technical support are available?
3. Is there a guarantee, warranty, or return policy?
4. Are there potential risks/harm associated with this product?
5. Are there contraindications for specific populations or etiologies?
6. What is recommended as sufficient training to be considered a qualified user of the product?

Additional Questions for Programs

1. Does the program offer CEUs? Are they an ASHA-approved provider?
2. Is the program presented by a credentialed or certified professional?
3. Does the time allotted seem appropriate for achieving the stated outcomes?
4. Who is the target audience (i.e., professional background, beginning vs. advanced)?
5. What is the refund/cancellation policy?
6. What instructional methods will be used? Will it be interactive or give a hands-on experience?
7. Is this topic relevant to current practice issues and trends?

See also: Tips for Choosing and Evaluating CE Courses

Additional Questions for Mobile Devices and Computer Applications ("Apps")

1. Is the particular mobile device the best tool to augment or enhance my client’s communication?
2. What type of connectivity is needed to use the mobile device?
3. Is the app from a trusted source?
4. How much does it cost?
5. Who will pay for the mobile device or app?
6. How long will I be able to use it?
7. Can I use it with more than one client?