Three Speech-language Pathology Graduate Programs, One Model: Using Systematic Instruction to Develop Students’ Clinical Decision-making Skills

To prepare graduate students to implement evidence-based practice effectively, educators must integrate instruction on rational clinical decision-making into course curricula. Three faculty members at different universities adopted an educational approach derived from the Rehabilitation Treatment Specification System (RTSS) to teach and assess clinical decision-making in the context of treating acquired cognitive-communication disorders for people with traumatic brain injury. Using treatment theory illustrated in the RTSS, the authors piloted instruction and assessment materials to examine potential usefulness of the approach and effects on student knowledge and confidence in clinical decision-making. The results indicated that the instructional approach effectively bolstered students’ knowledge of and confidence implementing memory-based cognitive-communication intervention. Additionally, using a case-based assessment tool, the authors were able to measure how students modified treatment activities in response to different contextual variables. Implications and suggestions for implementing theory-based instruction in graduate education are discussed.


Introduction
Educators in graduate-level speech-language pathology programs are challenged with the dual responsibility of facilitating knowledge transfer and ensuring students have the tools to apply that knowledge in clinical contexts. For example, preparing students to treat individuals with acquired cognitive-communication disorders requires explanation of basic factual information (e.g., How does traumatic brain injury affect cognitive-communication processes?), information about clinical practice (e.g., What are the assessment and treatment options for people with traumatic brain injury?), and instruction on clinical decision-making (e.g., how and why certain interventions may be preferred).
Learning and applying knowledge is essential for students entering a clinical environment rooted in evidence-based practice. As such, learning outcomes for graduate students in communication sciences and disorders (CSD) graduate programs focus on integrating information related to current scientific evidence, client/caregiver goals and values, and clinical expertise. In the context of traumatic brain injury (TBI) rehabilitation, teaching best current evidence may include a wide array of topics including the nature of TBI and resulting symptoms, examples of specific assessment and treatment activities, and related topics associated with community reintegration, psychosocial functioning, and independent living following TBI. Instruction to address understanding client/caregiver values may include concepts of counseling, interviewing, or implementing patient-centered care. Teaching clinical expertise means the instructor provides instruction and opportunity to practice clinical skills. Given the broad spectrum of knowledge and skill translation, educators must be well-versed on effective pedagogy to effectively train evidencebased practitioners.
Of the three corners in the evidence-based practice triangle (external scientific evidence, client perspectives, and clinical expertise), we would argue that the most challenging to teach and assess is "clinical expertise" (American Speech-Language-Hearing Association [ASHA], 2020). By definition, an "expert" has sufficiently learned and practiced skills to the degree they can rely on intuition instead of intentional, concrete analysis of a clinical problem (Benner, 1982). The temporal constraints of a typical master's degree program prevent the required intensity and variety of practice to facilitate true expertise and fluency. Expertise in healthcare practice is also characterized by a systematic approach to solving clinical problems. Among other attributes, experts are differentiated from novices based on their use of communication skills, level of confidence, use of self-reflection, and utilization of contextual/clinical resources (Black et al., 2010;Morrison & Symes, 2011). One study reveals expert clinicians choose treatment targets that are more specific and functional when compared with novices who tended to choose more general, less-personalized intervention targets (King et al., 2007). In the field of medical education, there is a tradition of comparing novice and expert approaches to solving medical problems. While both novices and experts tend to rely on predictable patterns to solve medical problems (i.e., differential diagnosis), experts tend to be more aware of situational factors (e.g., physical observations) and are more adroit at analyzing variations from predictable symptom patterns than novices (Moulton et al., 2007.) The differences between experts' and novices' approaches to making clinical decisions have also been explored in the field of speech-language pathology. For example, one study demonstrates how novices and experts differ in their use of diagnostic reasoning skills. Specifically, expert speech-language pathologists integrate prior experience, intuition, and awareness of contextual factors to plan and prioritize diagnostic objectives and integrate assessment findings into treatment planning compared to novices. Additionally, experts anticipate potential barriers and make contingency plans (Ginsberg et al., 2016). Given the role of clinical expertise in the ASHA evidence-based practice triangle, it seems graduate educators should seek to promote reasoning and decision-making skills within course curricula and teach students to think like experts (McAllister & Rose, 2000).
Research from the field of educational psychology has been instrumental in crafting preprofessional pedagogy. For example, Benjamin Bloom's Taxonomy of Educational Objectives (later referred to as the Taxonomy of Teaching, Learning, and Assessments) is arguably the most recognizable and comprehensive model for conceptualizing assessment of learning (Bloom, 1956). Of particular interest to graduate CSD education, Bloom's taxonomy provides a hierarchical framework to conceptualize progression of student learning through stages from novice to expert. As students learn, they are able to use new knowledge in increasingly complex, integrative ways. This means students' progress from remembering information, understanding information, applying information, analyzing information, evaluating information, and eventually are able to produce tangible evidence of their knowledge (Anderson et al., 2001;Bloom, 1956). Given the importance of promoting clinical expertise, it is incumbent upon educators to systematically assess student learning across all stages of the hierarchy.
Since 2008, the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) has continuously funded an interdisciplinary effort to improve understanding and communication about rational clinical decision-making. As a result, NIDILRR researchers have developed a schema for conceptualizing treatment theory, clinical activities, and treatment elements called Rehabilitation Treatment Specification System (RTSS; Dijkers, 2014;Hart et al., 2014;Whyte et al., 2014). The intent of this system is to use treatment theory as a common language between researchers and practitioners across different disciplines. The treatment theory component permits a top-down framework for guiding professionals and students to conceptualize clinical concepts, especially clinical goal development. There are four main components to RTSS (Turkstra et al., 2016): (a) rehabilitation aims (long-term goals, such as return-to-work, that offer guidance for selection of short-term goals, such as executive function skills to organize emails); (b) treatment ingredients (clinician and client activities that are catalysts for changing behavior); (c) mechanisms of action (the proposed manner in which the ingredients interact with the patient to alter performance, such as errorless learning); and (d) treatment targets (aspects of impairment, function, or participation that are being remediated or compensated for by treatment, such as accurate and timely completion of job duties). Figure 1 maps clinical decision-making onto the key components and theory of RTSS.
Training the four components of RTSS alongside course content should improve knowledge acquisition and clinical decision-making by providing a contextualizing structure of how to use the information. By training students in RTSS structure, students may be expected to approach clinical problems systematically (i.e., like an expert) and answer common critical thinking questions essential for clinical practice. Clinical questions include: 1. What is the clinical problem (treatment target)?
2. How will improvements in the treatment target improve the patient's life (rehabilitation aim)? 3. Why do I expect this intervention to work (evidence-based treatment ingredients and hypothesized mechanism of action; Browne & Keeley, 2014;Ferrett, 2015)?

Figure 1
Schematic of the Rehabilitation Treatment Specification System (RTSS) Note. Bidirectional arrows indicate that the discussion can go from targets to ingredients when identifying treatment goals or ingredients to targets when discussing/evaluating treatment options.
RTSS provides a taxonomy to communicate those rationales to instructors, supervisors, family members, and others. As such, RTSS provides a sufficiently robust scaffold to support student learning across the spectrum of Bloom's taxonomy and prepare students to think like expert clinicians.
Current project. The purpose of this project was to pilot teaching and assessment materials reflecting RTSS components for first-year speech-language pathology graduate students. The project was conducted in three separate graduate programs, but the content covered in each course similarly focused on treatment of acquired cognitive-communication disorders. As applied to the three authors' respective courses, we shared two objectives. First, we intended to teach evidencebased intervention approaches targeting memory impairment for individuals with TBI. Second, we sought to promote a rational, systematic approach to clinical decision-making that focused on teaching students how and, more importantly, why specific approaches should be implemented. By adopting RTSS, the overarching goal of this project was to determine the feasibility of implementing a generalizable pedagogic approach across three institutions and determine the potential effects on students' ability to analyze, evaluate, and apply information in a systematic, evidence-based manner.

Method
This study was conducted as a pilot, or "pre-trial," study to assess the potential impact of an RTSS teaching approach. In clinical research, pre-trial studies are conducted to develop and refine intervention and assessment materials as well as promote hypothesis generation about the potential effects of the treatment (Fey & Finestack, 2009;Robey, 2004). Since we implemented teaching activities and assessment methods typical for our courses and did not collect identifiable data from students, this pilot study was exempt from Institutional Review Board review.
Course/program Descriptions. This pilot study was conducted concurrently at three separate universities which are situated in the Northeast, South, and West regions of the United States. The three universities house speech-language pathology graduate programs that are accredited by ASHA. Each author is a faculty member who teaches a graduate course addressing management of cognitive-communication disorders at their respective institution. All three courses were offered for three semester credits during either the Summer 2018 or Spring 2019 semesters. The duration of instruction ranged from five to fifteen weeks.

Instructional Materials.
To facilitate students' acquisition and application of theoretical and clinical critical thinking skills, we incorporated RTSS into our acquired adult neurologic disorders course content focused on TBI and resulting memory deficits. The RTSS instructional material we presented included a brief description of the RTSS framework, emphasizing its major components: (a) rehabilitation aims (long-term goals, such as return-to-school, that offer guidance for selection of short-term goals); (b) treatment targets (aspects of impairment, function, or participation that are being remediated or compensated for by treatment); (c) treatment ingredients (clinician activities that instantiate client change and compose the therapeutic approach of short-term goals); and (d) mechanisms of action (the proposed manner in which the ingredients interact with the patient to alter performance).

Outcome Measures.
In the absence of existing measures, we developed three for this pilot study to assess student acquisition and application of knowledge pertaining to TBI symptoms, assessment and intervention practices, and RTSS components. The measures are: (a) Self-Assessment of Memory Treatment Knowledge and Skills Questionnaire (pre-/post-instruction); (b) Knowledge of Memory Treatment Ingredients (pre-/post-instruction); and (c) Assessment of Students' Application of the RTSS Model to a TBI Case Study (post-instruction). Information about each follows.

Self-Assessment of Memory Treatment Knowledge and Skills
Questionnaire. The Self-Assessment of Memory Treatment Knowledge and Skills Questionnaire is an 8-item self-report questionnaire measuring knowledge of treatment ingredients and confidence implementing treatment approaches (see Appendix A). The questionnaire was used as a pre-/post-test measure of self-efficacy, a key factor in critical thinking ability (Bandura, 1997). Each item is a statement about memory interventions and students rated their agreement using a seven-point Likert scale (1 = Strongly Disagree, 7 = Strongly Agree). This measure was developed for this pilot study to evaluate students' confidence in their knowledge of concepts central to RTSS including treatment aim, treatment target, and treatment ingredient. For example, two of the items were as follows: 'I am able to integrate contextual variables specific to individual patients while planning treatment for memory disorders;' 'I am confident in my ability to identify appropriate treatment targets for patients with TBI-related memory disorders given 3 months of therapy sessions.'

Knowledge of Memory Treatment Ingredients. The Knowledge of Memory Treatment
Ingredients measure required students to remember, understand, and evaluate key ingredients of five different memory interventions (see Appendix B). Students were provided with the names of five treatment approaches (e.g., external aid for memory) and were asked to: (a) list the key ingredients per approach (e.g., cueing; reinforcement), and (b) explain how each improves function of the person with TBI (e.g., enables a college student to complete homework assignments on time). This pre-/post-test measure was developed for this study and assessed the effects on student learning while using RTSS to explain key features of memory interventions.

Assessment of Students' Application of RTSS Model to TBI Case Study.
We developed a casebased assessment tool requiring students to evaluate, analyze, and apply relevant clinical information in the context of treating a hypothetical client with cognitive-communication impairment following TBI. The case study described how the client was injured during military service (e.g., TBI, physical impairments, anxiety); provided a detailed summary of impairment and activity restriction (e.g., memory impairment, loss of independence); and identified important contextual/environmental barriers with which the client was confronted (e.g., funding, home life). Students generated responses to open-ended requests for information and demonstrated critical thinking skills by modifying their treatment approach in response to contextual changes in the hypothetical case (e.g., dosage, care setting). For example, two of the probes regarding the case were as follows: 'Identify potential treatment aim and treatment targets;' 'Describe potential implications of service delivery on treatment ingredients.' This post-test measure gave us the ability to assess students' ability to integrate clinical information, make informed decisions, and individualize care based on their knowledge of how and why certain interventions work.

Analysis.
To assess potential effects of the instructional approach on acquisition of concepts related to memory intervention for individuals with TBI we analyzed change in our pre-/postinstruction measures (Self-Assessment and Treatment Ingredients) and analyzed our summative assessment (Case Study) using a criterion-referenced rubric.

Results
A total of 84 students were enrolled across the three courses and all were first-year CSD graduate students. There were thirty-two students from the Eastern university, twenty-eight students from the South, and twenty-four from the West. Sex distributions were consistent between the three programs: South = 27 females and 1 male; East = 32 females; West = 23 females and 1 male. Participant distribution is consistent with ASHA (2019) professional demographics, as 95.4% of speech-language pathologists are female.

Self-Assessment of Memory Treatment Knowledge and Skills Questionnaire.
As a measure of self-efficacy, we obtained self-ratings of students' comfort in their knowledge of persons with TBI and memory therapy. One data point was missing from the Southern university due to failure of one student to complete the follow-up survey. Students answered eight questions about their assessment and treatment abilities related to memory disorders. Responses were on a Likert scale of 1-7 with lower numbers reflecting a more negative attitude. With the same 7-point Likert scale for all eight questions, the range of possible scores was 8-56 with higher scores indicating more self-efficacy. Preliminary analysis indicated we could assume normality of the data, so a t-test was used to identify differences before and after instruction. Before instruction the mean assessment of the students was 20.99 (SD = 10.04). After instruction the mean student self-assessment was 40.95 (SD = 8.21). A t-test indicated a statistically significant change in student self-appraisal, t(83) = -16.12; p = 0.0001. Figure 2 illustrates change in self-assessment of memory treatment knowledge and confidence following instruction.

Figure 2
Graduate student performance on self-assessment of skills before and after RTSS module training Note. Figure illustrates the differences in self-assessments of content knowledge before and after module training in a bar chart with whiskers. Whiskers indicate standard error of measurement.
To examine differences between university settings we used an ANOVA. We found a significant difference between universities in self-assessments, F(2,82) = 6.01; p = 0.004 (see Figure 3). A post hoc Tukey Test indicated no difference between the universities from the West and South, but both West and South were significantly different than the university from the East, which had smaller increases in self-appraisal. The Western university differed from the Eastern university with a p = 0.006; while the Southern university differed from the Eastern university with a p = 0.029.

Knowledge of Memory Treatment Ingredients.
Next, we assessed changes in ingredient knowledge before and after instruction. Data were only available for the East and South schools as the school in the West ran their class in the Summer semester when this measure was still being developed. The schools in the East and South offered their classes in the Spring Semester. There were two students from the Southern university missing follow-up data (N = 60). Scores on this scale ranged in absolute value from 1-5 depending on the number of treatment ingredients generated per question. A Wilcoxon Signed Rank Test was used to test ingredient knowledge before and after instruction. Findings were significant with the median before training being zero ingredients (range: 0-3), and after training the median being 3 ingredients per approach (range: 0-5). We used a Mann-Whitney U to identify any possible differences between the Southern university and the Eastern university. Findings were not significant with a U = 311.5, two-tailed, p = 0.096, indicating no significant differences in the demonstration of ingredient knowledge between these two university groups.

Figure 3
Changes in self-assessment by university Note. A box and whisker plot showing the differences before module training and after module training by university setting. Points reference the mean score and whiskers indicate the standard deviation.

Assessment of Students' Application of RTSS Model to TBI Case Study.
Lastly, we examined the total difference in application of RTSS on the TBI case study using a rubric we developed (see Appendix C). The rubric was based on 100 possible points spread across five different criteria with higher scores indicating better performance. Because rubric scores did not pass tests of normalcy due to ceiling effects, we used a log transformation of the data before running an ANOVA. The ANOVA indicated that performance on rubrics did not differ significantly between the three programs where the course was taught, F(2,82) = 2.12, p = n.s. Overall performance on the case study was high (M = 92.5, SD = 12.42) indicating students demonstrated use of evidence-based practice and critical thinking skills.

Examples of Student Responses to Case Study Prompts.
One of the primary objectives of this study was to teach and assess students' application of RTSS-driven principles that would support their rational clinical decision-making. The following excerpts are from a student who demonstrated understanding of the interaction between treatment targets, treatment ingredients, and contextual variables on the case study assessment. When describing treatment aims and targets differentiated by contextual variables (high intensity versus low intensity), the student wrote that the overarching treatment aim is to "reduce the functional impact of the cognitive-communication deficit." Responding to the different treatment intensity conditions, the student selected different treatment approaches and offered a rational explanation. In the high intensity condition, she wrote, "I would target his decreased self-regulation through the use of metacognitive strategies and selfmonitoring strategies." In the low intensity condition, the student chose a more concrete approach, "One option...would be to train the use of external aid such as a memory wallet or visual schedule to compensate for his memory deficits." She added, "Given the (reduced number of sessions), I would suggest to the parents and caregivers that we incorporate some sort of home program.... This would provide more massed/distributed practice across contexts compared to only practicing once a week." This student demonstrated the capacity to integrate different contextual variables, prioritize the most important treatment targets, and anticipated implementation barriers. This example illustrates the student's mastery of RTSS component integration consistent with emerging clinical expertise.

Discussion
The current project sought to explore classroom application of the RTSS with first-year CSD graduate students enrolled in a course on acquired cognitive-communication disorders across three academic training programs. The pedagogical focus of this project expands on previous RTSSrelated work by Whyte et al. (2014), Hart et al. (2014), and Dijkers (2014) who first proposed a theory-driven, interprofessional, rehabilitation treatment taxonomy based on active ingredients to inform intervention decision-making. This work is consistent with Turkstra et al. (2016) who proposed translational applications of this taxonomy between speech-language pathology clinical and research settings. We applied RTSS as a framework for embedding instruction of evidencesupported clinical decision-making and reasoning skills into course curriculum. The RTSS model allowed us to teach and reinforce skills that reflect how expert clinicians may approach informed assessment and treatment for clients with cognitive-communication disorders. The results of this pilot study provide preliminary evidence that RTSS is helpful in preparing graduate studentclinicians to be evidence-based practitioners by explicitly targeting gaps between how novice and expert clinicians think about clinical practice.
Results support the use of RTSS in graduate training programs based on the following project outcomes: (a) students' self-efficacy regarding their post-TBI memory deficit knowledge and skills increased post-RTSS instruction; (b) students' knowledge of treatment ingredients or activities to address post-TBI memory deficits increased post-RTSS instruction; and (c) students' ability to integrate evidence-driven theory to clinical activity and treatment element problem solving and decision-making. Regarding students TBI knowledge and skills self-efficacy, knowledge of various TBI interventions, and expanded integration of TBI theory, practice, and critical thinking skills, the expectation is that all of these will continue to increase with subsequent academic courses and supervised clinical practice. This evolution will result in competent clinicians who have mastered the ability to translate their continually growing knowledge and skills into maximized outcomes for clients with TBI (Doble et al., 2019;Geitz et al., 2016). Although there were significant differences in self-efficacy scores between the school in the East as compared to the schools in the West and South, as Figure 3 shows, this school has a shallow slope in improvement from before to after because they started off with a higher self-efficacy score. We interpret this to mean that the general trend was consistent with the self-efficacy of the school in the East arriving at roughly the same levels as the other two programs.
While the RTSS principles were applied to assessment and treatment of cognitive-communication impairment following TBI, student outcomes may be maximized when the model is applied across a graduate program's entire curriculum. Such program-wide use would be expected to facilitate students' acquisition, application, and generalization of RTSS problem solving and decisionmaking competencies (Harvey-Northrop & Vinney, 2019; Morris et al., 2018). Similar to graduate educators who adopt program-wide instructional approaches (e.g., problem-based learning), RTSS may be useful for providing standard language and theory for training clinical decision making across courses (McAllister & Rose, 2000). From a pedagogic standpoint, two processes embedded within RTSS are particularly suitable for program-wide implementation. First, RTSS reinforces the importance of identifying treatment targets and selecting evidence-supported treatment approaches that are theoretically related to each other. In other words, the clinician must have reason to believe that chosen treatment ingredients will change a target behavior. Second, by accounting for "mechanism of action," RTSS encourages students to consider why certain interventions may or may not be effective, appropriate, or feasible given a client's communication need. These principles represent rational, informed, clinical decision-making across all age groups or clinical populations graduate students will see in clinical practice.
Limitations and Future Directions. Given the stated objectives of this first study exploring use of RTSS in an academic training program, no comparison was made of study outcomes with any other pedagogical methods. Therefore, a comparison study of multiple instructional methods would be a beneficial next step. While the authors of this work believe that using a RTSS instructional approach was beneficial to their students, a significant amount of time was required to develop the outcome measurement tools, present class material using an RTSS framework, and grade/score students' performance on each. It would be informative to objectively measure if the amount of required faculty time decreased with increased familiarity using RTSS teaching methods and outcome measurement tools when teaching TBI class content and compare that time required with time required for other class content (e.g., aphasia post-stroke). Finally, we were able to implement the same teaching methods for material related to cognitive-communication disorders in three different institutions successfully. It would be useful to determine whether other instructors/researchers could replicate the approach and whether RTSS would be as useful for teaching different course content. For example, the RTSS model may or may not facilitate the same student outcomes in an articulation course as in the acquired cognitive-communication disorder course targeted in this study.
It is incumbent on educators to prepare students to enter the field of speech-language pathology equipped with tools to help them make evidence-based treatment decisions. Rational decisionmaking is typical of expert clinicians and means the clinician understands both what to do and why. We believe the RTSS schema provides a clear, relevant, and generalizable scaffold for preparing students to think the way expert clinicians think and approach clinical problems rationally.

Disclosures
Bryan M. Ness Financial: Draws a salary from the university at which this study took place Non-financial: None Therese O'Neil-Pirozzi Financial: Draws a salary from the university at which this study took place Non-financial: None Peter Meulenbroek Financial: Draws a salary from the university at which this study took place Non-financial: None

RTSS Case Study Grading Rubric
Identify relevant background and assessment findings (Approx. 10 points)