A multidisciplinary carepath to standardize disparate concussion care was developed and implemented at the Cleveland Clinic. The Cleveland Clinic Concussion (C3) mobile application was used to enable the carepath through biomechanical outcomes characterizing cognitive and motor function. Patient outcomes improved while cost of care was reduced following implementation.
The evidence-informed standardization of care along disease lines is recommended to improve outcomes and reduce healthcare costs. The aim of this project is to 1) describe the development and implementation of the Concussion Carepath, 2) demonstrate the process of integrating technology in the form of a mobile application to enable the carepath and guide clinical decision-making, and 3) present data on the utility of the C3 app in facilitating decision-making throughout the injury recovery process. A multi-disciplinary team of experts in concussion care was formed to develop an evidence-informed algorithm, outlining best practices for the clinical management of concussion along three phases of recovery – acute, subacute, and post-concussive. A custom mobile application, the Cleveland Clinic Concussion (C3) app was developed and validated to provide a platform for the systematic collection of objective, biomechanical outcomes and to provide guidance in clinical decision-making in the field and clinical environments. The Cleveland Clinic Concussion app included an electronic incident report, assessment modules to measure important aspects of cognitive and motor function, and a return to play module to systematically document the six phases of post-injury rehabilitation. The assessment modules served as qualifiers within the carepath algorithm, driving referral for specialty services as indicated. Overall, the carepath coupled with the C3 app functioned in unison to facilitate communication among the interdisciplinary team, prevent stagnant care, and drive patients to the right provider at the right time for efficient and effective clinical management.
The practice of medicine and the delivery of health care are undergoing a fundamental transformation from volume to value1, driven by payers, hospitals, clinicians, and patients. Health care delivery is challenged by the lack of standardization, interoperability of systems, communication of information, and the coordination of services. These deficiencies complicate effective continuity of care that often result in unnecessary services and have been identified as primary sources of rising health care costs1,2. By contrast, coordinated disease- or condition-specific care has been shown to optimize outcomes while minimizing costs2. In a coordinated care model, patients with a given condition are managed by an experienced, interdisciplinary team according to best practice guidelines, and are referred to the right provider at the right time2.
In efforts to transform clinical practice from a primarily volume-based to a value-based model, the Cleveland Clinic embarked on an enterprise-wide initiative to standardize clinical practice along disease lines through the development and implementation of clinical carepaths3. A care pathway is defined as a standardized approach for the mutual decision-making and organization of care processes for a well-defined group of patients over a well-defined time4. The underlying tenet of a care pathway is standardization of care. The expectation is that with evidence-based care pathways and interdisciplinary care teams, standardization and best practice will be achieved over time, venue and provider resulting in improved efficiency, effectiveness, and value2. With the inclusion of validated outcomes, there is a built-in mechanism for iterating the carepath over time to further refine and improve patient care.
Concussion was identified as a condition in which the potential to streamline care and optimize patient management was significant. While concussive injuries are typically not life-threatening, their effects can be severe and debilitating, with direct and indirect costs estimated at $83 billion annually5. It is estimated that 80-90% of individuals with concussion recover within 7-14 days of injury6,7. However, consensus on the management of the 10-20% of individuals with persistent symptoms is lacking, and evidence supporting or refuting the efficacy of rehabilitation, medical interventions or the diagnostic value of imaging in individuals with post-concussive symptoms is sparse. Multiple consensus statements indicate that an interdisciplinary team approach is optimal to treat concussion6,8,9,10. In 2011, work on the Cleveland Clinic Concussion Carepath11, shown in Figure 1, was initiated. Our primary goal was to create a standardized, evidence-informed approach to the identification, evaluation and management of individuals with concussion. An interdisciplinary team of providers collaborated in the development of the carepath, building consensus based on available evidence in the literature and expert clinical opinion. Physician groups that were represented included: sports medicine, neurology, neurosurgery, rehabilitation medicine, neuroradiology, emergency medicine, primary care, pediatrics and family medicine. The carepath team also included athletic trainers (AT's), physical therapists (PT's), speech therapists, occupational therapists, nurses, and neuropsychologists. Finally, computer scientists and research scientists provided technical guidance and software development. The carepath algorithm, documentation template, and outcomes were developed over a course of 15 months from 2011 to 2012. To ensure appropriateness of outcome measures, the C3 application assessment modules were simultaneously being validated12,13,14,15,16.
The evidence-based Cleveland Clinic concussion carepath depicted in Figure 1 was developed to identify individuals with concussion on a trajectory of typical and delayed recovery, and specifically for the latter group, to create clear and objective criteria for referral to specialty services. Standardized outcome measures were established and proposed to be collected across disciplines. In an effort to facilitate utilization and compliance with the standardized evaluation and documentation requirements of the carepath, the Cleveland Clinic Concussion (C3) app was developed, tested, and deployed. The aim of this project is to 1) describe the development and implementation of the Concussion Carepath, 2) to demonstrate the integration of technology in the form of a mobile application to enable the carepath and guide clinical decision-making, and 3) to present preliminary data on the responsiveness of the C3 app in detecting change in neurological function post-concussion. We hypothesized that utilizing the C3 app would improve interdisciplinary communication aide in clinical decision making.
The protocol outlined below follows the guidelines of the Cleveland Clinic human research ethics committee.
1. Administering the C3 App
2. Assessment of Static and Dynamic Vision
3. Assessment of Information Processing Assessment using Simple and Choice Reaction Time Paradigms14
4. Processing Speed Test15
5. Assessment of Executive Function and Set Switching
6. Interpretation of C3 App to Guide Clinical Decision-Making
To investigate change in neurologic function following concussion, baseline and follow-up C3 assessments were analyzed in 181 student-athletes injured during the 2013-2014 athletic seasons. Detailed demographics of the 181 injured athletes are shown in Table 1. Data were stratified into two groups: those who recovered within three weeks of injury (N=92) and those who were still symptomatic three weeks after injury (N=89). When comparing the first post-injury assessments, Welch's two-sample t-tests revealed a significant difference between the two groups for the following C3 app modules: simple reaction time, (P<0.001); choice reaction time, (P<0.001); Trail Making Test B, (P=0.01); and for two of the six BESS stances quantifying postural sway (double limb stance on foam, P=0.02; tandem stance on foam, P=0.04). BESS errors were not significantly different between the two groups, (P=0.26). Results of the analysis are shown in Table 2. These results suggest that athletes who remained symptomatic performed significantly worse on C3 modules measuring information processing, executive function, set switching, and postural stability. Importantly, athletes performed comparably on all modules at baseline (Table 2), and differences were only seen at follow up, suggesting that the modules are effective in detecting change in neurologic function as a result of concussive injuries. Mean performance of the 181 athletes on the C3 modules at baseline and during each post-injury phase of recovery is depicted in Figure 8, stratified by typical versus prolonged recovery.
Typical | Prolonged | Total | |
N | 92 | 89 | 181 |
Age (SD) | 17 (1.29) | 18 (1.31) | |
Sex, male(%) | 73 (79%) | 66 (74%) | 139 (76.8%) |
Sport | |||
Football | 60 | 46 | 106 |
Soccer | 24 | 30 | 54 |
Baseketball | 2 | 4 | 6 |
Volleyball | 1 | 3 | 4 |
Wrestling | 2 | 2 | 4 |
Hockey | 1 | 1 | 2 |
Rugby | 0 | 2 | 2 |
Altro | 2 | 1 | 3 |
Table 1: Demographic table outlining characteristics of 181 student-athlete diagnosed with concussion.
BASELINE | FIRST POST-INJURY ASSESSMENT | |||||
Variable, mean (SD) | Typical | Prolonged | P-Value | Typical | Prolonged | P-Value |
BESS (total errors) | 14.5 (6.4) | 14.9 (7.2) | 0.65 | 10.9 (5.2) | 11.9 (5.7) | 0.26 |
Simple Reaction Time | 284.8 (24.4) | 287.6 (25.3) | 0.46 | 294.6 (40.7) | 330.1 (64.7) | 0.0003 |
Choice Reaction Time | 421.1 (57.7) | 425.7 (60.6) | 0.61 | 403.3 (60.8) | 459.0 (111.9) | 0.0009 |
Trail Making Test A | 24.2 (8.1) | 23.1 (7.0) | 0.31 | 20.6 (7.1) | 23.4 (8.3) | 0.05 |
Trail Making Test B | 47.4 (15.7) | 46.7 (16.2) | 0.76 | 38.4 (18.5) | 46.2 (16.2) | 0.01 |
Processing Speed Test | 58.0 (10.5) | 57.7 (10.2) | 0.81 | 65.0 (11.4) | 62.0 (11.8) | 0.17 |
Instrumented BESS (CC-PSI percentile) | ||||||
double limb firm | 46.6 (25.0) | 47.5 (27.4) | 0.83 | 51.6 (23.3) | 42.2 (32.1) | 0.06 |
single limb firm | 50.0 (22.5) | 53.0 (25.6) | 0.44 | 64.7 (24.8) | 53.4 (31.9) | 0.02 |
tandem stance firm | 55.6 (26.4) | 54.9 (27.0) | 0.87 | 56.7 (29.1) | 50.5 (26.9) | 0.18 |
double limb foam | 52.4 (28.7) | 49.7 (28.1) | 0.54 | 57.2 (29.3) | 56.1 (29.5) | 0.82 |
single limb foam | 55.6 (27.5) | 50.8 (29.6) | 0.29 | 52.9 (32.3) | 46.1 (31.3) | 0.21 |
tandem stance foam | 40.8 (22.6) | 40.6 (25.3) | 0.96 | 56.7 (28.6) | 47.6 (25.4) | 0.04 |
Graded Symptom Checklist* | 4.8 (11.2) | 18.4 (20.6) | 0.0001 | |||
Standardized Assessment of Concussion* | 26.7 (2.5) | 25.8 (4.20 | 0.17 | |||
*Variables only collected post-injury | ||||||
Bold indicates significant at P<0.05 | ||||||
Abbraviations: BESS (Balance Error Scoring System); CC-PSI (Cleveland Clinic Postural Stability Index) |
Table 2: Results of Welch's two-sample t-tests analyzing differences in performance at baseline (left panel) and at the first follow-up (post-injury) test (right panel) for student athletes who recovered within 21 days of injury (typical recovery) and those who recovered in greater than 21 days (prolonged recovery).
Figure 1: Cleveland Clinic Concussion Carepath algorithm depicting qualifiers guiding clinical care in the acute, subacute, and post-concussive phases post-injury. Fields with gold shading indicate points of care at which standardized, objective outcomes measured through the C3 app integrated with the carepath to guide clinical decision-making. Please click here to view a larger version of this figure.
Figure 2: The digital tablet is affixed to the sacrum of the participant to obtain a biomechanical measure of postural sway during performance of the six stances of the Balance Error Scoring System. Double limb stance on firm surface (Figure 2a), Single limb stance on firm surface (Figure 2b), Tandem stance on firm surface (Figure 2c), Double limb stance on foam surface (Figure 2d), Single limb stance on foam surface (Figure 2e), Tandem stance on foam surface (Figure 2f). Please click here to view a larger version of this figure.
Figure 3: Screen shot depicting instructions for administration of the static visual acuity test. Please click here to view a larger version of this figure.
Figure 4: Screen shots depicting the simple reaction time (Figure 4a) and choice reaction time (Figure 4b) paradigms. Please click here to view a larger version of this figure.
Figure 5: Screen shot depicting the Processing Speed Test (PST). Please click here to view a larger version of this figure.
Figure 6: Screen shots depicting the instructions for the Trail Making Test A (Figure 6a) and the digitized Trail Making Test A (Figure 6b), in addition to the Sample Trail Making Test B (Figure 6c), and the digitized Trail Making Test B (Figure 6d). Please click here to view a larger version of this figure.
Figure 7: Radar plot depicting baseline and post-injury performance on C3 modules for a representative athlete to aide in C3 app interpretation and guide clinical decision-making. Baseline performance is reflected by the perimeter of the polygon, while the red, yellow, and blue polygons represent performance at 2, 5, and 12 days post-injury, respectively. In the first post-injury assessment, information processing measured by the simple and choice reaction time modules did not appear to be impacted in this patient. However, deficits in balance, processing speed, and in the Trail Making Test were evident. Improvements in all aspects of function measured by the C3 app were evident over the course of recovery, as performance was at or near baseline by 12 days post-injury. Please click here to view a larger version of this figure.
Figure 8: C3 app data for student-athletes (N=181) who incurred a concussion stratified by those who recovered within 3 weeks (typical recovery group, N=92)) and those who took longer than 3 weeks to recover (protracted recovery group, N=89). Bar plots depict mean and standard error of the mean (SEM) performance at baseline and in each phase of recovery post-injury for following C3 app modules: simple reaction time (8a), choice reaction time (8b), Trail Making Test B (8c), Processing Speed Test (8d), Cleveland Clinic Postural Stability Index during BESS double limb stance on foam surface (8e), and Cleveland Clinic Postural Stability Index during BESS tandem stance on foam surface (8f). Please click here to view a larger version of this figure.
The development and implementation of the concussion carepath served numerous purposes in our health system's transformation toward value-based care1,2,3. The algorithm of care was critical in guiding clinical decision-making, and was supported by standardized, biomechanical outcomes in the form of the C3 app, which was used by all members of the interdisciplinary concussion team. These standardized outcomes provided qualifiers to monitor recovery patterns in patients, helped identify individuals at risk for protracted recovery, and drove referral for specialty services for those not recovering in a timely manner. As such, the technology-enabled carepath was designed for use by the multi-disciplinary clinical team to avoid stagnant care and to funnel patients to the right provider at the right time. Care and recovery pathways have been recently developed aimed at educating the general public in concussion care21. While the two care pathways overlap in content, the carepath described in this manuscript was designed for use by medical personnel.
A clinical workflow was created within the carepath algorithm representing patient management at three main phases: Acute (0-7 days post-injury), Subacute (8-21 days post-injury) and Post-Concussive (>21 days post-injury). The multi-modal C3 app served to provide a set of common data elements by which clinicians characterized injury status, measured recovery, and engaged in the coordination of care across each phase of injury/recovery. For example, based on epidemiological data indicating typical time to recover from concussion20 and modifiers/co-morbidities that often contribute to prolonged recovery20, the carepath team recommends referral to specialty services to assist with symptom management for those individuals >8 days post-injury who demonstrated a lack of progress toward recovery (e.g., no improvement in C3 modules) and presented with modifiers that may prolong recovery. In the absence of prognostic modifiers, referral for specialty services was recommended for individuals who remained symptomatic >21 days post-injury and did not demonstrate improvements in performance in C3 modules relative to post-injury or did not progress toward baseline performance level if a baseline test was available. These standardized outcomes related to symptoms, neurocognitive function and postural stability using the C3 application across all members of the interdisciplinary team were incorporated into the carepath and into electronic health record (EHR) documentation templates to ensure the collection of objective clinical and subjective patient-reported outcomes to guide clinical decisions11,17.
The C3 app metrics in combination with the carepath algorithm were critical in preventing stagnant clinical care by empowering physicians to refer patients who were not recovering in a timely manner to the right provider at the right time. Specific cut-off scores to trigger referrals were not explicitly identified in the carepath as not all outcome values had sufficient normative data. In general, if the patient was beyond one standard deviation away from an established age-gender mean value at follow-up, a referral to the next level provider was indicated.
Our preliminary analysis indicated that the assessment modules within the C3 app were responsive in detecting differences in performance between athletes who are had recovered compared to those who remained symptomatic post-concussion. A limitation to our methodology was that we cannot determine sensitivity or specificity of these modules with the current data set, as only injured athletes are represented. Correlations between baseline concussion scores and other objective outcomes have been previously reported22, and work is under way to more precisely determine values for each module of the C3 app that indicate meaningful change. Additionally, predictive models are currently being built using clinical and C3 data to identify student-athletes at risk for protracted recovery.
Concussion care is unique in that the initial management of the injury often occurs offsite, with AT's positioned at schools and venues providing the first line of care. A limiting factor in administering medical care remotely is poor access to the EHR and limited direct interaction with other members of the interdisciplinary care team. The EIR module within the C3 app allowed for the detailed documentation of the athlete's injury, their initial disposition, and medical management provided by the AT. Thus, it facilitated hand-offs among providers by linking the care provided by the AT offsite with the medical management of the athlete in clinic and their post-injury rehabilitation. Furthermore, the electronic template eliminated the outdated paper-pencil documentation format commonly utilized in the school setting, promoting more complete documentation accessible within the EHR. Analysis of detailed injury documentation is under way to better understand circumstances surrounding concussive injuries with the aim of mitigating risk and improving outcomes.
The development and implementation of the carepath, and its enablement via the C3 app, served to standardize the care of concussion across the Cleveland Clinic enterprise along evidence-based best practices. Overall, the C3 app was responsive in discriminating between symptomatic and non-symptomatic athletes post-injury and guided referral to specialty services to promote active rehabilitation for athletes with prolonged symptoms. Detailed analyses quantifying the clinical and economic impact of the carepath are ongoing.
The authors have nothing to disclose.
We wish to thank the Jason Cruickshank, Bob Gray, and the Cleveland Clinic Athletic Trainers for their support and assistance with data collection. This study was supported by the Edward F. and Barbara A. Bell Family Endowed Chair to JLA.
Foam Balance Pad | Airex, Sins, Switzerland | dense foam balance pad used during balance testing | |
iPad Digital Table | Apple |