Primary Image

Coma Recovery Scale-Revised

Coma Recovery Scale - Revised

Last Updated

Atomized Content

Purpose

CRS-R a standardized neurobehavioral assessment measure designed for use in patients with disorders of consciousness. The scale is intended to be used to establish diagnosis, monitor behavioral recovery, predict outcome, and assess treatment effectiveness.

Link to Instrument

Instrument Details

Acronym CRS-R

Area of Assessment

Cognition
Language
Vision & Perception
Communication
Functional Mobility

Assessment Type

Observer

Cost

Free

Cost Description

<$20 for common objects. Use of the CRS-R is free under some circumstances, but a license is required for research or commercial use. More information may be obtained here: https://spauldingrehab.org/research/programs-labs/rehabilitation-outcomes-center/crs-r.

CDE Status

NINDS CDE Notice of Copyright
JFK Coma Recovery Scale- Revised

Classification

Supplemental for Traumatic Brain Injury (TBI).

Diagnosis/Conditions

  • Brain Injury Recovery

Populations

Key Descriptions

  • The CRS-R consists of 6 subscales designed to assess auditory function, receptive and expressive language, visuoperception, communication ability, motor functions, and arousal level.
  • The 6 subscales are comprised of hierarchically-arranged items reflecting brainstem, subcortical, and cortically-mediated behaviors:
    1. Auditory
    2. Visual
    3. Motor
    4. Oromotor
    5. Communication
    6. Arousal Functions
  • Minimum score: 0
    Maximum score: 23
  • Lowest item on each subscale represents reflexive activity while the highest item represents cognitively-mediated behaviors.
  • Scoring is standardized based on the presence or absence of operationally-defined behavioral criteria. Most items must be administered to obtain a score, although some behaviors (e.g., speech) can be scored when they occur spontaneously.
  • Behavioral responses must be clearly-discernible before they are scored present.
  • While the CRS-R total score should not be relied upon to establish a diagnosis, a total score of 10 or greater indicates a diagnosis of MCS or eMCS.
  • The CRS-R can also be used to differentiate patients into “MCS+” and “MCS-” subgroups, based on the presence or absence of receptive or expressive language function. (Thibaut, 2019)

Number of Items

23

Equipment Required

  • Instruction manual
  • Scoring sheet
  • 2 common objects (cup, comb, etc.)
  • An object that produces a loud noise
  • ADL objects (toothbrush, phone, etc)
  • Hand mirror
  • Brightly coloured object
  • Baseball sized ball
  • Pencil
  • Tongue depressor

Time to Administer

15-30 minutes

Required Training

Reading an Article/Manual

Required Training Description

While there is no mandated training, a recommended training program, “Ten Guiding Principles for Administration and Scoring” and the FAQs are available at the top of this page under “Downloads”. Information about the answer key is in the "Considerations" section.

The CRS-R should be performed by a trained clinician or research investigator.

Age Ranges

Child

6 - 12

years

Adolescent

13 - 17

years

Adult

18 - 64

years

Elderly Adult

65 +

years

Instrument Reviewers

Initially reviewed by Erin Donnelly, PT, MSPT, NCS and the TBI EDGE task force of the Neurology Section of the APTA in 6/2012.

Updated in 2020 by:

Yelena Bodien, PhD

Srivatsan Uchani, BA

Alyssa Taubert, OTD, CBIS

Joseph Giacino, PhD

Camille Chatelle, PhD  

Affiliation: Spaulding Rehabilitation Hospital, Boston MA

ICF Domain

Body Structure
Body Function

Measurement Domain

Cognition
Motor
Emotion

Professional Association Recommendation

  • American Congress of Rehabilitation Medicine (ACRM)
  • National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR)
  • American Academy of Neurology (AAN)
  • National Institute of Neurologic Disorders and Stroke (NINDS)
  • Neurology section of the American Physical Therapy Association’s Multiple Sclerosis Taskforce (MSEDGE), Parkinson’s Taskforce (PD EDGE), Spinal Cord Injury Taskforce (SCI EDGE), Stroke Taskforce (StrokEDGE), Traumatic Brain Injury Taskforce (TBI EDGE), and Vestibular Taskforce (Vestibular EDGE)

Recommendations for use of the instrument from the Neurology Section of the American Physical Therapy Association’s Multiple Sclerosis Taskforce (MSEDGE), Parkinson’s Taskforce (PD EDGE), Spinal Cord Injury Taskforce (PD EDGE), Stroke Taskforce (StrokEDGE), Traumatic Brain Injury Taskforce (TBI EDGE), and Vestibular Taskforce (Vestibular EDGE) are listed below. These recommendations were developed by a panel of research and clinical experts using a modified Delphi process.

For detailed information about how recommendations were made, please visit:  http://www.neuropt.org/go/healthcare-professionals/neurology-section-outcome-measures-recommendations

Abbreviations:

 

HR

Highly Recommend

R

Recommend

LS / UR

Reasonable to use, but limited study in target group  / Unable to Recommend

NR

Not Recommended

Recommendations based on level of care in which the assessment is taken:

 

Acute Care

Inpatient Rehabilitation

Skilled Nursing Facility

Outpatient

Rehabilitation

Home Health

TBI EDGE

R

HR

HR

HR

HR

Recommendations for use based on ambulatory status after brain injury:

 

Completely Independent

Mildly dependant

Moderately Dependant

Severely Dependant

TBI EDGE

N/A

N/A

N/A

N/A

Recommendations for entry-level physical therapy education and use in research:

 

Students should learn to administer this tool? (Y/N)

Students should be exposed to tool? (Y/N)

Appropriate for use in intervention research studies? (Y/N)

Is additional research warranted for this tool (Y/N)

TBI EDGE

Yes

Yes

Yes

Not reported

Considerations

  • The examiner should communicate with medical or nursing staff to identify any contraindications or precautionary measures that should be taken before initiating CRS-R assessment.
  • There are no specific guidelines governing the frequency of CRS-R administration. Best practice suggests that up to five assessments may be required to capture the optimal level of function. (Wannez 2017)
  • The frequency of assessment is also dependent upon the rate of change in performance on the CRS-R, which is usually associated with the length of time post-injury.
  • We recommend discontinuing use of the CRS-R when all three of the following behaviors have been elicited, concurrently, on three consecutive examinations conducted over two weeks:
    • Consistent movement to command (Auditory Subscale = 4)
    • Reliable yes-no responses (Communication Subscale = 2)
    • Focused attention (Arousal Subscale = 3)
  • If there is doubt as to whether the behavior meets the required criteria, the item should not be scored as being present and the next item down should be administered. A rule-of-thumb for determining level of certainty is whether the examiner believes that at least nine out of ten observers would agree the response criteria were met.
  • The CRS-R should be administered and scored as described in the manual. However, certain items or subscales may have to be omitted due to patient-specific factors. If one or more subscales are omitted, the total score cannot be obtained.
  • When CRS-R items cannot be administered or scored in a valid manner, a Test Completion Code should be used to indicate that the examination is confounded.
  • The CRS-R has been translated and re-validated in Spanish (Tamashiro 2014), Italian (Estraneo 2015), French (Schnakers 2008b), Portuguese (Simoes 2011), Norwegian (Lovstad et al., 2010), Russian (Iazeva 2018), German (Maurer-Karattup 2010), Polish (Binder 2018), Korean (Han 2018), and Chinese (Zhang 2019). The CRS-R is also available in Dutch, Swedish, Danish, and Greek, but has not been re-validated in these languages.
  • The pediatric version of the CRS-R, the Coma Recovery Scale for Pediatrics (CRS-P) should be used when assessing children between the ages of one and five who have not yet completed language and motor development (Slomine 2019).
  • In a review of scales that assess DoC, CRS-R was the only scale recommended with minor reservations. This recommendation was supported by expert consensus that the CRS-R has excellent content validity and acceptable standardized administration and scoring procedures. Studies provide evidence that the CRS-R probably has good interrater reliability and good internal consistency and possibly has excellent test-retest reliability. (Seel 2010)
  • The CRS-R satisfies all the criteria required for interval measurement (ie, unidimensionality, local independence, item invariance, absence of differential item function across diagnostic groups) (La Porta 2013; Gerrard 2014)

For more information or to request the Answer Key for the written test portion of the training module please contact:

Michael J.G. Bergin, PhD

Laboratory Manager

Neurorehabilitation Laboratory

Spaulding Rehabilitation Hospital Neurorehabilitation Lab
mjbergin@partners.org

Or

Yelena G. Bodien, PhD

Research Scientist

Spaulding Rehabilitation Hospital Neurorehabilitation Lab

ybodien@mgh.harvard.edu

Do you see an error or have a suggestion for this instrument summary? Please e-mail us!

Brain Injury

back to Populations

Cut-Off Scores

Bodien, 2016; n = 252; Mean Age = 49 years (19.7); Mean Time Post-TBI = 48 (53) days; traumatic or non-traumatic DoC

  • A cut-off score of 8 provides the best balance between sensitivity and specificity for detecting consciousness, accurately classifying 93% of cases. A total score of 8 yields a true positive rate of 93%, true negative rate of 96% and diagnostic accuracy rate of 93%
  • CRS-R total score of 10 or higher yielded a sensitivity of 0.78 for correct identification of patients in MCS or EMCS, and specificity of 1.00 for correct identification of patients who did not meet criteria for either of these diagnoses (i.e., diagnosed with VS or coma). All patients with a total score of greater than or equal to 10 are therefore MCS or eMCS, per CRS-R criteria.
  • The “optimal” total score cut-off will vary depending on the user's objective

CRS-R Total Score

Cut-off

7

8

9

10

11

Sensitivity

0.97

0.93

0.88

0.78

0.73

Specificity

0.80

0.963

0.97

1

1

Accuracy

0.921

0.937

0.905

0.841

0.802

Chatelle, 2016; n = 1190; Mean Age = 43 (20); Mean Time Post-injury = 162 (568) days; DoC patients; English- and French-speaking sample

  • Determined CRS-R subscore combinations that are unlikely to co-occur
  • Unlikely subscore combinations should be examined for administration error, scoring error, or underlying confounding factors that could invalidate the assessment

Disorder of Consciousness (TBI, CVA, hypoxic-ischemic BI, tumor): (Giacino J, Kalmar K, Whyte J, 2004; n= 80; mean age= 38.86 years(range 17-79 years old); mean time post injury= 58.43 days, range 21-249 days and Lovstad et al, 2010; n=31 with severe brain injury, median age 33 years, median days post injury 143).

 

CRS-R subscale

Vegetative State

Minimally Conscious State

Emergence from Minimally Conscious State (MCS+)

Auditory 

Less than or equal to 2 and

3-4 OR 

 

Visual 

Less than or equal to 1 and

2-5 OR 

 

Motor 

Less than or equal to 2 and

3-5 OR 

OR

Oromotor/verbal 

Less than or equal to 2 and

3 OR 

 

Communication 

2-3

In order to be designated in the vegetative state all of the scores in column 2 must be met, however minimally conscious state can be achieved by the demonstration of only one of the score ranges in column 3, likewise emergence from MCS occurs with higher scores in motor or communication subscales.

Normative Data

Chatelle, 2016; n = 1190; Mean Age = 43 (20); Mean Time Post-injury = 162 (568) days; DoC patients; English- and French-speaking sample)

  • Mean admission CRS-R total score= 8.5±5.1

 

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

  • Median CRS-R total score = 12.00

 

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • Median CRS-R total score after the first visit was 8.5 [5.0; 14.75] and after the second visit was 10.0 [5.0; 17.75], р<0.0001
  • Total CRS-R score was 5 [4.5; 6.0] in the group of VS patients vs. 13 [10; 19] in the MCS group (p<0.0001)

 

Test/Retest Reliability

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

  • Test-retest reliability was excellent for CRS-R total scores (ρ=.94, P<.001), demonstrating adequate stability in patient performance over a brief assessment interval (i.e., 36h)
  • Cross-correlation, representing the relationship between scores obtained by different raters on different days, was the lowest of the 3 pairs of ratings (ρ=.79, P<.001)
  • No systematic difference in the scores obtained by different raters on different days (P=.80) or by different raters on the same day (P=.10)
  • Scores obtained by the same rater (ie, rater A) on different days, however, differed significantly (P=.02)
  • Test-retest reliability was high for the total CRS-R score and there was relatively good agreement between the scores of different raters on different days
  • Test-retest reliability was adequate for all subscales except the oromotor/verbal, on which scores were systematically higher on day 2
  • In view of these findings, scores on the visual and oromotor/verbal subscales should be used cautiously. Taken together, the results of these analyses suggest that the CRS-R can be used reliably

Test-Retest Reliability of Dichotomized CRS-R Subscale Scores (n=20)

Subscale

Cohen kappa

95% CI

P

Rater Agreement

Auditory

0.63

±.35

.00

85%

Visual

0.90

±.19

.00

95%

Motor

1.00

±.00

.00

100%

Oromotor/Verbal

0.23

±.51

.17

70%

Communication

0.89

±.22

.00

95%

     

Han, 2018; n = 39; Mean Age = 56.9 (16.9); Time Post-injury = 125.6 (128.8) days; traumatic and non-traumatic brain injury; Korean sample

  • Level of test-retest agreement was very high (97.4%; 38 of 39 cases)

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • Test-retest consistency was high with the correlation coefficient r=1 (р<.0001), indicating the stability of patient’s assessment during a short observation period over one day

La Porta, 2013; n = 129; Mean Age = 47 (20); Mean Time Post-injury = 421 (599) days; traumatic and non-traumatic brain injury; Italian sample

CRS-R Test-Retest Reliability by Paper

 

Giacino and Kalmar (2004)

 

 

Schnakers et  al (2008)

 

 

Study design and setting

 

 

 

 

 

 

No. of centers

1

 

 

5

 

 

Assessment setting

 R

 

 

A, R,

 

 

Sample size

80

 

 

77

 

 

No. of raters

2

 

 

24

 

 

Sample size/rater ratio

40

 

 

3.2

 

 

Reliability

IRR

TRT

ICR

IRR

TRT

ICR

Auditory subscale

K=.86

K = 0.63  t

 

K= .82*

 

 

Visual subscale

K= .58 t

K=0.90

 

K=.85

 

 

Motor subscale

K= . 78*

K= 1.00

 

K=.93

 

 

Oromotor subscale

K= .77*

K= 0.23t

 

K=.92

 

 

Communication subscale

K=.88

K=0.89

 

K=.98

 

 

Vigilance subscale

NA

NA

 

K= . 74*

 

 

Total score

p= .84 *

p=0.94

a= .84*

K= .8 0*

NA

NA

 

 

Lovstad et al (2010)

 

 

Simoes et al (2011)

 

 

Sacco et al (2011)

 

 

Study design and setting

 

 

 

 

 

 

 

 

 

No. of centers

6

 

 

1

 

 

1

 

 

Assessment setting

R

 

 

A

 

 

R

 

 

Sample size

31

 

 

20

 

 

38

 

 

No. of raters

8

 

 

2

 

 

2

 

 

Sample size/rater ratio

3.8

 

 

10

 

 

19

 

 

Reliability

IRR

TRT

ICR

IRR

TRT

ICR

IRR

TRT

ICR

Auditory subscale

K=.90

K= . 71*

 

ICC=0.99

ICC= .86

 

Kw = . 6St

Kw = 0.80*

 

Visual subscale

K= .46t

K=.86

 

ICC=l.00

ICC= .88

 

Kw = . 71*

Kw = 0.84*

 

Motor subscale

K= .67t

K= .73*

 

ICC=0.98

ICC=.81*

 

Kw = . 79*

Kw = 0.96

 

Oromotor subscale

K=.89

K= . 71*

 

ICC=0.96

ICC= .82*

 

Kw = .44t

Kw = 0.8 5

 

Communication subscale

K= .62t

K=.89

 

ICC=0.97

ICC= .82*

 

Kw = .88

Kw = 0.88

 

Vigilance subscale

NA

NA

 

ICC=0.98

ICC= .84*

 

Kw = .51t

Kw  = l. 00

 

Total score

K=.94

NA

a= .74*

ICC=0.99

ICC= .87

NA

p= .81 *

p=0.97        

a= .81 *

 

NOTE. Where several reliability values were available, we reported only the largest ones. Comparison of the various studies may be difficult in view of the fact that the classical psychometric properties reported are strictly sample-dependent and several reliability coefficients were used across different studies.

Abbreviations: a, Cronbach alpha; A, acute setting; CRS-R, CRS-R total score; ICC, intraclass correlation coefficient; ICR, internal consistency reliability; IRR, interrater reliability; K, Cohen Kappa; Kw, weighted Kappa; NA, not applicable; p, Spearman correlation coefficient; R, rehabilitation setting; TRT, test-retest reliability.

* Values compatible only with measurement at the group level (≥ .70 <.85), not at the individual level (≥ .85 ).

t Values not sufficiently reliable for any measurement (< . 70), including measurement at the group level.

 

Lovstad, 2010; n = 31; Mean Age = 33 (15); Median Time Post-injury = 143, range = 21–2130 days; traumatic and non-traumatic brain injury; Norwegian sample

  • Test-retest reliability for rater A from day 2 to 3 was very good (ρ = 0.83) and good for rater C from day 1 to 3 (ρ = 0.77)
  • Test-retest agreement was very good for rater A from day 2 to 3 (ICC = 0.87, 95% CI 0.75–0.94) and good for rater C from day 1 to 3 (ICC = 0.78, 95% CI 0.58–0.90)
  • Test-retest agreement from days 1 to 3 was also higher for the moderately experienced raters versus newly trained raters
  • Test-retest correlations within the newly trained rater group were not significant (ρ = 0.15 and P = .09)

 

Experienced Rater A Day 2 and 3

Less-experienced Rater C Day 1 and 3

 

Spearman

Cohen’s kappa

Spearman

Cohen’s kappa

Auditory

0.84

0.71

0.33

0.35

Visual

0.80

0.86

0.74

0.57

Motor

0.85

0.73

0.59

0.81

Oromotor

0.90

0.71

0.86

0.87

Communication

0.90

0.89

0.91

0.89

Arousal

0.59

NA

0.70

NA

 

Zhang, 2010; n = 169; Median Age = 58, range = 18–86; Time Post-injury: range = 70-99; traumatic and non-traumatic brain injury; Chinese sample

  • Good test-retest reliability for CRS-R total score and subscale scores (intra-class correlation coefficient [ICC] = 0.87 and ICC = 0.66-0.84, respectively)

Interrater/Intrarater Reliability

Binder, 2018; n = 20; Median Age = 38.0 (14.39); Mean Time Post-TBI = 1.5 (0.5) days; severe TBI; Polish sample

  • Inter-rater reliability for CRS-R total scores was excellent: (ρ = 0.76, p < 0.001). Inter-rater reliability for subscales was fair to excellent

 

Estraneo, 2015; n = 122; Mean Age = 54.58, range = 14-88; Time Post-injury = 25 - >365 days; traumatic or non-traumatic brain injury; Italian population

  • Italian version of CRS-R has good-to-excellent interrater reliability for all subscales, particularly Communication (k=1)

Inter-rater agreement on single subscales of CRS-R, and on diagnostic classification based on CRS-R, expressed as Fleiss generalized Kappa statistic (and 95% Confidence Interval)

Subscale

All Raters (N =27)

Auditory

0.76 (0.66 – 0.83)

Visual

0.73 (0.63 – 0.81)

Motor

0.76 (0.66 – 0.83)

Oromotor

0.73 (0.63 – 0.81)

Communication

1

Arousal

0.72 (0.63 – 0.80)

Diagnostic
Classification

0.79 (0.71 – 0.84)

 

  • Italian version of the CRS-R demonstrated high sensitivity (indicating proportion of subjects diagnosed as VS by CRS-R: 91.9%, 95% CI: 82.4-96.5)
  • Also demonstrated high specificity (indicating proportion of subjects diagnosed as not VS by CRS-R: 86.7%, 95% CI: 82.4-96.5)

 

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

Interrater Reliability of Dichotomized CRS-R Subscale Scores (n=20)

Subscale

Cohen kappa

95% CI

P

Rater Agreement

Auditory

.86

±.27

.01

95%

Visual

.58

±.33

.03

80%

Motor

.78

±.28

.01

90%

Oromotor/Verbal

.77

±.42

.03

95%

Communication

.88

±.24

.00

95%

 

Han, 2018; n = 39; Mean Age = 56.9 (16.9); Time Post-injury = 125.6 (128.8) days; traumatic and non-traumatic brain injury; Korean sample

  • Inter-rater reliability (k=0.929, p<0.01) and intra-rater reliability (k=0.938, p<0.01) were both high for total K-CRSR scores
  • Inter- and intra-rater agreement rates were very high (94.9% and 97.4%, respectively)
  • Inter-rater reliability of the total score for K-CRSR and its subscales was very high (Ƙ=0.93, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.90, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.90, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.86, p<0.01)
  • Agreement between the scores of rater B on day 1 and the scores of rater A on day 3 was very high (Ƙ=0.908, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.91, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.85, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.90, p<0.01)
  • intra-rater reliability of the total score for K-CRSR and its subscales was very high (Ƙ=0.94, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.95, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.95, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.91, p<0.01)
  • Reliability was in perfect agreement with the score for the communication subscale

 

 

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • Inter-rater reliability for the CRS-R total score (κ=0.99, p<0.001) and subscale scores was good

Subscales

Cohen’s kappa

95% Confidence Interval

 

p for Cohen’s kappa

Total Score

0.99

0.876

1.0

<0.001

Auditory

0.934

0.848

1.0

<0.001

Visual

0.873

0.763

0.957

<0.001

Motor

0.979

0.919

1,0

<0.001

Oromotor

0.961

0.854

1,0

<0.001

Communication

1.0

1.0

1.0

<0.001

Arousal

1.0

1.0

1.0

<0.001

La Porta, 2013; n = 129; Mean Age = 47 (20); Mean Time Post-injury = 421 (599) days; traumatic and non-traumatic brain injury; Italian sample

  • Summary table of CRS-R reliability studies through 2011

 

Summary of reliability study results of the CRS-R, adapted from La Porta (2013)

Giacino and Kalmar (2004)        Schnakers et  al (2008)              Lovstad et al (2010)               Simoes et al (2011)

 

 

Sacco et al (2011)

 

 

Study design and setting

 

No. of centers

1

5

6

1

1

Assessment setting

R

 

 

A, R, NH

 

 

R, NH

 

 

A

 

 

R

 

Sample size

80

 

 

77

 

 

31

 

 

20

 

 

38

 

No. of raters

2

 

 

24

 

 

8

 

 

2

 

 

2

 

Sample size/rater ratio

40

 

 

3.2

 

 

3.8

 

 

10

 

 

19

 

Reliability

IRR

TRT

ICR

IRR

TRT

ICR

IRR

TRT

ICR

IRR

TRT

ICR

IRR

TRT                    ICR                 

Auditory subscale

K=.86

K = 0.63  t

 

K= .82*

 

 

K=.90

K= . 71*

 

ICC=0.99

ICC= .86

 

Kw = . 6St

Kw = 0.80*

Visual subscale

K= .58 t

K=0.90

 

K=.85

 

 

K= .46t

K=.86

 

ICC=l.00

ICC= .88

 

Kw = . 71*

Kw = 0.84*

Motor subscale

K= . 78*

K= 1.00

 

K=.93

 

 

K= .67t

K= .73*

 

ICC=0.98

ICC=.81*

 

Kw = . 79*

Kw = 0.96

Oromotor subscale

K= .77*

K= 0.23t

 

K=.92

 

 

K=.89

K= . 71*

 

ICC=0.96

ICC= .82*

 

Kw = .44t

Kw = Q.8 5

Communication subscale

K=.88

K=0.89

 

K=.98

 

 

K= .62t

K=.89

 

ICC=0.97

ICC= .82*

 

Kw = .88

Kw = 0.88

Vigilance subscale

NA

NA

 

K= . 74*

 

 

NA

NA

 

ICC=0.98

ICC= .84*

 

Kw = .51t

Kw  = l. QQ

Total score

p= .84 *

p=0.94

a= .84*

K= .8 0*

NA

NA

K=.94

NA

a= .74*

ICC=0.99

ICC= .87

NA

p= .81 *

p=0.97                 a= .81 *

NOTE. Where several reliability values were available, we reported only the largest ones. Comparison of the various studies may be difficult in view of the fact that the classical psychometric properties reported are strictly sample-dependent and several reliability coefficients were used across different studies.

Abbreviations: a, Cronbach alpha; A, acute setting; CRS-R, CRS-R total score; ICC, intraclass correlation coefficient; ICR, internal consistency reliability; IRR, interrater reliability; K, Cohen Kappa; Kw, weighted Kappa; NA, not applicable; p, Spearman correlation coefficient; R, rehabilitation setting; TRT, test-retest reliability.

* Values compatible only with measurement at the group level (≥ .70 <.85), not at the individual level (≥ .85 ).

t Values not sufficiently reliable for any measurement (< . 70), includinq measurement at the qroup level.

Lovstad, 2010; n = 31; Mean Age = 33 (15); Median Time Post-injury = 143, range = 21–2130 days; traumatic and non-traumatic brain injury; Norwegian sample

  • Good interrater agreement between the experienced

rater (ρ =0.77)

  • Interrater correlations within the newly trained rater group were not significant (ρ = 0.15 and P = .09)
  • Interrater agreement was not affected by whether a rater had prior exposure (ρ = 0.68, P = .01, n = 16) or no prior exposure (ρ = 0.67, P = .03, n = 12)

 

 

Portaccio, 2018a; n = 110; Mean Age = 58.7 (16.2); Time Post-injury = 2.1 (2.1) months; traumatic and non-traumatic brain injury; Italian sample

  • Across 6 raters, Kappa coefficient for total scores yielded a rate of agreement of 0.827

 

Schnakers, 2008b; n = 77; traumatic brain injury; French-speaking sample

  • Inter-rater reliability for the CRS-R total score was good (k= 0.80).
  • Mean kappa values:
    • good to excellent for the auditory (k= 0.82),
    • visual (k= 0.85), motor (k= 0.93),
    • oromotor (k= 0.92),
    • communication (k=0.98) and
    • arousal (k=0.74) sub-scales

 

Slomine, 2019; n=33; Age range: 8-59 months; typically developing children

  • Inter-rater reliability of the Coma Recovery Scale for Pediatrics (CRS-P) subscale scores was adequate (Kw = .87–1.00)

 

Tamashiro, 2014; n = 35; severe acquired brain injury; Spanish-speaking sample

  • Inter-rater reliability was good for total CRS-R scores (Cronbach α = 0.973, p = 0.001).
  • Sub-scale analysis showed moderate-to-high inter-rater agreement

 

Zhang, 2010; n = 169; Median Age = 58, range = 18–86; Time Post-injury: range = 70-99; traumatic and non-traumatic brain injury; Chinese sample

  • Inter-rater reliability was high (ICC = 0.719; p < 0.01)

Internal Consistency

Binder, 2018; n = 20; Median Age = 38.0 (14.39); Mean Time Post-TBI = 1.5 (0.5) days; severe TBI; Polish sample

  • Internal consistency was excellent (Cronbach’s α = 0.85)

 

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

  • Relationship between CRS-R total score and the individual subscale scores was investigated using Cronbach α. Resulted in an α value of .83, indicating that CRS-R has excellent internal consistency

 

CRS-R Subscale Intercorrelations (N=80)

Subscale

Auditory

Visual

Motor

Oromotor/

Verbal

Communication

Arousal

Auditory

1.00

 

 

 

 

 

Visual

0.70

1.00

 

 

 

 

Motor

0.60

0.60

1.00

 

 

 

Oromotor/ Verbal

0.51

0.29

0.47

1.00

 

 

Communication

0.62

0.54

0.49

0.65

1.00

 

Arousal

0.50

0.43

0.31

0.44

0.63

1.00

 

Han, 2018; n = 39; Mean Age = 56.9 (16.9); Mean Time Post-injury = 125.6 (128.8) days; traumatic and non-traumatic brain injury; Korean sample

  • Cronbach’s alpha showed a high degree of internal consistency. Specifically, it was 0.9 for rater A on day 1, 0.9 for rater B and 0.9 for rater A on day 3

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • Internal consistency of the total score using Cronbach’s alpha was high 0.87 (p=0.0001). More precisely, α=0.87 and 0.89 for the first and second visit respectively

 

La Porta, 2013; n = 129; Mean Age = 47 (20); Mean Time Post-injury = 421 (599) days; traumatic and non-traumatic brain injury; Italian sample

  • Rasch Analysis demonstrated excellent internal consistency (Cronbach’s alpha= .83, as per Giacino et al., 2004)

 

Lovstad, 2010; n = 31; Mean Age = 33 (15); Median Time Post-injury = 143, range = 21–2130 days; traumatic and non-traumatic brain injury; Norwegian sample

  • Good to Excellent: Cronbach’s α for each of the 5 assessments ranged from 0.68 to 0.85, with a mean of 0.74, and corresponding 95% CI [0.6–0.9]

Zhang, 2010; n = 169; Median Age = 58, range = 18–86; Time Post-injury: range = 70-99; traumatic and non-traumatic brain injury; Chinese sample

  • Internal consistency for the total score was excellent (Cronbach's α = 0.84)

Criterion Validity (Predictive/Concurrent)

Concurrent

Casarotto, 2016; n = 150; Age range = 18–80 years; Median Time Post-TBI = 4 (1-7) days; traumatic or non-traumatic brain injury; Italian sample)

  • The Perturbation Complexity Index, derived by analyzing Transcranial Magnetic Resonance- evoked EEG signal has a sensitivity of 94.7% for detecting MCS patients on the CRS-R.

 

Di Perri, 2016; n = 58; Age Range = 11-82; Time Post-injury = All more than 28 days; traumatic or non-traumatic brain injury; French-speaking sample

  • Positive functional magnetic resonance imaging (fMRI) default mode network connectivity correlated with CRS-R scores

 

Estraneo, 2015; n = 122; Mean Age = 54.58, range = 14-88; Time Post-injury = 25 - >365 days; traumatic or non-traumatic brain injury; Italian population

  • Italian version of CRS-R shows high sensitivity (indicating proportion of subjects designated as VS by the CRS-R: 91.9%, 95% CI: 82.4-96.5) and high specificity (indicating proportion of subjects designated as not VS by CRS-R: 86.7%, 95% CI: 82.4-96.5) with reference to clinical consensus diagnosis
  • Good concurrent validity between CRS-R and DRS. Spearman correlation coefficients between CRS-R and DRS significant for total sample (p < 0.001)

 

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

  • CRS-R total scores correlated significantly with total scores on the CRS and DRS indicating acceptable concurrent validity
  • Spearman coefficients were significant between the CRS-R and the CRS (ρ=.97, P<.001) and between the CRS-R and DRS (ρ=−.90, P<.001)

 

Han, 2018; n = 39; Mean Age = 56.9 (16.9); Time Post-injury = 125.6 (128.8) days; traumatic and non-traumatic brain injury; Korean sample

  • Total score for the Korean CRSR (K-CRSR) was strongly correlated with the total score for K-GCS, and the Spearman correlation coefficient was significant (r=0.894, p<0.01)

 

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • CRS-R showed good criterion validity between two other standardized behavioral scales (adequate correlation with GCS, r=0.597 and excellent correlation with FOUR Score, r=0.900)
  • CRS-R demonstrated a significantly higher sensitivity in differential diagnosis of DOC, as compared to GCS, and FOUR Score (p<0.001)
  • All selected CRS-R subscales showed significant (p<0.05) very high, high or moderate correlation with similar items on the other scales, except for visual function on the GCS (r= 0.257; p<0.05), where the correlation was significant but poor
  • Chi-squared analysis showed that the proportion of patients diagnosed with MCS by the CRS-R was significantly higher as compared to the GCS (χ2 =25; p<0.0001)

 

Schnakers, 2008b; n = 77; traumatic brain injury; French-speaking sample

  • Excellent concurrent validity between CRS-R and GCS, FOUR and WHIM in acute and between CRS-R and FOUR and WHIM in chronic patients.
  • The French version of the CRS-R demonstrated significantly higher sensitivity in detecting MCS patients, as compared to the GCS, FOUR & the WHIM

CSR-R

GCS

FOUR

WHIM

Both Stages

.59

.63

.76

Acute Stage

.72

.61

.68

Chronic Stage

.46

.69

.87

Lovstad, 2010; n = 31; Mean Age = 33 (15); Median Time Post-injury = 143, range = 21–2130 days; traumatic and non-traumatic brain injury; Norwegian sample

  • CRS-R sensitivity, which indicates the proportion of persons diagnosed as VS on the DRS that were also classified as VS on the CRS-R, was 0.54- 0 0.62.
  • CRS-R Specificity, which indicates the proportion of persons classified as not being in VS by the CRS-R or by the DRS, was 0.89-1.0 for the same experienced raters (see Table 5). The high specificity values indicate that those classified as in an MCS by the DRS are also categorized as MCS by the CRS-R. The rather low sensitivity values reflect that the CRS-R detected signs of consciousness (ie, MCS) while the DRS failed to do so.

 

Stender, 2014; n = 126; traumatic and non-traumatic brain injury; French-speaking sample

  • F-FDG PET had high sensitivity for identification of patients in a minimally conscious state (93%, 95% CI 85–98) and high congruence (85%, 77–90) with behavioral CRS–R scores

 

Tan, 2019; n = 22; Mean Age = 52, range = 16–72; Time Post-injury: range = 1.5-8 months; traumatic and non-traumatic brain injury; Chinese sample

  • Structural MRI findings correlate with CRS-R total score and DoC diagnosis

 

Tamashiro, 2014; n = 35; severe acquired brain injury; Spanish-speaking sample

  • Total CRS-R scores correlated significantly (p < 0.05) with total GCS (r = 0.74) and DRS (r = 0.54) scores, indicating acceptable concurrent validity

 

Zhang, 2010; n = 169; Median Age = 58, range = 18–86; Time Post-injury: range = 70-99; traumatic and non-traumatic brain injury; Chinese sample

  • Concurrent validity was good between CRS-R total scale and GCS total scale
  • Diagnostic validity was excellent compared with GCS (emerged from UWS: 24%; emerged from MCS: 28%)

 

Predictive

Coleman, 2007; n = 14; Age Range = 22-67; Time Post-injury = 2-122 months; traumatic or non-traumatic brain injury

  • Patients who responded to complex language stimuli during functional magnetic resonance imaging (fMRI) had higher CRS-R scores at 6-month follow-up

 

Giacino 2019; n= 97;   Mean Age = 37.2 (15.4);  Median time post injury = 47 days (37–65); traumatic brain injury

  • The odds of recovering a specific target behavior were 3.2 (95% CI: 1.2–8.1) to 7.8 (95% CI: 2.7–23.0) times higher for patients in MCS than for those in VS. Patients with preserved language function (MCS+) recovered the most behaviors (p ≤ 0.002) and had the least disability (p ≤ 0.002) at follow-up. 

 

Hamilton, 2018; n = 70; Age Range = 19-69; Time Post-injury = 135.33 (76.02) days; traumatic and non-traumatic brain injury)

  • At week three of inpatient rehabilitation, the CRS-R scores of greater than or equal to 9 predicted emergence from MCS in an average of 72.9% of veterans

 

Lucca, 2019; n = 180; Mean Age = 51.43 (range: 33.7-72.3); Mean Time Post-injury = 6.8 (2.0) weeks; traumatic and non-traumatic brain injury; Italian sample

  • CRS-R diagnosis, CRS-R total scores and etiology were significantly associated with emergence from MCS

 

Portaccio, 2018a and 2018b; n = 110; Mean Age = 58.7 (16.2); Time Post-injury = 2.1 (2.1) months; traumatic and non-traumatic brain injury; Italian sample

  • Patients who transitioned from VS to MCS/eMCS or MCS to eMCS by rehabilitation discharge had higher score on the CRS-R at admission (9.7 ± 6.0 vs 7.5 ± 4.8, P < .047)
  • A higher CRS-R change score by week 4 of inpatient rehabilitation predicted likelihood of transition from VS to MCS/eMCS or MCS to eMCS

Construct Validity

Gerrard, 2014; n = 180; Mean Age = 36.6 (15.4); Time Post-injury = 4-16 weeks; TBI patients)

  • CRS-R shows evidence of construct validity and empirical support for the theoretical hierarchy of behaviors assessed within each subscale

 

Variety of Diagnoses affecting consciousness: (Schnakers et al., 2009, n=103 prospective patients with variety of diagnoses affecting consciousness, 55 (19) years, Belgian sample. Consensus medical diagnoses for vegetative, minimally conscious or “uncertain” were compared with diagnostic categories derived from CRS-R testing. ) 

Of 44 patients diagnosed in VS, 18 were in MCS; of 41 in MCS, 4 had emerged from MCS, and majority of “uncertain” diagnoses were in MCS (89%). The use of the CRS-R standardizes assessment and identified misclassified patients who have greater levels of consciousness than recognized by medical consensus.

Content Validity

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury

 

  • Content validity assessment established a strong relationship between the CRS-R and the GCS, and moderate correlation between the CRS-R and the FOUR
  • Met all criteria of the Aspen workgroup definitions for MCS (Giacino 2002)

 

Thibaut, 2019; n=120; Mean Age = 47 ± 19 years;  time post-injury: 40 ± 23 days; traumatic  and non-traumatic injury

  • MCS patients with no behavioral evidence of language function (i.e., MCS-) were more functionally impaired than patients with behavioral evidence of language function (MCS+) at time of transition from VS and at discharge from inpatients rehabilitation.
  • Provides diagnostic criteria for MCS+ and MCS-

Floor/Ceiling Effects

The measure was designed to assess patients at Rancho Levels of Cognitive Functioning I-IV, so patients who are beyond these levels are not appropriate for its use.

 

Giacino, 2004; n = 80; Mean Age = 38.86 (13.18) [validity study], 36.70 (12.40) [reliability study]; Time Post-injury = 58.43 (30.80) [validity  study], 57.15 (26.90) [reliability study]; traumatic and non-traumatic brain injury)

  • Majority of CRS-R total scores fell in the middle range, demonstrating that the scale is not subject to ceiling or floor effects

 

La Porta, 2013; n = 129; Mean Age = 47 (20); Mean Time Post-injury = 421 (599) days; traumatic and non-traumatic brain injury; Italian sample

  • Rasch analysis revealed negligible floor and ceiling effects

 

Slomine, 2019; n=33; Age range: 8-59 months; typically developing children

  • All 4-year-olds, 75% of 3-year-olds, 10% of 2-year-olds, and 0% <2 years scored at the CRS-P ceiling.

Responsiveness

Giacino, 2012; n=; Mean Age = 35.5 (15); Median Time post- injury = 48 (380=-66); traumatic brain injury

  • patients who received amantadine had a higher rate of recovery across the six behaviors at the highest end of each CRS-R subscale.
  • after washout, behavioral recovery remained more favorable in the amantadine group across five of the six behaviors monitored.

 

Iazeva, 2018; n = 58; Median Age = 46 (18); Mean Time Post-injury = 2.5 (1) months; traumatic and non-traumatic brain injury; Russian sample

  • Median CRS-R total score after the first visit was 8.5 [5.0; 14.75] and after the second visit was 10.0 [5.0; 17.75], р<0.0001. In 11 patients (18,3%), this change was associated with a change in clinical diagnosis, suggesting sufficient sensitivity in capturing changes in diagnosis

 

Schiff, 2009; n=1; Age = 38; Time post injury = 6 years; traumatic brain injury

  • In a case-study, a patient showed the first instances of functional object use on the Motor subscale (score = 6) and intelligible verbalization on the Oromotor subscale (score = 3) after deep brain stimulation to the central thalamus. No previous episodes of intelligible verbalization had been observed during a series of 33 evaluations conducted across the 6 months of observation preceding the impact

Bibliography

Binder, M., Górsk, U., Wójcik-Krzemień, A., & Gociewicz, K. (2018). A validation of the Polish version of the Coma Recovery Scale-Revised (CRSR). Brain Injury, 32(2), 242-246. doi: 10.1080/02699052.2017.1406991.

Bodien, Y.G., Carlowicz, C.A., Chatelle, C., & Giacino, J.T. (2016). Sensitivyt and specificity of the Coma Recovery Scale- Revised Total Score in Detection of Conscious Awareness. Archives of Physical Medicine and Rehabilitation, 97(3), 490-492.e1. doi:  https://doi.org/10.1016/j.apmr.2015.08.422

Casarotto S, Comanducci A, Rosanova M, Sarasso S, Fecchio M, Napolitani M, Pigorini A, GCasali A, Trimarchi PD, Boly M, Gosseries O, Bodart O, Curto F, Landi C, Mariotti M, DevalleG, Laureys S, Tononi G6, Massimini M. Stratification of unresponsive patients by an independently validated index of brain complexity. Ann Neurol. 2016; 80(5):718-729. doi:10.1002/ana.24779.

Chatelle, C., Bodien, Y.G., Carlowicz, C., Wannez, S., Charland-Verville, V., Gosseries,O., Laureys, S., Seel, R.T., Giacino, J.T. (2016) Detection and Interpretation of Impossible and Improbable Coma Recovery Scale-Revised Scores. Archives Physical Medicine Rehabilitation, 97(8), 1295-1300.e4. doi: 10.1016/j.apmr.2016.02.009.

Coleman MR, Rodd JM, Davis MH, Johnsrude IS, Menon DK, Pickard JD, Owen AM. Do vegetative patients retain aspects of language comprehension? Evidence from fMRI. Brain 2007;130:2494-2507.

Cortese, M.D., Riganello, F., Arcuri, F., Pugliese, M., Lucca, L., Dolce, G., Sannita, W.G. (2015). Coma Recovery Scale-R: variability in the disorder of consciousness. BMC Neurology, 8, 15:186. doi: 10.1186/s12883-015-0455-5.

Di Perri C, Bahri MA, Amico E, Thibaut A, Heine L, Antonopoulos G, Charland-Verville V,Wannez S, Gomez F, Hustinx R, Tshibanda L, Demertzi A, Soddu A, Laureys S. Neural correlates of consciousness in patients who have emerged from a minimally conscious state: across-sectional multimodal imaging study. Lancet Neurol. 2016;15(8):830-842. doi:10.1016/S1474-4422(16)00111-3.

Estraneo, A., Moretta, P., De Tanti, A., et al. (2015). An Italian multicentre validation study of the coma recovery scale-revised. Eur J Phys Rehabil Med, 51(5):627–634.

Gerrard, P., Zafonte, R., Giacino, J.T. (2014). Coma Recovery Scale-Revised: evidentiary support for hierarchical grading of level of consciousness. Archives Physical Medicine Rehabilitation, 95(12), 2335-41. doi: 10.1016/j.apmr.2014.06.018. Epub 2014 Jul 7.

Giacino, J.T., & Kalmar, K. (1997). The Vegetative and Minimally Conscious States: A Comparison of Clinical Features and Functional Outcome. Journal of Head Trauma Rehabilitation, 12(4), 36–51. doi: 10.1097/00001199-199708000-00005.

Giacino, JT., Ashwal, S., Childs, N., Cranford, R., Jennett, B., Katz, DI., Kelly, JP., Rosenberg, JH., Whyte, J., Zafonte, RD., Zasler, ND., & Aspen Neurobehavioral Conference Workgroup. The minimally conscious state: Definition and diagnostic criteria. Neurology, 2002; 58(3), 349–353.https://doi.org/10.1212/WNL.58.3.349.

Giacino, J.T., Kalmar, K., & Whyte, J. (2004) The JFK Coma Recovery Scale-Revised: Measurement characteristics and diagnostic utility. Archives of Physical Medicine and Rehabilitation, 85(12), 2020-2029. DOI: https://doi.org/10.1016/j.apmr.2004.02.033.

Giacino, J.T., Katz, D.I., Schiff, N.D, Whyte, J., Ashman, E.J., Ashwal, S., Barbano, R., Hammond, F.M., Laureys, S., Ling, G.S.F., Nakase-Richardson, R., Seel, R.T., Yablon, S., Getchius, T.S.D., Gronseth, G.S., & Armstrong, M.J. (2018). Practice guideline update recommendations summary: Disorders of consciousness: Report of the Guideline Development, Dissemination, and Implementation Subcommittee of the American Academy of Neurology; the American Congress of Rehabilitation Medicine; and the National Institute on Disability, Independent Living, and Rehabilitation Research. Neurology, 91(10), 450-460. doi: 10.1212/WNL.0000000000005926.

Giacino JT, Sherer M, Christoforou A, Maurer-Karattup P, Hammond FM, Long D, Bagiella E. Behavioral Recovery and Early Decision Making in Patients with Prolonged Disturbance in Consciousness after Traumatic Brain Injury. Journal Neurotrauma. 2019. doi: 10.1089/neu.2019.6429.

Godbolt, A. K., Stenson, S., et al. (2012). "Disorders of consciousness: Preliminary data supports added value of extended behavioural assessment." Brain Injury 26(2): 188-193. 

Hamilton JA, Perrin PB, Campbell TA, Danish SJ, Goldstein AL. Predicting emergence from a disorder of consciousness using the Coma Recovery Scale-Revised. Neuropsychol Rehabil.2018; 16:1-15. doi: 10.1080/09602011.2018.1461656.

Han, H.J., Kim, E.J., Lee, H.J., Pyun, S.B., Joa, K.L., & Jung, H.Y. (2018). Validation of Korean Version of Coma Recovery Scale-Revised (K-CRSR). Annals Rehabilitation Medicine, 42(4), 536–541. doi: 10.5535/arm.2018.42.4.536

Iazeva, E.G., Legostaeva, L.A., Zimin, A.A., Sergeev, D.V., Domashenko, M.A., Samorukov, V.Y., Yusupova, D.G., Ryabinkina, J.V., Suponeva, N.A., Piradov, M.A., Bodien ,Y.G., & Giacino, J.T.(2018).  A Russian validation study of the Coma Recovery Scale-Revised (CRS-R). Brain Injury, 2, 1-8. doi: 10.1080/02699052.2018.1539248.

La Porta, F., Caselli, S., Ianes, A.B., Cameli, O., Lino, M., Piperno, R., Sighinolfi, A., Lombardi, F., & Tennant, A. (2013). Can we scientifically and reliably measure the level of consciousness in vegetative and minimally conscious States? Rasch analysis of the coma recovery scale-revised. Archives of Physical Medicine and Rehabilitation, 94, 527-535.

Lovstad, M., Froslie, K., Giacino, J., Skandsen, T., Anke, A., & Schanke, A. (2010). Reliability and Diagnostic Characteristics of the JFK Coma Recovery Scale–Revised: exploring the influence of rater's level of experience. Journal of Head Trauma Rehabilitation, 349-356.

Lucca, L.F., Lofaro, D., Pignolo, L., Leto, E., Ursino, M., Cortese, D., Conforti, D., Tonin, P., & Cerasa, A. (2019). Outcome prediction in Disorders of Consciousness : The role of Coma Recovery Scale Revised. BMC Neurology, 19 (68). PMID: 30999877.

Maurer-Karattup P, Giacino J, & Luther M, Eifert B. Diagnosis of disorders of consciousness with the German version of Coma Recovery Scale-Revised (CRS-R). Neurologie und Rehabilitation. 2010;16. 232-246.

Portaccio, E., Morrocchesi, A., Romoli, A.M., Hakiki, B., Taglioli, M.P., Lippi, E., Di Renzone, M., Grippo, A., Macchi, C. (2018a) Intensive Rehabilitation Unit Study Group of the IRCCS Don Gnocchi Foundation, Italy. Improvement on the Coma Recovery Scale-Revised During the First Four Weeks of Hospital Stay Predicts Outcome at Discharge in Intensive Rehabilitation After Severe Brain Injury. Archives of Physical Medicine and Rehabilitation, 99(5), 914-919. doi: 10.1016/j.apmr.2018.01.015.

Portaccio E, Morrocchesi A, Romoli AM, Hakiki B, Taglioli MP, Lippi E, Di Renzone M, Grippo A, Macchi C. Score on Coma Recovery Scale-Revised at admission predicts outcome at discharge in intensive rehabilitation after severe brain injury. Brain Inj. 2018b;32(6):730-734. doi: 10.1080/02699052.2018.1440420.

Sacco S, Altobelli E, Pistarini C, Cerone D, Cazzulani B, Carolei A. Validation of the Italian version of the Coma Recovery Scale-Revised (CRS-R). Brain Inj 2011;25:488-95

Schiff ND, Giacino JT, Kalmar K, Victor JD, Baker K, Gerber M, Fritz B, Eisenberg B, O’Connor JO, Kobylarz EJ, Farris S, Machado A, McCagg C, Plum F, Fins JJ, Rezai A. Behavioral improvements with thalamic stimulation after severe traumatic brain injury. Nature 2007;448(Aug 2):600-604.

Schnakers C, Majerus S, Giacino J, Vanhaudenhuyes A, Bruno MA, Boly M, Moonen G, Damas P, Lambermont P, Lamy M, Damas F, Ventura M, Laureys S. A French validation study of the Coma Recovery Scale- Revised. Brain Injury, 2008b;22(10):786-792.

Schnakers C, Vanhaudenhuyse A, Giacino J, Ventura M, Boly M, Majerus S, Moonen G, Laureys S. Diagnostic accuracy of the vegetative and minimally conscious state: Clinical consensus versus standardized neurobehavioral assessment. BMC Neurol 2009;9:35.

Seel RT, Sherer M, Whyte J, Katz DI, Giacino JT, Rosenbaum AM, Hammond FM, Kalmar K, Pape TL, Zafonte R, Biester RC, Kaelin D, Kean J, Zasler N. American Congress of Rehabilitation Medicine, Brain Injury-Interdisciplinary Special Interest Group, Disorders of Consciousness Task Force. Assessment scales for disorders of consciousness: evidence-based recommendations for clinical practice and research. Arch Phys Med Rehabil. 2010 Dec;91(12):1795-813. doi: 10.1016/j.apmr.2010.07.218.

Simões JF, Jesus LM, Voegeli D, Sá-Couto P, Fernandes J, Morgado M.  Assessment of comatose patients: a Portuguese instrument based on the Coma Recovery Scale - revised and using nursing standard terminology. J Adv Nurs. 2011 May;67(5):1129-41. doi: 10.1111/j.1365-2648.2010.05559.

Slomine BS, Suskauer SJ, Nicholson R, Giacino JT. Preliminary validation of the coma recovery scale for pediatrics in typically developing young children. Brain Inj. 2019 Aug 28:1-6. doi: 10.1080/02699052.2019.1658221.

Stender J, Gosseries O, Bruno MA, Charland-Verville V, Vanhaudenhuyse A, Demertzi A, Chatelle C, Thonnard M, Thibaut A, Heine L, Soddu A, Boly M, Schnakers C, Gjedde A4, Laureys S. 1. Diagnostic precision of PET imaging and functional MRI in disorders of consciousness: a clinical validation study. Lancet. 2014;384(9942):514-22. doi: 10.1016/S0140-6736(14)60042-8. Epub 2014 Apr 15.

Tamashiro, M., Rivas, M.E., Ron, M., Salierno, F., Dalera, M., & Olmos, L. (2014). A Spanish validation of the Coma Recovery Scale-Revised (CRS-R). Brain Injury, 28(13-14), 1744-7. doi: 10.3109/02699052.2014.947621.

Tan X, Zhou Z, Gao J, Meng F, Yu Y, Zhang J, He F, Wei R, Wang J, Peng G, Zhang X, Pan G, Luo B. Structural connectome alterations in patients with disorders of consciousness revealed by 7-tesla magnetic resonance imaging. Neuroimage Clin. 2019;22:101702. doi: 10.1016/j.nicl.2019.101702.

Thibaut A, Bodien YG, Laureys S, Giacino JT. Minimally conscious state "plus": diagnostic criteria and relation to functional recovery. J Neurol. 2019 Nov 26. doi: 10.1007/s00415-019-09628-y.

Wannez S, Heine L, Thonnard M, Gosseries O, Laureys S; Coma Science Group collaborators. The repetition of behavioral assessments in diagnosis of disorders of consciousness. Ann Neurol. 2017;81(6):883-889. doi: 10.1002/ana.24962.

Wilde EA, Whiteneck GG, Bogner J, Bushnik T, Cifu DX, Dikmen S, French L, Giacino JT, Hart T, Malec JF, Millis SR, Novack TA, Sherer M, Tulsky DS, Vanderploeg RD, von Steinbuechel N. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. 2010;91(11):1650-1660.e17. doi: 10.1016/j.apmr.2010.06.033.

Zhang, Y., Wang, J., Schnakers, C., He, M., Luo, H., Cheng, L., Wang, F., Nie, Y., Huang, W., Hu, X., Laureys, S., & Di, H. (2019). Validation of the Chinese version of the Coma Recovery Scale-Revised (CRS-R). Brain Injury, 33(4), 529-533. doi: 10.1080/02699052.2019.1566832.