Validation and reproducibility assessment of 10-word memory sets in the Russian version of SCAT-5: A cross-sectional single-center study

INTRODUCTION

Concussion is the most common traumatic brain injury sustained in sports. While the expert community lacks a unified consensus on the incidence of such injuries, recent epidemiological studies [1–3] have identified team sports as having the highest risk of concussion. Furthermore, a systematic review and meta-analysis by Pfister et al. confirms that the primary risk factor is the high probability of physical contact between athletes [4]. Despite its prevalence, concussion is a clinical diagnosis, as it lacks specific brain matter changes on computed tomography or magnetic resonance imaging. In high-performance sports, diagnosing concussions can be complicated by athletes concealing symptoms from coaching staff and medical personnel, a lack of awareness, and the high-pressure environment where sports physicians have limited decision-making time. To address these challenges, a special clinical tool was developed in 2004 — the Sport Concussion Assessment Tool (SCAT) protocol¹ [5].

The protocol comprises several diagnostic components, including a cognitive function test designed to assess immediate memory and delayed recall. For this purpose, a standardized list of unrelated words is used. Noteworthy is that the original SCAT protocol was developed and published in English. A number of foreign studies focused on its translation and cultural adaptation note challenges stemming from the ethnic and sociocultural nuances in its application [6][7]. In addition to linguistic features, it is essential to consider that while the SCAT-5 protocol offers a choice between a 5-word or a 10-word list, the SCAT-6 protocol provides only a 10-word list. This transition aims to eliminate the saturation effect (ceiling phenomenon) during memory assessment [8]. This phenomenon occurs when a task is too simple, resulting in near-perfect scores for most subjects and failing to distinguish individual differences, which can potentially distort the outcome of concussion evaluation [9][10]. To ensure objectivity during repeated assessments, the protocol provides three distinct word lists of comparable difficulty to avoid the saturation effect.

Currently, there are no Russian-language studies devoted to the development and validation of word lists for memory assessment within the SCAT protocol. No experimental data are available to confirm that Russian-language word lists are equivalent in difficulty or provide sufficient reproducibility for monitoring cognitive dynamics in athletes, particularly following a diagnosed concussion.

Successive versions of the original English-language SCAT protocol have introduced specific refinements and updates. Starting with SCAT-2, standardized word lists were used to assess immediate memory; however, in the fifth and sixth versions, each list was expanded to 10 words. Notably, the SCAT-5 and SCAT-6 use identical word sets and word order.

The present study aims to evaluate the comparability and equivalence of the three proposed Russian-language 10-word lists for memory assessment in athletes within the SCAT-5 protocol.

MATERIALS AND METHODS

A clinical examination was conducted on 446 athletes (373 males, 73 females; median age 37 [ 26; 43] years), including both active and retired competitors. Assessments were performed between 2022 and 2025 using the SCAT-5 protocol² [11]. The study included representatives of various sports: motorsports (n = 326, 73.1%), rugby (n = 72, 16.1%), handball (n = 25, 5.6%), Muay Thai (n = 16, 3.6%), boxing (n = 5, 1.1%), hockey (n = 1, 0.3%), and yoga (n = 1, 0.2%). All active athletes underwent a baseline (pre-competition) evaluation, with no abnormalities detected, while retired athletes underwent an isolated SCAT-5 protocol examination.

The athletes were randomly assigned to one of three groups based on the specific word list used for immediate memory and delayed recall assessment. Group 1 was evaluated using List G (n = 177, median age 37 [ 26; 43] years); Group 2, using List H (n = 129, median age 35 [ 24; 43] years); Group 3, using List I (n = 140, median age 38 [ 28; 43] years). No significant differences in the level of education were observed between the groups.

The SCAT-5 protocol for off-field or medical office evaluation includes a comprehensive assessment of the athlete’s clinical history, symptoms, and cognitive function (including orientation, short-term memory, and concentration). It also encompasses a neurological screen (including coordination) and a delayed recall assessment to evaluate long-term memory.

For symptom assessment, the athletes were required to independently complete the SCAT-5 symptom checklist. It includes headache, “pressure in the head,” neck pain, nausea or vomiting, dizziness, blurred vision, balance problems, sensitivity to light and loud sounds, feeling slowed down, feeling like “in a fog,” feeling “not right,” difficulty concentrating and remembering, fatigue, lack of energy, confusion, drowsiness, increased emotionality, irritability, depression, nervousness or anxiety, and sleep problems. Each symptom was rated based on its current severity: 0 — none, 1–2 — mild, 3–4 — moderate, 5–6 — severe. The evaluation yielded the total number of symptoms (maximum of 22 points) and the symptom severity score (maximum of 132 points).

Immediate memory was assessed over three consecutive trials. In each trial, the athlete attempted to recall one of three read-out word lists; scoring ranged from 0 to 10 points (one point for each correctly recalled word). Delayed recall (long-term memory) was evaluated five minutes after the completion of the immediate memory test. The athletes were asked to recall as many words as possible from the same list, with scoring performed on the same scale.

Additionally, integrated scores were calculated for both memory phases. Immediate memory was represented by the median of the sum of scores across all three recall trials, while the delayed recall score was defined as the median of the five-minute delayed memory test results for each participant in each group.

Three Russian-language word lists were used for memory assessment; these word sets had been adapted by specialists from the National Center of Sports Medicine in accordance with the international SCAT-5 guidelines^3,4. The lists are as follows: List G (Forest-Tooth-Country-Grain-Fragrance-Puppy-Lantern-Taxi-Bazaar), List H (Game-Ship-Bear-Air-Coffee-Month-Skin-Step-Carpet-Brother), and List I (Boy-Winter-Notebook-Belt-Bath-Cloud-Glass-Candle-Breakfast-Cat). The adaptation process and word selection for the Russian-language version of the SCAT-5 protocol were also guided by [12]. Notably, these word lists are not merely direct translations; they include monosyllabic Russian words, whereas the English-language lists consist only of disyllabic words.

Statistical analysis was performed using IBM SPSS Statistics 23.0. The Lilliefors-corrected Kolmogorov–Smirnov test was applied to assess the distribution of quantitative parameters. Since the data followed a non-normal distribution, quantitative parameters are presented as medians and quartiles, while qualitative data are expressed as relative frequencies with 95% confidence intervals. Intergroup comparisons were conducted using the Kruskal–Wallis test, followed by pairwise comparisons via the Bonferroni-corrected Mann–Whitney U test. Discrete variables were compared using the χ² test. Relationships between quantitative variables were assessed using Spearman’s rank correlation coefficient. Statistical significance was set at p < 0.05.

RESULTS

Comparative analysis of immediate memory scores in athletes

Figure 1 illustrates the comparative analysis of word recall performance across three different groups of athletes during the first trial. Memorization and correct recall of words at serial positions 2, 3, 5, and 8 did not differ statistically significantly across Groups 1, 2, and 3 (p > 0.05). However, several position-specific variations were observed. Group 2 exhibited poorer recall of the first word but superior memorization of the seventh word compared to Groups 1 and 3 (both p < 0.001). The fourth word was recalled more frequently by Group 3 than by Groups 1 and 2 (p < 0.001). Noteworthy is that Group 1 athletes demonstrated better recall of the sixth word (vs. Group 2, p < 0.001; vs. Group 3 p = 0.004), but statistically significantly poorer performance for the tenth compared to the other groups (p = 0.006; p < 0.001). Finally, the ninth word was best recalled by Group 2 and worst by Group 3 (p < 0.001).

Analysis of the second immediate memory trial revealed no statistically significant differences in the recall of words at serial positions 2, 3, 5, 7, 8, and 9 (p > 0.05) (Fig. 2). However, Group 2 demonstrated significantly poorer recall of the first word compared to Groups 1 and 3 (both p = 0.002). The fourth word was recalled better by athletes from Group 3 than by Groups 1 and 2 (p < 0.001 and p = 0.008, respectively). Conversely, the sixth word was recalled statistically significantly worse by Group 2 than by the other two groups (vs. Group 1, p < 0.001; vs. Group 3, p = 0.031). Finally, the tenth word was best recalled by Group 2 and worst recalled by Group 1 (p = 0.005).

Analysis of the third immediate memory trial showed no statistically significant differences between the groups in the recall of words at serial positions 2, 3, 5, 7, 8, 9, and 10 (p > 0.05) (Fig. 3). Noteworthy is that Group 2 demonstrated significantly poorer recall of the first and sixth words compared to Groups 1 and 3 (p < 0.001; p = 0.011; p < 0.001; p = 0.003). The fourth word was recalled statistically significantly better by Group 3 than by Groups 1 and 2 (both p < 0.001).

Comparative analysis revealed that the number of successfully recalled words increased with each successive trial. However, the first and sixth words from List H were recalled less frequently by Group 2 compared to Groups 1 and 3, even by the third trial. The fourth word was recalled better by Group 3 than by the athletes in the other groups, who recalled words of the same serial position from other lists, regardless of the trial sequence. Figure 4 presents the integrated scores for three immediate recall trials across the study groups.

Intergroup analysis showed no statistically significant differences between integrated immediate memory scores (Groups 1–2: p = 0.728; Groups 1–3: p = 0.024; Groups 2–3: p = 0.052 for ). Notably, the ceiling effect (the maximum possible score over three trials) was observed in only one athlete from Group 1 (0.2%).

Comparative analysis of delayed recall performance across athlete groups

Analysis of delayed recall performance revealed no statistically significant differences in the memorization and recall of words at serial positions 2, 4, 5, 8, and 9, regardless of the group in which the SCAT-5 protocol examination was conducted (p > 0.05) (Fig. 5). However, several position-specific variations persisted. Group 1 athletes recalled the sixth word statistically significantly more often than those in Groups 2 and 3 (both p < 0.001), while Group 3 demonstrated a higher recall frequency for the seventh word compared to the other groups (both p < 0.001). Group 1 showed significantly poorer performance for the tenth word compared to Groups 2 and 3, who recalled words with the same serial position (p < 0.001 and p < 0.001, respectively). The first word was recalled best by Group 3 and worst by those in Group 2 (p = 0.010 and p = 0.003, respectively). Finally, the third word was recalled statistically significantly better by Group 2 and worse by Group 3 (p = 0.119 and p = 0.010, respectively).

Across all word lists, delayed recall performance followed a consistent pattern: participants demonstrated high recall for the first five words well, followed by a decline for words six through nine, and a slight improvement for the last (tenth) word. The median delayed recall scores were 5 [ 4; 7], 5 [ 4; 6.5], and 5 [ 4; 7] for Groups 1, 2, and 3, respectively. Intergroup analysis confirmed no statistically significant differences between the integrated delayed recall scores (Groups 1–2: p = 0.934; Groups 1–3: p = 0.29 ; Groups 2–3: p = 0.378).

Correlation between memory performance and symptom severity

An analysis was conducted to evaluate the relationship between the immediate memory performance in athletes and the total number of symptoms (Fig. 6A), their severity (Fig. 6B), and delayed recall scores (Fig. 6C).

Immediate memory performance correlated strongly with delayed recall scores (r_s = 0.625, p < 0.001). However, no statistically significant correlations were found between the immediate memory and the number of identified symptoms or symptom severity (r_s = –0.011, p = 0.824; r_s = –0.015, p = 0.747).

A strong positive correlation between immediate and delayed recall performance was observed across all athlete groups (Group 1: r_s = 0.629, p < 0.001; Group 2: r_s = 0.663, p < 0.001; Group 3: r_s = 0.591, p < 0.0). These data are presented in Figure 7.

While immediate memory performance showed a positive correlation with delayed recall across all study groups, no statistically significant associations were observed between immediate memory performance and the number of identified symptoms or symptom severity.

DISCUSSION

According to published data, motorsports account for 6.3% to 23% of all medical consultations for concussion across all sports [13]. This indicates the increased risk of concussion in motorsports and justifies the monitoring of athletes using the SCAT protocol, while also confirming the representativeness of the study sample [14]. The present sample size (n = 446) is comparable to foreign studies evaluating baseline SCAT performance [15][16]. A key strength of this study is the detailed analysis of both immediate memory and delayed recall in athletes, conducted through a word-by-word comparison of different word lists.

Diagnosing concussion in sports remains challenging due to the subjectivity of athletes’ complaints, the diversity of clinical manifestations, rapid symptom development, the absence of structural changes on neuroimaging, and the lack of a reliable brain injury bio­marker [17–22]. In this connection, the medical community has developed and implemented clinical assessment protocols, with the SCAT currently being the most relevant and widely recognized tool [23]. A significant advantage of the SCAT protocol is its integrated diagnostic approach: mBESS (modified Balance Error Scoring System) and SAC (Standardized Assessment of Concussion). Linguistic adaptation challenges primarily affect the SAC component, which is designed to assess cognitive function. In this study, we used a more complex 10-word version of the immediate memory test rather than the five-word list. This approach minimizes the ceiling (saturation) effect, which has been repeatedly confirmed in international literature [15][16][24–28].

Athletes across all groups demonstrated a progressive increase in recalled words with each successive immediate memory trial. The absence of a statistically significant difference in the integrated scores suggests that the selected Russian-language word lists are of equivalent difficulty, aligning with findings reported by other authors [15][16][29].

Noteworthy is that words at serial positions 1, 2, 3, 6, and 10 were more successfully recalled by the athletes across all trials, regardless of the list used. This may be related to the patterns of human memory function [30][31]. Previous studies have repeatedly mentioned positional effects (serial position effects), where items at the beginning and end of a list are better retained. However, the successful recall of the sixth (central position) demonstrates a departure from this concept [32][33]. Furthermore, the fourth word from List I, used in Group 3, shows a pronounced tendency to be more easily memorized by athletes, which may be related to nuances in the cultural adaptation of the original SCAT-5 protocol [6][34]. Nevertheless, given that the integrated memory scores were comparable to international data, as well as the absence of statistically significant intergroup differences in immediate memory assessment, the impact of cultural adaptations on memory assessment results appears to be mitigated [15][16][29][35][36].

In addition to immediate memory performance, the comparability of different word lists was also observed during delayed recall assessment. In the present study, no statistically significant differences were found between the groups regarding the total number of successfully recalled words. Furthermore, the delayed recall scores are consistent with findings reported by other researchers [16][37][38].

A notable finding of this study is the absence of a correlation between the athletes’ memory performance and the number and severity of identified symptoms.

For instance, McLeod et al. [28] note that an athlete’s complaints may be influenced not only by the acute phase of a concussion but also by various anamnestic factors (e.g., overtraining, life stresses, hormonal changes, sleep disturbances, and mental health issues). Therefore, the context of SCAT examination must be considered. Iverson et al. [39] reached similar conclusions regarding the influence of situational factors, finding a positive correlation only between the total number of symptoms and the concussion severity, while no such associations were found between other SCAT-5 components. Notably, the study by Iverson et al. was conducted during the pre-season period when none of the athletes were recovering from a head injury.

Despite the use of different word sets to assess memory function, we found no statistically significant differences between groups in the integrated immediate memory and delayed recall scores. Furthermore, a detailed analysis of word reproducibility confirmed that most corresponding serial positions were comparable across groups. The statistically significant differences observed for individual words may become the subject of future research aimed at optimizing the used word lists.

Several limitations of this study should be noted. First, the cohort was dominated by a single discipline (motorsports). Second, subjectivity is inherent in any clinical examination, as both athletes and physicians may introduce bias during the assessment of symptoms and cognitive functions. Third, the single-center design limited the inclusion of a larger number of athletes. Finally, the marked gender imbalance (predominance of male athletes) may affect the validity of the studied word lists when applied to female athletes.

Future research should focus on expanding the study sample size. It is also essential to identify possible associations between specific diseases, conditions, injuries, and individual performance metrics of the protocol. Furthermore, incorporating objective diagnostic tools, such as neuroimaging, will also be necessary to confirm the obtained results.

CONCLUSION

Any of the three proposed word lists may be used for assessing immediate memory and delayed recall within the SCAT-5 protocol in examined athletes. This will ensure the necessary methodological validity comparable to the original English version in obtaining baseline test scores of athletes during pre-season preparation, periodic monitoring, acute injury evaluation, and throughout the post-concussion recovery period.

The use of a 10-word list eliminates the ceiling effect and enhances the diagnostic significance of the SCAT-5 protocol. Any of the proposed word lists can be used in more advanced versions of the protocol, such as the SCAT-6, where a 10-word series is used for memory assessment.

The memory performance of athletes is independent of the number of symptoms or symptom severity, which facilitates the diagnosis of concussion using the SCAT protocol. Nevertheless, the potential for linguistic refinement of individual words should be considered when translating the SCAT-6 into Russian.

Any of the three word lists evaluated in this study is interchangeable, as evidenced by the lack of statistically significant differences in the integrated immediate memory scores, delayed recall scores, or in the detailed analysis of word-by-word reproducibility between groups. Therefore, each word list has the same validity.

Authors’ contributions. All authors confirm that their contributions meet the ICMJE criteria for authorship. The primary contributions are as follows: Vladimir S. Feshchenko — conceptualization, research supervision and verification, manuscript editing and verification, final manuscript approval; Boris A. Polyaev — project administration, manuscript editing and verification; Anton V. Slivin — data processing, formal analysis, visualization, manuscript drafting; Mikhail I. Polyakov — data collection and processing, manuscript drafting, investigation and methodology; Igor T. Vykhodets — critical analysis, formulation of the general goals and study objectives.