Prospects for applying eye tracking in functional state assessment of pilots

INTRODUCTION

According to research [1][2], measures aimed at preserving pilot health and predicting its dynamics are essential elements of modern aviation flight safety. In this regard, functional state assessment of pilots and their performance during professional activity have become routine and mandatory tasks of aviation physicians.

Health-assessment activities are conducted not only directly during flight operations but also on aviation simulators and when conducting psychophysiological training of pilots for the effects of various flight factors [3–5]. The dynamics of the functional state of a pilot’s body is monitored during application of special research methods, medical examination of flight personnel by medical boards [6], training of cadets at flight training organization [7], as well as when simulating the influence of extreme aviation flight factors in ground-based conditions [7][8].

At present, there exist a substantial number of methods for assessing the functional state and professional performance of pilots. Russian researchers have developed both monoparametric approaches, where physiological, psychophysiological, and psychological indicators of an individual are recorded and comprehensively evaluated, as well as multiparametric solutions. For example, a large body of work is dedicated to assessing heart rate regulation as an integral indicator of a pilot’s functional state, with the possibilities of mathematical and spectral analysis of heart rate variability being described in detail [8][9]. Research into the potential of computerized stabilometry for evaluating vestibular function as a statokinetic functional system in humans is underway [10]. In addition, numerous other techniques and approaches have been studied and developed with the purpose of proposing integrated, valid, and highly convenient practical solutions for aviation physicians to monitor the functional state of pilots.

Eye tracking is a highly promising technology for assessing the functional state of pilots based on oculomotor activity. This technology has been studied for several decades [11], finding wide application in medicine, e.g., in neurophysiology, neurosurgery, psychiatry, and ophthalmology [12][13], as well as in a number of paramedical fields [14]. It should be noted that the capabilities of eye tracking in aviation medicine have already been addressed; however, these works primarily focused on the issues of information perception and attention distribution, which are key for aviation ergonomics. For instance, assessment of the functional state and performance of civil aviation pilots [15][16] and cadets during flights on aviation simulators was undertaken.¹ A distinctive feature of this work was the evaluation of a pilot’s oculomotor activity during a flight on an aviation simulator. According to prominent Russian researchers, human oculomotor activity is directly linked to the process of solving cognitive tasks, with the efficiency of their solution being strongly affected by various internal and external factors [12]. Thus, eye tracking holds considerable potential for assessing the impact of various factors associated with the professional activity of pilots on their functional state and performance.

Considering the above, the psychophysiological foundations for using oculomotor activity indicators in diagnosing the functional state of pilots during their professional activities are of great interest. In an attempt to address this problem, we conducted a study to assess the functional state of pilots during ground-based training on aviation simulators. The aim was to evaluate the possibility of using eye tracking for assessing the functional state of pilots.

MATERIALS AND METHODS

The study involved 22 male pilots from Southeast Asia aged 24–47 years (mean age 28.8 ± 2 years). At the time of examination, the participants had no acute conditions or exacerbations of chronic diseases, including those affecting the visual and nervous systems. All pilots have had practical experience in real flights and simulator training with simulated emergency situations in flights on Yak-130 aircrafts.

During the study, the pilots underwent training on Yak-130 aviation simulators. Each pilot performed one flight according to a predetermined route and flight parameters. During the flight, the flight director simulated the impact of an external stress factor by creating an abnormal emergency situation, such as failures of control surfaces and systems, engine failures, deployment of the drag chute in flight, etc. Only one abnormal situation was simulated per flight. In the emergency situation, the pilot had to perform a series of actions according to the algorithm and complete the flight. Upon completion of the task, the flight director evaluated the pilot’s actions on a 10-point scale in accordance with the protocol established by the governing documents.

The participants were divided into two groups based on the results of the simulator flight (Yak-130, Russia).

Group 1 included 10 pilots with an average score of 7.3 within the range of 7–8 points. The mean age in this group was 25.5 ± 2 years, and the mean total flight time was 261 h. Group 2 consisted of 12 pilots who received an average score of 8.2 within the range of 8–9 points. Their mean age was 31.5 ± 2 years, and their flight time was 603 h. Thus, the formed groups differed in terms of not only simulator training results but also significant differences in age and professional skills. The latter factor was likely to affect the former factor.

In order to assess the dynamics of the functional state of pilots under the influence of a stress factor, they were examined 15 min before performing the simulator flight and within 15 min immediately after completing the flight. The functional state of pilots during the pre-flight period and after the simulator flight was diagnosed according to a unified plan. This plan involved assessment of heart rate regulation (HRR) and the functional level of the central nervous system (CNS), as well as recording oculomotor activity in the pilots while performing a cognitive task.

HRR was assessed through the analysis of statistical indicators of the cardiorhythmogram: mode (Mo, ms), reflecting the functional level of the sinus node; amplitude of mode (AMo, %), indicating the activity of the sympathetic nervous system; variation range (VR, ms), associated with the activity of the parasympathetic division of the autonomic nervous system; stress index of regulatory systems (SI, rel. units), which reflects the activity of sympathetic rhythm regulation. Spectral indicators included power of high-frequency oscillations (HF, ms²), reflecting parasympathetic influence on the rhythm; low-frequency oscillations (LF, ms²), associated with sympathetic nervous system activity; very low-frequency oscillations (VLF, ms²), associated with humoral influence on heart rhythm; the LF/HF ratio (rel. units), indicating the balance of sympathetic and parasympathetic nervous system influences on the rhythm; total power of the spectrum (TP, ms²), reflecting the total autonomic influence on heart rhythm; and the integral indicator of regulatory system activity (PARS, rel. units) [17], recorded over 5 min using the VNS-Ritm equipment (Russia).

The CNS state was investigated using the NS-Psikhotest equipment (Russia) by recording simple and complex sensorimotor reactions (SSR and CSR), reaction to a moving object (RMO), critical flicker fusion frequency (CFFF). The stability and concentration of attention were assessed by comparing the results obtained by the Noise Immunity and Attention Assessment methods. During the Schulte–Platonov Black-Red Tables test, attention capacity (AC), distribution (AD), and switching (AS) were determined on a five-point scale. Here, attention capacity was assessed across the ranges: 0–29, 30–37, 38–50, 51–60, 61 and above points. Attention distribution was assessed as follows: 0–43, 44–56, 57–87, 88–106, 107 and above points. Attention switching was determined across the ranges: 0–9, 10–17, 18–31, 32–40, 41 and above points.²

Recording of oculomotor activity was conducted using an eye tracker (SMI RED250, Germany) while the pilot performing a cognitive task in the form of the Schulte–Platonov Black-and-Red Tables test. Analysis of the recorded data was performed using the standard SMI BeGaze software for this equipment. Quantitative indicators of fixations and saccades (number and frequency) were analyzed; temporal parameters included duration of fixations and saccades (short — < 200 ms, medium — 200–350 ms, and long — > 350 ms fixations); saccadic velocity characteristics incldued peak velocity, average velocity, and amplitude [12].

Statistical analysis was performed using the Microsoft® Excel-2016 software, the SPSS 26 application software package, with calculation of the Wilcoxon test. The values of all indicators are presented as the median (M_e) with the upper and lower quartiles [Q₁; Q₃], corresponding to the 25^th and 75^th percentiles of the distribution. The indicators recorded before and after the simulator flight were compared. All differences were considered statistically significant at a significance level of p < 0.05. Correlation analysis was performed using Spearman’s rank correlation coefficient, with the indicators being considered statistically significant at a level of p < 0.05.

The pilots were examined in the morning in a quiet, well-ventilated separate room at an air temperature of +22°C. All participants provided voluntary consent to participate in the study, which was conducted in accordance with biomedical ethics standards (Declaration of Helsinki and European Community directives, 8/609 EC).

RESULTS

When investigating oculomotor activity (quantitative and temporal characteristics of fixations) in pilots before and after simulating an emergency situation on an aviation simulator, no statistically significant changes were detected in either group, except for a 9% decrease (p < 0.05) in the number of fixations in Group 2. The corresponding data are presented in Table 1.

The results of assessing gaze movement characteristics — quantitative, temporal, and velocity parameters of saccades — are reflected in Table 2. Statistically significant changes were registered in Group 1. These were presented as a decrease in saccadic velocity parameters: a 9% decrease in peak saccadic velocity (p < 0.05), a 6% decrease in average velocity (p < 0.05), and a 17% decrease in saccadic amplitude (p < 0.05), while quantitative and temporal characteristics showed no significant changes. In Group 2, significant changes were noted only in an 8% decrease in the number of saccades (p < 0.05).

In order to assess the dynamics of the functional state of pilots during simulation of an in-flight emergency and to provide a physiological rationale for the nature of changes in oculomotor activity indicators, an assessment of HRR was conducted.

In Group 1, the following statistically significant changes were recorded: an 18% increase in the VR indicator (from 193 to 227 ms), a 12% increase in TP (from 3756 to 4194 ms²), a 46% increase in VLF (from 1200 to 1780 ms²), a 25% increase in the PARS indicator (from 3.0 to 3.75 points), and a 26% decrease in HF (from 1629 to 1250 ms²), and a 16% decrease in SI (from 155 to 131 rel. units) (Fig. 1).

In Group 2, statistically significant changes were also noted: an increase in AMo by 32% and SI by 67% (from 293 to 488 rel. units); a decrease in TP by 33% (from 2602 to 1749 ms²), LF by 39% (from 1153 to 699 ms²), and the LF/HF ratio by 30% (from 1.93 to 1.35 rel. units) (Fig. 2).

The impact of the pilot’s actions in an emergency situation during simulated flight conditions on an aviation simulator on their functional state was assessed by changes in the functioning level of the CNS. To that end, the rate of visual-motor reactions, the mobility of nervous processes in the visual analyzer, the balance of excitation and inhibition processes, as well as attention characteristics were recorded.

In Group 1, pilots exhibited significant changes in a number of indicators: a 37% decrease in the absolute negative value of RMO time (from –12.7 to –8.0 ms), a 50% increase in attention distribution indicators (from 71.9 to 108.0 points), and a 94% increase in attention switching indicators (from 39.4 to 76.3 points) compared to baseline values (p < 0.05). Other psychophysiological indicators, including attention stability and concentration, showed no statistically significant changes (Fig. 3).

When assessing sensorimotor reactions in Group 2, the pilots also showed significant changes: a 2.2-fold increase in the absolute negative value of RMO from –6.1 to –13.0 ms (p < 0.05) and a 2.6-fold increase in attention switching from 6.8 to 17.7 points (p < 0.05). Other psychophysiological indicators, similar to those in Group 1, showed no statistically significant changes (Fig. 4).

In order to identify statistical relationships between oculomotor activity, heart rate regulation characteristics, and the CNS functioning level during cognitive task performance, a correlation analysis was performed. The data obtained before, after the start, and immediately upon the completion of the simulation flight were investigated. The most significant results of the correlation analysis are presented in Table 3.

Tight inverse correlations between velocity indicators of oculomotor activity and attention capacity and distribution were established, along with tight direct correlations with the indicator of regulatory system activity and the balance ratio of low- to high-frequency oscillation power.

DISCUSSION

The analysis of gaze stability based on fixation characteristics, which revealed no statistically significant changes, indicates the absence of a target for these oculography parameters in the present study. The number and duration of fixations largely reflect the visual analyzer state when solving complex cognitive tasks or under challenging conditions for their execution [12–14]. In the conducted study, cognitive tasks were solved before and after simulating an emergency situation, rather than during the performance of the flight task on the simulator, when the pilot is exposed to the maximum spectrum of professional stress factors. Consequently, various parameters of gaze fixation did not show significant changes.

At the same time, the assessment of gaze movement results revealed a significant decrease in the velocity parameters of saccades in pilots with lower performance results on the simulator (Group 1). According to a number of authors, saccades exert a certain reciprocal influence on cognitive function, and a decrease in saccadic velocity is observed in neurophysiological disorders [12]. In the presented study, the presence of diseases or pathological conditions in the pilots was excluded at the stage of forming the subject group. Therefore, the described features of gaze movement may be due to changes in the functional state of Group 1 pilots under the influence of stress factors during the simulation of an emergency situation, which inevitably exerts autonomic influence on the functioning of all human analyzers, including the visual ones. The most likely functional state in Group 1 with lower simulator flight results after the flights is fatigue. This state, although being characterized as normal, is accompanied by the onset of a decline in human performance and the functional level of physiological systems.

The analysis of HRR as an integral indicator of the functional state of pilot health allowed us to draw several conclusions. Pilots in Group 1 showed pronounced tension in the regulatory systems with a tendency towards strain of the adaptation mechanisms. In a study by Baevsky et al., strain was characterized by somewhat discordant changes in HRR indicators [8]. On the one hand, Group 1 showed a decrease in the index of tension of regulatory systems and an increase in the variation range, reflecting increased parasympathetic influence. On the other hand, there was an increase in total spectral power, indicating an overall reduction in autonomic influence and centralization of rhythm control. However, this reduction is primarily due to a decrease in parasympathetic influence, which somewhat contradicts the data from variational pulse oximetry. The increase in the PARS indicator reflects a shift in the tension of regulatory systems into the area of pronounced tension [8][17].

From a physiological standpoint, concordant changes in heart rate variability parameters and saccadic velocity indicators can be observed, reflecting a decrease in cognitive activity and likely fatigue of the visual analyzer.

The assessment of HRR in Group 2 before and after simulating emergency situations on a simulator indicated an increase in the centralization of rhythm control and overall autonomic influence thereon. This is confirmed by a significant increase in SI, sympathetic influence (AMo), and a decrease in total spectral power. The decrease in TP is due to a statistically significant reduction in sympathetic influence; however, a trend toward a synchronous decrease in parasympathetic and humoral influence can be observed [8].

The above HRR results, considering the absence of significant changes in fixation and saccade parameters during oculography, allow us to conclude that pilots in Group 2 developed a normal functional state with moderate load on adaptation mechanisms and body regulatory systems, while maintaining professional performance when performing simulation flights.

The results obtained when studying the CNS functioning level and attention characteristics in pilots before and after simulating emergency situations on a simulator demonstrated different baseline values in the two groups. In Group 1, the pilots (with lower simulator flight results) exhibited a significant predominance of excitation over inhibition processes, unlike the pilots in Group 2. The same situation was observed when considering attention characteristics. Group 1 registered indicators corresponding to low mobility of nervous processes, while in Group 2 its higher mobility was noted. The attention characteristics fully confirm the difference in the baseline functional state of the pilots, as described by the reaction to a moving object. These differences can be explained by the varying level of professional skills of the pilots in the two groups and their level of psychological readiness to act in conditions of emergency and abnormal flight situations.

The dynamics of these parameters, recorded immediately after the flight in Group 1, reflects an even greater decrease in the functioning level of the CNS compared to baseline. This can be interpreted as strain of the body’s adaptation mechanisms [17]. This fully corresponds to the results obtained from oculography and the analysis of saccadic velocity indicators in Group 1, which indicate a decrease in cognitive activity and, most likely, the development of fatigue.

In Group 2, after flights on a simulator, an increased predominance of excitation processes in the CNS was noted, which can be characterized as the mobilization of psychophysiological reserves. The absence of significant changes in attention characteristics, except for a slight decrease in switching capacity (from level 5 to 4), cannot indicate a significant reduction in the mobility of nervous processes.

These results, considering the absence of significant changes in saccadic velocity indicators and gaze fixation characteristics in Group 2 pilots, indicate the preservation of their normal functional state while maintaining their professional performance, which is ensured by moderate tension in the regulatory systems [17].

The conducted correlation analysis of oculomotor activity parameters with HRR indicators and CNS functioning level indicated a close statistical relationship of the velocity indicators of oculomotor activity with the PARS integral heart rate variability indicator. An increase in its values reflects tension in the body’s regulatory mechanisms. Inverse correlations were established with attention properties, such as capacity and distribution, which reflects a connection between the activation of oculomotor activity and the complexity of cognitive tasks, as well as the mobilization of psychophysiological reserves. To a lesser extent, saccadic velocity parameters were associated with indicators of the spectral analysis of rhythmograms and the CNS functioning level.

CONCLUSIONS

The results obtained when assessing the oculomotor activity of pilots during cognitive task performance under simulated emergency conditions on an aviation simulator allow us to draw the following conclusions:

1. The analysis of oculomotor activity in conjunction with heart rate regulation parameters, CNS functioning level, and attention characteristics indicated the development of fatigue in pilots with lower simulator flight results. At the same time, pilots from Group 2 with a higher level of psychophysiological reserves demonstrated preservation of a normal functional state. Oculomotor activity indicators show statistically significant changes against the background of functional state dynamics during professional activity.
2. The dynamics of velocity indicators of oculomotor activity in pilots, such as saccadic velocity and amplitude, are co-directional with changes in heart rate regulation. This reflects a certain degree of load on the body’s regulatory systems and the mobilization of physiological reserves. This is demonstrated by the dynamics of the index of regulatory systems tension, the indicator of regulatory system activity, as well as the spectral parameters of the cardiorhythmogram.
3. Changes in oculomotor activity indicators are co-directional with the dynamics of attention capacity and distribution, the balance and mobility of nervous processes. This reflects the functioning level of the central nervous system.

Velocity indicators of oculomotor activity have a tight, statistically significant correlation with the PARS integral heart rate regulation indicator, as well as with attention capacity and distribution. This indicates their dependence on the body’s overall functional state. Analysis of oculomotor activity indicators allows the functional state of pilots to be assessed. In order to fully implement the potential of eye tracking technologies for assessment of the functional state of pilots, it seems advisable to evaluate the dynamics of oculomotor activity indicators during the direct impact of professional activity factors, as well as during analytical mental work.

Authors’ contributions. All the authors confirm that they meet the ICMJE criteria for authorship. The most significant contributions were as follows: Oleg A. Annenkov — materials collection, manuscript writing; Andrey А. Blaginin — literature review; Evgeniy V. Ivchenko — study design; Dmitry V. Ovchinnikov — data interpretation, draft writing; Semyon P. Lyashedko — statistical analysis.