Informativity assessment of websites of territorial attestation commissions on the attestation procedure for healthcare specialists

INTRODUCTION

The quality of information systems determines the efficiency of organizations. This requires permanent improvement of such systems [1], including the quality of information content [2].

The website content is information intended for the target audience to meet their requirements within a specific field of activity [3]. The study [4] of web queries showed that information search accounts for 80% of all web operations. At the same time, the main deficiency of information content is known to be the overabundance of irrelevant and insignificant information [5]. A serious difficulty faced by developers of information content consists in recognizing the usefulness of the material and selecting targeted information [6].

The quality of information content is acquiring particular importance in the context of the growing amount of online information, leading to information overload and social media fatigue [7][8]. Information content that motivates Internet users to take certain actions or influences their behavioral intentions is the “golden fleece” of any organization [9][10]. Convincing information content is capable of changing the users’ attitudes and encouraging them to interact with the offeror [11][12].

Due to the current decline in medical professionals’ motivation and interest in the qualification institution, their unwillingness to engage in additional work-related activities, fear of reducing the amount of free time, lack of understanding of all certification aspects and processes, as well as the desire to avoid complex procedures, the information content of the websites of certification organization should be as useful and convincing as possible [13][14].

The Order of the Ministry of Health of the Russian Federation (No. 458n dated August 31, 2023) “On Approval of the Procedure and Terms for Medical and Pharmaceutical Professionals to Obtain a Qualification Grade”¹ requires governmental agencies and organizations that have established attestation commissions to post information about these commissions, visiting hours, and document submission order, as well as the composition of expert groups, on their official websites.

In this study, we set out to assess the quality of information content on the websites of the healthcare authorities of the constituent entities of the Russian Federation (RF subjects) and the respective territorial certification commissions regarding the assignment of qualification categories.

MATERIALS AND METHODS

An information search about the procedure for certifying medical professionals on the websites of the healthcare executive authorities of 83 RF subjects and on the websites and territorial attestation commissions was carried out.

We evaluated the information availability on the websites of territorial certification commissions, as specified in Table 1. Information from paragraphs 34.1–34.8, 37, and 39 describes the rules for submitting documents for certification, while paragraphs 35 and 36 describe the procedure for submitting them. The information availability from the Order on the website² was evaluated positively (“yes”) provided that it met all the requirements of the relevant clause in the regulatory document.

A survey was conducted in absentia among 47 medical professionals with at least two years of relevant professional experience, who were enrolled in the Postgraduate Medical Education Academy. The survey focused on 47 websites identified during the initial assessment as containing information about the rules and procedures for submitting documents for certification. Each respondent reviewed the certification information posted on the websites of three different RF subjects. Thus, the information on each website was evaluated independently by three respondents. The corresponding website links were sent to the respondents via email. The distribution of websites among the respondents was carried out in a random manner using the Randomus online random number generator (https://randomus.ru).

The number of respondents exceeded the number required for pilot studies, with a significance level of p = 0.05 [15]. Among the respondents, there were 25 (53.2%) men and 22 (46.8%) women. The average age of the respondents was 32.3 ± 4.94 years; the difference in age between men (33.7 ± 5.32 years) and women (30.8 ± 3.94 years) was significant (t = 3.69; p = 0.011). Among 47 respondents, 24 (51.1%) were surgeons, including dentists, and 23 (48.9%) were general practitioners, including dentists. There was no difference in the age of respondents with surgical (32.5 ± 5.36 years) and therapeutic (32.1 ± 4.47 years) specialties (t = 0.49; p = 0.388). Among the respondents, 25 (53.2%) worked in medical organizations providing outpatient care, and 22 (46.8%) worked in hospitals. There was no difference in age between the respondents of these two groups: 31.8 ± 4.44 years and 32.9 ± 5.42, respectively (t = 1.39; p = 0.316). The respondents’ distribution by selected characteristics is presented in Table 2.

The survey was conducted using a questionnaire developed by the Department of Economics and Marketing at the Postgraduate Medical Education Academy. The questionnaire contained two questions about the sufficiency and clarity of the website information about the rules and procedures for submitting documents for certification, as follows:

1. How complete is the information provided to answer your questions about the rules and procedures for submitting documents for certification?
2. To what extent is the information presented clear to you, and does it require additional explanation or search for information on other resources?

To formalize the respondents’ opinions on the sufficiency and clarity of the information about the rules and procedures for submitting documents for certification, a five-point rating scale was used, where “1” represents an extreme lack of information and “5” represents an absolute sufficiency and clarity of the presented information.

Statistical processing was performed using the SPSS software (IBM Company). The availability of information about the list of certification specialties, the rules and procedures for submitting documents, and sample tests was evaluated using frequency analysis. The analysis of respondents’ opinions about the sufficiency and clarity of information about the rules and procedures for submitting documents for certification was conducted using frequency analysis and the calculation of average values (mean, standard deviation, and median). Since the variables “sufficiency” and “clarity” did not have a normal distribution (p ≤ 0.001), the Mann–Whitney and Kruskal–Wallis criteria and Spearman’s rank correlation coefficient were used.

RESULTS

The conducted search revealed that in 17 RF subjects, the territorial attestation commissions have separate websites, including those without a link on the websites of the authority organ. The analysis showed that most websites of territorial attestation commissions contain the basic information required by the Order³ (Table 3). However, only 47 websites (56.6%) contained information on all Order paragraphs regarding the rules and procedures for submitting documents. Only in 14 (16.9%) RF subjects, the websites of healthcare executive authorities and territorial attestation commissions simultaneously provide information on the rules and procedures for submitting documents, the list of attestation specialties, attestation test samples, and reference to the current Order of the Ministry of Health of the Russian Federation (No. 458n dated August 31, 2023)⁴, as well as information on the current dates of meetings of attestation/expert commissions.

Out of 18 websites that featured sample tests, 5 (6.1%) websites provided a link to the Central Commission for the Certification of Medical Workers for the Assignment of Qualification Categories. Only 21 (25.3%) websites had an active link to the current order of the Ministry of Health of the Russian Federation, and 13 (15.7%) websites mentioned that the procedure for medical and pharmaceutical workers to pass certification for obtaining a qualification category was regulated by the current order. 34 (41.0%) websites did not contain information about the current regulatory legal act, and 3 websites contained an invalid order issued by the Ministry of Health of the Russian Federation on April 23, 2013, No. 240N.

According to the survey results, the average score given by the respondents for the sufficiency of information about certification of medical professionals on 47 websites was 3.13 ± 1.04 (median 3.0). Concerning the content clarity, the score was 2.98 ± 1.02 (median 3.0).

Regarding the sufficiency of information, the respondents gave the highest score of 4.33 to 5 (10.6%) websites, and the lowest score of 1.33 to 4 (8.5%) websites. According to the respondents, 2 (4.3%) websites received the highest score of 4.68 for the clarity of information, while 1 (2.1%) website received the lowest score of 1.0.

According to the respondents, the clarity of information about the certification of medical professionals is worse than its sufficiency (p = 0.009). The opinions of the respondents about the sufficiency and clarity of information on different websites differ significantly: 102.155 ≤ χ² ≤ 110.978 (p ≤ 0.001). When comparing the opinions of respondents on the same websites, all respondents gave the same rating for the sufficiency (p = 0.881) and clarity (p = 0.976) of the information presented.

The respondents’ opinions did not depend on their age when evaluating the websites based on information sufficiency (p = 0.416) and content clarity (p = 0.706); no statistically significant differences were found during the correlation analysis.

The opinions of respondents of different genders about the sufficiency (p = 0.163) and clarity (p = 0.148) of the information were identical. The opinions of respondents of surgical and therapeutic specialties about the sufficiency (p = 0.901) and clarity (0.947) of the content were also identical. The opinions of respondents working in hospitals and medical organizations providing outpatient care about the sufficiency (p = 0.956) and clarity (0.983) of the websites were also identical.

DISCUSSION

According to the results obtained, out of 83 RF subjects, only 14 (16.9%) have the required information about the rules and procedures for submitting documents, the list of attestation specialties, attestation test samples, and active links to the current Order of the Ministry of Health of the Russian Federation (No. 458n dated August 31, 2023), as well as information about the current dates of the attestation/expert commission meetings.

Information on all Order paragraphs regarding the rules and procedure for submitting documents is presented on 47 (56.6%) websites. When evaluating these websites, the respondents considered the sufficiency of information on the rules and procedure for submitting documents to be satisfactory, while the content clarity was significantly lower than satisfactory. Thus, even at the stage of preparing for attestation for a qualification grade, the applicant will still have questions about the rules and procedure for submitting documents, requiring additional explanations and/or search for information on other resources.

It should be noted that the consistency of the respondents’ opinions about the sufficiency and clarity of information on the same websites indicates their impartiality. The lack of correlation between the respondents’ opinions and their age, gender, specialty, or the type of medical facility eliminates the influence of sociodemographic factors. Given that the average age of the respondents was 32 years, we cannot assume that they lack competencies in using online resources.

The discrepancy between the respondents’ assessments of the sufficiency and clarity of information on different websites indirectly reflects the attitude of the attestation commissions towards the quality of the information content on their websites. In our opinion, the poor presentation, delayed updates, low clarity, and insufficient information about certification procedures demonstrate the indifference of some regional healthcare executive authorities towards the institution of medical qualification.

Given that the number of respondents corresponds to that required for a pilot study, the opinion on the sufficiency and clarity of the information about certification of medical professionals needs to be confirmed by surveying a larger number of specialists.

The survey was conducted by medical professionals who have good skills of online information search, which may distort the results in favor of higher ratings. Despite the survey being conducted remotely, it is possible that the respondents who were currently studying at the same organization might discuss the subject of the survey. Additionally, there may be a bias towards positive or negative attitudes towards the institution of qualification categories.

CONCLUSION

The information about the attestation procedure of medical professionals, which is available on the websites of the healthcare executive authorities of the Russian Federation subjects and on the websites of the respective territorial attestation commissions, needs to be corrected and updated.