Introduction

Limited financial resources, poor infrastructure and, above all, staff shortages paired with increased demand and growing awareness of patients are the key problems that the healthcare sector in Poland is currently faced with. Determining the correlation between the competencies of healthcare professionals and patient care effectiveness in primary healthcare may be of significance to the management process of healthcare organisations and it may trigger the adoption of necessary actions aimed at improving the effectiveness of patient care [1].

The essence of management is, among others, the productive management of staff knowledge and competence. The correlation between medical staff competencies and patient care effectiveness in primary care is a highly complex issue [2]. Existing research focused primarily on the efficiency of entire healthcare organisations or entire healthcare systems without paying much attention to the effectiveness of the people performing the different diagnostic and therapeutic jobs, despite the fact that the effectiveness of public healthcare facilities is the sum total of the effectiveness of the individual staff members.

Objective

The objective of the study was to establish a correlation between the competencies of primary care physicians and the effectiveness of patient care. To accomplish this, the study authors analysed two types of competencies of medical staff (hard and soft skills) with the use of the original patient care effectiveness indicators.

Materials and methodology

The research was conducted as part of a pilot study held at the Medical and Diagnostic Centre (MDC) in Siedlce. As of 31 December 2020, 85,000 patients were enrolled with the MDC primary healthcare physicians in the Eastern Mazovia and Lublin provinces [3].

The study population consisted of primary healthcare physicians employed at the MDC in Siedlce. The approximate number of primary healthcare physicians working at the MDC is 65 (depending on the criteria used). The sample is exclusive of primary healthcare physicians working at MDC for less than six months, physicians with fewer than 150 patient declarations, and physicians on sick leave, vacation leave and collaborating with the centre as needed basis. These physicians were excluded from the study group for the reasons of total absence of data or the availability of data that could not be used to properly evaluate their effectiveness. As such, the final research sample consisted of 29 respondents who treated 94% of the MDC patient population.

The target definitions and measurement methods of the subject matter of the analysis i.e. staff competencies and patient care effectiveness were developed for the purposes of this study. Below is a presentation of our understanding of the key concepts, as well as our data collection and analysis methodology.

Hard skills

The operational dimension of knowledge is represented by competence and experience which are measurable characteristics required to perform a given type of work [4, 5]. In our study, the degree of competence was measured with the number of specialisations of the given physician and total work experience of the given physician along with his or her experience of working at MDC (in years). The data were collected in a survey. The physicians were asked to individually provide the number of their medical specialisations and their work experience in years in a questionnaire. The research methodology applied in the study was the computer-aided-web-interviewing (CAWI) technique. The survey was made available to the physicians via the online research tool webankieta.pl. The study was aimed at determining the level of social competencies or skills of the physicians from our study sample. The research was conducted between 12.01.2020 and 12.03.2020 and each of the 29 respondents received an individual link and identification token. We got responses from 23 respondents with the other 6 refusing to fill in the questionnaire. Out of the 23 questionnaires two were incomplete – the respondents failed to answer all the questions. Eventually, we used data obtained from 21 primary physicians working at the MDC. To obtain possibly the most comparable results we standardised the “total work experience” and “work experience at MDC” indicators by dividing the value provided by each physician by the highest value recorded in the sample. As a result, each respondent was assigned an index value expressed as a percentage of the highest result.

Soft skills

The operational dimension of competencies is manifested as social skills. The most popular method of measuring social skills in Poland is the Social Competencies Questionnaire (Kwestionariusz Kompetencji Społecznych or SKK). The Social Competencies Profile (Profil Kompetencji Społecznych or PROKOS) is the updated version of the questionnaire. In our study, we used the PROKOS questionnaire developed in 2013 by Anna Matczak and Katarzyna Martowska in the version addressed to the working population [6]. The PROKOS questionnaire features 90 expressions in the infinitive denoting different activities. The respondent is asked to rate on a four-point scale the effectiveness of his or her performance of the given activity: 60 out of these items refer to activities performed to cope in various social settings that may be considered difficult and the other 30 are buffer questions regarding competencies other than social. Apart from providing an overall index value, the PROKOS questionnaire allows researchers to obtain detailed indices on different social competencies: assertiveness (scale A), cooperation (scale C), sociability (scale S), social resourcefulness (scale R) and social activism (scale A). The indices were obtained in the course of a factor analysis, which demonstrates their high cognitive value.

Patient care effectiveness

Since there are no effectiveness indicators to measure patient care effectiveness at the level of individual physicians apart from the measurement of patient satisfaction, we adopted our own definition grounded on the concept of value-based medicine. Value is defined here as the relationship between quality and cost. Value grows along with the improvement of quality or a drop in healthcare costs. In operational terms, quality measurement can be expressed in three areas: structure, process and result [7]. As part of each of the areas we selected certain characteristics of physicians that were used to develop the effectiveness indicators (Graph 2.). To calculate the indicators we used anonymised data obtained from the IT systems of the MDC. The research method adopted was the cross-sectional study. To facilitate the comparison and modelling of data each of the indicators was relativized via dividing each result by the highest result of obtained. Consequently, the values of each indicators are expressed as percentages denoting a percentage of the highest result. In line with the concept of effectiveness coined by Porter, effectiveness is patient value expressed as the relationship between health outcomes and healthcare costs [8, 9].

Statistical analysis

In this study, a special type of regression was used - multivariate regression, i.e. one in which one dependent variable (a given effectiveness index) is influenced by several independent variables (indices of hard competences and soft competences). As a result of the analysis carried out in this way, models were created describing the relationships between several hard and soft competences of doctors and individual effectiveness indexes. The least squares method, based on the analysis of variance, was used to select the best-fitted model.

Results

Comparison of hard skills

To evaluate the top results obtained in the course of an analysis of hard skills the total work experience of physicians in years was compared with their work experience at MDC in years. This was done by plotting the values of two indicators relativized at an earlier stage of the study. To facilitate the analysis, the total work experience in years was expressed as a line (the line connects the index values for the individual physicians), while data about the work experience at MDC was expressed as points. Axis X represents the individual physicians and axis Y shows their results in terms of work experience relativized to 100.

Graph 1. Comparison of total work experience in years and work experience at MDC in years for 21 physicians studied on a 0 to 100 scale.

Legend: blue line- total work experience; red line - work experience at MDC

Source: Own study based on research results

The graph shows there is no correlation between total work experience and work experience at MDC.

Comparison of soft skills

Below is a presentation of the comparison of average results obtained for primary healthcare physicians at the MDC and for physicians from the normalized sample and additional validation obtained from a study conducted by Martowska and collaborators [10]. The study was standardized and conducted on the professional group of physicians, which permitted the comparison of results.

Table 1. Comparison of average results for MDC physicians and for physicians from the normalized group and additional validation

Source: Own study based on research results

As shown above, assertiveness, cooperation, sociability and social resourcefulness of the participating physicians from the MDC were considerably lower than the competencies of the physicians from the normalized study. Similar results were obtained only in the case of the skill of social activism.

Comparison of patient care effectiveness in primary healthcare

Below is a presentation of aggregated data for the 14 original effectiveness indicators as measured for the individual physicians.

Graph 2. Comparison of 14 original effectiveness indicators for the 21 physicians studied on a 0 to 100 scale per indicator.

Legend:

Patient Population Coverage

Working Time Effectiveness

Complex Visits Effectiveness

Routine and Advanced Health-Check Effectiveness

Number of DILO Cards Effectiveness (Cards for Diagnostics and Oncological Treatment)

Average Chronic Patient Life-Span Effectiveness

Percentage of Mammograms

Creative Destruction

Key Visits Effectiveness

Number of Complex Visits Effectiveness

Care Plan Execution Effectiveness

Average Patient Life-Span Effectiveness

Percentage of Cervical Screening Tests

Percentage of CVD Screening Tests

Source: Own study based on research results

Apart from the presentation of results for each of the effectiveness indicators, a factor analysis was also performed, which was conducive to the development of a scale. The factor analysis performed for the 14 original indicators demonstrated that a highly reliable scale can be created using 5 of the original indicators. The following indicators were used to develop the scale Complex Visits Effectiveness, Key Visits Effectiveness, Complex Visits Number Effectiveness, Average Patient Life-Span Effectiveness, and Patient Population Coverage.

The resulting scale must be verified. The first verification will be performed in the final report from the study where the scale will be used to model the correlation between hard and soft skills. Once this is done, we will be able to tell whether the scale can successfully be used in lieu of the five indicators that comprise it, and perhaps also in lieu of the other indicators.

Correlation between the competencies of medical staff and patient care effectiveness in primary healthcare

Single-factor analyses were also performed in the study, namely analyses in which the diversification of one index was explained with a single variable. As a result, it was possible to establish a correlation between the effectiveness indicators and social competencies. The first coefficient calculated for the two variables, i.e. for the given scale of social competencies and for the individual effectiveness indicator was the Pearson Correlation Coefficient.

Assertiveness

The first stage of the analysis was the calculation of the correlation coefficient between assertiveness and the individual effectiveness indicators. None of the correlations proved relevant. This means that the competence of assertiveness most likely has no impact on the diversification of the effectiveness of physicians.

Table 2. Pearson Correlation Coefficient between competence of assertiveness and the individual effectiveness indicators.

source: own study based on data obtained from CMD and survey results

Cooperation

The first stage of the analysis was the calculation of the correlation coefficient between cooperation and the individual effectiveness indicators. None of the correlations proved relevant. This means that the competency of cooperation most likely has no impact on the diversification of the effectiveness of physicians.

Table 3. Pearson Correlation Coefficient between cooperation and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Sociability

The first stage of the analysis was the calculation of the correlation coefficient between sociability and the individual effectiveness indicators. Sociability proved to be the only competency to correlate with one of the effectiveness indicators i.e. with Creative Destruction. It was a moderate inverse correlation. The other correlations proved irrelevant, which means that the competency of sociability most likely has no impact on the diversification of the effectiveness of physicians.

Table 4. Pearsons Correlation Coefficient between sociability and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Social resourcefulness

The first stage of the analysis was the calculation of the correlation coefficient between social resourcefulness and the individual effectiveness indicators. None of the correlations proved relevant. This means that the competency of social resourcefulness has little impact on the diversification of the effectiveness of physicians.

Table 5. Pearsons Correlation Coefficient between social resourcefulness and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Social activism

The first stage of the analysis was the calculation of the correlation coefficient between social activism and the individual effectiveness indicators. None of the correlations proved relevant. This means that the competency of social activism most likely has no impact on the diversification of the effectiveness of physicians.

Table 6. Pearsons Correlation Coefficient between social activism and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Overall social competencies

The first stage of the analysis was the calculation of the correlation coefficient between overall social competencies and the individual effectiveness indicators. Of relevance was only the correlation between social competencies and the Creative Destruction indicator. It is an inverse correlation, which means that the value of the indicators drops with the growth of social competencies. The other correlations proved irrelevant, which means that social competencies most likely have no impact on the diversification of the effectiveness of physicians.

Table 7. Pearsons Correlation Coefficient between overall social competencies and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Evaluation of the correlation between the social competencies of medical staff and patient care effectiveness in primary healthcare

The sample that we eventually managed to conduct the study on was too small to perform accurate statistical calculations. In effect, the statistical tests did not demonstrate the existence of any significant correlations between social competencies and patient care effectiveness.

Of statistical significance was only one correlation, namely that between sociability and overall social competencies and the Creative Destruction indicator, however, an analysis of a scatter graph showed that the correlation does not actually exist.

A scatter graph was done for each pair of variables which made it possible to analyse the interdependence of the variables in the study sample. Such observation, although not based on hard data, permitted us to identify certain trends that might be statistically confirmed in a study conducted on a larger sample of respondents.

The social competencies of assertiveness and cooperation, in literature found to be most significant in the work of physicians, seem to offer the highest chance for discovering certain patterns.

The assertiveness competency demonstrates a correlation with the following variables:

·         Percentage of CVD Screening Tests;

·         Care Plan Execution Effectiveness;

·         Average Patient Life-span Effectiveness.

The cooperation competency demonstrates a correlation with the following variables:

·         Key Visits Effectiveness;

·         Percentage of Cervical Screening Tests;

·         Percentage of Mammograms;

·         Percentage of CVD Screening Tests.

Interestingly, on the scatter graphs the cooperation competency inversely correlates with the Routine and Advanced Health-Check Effectiveness indicator and the Number of DILO Cards indicator. Moreover, the sociability competency demonstrates an inverse correlation between the Key Visits Effectiveness indicator, social resourcefulness correlates inversely with the Working Time Effectiveness indicator, while social activism correlates with the Percentage of Cervical Screening Tests and the Care Plan Execution Effectiveness indicators. Therefore, analyses of scatter graphs show that given a larger study, it will be possible to identify correlations between the different social competencies and the values of some of the effectiveness indicators. Clearly, the individual competencies (especially assertiveness and cooperation) will be of more significance to the diversification of effectiveness than overall social competencies.

Evaluation of the correlation of hard skills of medical staff and patient care effectiveness

Variance analysis tests did not show any diversification of patient care effectiveness between the results for women and men or the results for physicians with a varied number of specialisations [11].

The only statistical correlation in terms of work experience was the statistically relevant correlation between work experience and the Creative Destruction effectiveness indicator, however, owing to the imperfect nature of the indicator no conclusions can be reached from the correlation. To identify potential correlations between variables that are not visible in statistical tests, an analysis of scatter graphs for the total work experience and work experience at MDC and the individual patient care effectiveness indicators was performed.

Table 8. Pearson Correlation Coefficient between work experience and the individual effectiveness indicators

source: own study based on data obtained from CMD and survey results

Table 9. Pearson Correlation Coefficient between work experience at MDC and the individual patient care effectiveness indicators

source: own study based on data obtained from CMD and survey results

The scatter graph suggests that given a larger study sample it will most likely be possible to identify correlations between work experience and the Patient Population Coverage indicator. This correlation can be explained with the fact that higher seniority permitted the physicians to attract a higher number of patients. Interestingly, no such correlation was identified between the work experience at MDC and the Patient Population Coverage indicator data, although total work experience and work experience at MDC do correlate. Perhaps, in reality, the number of patients depends more on the physician himself rather than the organisation he represents. Furthermore, work experience inversely correlates with the Percentage of CVD Screening Tests indicator as if more senior physicians were less likely to refer their patients for screening tests.

The work experience at MDC indicator seems to correlate with the following effectiveness indicators: Key Visits Effectiveness, Complex Visits Effectiveness and Routine and Advanced Health-Check Effectiveness, namely, all the factors affecting the remuneration of physicians at MDC. The physicians seem to learn from the rules applied in their organisation.

An inverse correlation between work experience at MDC and the Number of DILO Cards indicator, which may mean that more senior physicians working at MDC are less likely to diagnose cancer.

Evaluation of the modelling results

Statistical modelling is even more prone to failure of meeting the study objectives than single-factor tests such as the variance analysis, therefore, these results – given the very small sample size – are not very reliable.

Nevertheless, a correlation between the hard and soft skills studies and the following four effectiveness indicators related to the non-standard activities performed by physicians employed at the MDC was identified:

• ·         Key Visits Effectiveness;
• ·         Complex Visits Effectiveness;
• ·         Number of Complex Visits Effectiveness;
• ·         Routine and Advanced Health-Check Effectiveness.

Discussion and conclusions

When evaluating the original effectiveness indicators developed for the purposes of this study one must focus on their internal construction and the distribution of the results obtained. Given the small research sample, it is impossible to determine the factors impacting the indicators which – as found in literature – would demonstrate their usefulness.

Above all, noteworthy is the relatively uniform distribution of the different indicators (consistent with the Gaussian distribution), obtained mainly owing to their relativization (i.e. dividing each result by the highest result). Exceptions to the rule are the Average Patient Life-Span Effectiveness and the Average Chronic Patient Life-Span Effectiveness indicators. They seem to be very significant, however, the minor differences between the values prevented the study authors from identifying any differences between the physicians. Problematic was also the Creative Destruction indicator the results of which were both concentrated and extreme. Such distribution of values of this indicator was caused – to much too high a degree – by the short employment period in the given health care organisation (the need to build a patient data base from a very low figure), which hindered the interpretation of the differences observed.

The ultimate plan is to conduct an extended study on a representative sample of the health care organisations from across Poland and accounting for the necessity to modify the research methodology in view of the likely absence of as reliable and as systematically collected data as that obtained from the Medical and Diagnostic Centre in Siedlce. In the future, the study will also be extended to include nursing staff.

In light of the fact that this was a pilot study, the sample comprised of primary care physicians employed at the Medical and Diagnostic Centre in Siedlce only. This permitted the assessment of the reliability of the tools designed for the purposes of the study. However, unfortunately, it did not offer the opportunity to identify any dependencies of variables nor to develop reliable measurement scales for the effectiveness of patient care. This is why the research authors intend to conduct a study on a larger sample of physicians from health care organisations from across Poland.

The inclusion of social competencies into the study must be evaluated positively. Although no statistical correlation was observed between them and the effectiveness indicators, an analysis of the scatter graphs suggests that there is a chance of identifying such correlations in a study conducted on a larger sample of respondents.

The key conclusions of the study in terms of the evaluation of the competencies of medical staff are presented below.

1. The social competencies of primary healthcare physicians participating in the study analysed aggregately and individually were visibly lower than the competencies of physicians participating in the normalized study. One exception was the social activism competency which proved to be at a similar level both for MDC physicians and the normalized study participants.

2. The scales of the individual social competencies were highly reliable, as was the case with the normalized study, which demonstrates that the questionnaire was successful and the study results must be found correct.

3. The individual soft skills of all primary healthcare physicians strongly correlated with one another, as was the case with the normalized study. This is yet another argument speaking in favour of the reliability of the results obtained and, furthermore, it shows that if physicians display a high degree of one type of skills, they are also highly skilled in other areas. The only exceptions were sociability and social resourcefulness which showed statistically irrelevant correlations.

4. As regards the analysis of hard skills, no correlation was shown for the total work experience and the work experience at MDC of primary healthcare physicians. 

The key conclusions of the study in terms of effectiveness of patient care are presented below.

1. The factor analysis performed for the 14 original indicators demonstrated that 5 of the original indicators can be used to create a highly reliable scale. The indicators in question are the following:

Ø  Complex Visits Effectiveness;

Ø  Key Visits Effectiveness;

Ø  Complex Visits Number Effectiveness;

Ø  Average Patient Life-span Effectiveness;

Ø  Patient Population Coverage

2. Of significance is also the fact that the above indicators included in the scale belong to all three analysed quality areas, namely structure, process and effects, which is reflected in theory.

3. It is noteworthy that the indicators used for the calculations were relativized. As such, an attempt ought to be made to make calculations based on non-relativized data to verify whether they still correlate to a similar degree.

4. Summing up, it must be found that the development of original effectiveness indicators is highly innovative in terms of approach and study topic in Poland. Moreover, the identification of a strong scale comprised of five original effectiveness indicators is an important step towards developing a uniform indicator for measuring the effectiveness of patient care in the primary care system, which will provide a significant contribution to this area of science.

Conflict of interest: none declared.