Assessment of Factors Influencing Educational Effectiveness in Higher Educational Institutions

Abstract

This paper discusses the results of the study of educational effectiveness at the tertiary level. It contains an overview of the research considering the quality and effectiveness of higher education development, the issues of creating criteria for their assessment and measurement. The four groups of factors (“Man”, “Milieu”, “Material”, “Method”) for evaluating educational effectiveness were determined, totally including 65 points. The results of the students’ surveys revealed the most significant factors as well as the least crucial ones influencing educational effectiveness in higher educational institutions. The total number of respondents was 279. It was found that the subgroup of factors “Teacher’s competence” is the most important from the viewpoint of the respondents, while the subgroup “Infrastructure facilities” is the least valuable one. The priority measures list was given, the implementation of which will make it possible to increase educational effectiveness in higher educational institutions. Methods such as Ishikawa diagram building, brainstorming and the survey of students were used for achieving the objectives of this research.

1. Introduction

Education has an important impact on a country’s economic prosperity. Educational effectiveness has a direct impact on the level of competitiveness of any country. In order to improve competitiveness, it is necessary to study the national education system and develop measures for improving its effectiveness [1]. According to Liu and Xu’s studies devoted to the impact of the education level on the level of national competitiveness, conducted among 53 countries of the world using data from 2000 to 2014, the highest level of education was observed in countries such as Singapore, the Philippines, India, China, Indonesia, Taiwan, South Korea, the Russian Federation and Japan. The lowest level of education was marked in Iceland, Portugal, Denmark, Brazil, South Africa and Austria [1] (p. 6). Currently the issue of measuring the quality and effectiveness of education is a crucial point due to the challenges caused by globalization processes, the COVID-19 pandemic and other factors [2,3,4,5].

Nowadays, there is no universal definition of educational quality. It seems to be that the most appropriate definition is given by V.I. Zvonnikov. “The quality of education acts, firstly, as a relapse of activity, particularly as a combination of its characteristics, ranging from the ability to meet established and expected needs. Secondly, as a process aimed at achieving planned results taking into account student internal potential and external conditions” [6]. From this point of view, firstly, the educational quality can be assessed by forming a system of characteristics. Secondly, these characteristics should be determined taking into account students’ needs [7].

For improving the quality of education, researchers propose to implement tools used by industrial enterprises such as Lean Six Sigma [8,9], lean management in higher educational institutions [10,11,12] or Quality Management Systems [13,14,15]. These studies are devoted to improving the level of educational process quality by increasing its value, as well as finding out and reducing various losses such as over-production, over-processing, waiting, transportation, etc. The search and loss reduction make it necessary to build a system of constant studies of university internal processes.

Rao Naveed et al. [13] compare and contrast a quality-focused education institute with an ordinary institute (

Table 1. Comparison of characteristics of quality educational institute to ordinary education institute.

Quality-Focused Educational Institute	Ordinary Educational Institute
Student-focused	Focused on other needs, e.g., finances
Proactive approach to problem identification and mitigation	Adopt fire-fighting approach once problems appear
Invest in staff (academic, services, support)	Less focused on staff development
Prepare and follow quality strategies	Lack of vision in regard to quality strategy
Attention to feedback collection and following up with corrective measures	Less focus on feedback gathering and responding to feedback
Has quality policies and plans in place	No quality policies or plans
Quality improvements are led by top management and followed by everybody	Only few people are responsible for assuring quality process
Plan long term in regard to curricula and student employability, etc.	No, or only short-term, planning for students
Has distinctive vision in line with contemporary and future student/staff needs and challenges	Lacks distinctive vision to address challenges to staff and students

Source: [13].

).

Among the quality characteristics focused on educational institutes given in Table 1, the most noteworthy ones are student orientation and a proactive approach to problem identification and mitigation, which are possible only with an established system of attention to feedback collection and following up with corrective measures. It should be taken into consideration that in order to form a distinctive vision in line with the contemporary and future student/staff needs and challenges, the educational institution must conduct constant research on such needs.

Researchers studying the issues of educational quality stress the importance of education system evaluation [16,17,18]. They emphasize that the education system assessment should be carried out taking into account the separation of two concepts—effectiveness and efficiency [16,19,20]. Effectiveness refers to the ability of an educational institution to achieve institutional goals, whereas efficiency refers to achieving institutional goals with the best utilization of resources [21]. According to the United Nations Educational, Scientific and Cultural Organization (UNESCO) definition, the efficiency of an educational system is the measure of the relation of investment to the result of education [22].

However, we believe that economic indicators cannot be indicators of higher education effectiveness. We should agree with scholars who suppose that education system efficiency is very frequently measured by economic models [3,16]. If the main criteria for the assessment of educational quality are labor productivity, time spent control, cost savings, increased income or other financial displays, this can lead to the breaking of Deming’s basic principles (e.g., “eliminate numerical quotas/goals” or “drive out fear”) [23]. The authors share the opinion of Naveed et al. [13]. They claim that the measurement of the education quality level cannot be carried out with the power of quantitative indicators, and it is also necessary to use qualitative indicators.

The other authors emphasize the importance of students’ assessment of the university’s educational activity [16,24,25]. Students are one of the main stakeholders of the educational system [13,16,26]; therefore, forecasting and assessing their expectations is a crucial role of university activities as part of improving the quality of education. Some researchers identify students’ satisfaction with their requirements as an extremely important factor of educational effectiveness in higher educational institutions [3,25,27].

We suppose that for developing decisions aimed at increasing the effectiveness of higher education, it is necessary to consider the educational process not at the level of a country or region but at the level of a separate organization. Investigators dealing with educational effectiveness stress the value of studies focused on assessing the level of investment in education and the lack of research aimed at assessing the effectiveness of educational processes in a particular higher institution [1,28]. T. Agasisti has the same opinion. He believes that research on educational effectiveness should be focused not only on measuring performance but also investigating the internal processes of higher educational institutions [29].

In order to properly provide the measurement of the educational quality and effectiveness at the tertiary level, university authorities should create a higher educational management system that could ensure the transformation of an ordinary institute into a quality-focused education institute. In this regard, it seems crucial to organize not only the educational process but also to build the infrastructure of educational institution being able to motivate students for academic activities within the framework of the learning process [30,31,32].

The evaluation of the effectiveness of educational processes causes a problem of criteria determination for providing assessment. A number of research has been conducted to develop a variety of evaluating criteria (

Table 2. Factors and indicators affecting education effectiveness.

Authors	Quality Indicators (Factors)
T.P. Kuzminskaya, N.N. Burova [33]	High-qualified teaching staff (TS); educational and methodological support, training methods and technology; infrastructure facilities; intellectual potential; students as the principal object of the educational process; graduates as a channel for the communication of the university with professional spheres
O.A. Shendaleva [34]	The stability of the educational process; using a rating system for assessing students’ performance; high-qualified teaching staff; motivating teachers and students to present and acquire knowledge; adequacy, inter-relationship, the sequence of curriculum implementation; the provision of information and methodological materials
Kaur H., Bhalla G.S. [35]	Academic environment; college administration; student support services; learning material; infrastructure facilities; placement services; extracurricular activities; financial administration
Gundić A., Županović D., Grbić L., Baric M. [16]	Study program (learning outcomes; student–teacher ratio); teaching staff (teachers’ professional development; teachers’ competences; teachers’ workload; teachers’ mobility); students’ mobility; teaching resources (educational facilities; equipment; teaching material)
Dyatlov A. V., Kovalev V. V. [36]	The intensification of educational and scientific activity; stimulating inter-university and intra-university competition among teaching staff; improving the professionalization of university management
Ramírez-Correa, P.; Peña-Vinces, J.C.; Alfaro-Pérez, J. [37]	Operating expenses; student enrollments; the number of Institute for Scientific Information (ISI) publications; operating income

Source: Compiled by authors.

).

The disadvantage of these factors and criteria is the complexity of their assessment, which is due to the complexity of the quantitative measurement of many factors (for example, intellectual potential, the stability of the educational process, stimulating inter-university and intra-university competition, improving the professionalization of university management or the academic environment). We suppose the assessment of each of the above indicators is the subject of a specific study. To identify educational issues and develop measures to solve problems of educational quality, it is necessary to develop a list of specific factors affecting the educational process in a particular organization.

This study focuses on eliminating the problem of research shortage aimed at assessing the effectiveness of the educational processes in a particular organization. The object of this study is the academic process at Empress Catherine II Saint Petersburg Mining University. The subject of this study is the effectiveness of the training process at Empress Catherine II Saint Petersburg Mining University. The purpose of this research is to identify the factors having the greatest and least impact on the effectiveness of the educational process of a higher educational institution from students’ point of view. The objectives of this study are the following:

-

Develop a list of factors having an impact on the effectiveness of the educational process at higher education institutions;

-

Provide a factor assessment;

-

Identify the factors having the greatest influence on the effectiveness of the educational process at the university;

-

Identify the factors having the least impact on the effectiveness of the educational process at the university.

The obtained results will make it possible to compare university teachers’ experience (the co-authors’ experience is from 12 to 21 years) with students’ satisfaction and make certain adjustments to the educational process at the tertiary level.

2. Materials and Methods

The authors suggested 65 factors that can influence the effectiveness of the educational process (

Table 3. The factors having an impact on the effectiveness of the educational process.

Factor Group	Subgroup of Factors	Name of Factor
1. Man	1.1. Teachers’ competences	1.1.1. Teacher does not know an academic discipline well
		1.1.2. New academic course for teacher
		1.1.3. Teacher delivers boring lectures
		1.1.4. Teacher does not care about students’ understanding
	1.2. Teachers’ low level of academic ethics	1.2.1. Teacher is late for classes
		1.2.2. Teacher skips classes
		1.2.3. Teacher finishes classes earlier
		1.2.4. Teacher is not demanding
		1.2.5. Teacher is overly demanding
		1.2.6. Teacher’s unfair treatment of students
		1.2.7. Teacher does not meet the standards of academic ethics (rude, intemperate, impolite)
	1.3. Students’ low level of training	1.3.1. Poor school performance
		1.3.2. Low ability level
		1.3.3. Inability to express oneself
	1.4. Students’ low level of academic ethics	1.4.1. Student is late for classes
		1.4.2. Student skips classes because they have to work
		1.4.3. Student breaks deadlines for completing assignments
2. Milieu	2.1. Academic facilities	2.1.1. Classroom has poor acoustics
		2.1.2. Windowless classroom
		2.1.3. Classroom is not equipped with media
		2.1.4. Media is not connected to Internet
		2.1.5. Outdated media hardware
		2.1.6. Lack of required software for academic discipline
		2.1.7. Outdated software in computer classroom
		2.1.8. Lack of required equipment for practical training
		2.1.9. Too small room for classes
		2.1.10. Too large room for classes
	2.2. Infrastructure facilities	2.2.1. Shortage of laboratories
		2.2.2. Lack of staff room facilities
		2.2.3. Not enough space in staff rooms
		2.2.4. No extra chairs for students in staff rooms
	2.3. Class schedule	2.3.1. Too busy class schedule
		2.3.2. Too short breaks
	2.4. High teachers’ workload	2.4.1. High academic workload of teacher
		2.4.2. High scientific workload of teacher
		2.4.3. Large amount of bureaucracy
		2.4.4. Need to move between university buildings
		2.4.5. Conducting a lot of useless work
	2.5. Methodological issues	2.5.1. Teacher has a large number of classes daily
		2.5.2. Academic discipline is adequately placed in academic curriculum
		2.5.3. Ban on student expulsion (it forces teacher to be not demanding)
	2.6. Administration control	2.6.1. Need to wear uniforms 2.6.2. Monitoring time spent in workplace
3. Material	3.1. Classroom facilities	3.1.1. Lack of whiteboard markers
		3.1.2. Lack of whiteboard cleaner
		3.1.3. Paper shortage
		3.1.4. Shortage of stationery goods
		3.1.5. Shortage of printer cartridges
		3.1.6. No possibility to print out students’ work (summaries, coursework, etc.)
	3.2. Methodological materials	3.2.1. Lack of guidelines for classes, coursework, etc. 3.2.2. Outdated course content
4. Method	4.1. Teaching methods	4.1.1. Low level of teaching effectiveness
		4.1.2. Academic curriculum does not meet business requirements
		4.1.3. Academic curriculum does not meet employer requirements
		4.1.4. Insufficient association of lectures with laboratory classes
		4.1.5. Outdated information provided by instructor
		4.1.6. Discrepancies between learning material and examination database
	4.2. Examination form	4.2.1. Test exam
		4.2.2. Not fixed exam grade scale
	4.3. Credit test forms	4.3.1. Credit test form
		4.3.2. Oral interview
		4.3.3. Credit test is assessed by attending classes
		4.3.4. Too much learning material to have credit test
	4.4. Control methods	4.4.1. Lack of monitoring during semester
		4.4.2. Too much test work, calculation and graphic work during semester

Source: Created by authors.

). The Ishikawa diagram construction method was used for providing the structure of factors.

The factors are presented in 4 groups:

• Man.
• Milieu.
• Material.
• Method.

The brainstorming method was used to identify the factors. The 4 associate professors with working experience of more than 20 years of teaching took part in developing the list of factors.

This study was conducted from September to November 2023. This research was carried out based on the results of a student survey. To participate, the students were offered the chance to complete a paper-based questionnaire. The students were asked to mark in the Table 3 factors that currently have an impact on the effectiveness of their knowledge acquisition. The number of factors that can be marked was not limited.

The respondents were bachelor and specialist degree students (1–4 years of studying), as well as master’s degree students (2 years of studying) of economic and technical profiles of St. Petersburg Mining University. The total number of respondents was 279 (

Table 4. Respondents characteristics.

No.	Degree	Profile	Year of Studying	Number of Students
1	master’s degree	technical	2	26
2	master’s degree	technical	2	7
3	bachelor	economic	4	9
4	bachelor	economic	4	5
5	bachelor	economic	3	13
6	bachelor	economic	2	4
7	bachelor	economic	1	28
8	bachelor	technical	4	52
9	bachelor	technical	4	17
10	bachelor	technical	4	17
11	bachelor	technical	4	34
12	bachelor	technical	4	9
13	bachelor	technical	4	14
14	specialist	technical	4	19
15	specialist	technical	4	25
	Total amount:			279:

Source: Created by authors.

).

The Empress Catherine II Saint Petersburg Mining University is one of the oldest mineral and mining engineering institutions in the world. Annually, about 2 thousand students graduate from it [38,39]. It has well-developed infrastructure facilities providing high-quality education [40,41,42]. More than 170 graduates of the Empress Catherine II Saint Petersburg Mining University have become laureates of the State Award for their contribution to the development of science and technology. More than 5000 dissertations have been defended at the Empress Catherine II Saint Petersburg Mining University for the past 80 years [43,44]. Despite the applied focus of research, scientists at the Saint Petersburg Mining University pay attention in their research to improving the effectiveness of education, in particular, studying trends in the development of mining education abroad [45] or comparative characteristics of Russian and Western education in the mining industry [46]. In 2023, the university took third place in the world rating among the best universities training specialists in the profile of “Mineral & Mining Engineering” [47].

The research hypotheses are as follows:

• Group 1 “Man” has the greatest influence on the quality of the educational process.
• The most important subgroup of factors among the first group is subgroup 1.1. “Teachers’ Competences”.
• The group “Milieu” is the least important factor affecting the effectiveness of the educational process.
• The group “Material” is the third most important factor affecting the effectiveness of the educational process.
• Group 4 “Method” is the second most sufficient group of factors.
• The most significant subgroup of factors in Group 4 is subgroup 4.1 “Teaching methods”.

In general, we consider that the quality of the educational process is more influenced by a teacher’s personality and his/her professional skills in conducting classes than infrastructure facilities and control methods.

3. Results

As a result of the conducted survey, the respondents selected 4903 factors in total. On average, each respondent chose 17.6 factors. Thus, this indicates a fairly high interest of the respondents to participate in the questionnaire and proves the quality of the obtained results. The quality of the survey is confirmed by the absence of the questionnaires in which no factor was marked, only one factor was marked or all factors were marked.

First of all, it is crucial to determine the group of factors that is the most frequently marked by the respondents. We calculated the average number of the selected factors in the subgroups of each group. Figure 1 shows the results of these calculations.

Figure 1 shows that the total number of the selected points in the groups is proportional to the number of factors in the groups (17 factors in Group 1; 26 in Group 2; 8 points in Group 3 and 14 in Group 4). Therefore, the importance of groups should be determined not by the total number but by the average number of selected points of each group.

Group 4 “Method” is the leader in the average number, despite the fact that the total number of the selected points takes third place. Group 3 “Material”, which is the minimum in terms of the total number of factors, takes second place by the average number. Group 1 “Man” ranks in second position in the total number of selected points and third in its average number. According to the total number of selected factors, Group 2 is the leader. However, the group “Milieu” obtained the smallest amount of the selected factors.

Figure 2 shows the results of the subgroup factor analysis by the total and average number of selected factors in the subgroup. The subgroups are ranked by the decreasing average number of selected factors.

As Figure 2 presents, subgroup 1.1. “Teachers’ Competences” is the first most important point for the polled students. Subgroup 4.1. “Teaching methods” takes second place both by total and average numbers. The third and fifth places, respectively, are occupied by subgroup 4.2. “Examination form” and subgroup 4.4. “Control Methods”. The next place is taken by subgroup 3.1. “Classrooms facilities”.

A number of subgroups should be noted in which both the total and average number of factors are in an intermediate position. They are subgroups such as 2.3. “Class schedule”, 3.2. “Methodological materials”, 1.4. “Students’ low level of academic ethics”, 2.5. “Methodological issues” and 4.4. “Control Methods”.

The subgroups with the lowest average and overall value are the following: 2.2. “Infrastructure facilities”; 2.6. “Administration control” and 1.3. “Students’ low level of training”. On the other hand, subgroup 2.1. “Academic facilities” has the highest overall value of the selected factors (due to the fact that it includes the maximum number of points, particularly 10), but on average, the respondents chose these factors relatively infrequently. The results of the survey analysis by the number of selected factors are presented in

Table 5. Factor analysis results.

Student Group	Factor Number	Respondent Percentage, %	Student Group	Factor Number	Respondent Percentage, %
1	1.1.3.	75.27	34	3.1.3.	22.58
2	3.1.1.	62.72	35	2.4.4.	21.15
3	2.1.1.	56.27	36	2.6.1.	21.15
4	4.4.2.	55.91	37	1.1.2.	20.43
5	3.2.1.	53.41	38	2.3.2.	19.71
6	4.2.2.	50.18	39	4.3.2.	19.71
7	3.1.2.	49.82	40	2.2.1.	18.28
8	3.1.6.	49.10	41	2.2.4.	18.28
9	4.1.6.	48.75	42	4.2.1.	17.92
10	1.1.4.	48.39	43	2.4.3.	16.49
11	1.2.7.	47.67	44	1.3.2.	16.13
12	1.2.5.	47.31	45	2.1.4.	15.41
13	4.1.2.	45.52	46	1.2.1.	15.05
14	1.2.6.	43.73	47	2.1.2.	14.34
15	2.3.1.	43.37	48	1.2.2.	13.98
16	2.4.5.	43.37	49	2.1.9.	13.98
17	4.1.1.	43.37	50	4.1.3.	13.98
18	4.3.4.	38.35	51	2.1.5.	13.62
19	2.5.2.	37.99	52	2.6.2.	12.90
20	2.1.6.	36.92	53	1.2.4.	12.19
21	1.1.1.	35.13	54	4.3.1.	10.75
22	2.5.1.	35.13	55	4.4.1.	10.75
23	4.1.5.	31.90	56	1.3.1.	8.96
24	1.4.2.	30.47	57	3.1.4.	8.60
25	2.1.7.	30.11	58	2.4.2.	6.45
26	2.1.8.	29.39	59	3.1.5.	6.45
27	4.1.4.	29.03	60	2.1.10.	5.38
28	1.4.3.	27.96	61	3.2.2.	5.38
29	1.3.3.	27.60	62	2.5.3.	4.66
30	2.1.3.	24.73	63	2.2.3.	2.51
31	2.4.1.	23.30	64	1.2.3.	1.08
32	4.3.3.	23.30	65	2.2.2.	1.08
33	1.4.1.	22.58	-	-	-

Source: Created by authors.

.

According to these, the list of factors can be divided into some groups of importance: a group of the 6 most important factors (more than 50% of respondents); a group of 40 factors of medium importance (15–50% of respondents) and a group of the 19 least important factors (less than 15% of respondents).

For checking whether the difference in the proportion of respondents for the six most important factors forming the first group is statistically significant (the proportion of respondents is more than 50%), we formulate the following five null hypotheses:

H₀₁:

p_1.1.3 = p_3.1.1;

H₀₂:

p_3.1.1 = p_2.1.1;

H₀₃:

p_2.1.1 = p_4.4.2;

H₀₄:

p_4.4.2 = p_3.2.1;

H₀₅:

p_3.2.1 = p_4.2.2.

The variable φ is used as a criterion for justifying hypotheses H₀₁, H₀₂, H₀₃, H₀₄ and H₀₅ (Equation (1))

$$\mathsf{\phi }={\sum }_{i=1}^2{\sum }_{j=1}^2\frac{{(m_{ij}+m_{ij}^{teor})}^2}{m_{ij}^{teor}}$$

(1)

where

m_ij—the frequency of the respondents’ selection of the compared factor (the first factor—m₁₁; the second factor—m₁₂); the frequency of the event when the factor being compared was not selected (the first factor—m₂₁; the second factor—m₂₂).

$m_{ij}^{teor}$—the theoretical frequency of the respondents’ selection of the compared factor (the theoretical frequency of the event when the compared factor was not selected).

Table 6. Hypotheses’ justification formulas.

Survey Results	Factor
	1	2
Chosen factor	$m_{11}$	$m_{12}$
	$m_{11}^{teor}=n_1\frac{ m_{11}+m_{12}}{n_1+n_2 }$	$m_{12}^{teor}=n_2\frac{ m_{11}+m_{12}}{n_1+n_2 }$
Unchosen factor	$m_{21}=n_1-m_{11}$	$m_{22}=n_{12}-m_{12}$
	$m_{21}^{teor}=n_1-m_{11}^{teor}$	$m_{22}^{teor}=n_1-m_{12}^{teor}$
Respondent number	$n_1$	$n_{12}$

Source: Created by authors.

contains data for the justification of each of the proposed hypotheses.

For example, for justifying the hypothesis, the proposed table will provide the following data (

Table 7. Hypotheses’ justification figures.

Survey Result	Factor
	1.1.3	3.1.1.
Chosen factor	210	175
	192.5	192.5
Unchosen factor	69	104
	86.5	86.5
Respondent number	279	279

Source: Created by authors.

).

The variable φ has χ2 distribution with a degree of free choice k = ν − 1 = 2 − 1 = 1, where ν is a number of compared factors.

The critical point φcr forms a right-handed interval, i.e., (φcr, +∞). If φ $\in$ φcr, +∞, then hypothesis H₀₁ (determining the equality of respondents’ shares) is not proven, as the observed difference is statistically significant. Otherwise, we do not reject this hypothesis if the difference in shares is statistically insignificant.

The justification of hypotheses H₀₁, H₀₂, H₀₃, H₀₄ and H₀₅, was carried out with the value α = 0.05. The critical point for such a value is φcr = 3.84.

Table 8. Results of hypotheses’ justification.

№	Hypothesis	φ	Conclusion
1	H01	10.26	Unproved hypothesis, the observed difference is statistically significant
2	H02	2.41	Proved hypothesis, the observed difference is statistically insignificant
3	H03	0.01	Proved hypothesis, the observed difference is statistically insignificant
4	H04	0.35	Proved hypothesis, the observed difference is statistically insignificant
5	H05	0.58	Proved hypothesis, the observed difference is statistically insignificant

Source: Created by authors.

contains the calculated values and the results of the hypotheses’ justification.

Thus, we can conclude that the most important factor for the respondents is 1.1.3. The other factors included in the first group (factors 3.1.1., 2.1.1., 4.4.2., 3.2.1. and 4.2.2.) have approximately the same importance for the polled students.

The data of Table 5 show that the overwhelming majority of the respondents consider the human factor to be the most significant one, namely, the boring manner of lecture delivering. However, it is noteworthy that the importance of a teacher’s interesting way of presenting material is more sufficient than the coherence of its presentation and good subject competence (factors 1.1.4. and 1.1.1., 10th and 21st places, respectively).

Factor 3.1.1. “Lack of whiteboard markers” occupies second place. The same group’s points 3.2.1. “Lack of guidelines”, 3.1.2. “Lack of whiteboard cleaner” and 3.1.6. “No possibility to print out students’ works” take fifth, seventh and eighth places, respectively.

The poor classroom acoustics factor of Group 2 “Milieu” takes third place.

However, except for the factors mentioned above, the other most crucial ones are 4.4.2 “Too many test works, calculation and graphic works during semester” and 4.2.2. “Not fixed Exam Grade Scale”.

The group of 40 factors of medium importance presents the following data: Group 1 factors contain 65%, while Group 2 factors have a slightly low percentage of only 62%; Group 3 factors demonstrate the least percentage (37%); Group 4 factors have almost the same level as Group 1 and Group 2, 64%, respectively. Analyzing the factors of medium importance, the values of which are close to the most significant ones, except for 3.1.2. and 3.1.6 points mentioned above, we can determine the following:

-

Discrepancies between learning material and examination database (4.1.6.), and it determines the importance of exam preparation based on lecture materials;

-

Teacher does not meet the standards of academic ethics (1.2.7.);

-

Teacher is overly demanding (1.2.5.);

-

Teacher’s unfair treatment of students (1.2.6.);

-

Academic curriculum does not meet business requirements (4.1.2.), and it is especially important for industrial engineering higher institutions;

-

Academic discipline is adequately placed in academic curriculum (2.5.2.), and a teacher is non-responsible for this point.

It is easy to suppose that most of the mentioned factors are those ones directly or indirectly dependent on a teacher’s personality.

The group of 19 factors selected by less than 15% of the respondents is mostly represented by Group 2 “Milieu”. The least important factors are those such as the lack of staff room facilities or not enough space in them (2.2.2. and 2.2.3.), finishing classes early (1.2.3.), ban on student expulsion (2.5.3), outdated course content (3.2.2.), too large a room for classes (2.1.10.), a shortage of printer cartridges and stationery goods (3.1.5. and 3.1.4.), a high scientific workload of the teacher (2.4.2.) and poor school performance (1.3.1.).

Thus, it is obvious that most of the unimportant factors do not directly affect students’ interests (lack of staff rooms and cramped quarters, high workload of teachers, lack of printer cartridges and stationery goods). The students consider them as facilitating the learning process (finishing classes early, ban on expulsion of students).

It should also be noted that 2% of all responses are the “other” factor. The largest number of respondents (11%) chose “other” as a comment on subgroup 2.3 “Class schedule”. This subgroup indicates the following sufficient factors (they are given as the number decreases):

-

Inconvenient daily class schedule;

-

Class schedule changes during the semester;

-

Classes in different university buildings for one day;

-

Short lunch break.

Moreover, 5% of the respondents chose “other” in subgroup 2.2. “Infrastructure facilities”. Basically, the following factors were marked:

-

Lack of places for self-study work;

-

Lack of recreational facilities;

-

Shortage of water coolers in university recreational zone.

More than 4% of the polled students marked “other” in subgroup 4.4 “Control methods”. They are the following:

-

A large amount of coursework;

-

Complex tests.

The Ishikawa diagram was compiled (Figure 3) based on the survey data processing. It includes the most significant factors of the four groups. The factor number means its place in the overall list, ordered by the degree of importance to the students (1 is the most important one).

4. Discussion

Overall, Hypothesis 1 was not confirmed as Group 1 occupies third position regarding the average number of selected factors. However, Hypothesis 2 was proven, as the most crucial subgroup is 1.1. “Teachers’ Competences”, and it contains the most sufficient point 1.1.3 “Teacher delivers boring lectures”.

According to the obtained survey results, Group 2 “Milieu” is the least influential one on the effectiveness of the educational process. So, it verifies Hypothesis 3.

The obtained results were also contrary to Hypothesis 4 as Group 3 “Material” was in second position by the number of factors chosen by students.

There is undeniable proof that Hypothesis 5 was inaccurate as according to the obtained results, Group 4 “Method” takes the leading position among the most sufficient groups of factors.

Hypothesis 6 was supported by the survey results. The most crucial subgroup of Group 4 is 4.1 “Teaching methods”.

The respondents believe that subgroups such as the low qualification of the instructor, methods of conducting classes and providing regular tests of knowledge have the greatest impact on the quality and effectiveness of the educational process. According to the respondents, it is a noteworthy fact that teaching methods and the manner of conducting classes have more of a negative impact on the educational process than teachers’ low level of academic ethics. A teacher can start classes on time, not cancel them and not finish ahead of schedule, while their lectures are boring and tedious. However, a properly compiled class schedule, student’s low level of academic ethics and high quality of methodological materials are identified as factors of medium importance for the educational process.

Overall, the polled students believe that subgroups of factors such as infrastructure facilities, strict tests of knowledge and a student’s low level of training do not have a sufficient impact on the effectiveness of the educational process.

The key factors are the manner of conducting classes, academic and classroom facilities (for instance, the lack of whiteboard markers and cleaners), poor classroom acoustics, a student’s high academic workload and exam grade scale.

Mostly, Group 2 “Milieu” contains the least important factors.

Measures and activities for improving students’ knowledge efficiency should include the following:

• Development of teaching staff’s professional competence. A boring, incoherent delivery of learning material can be partially eliminated by providing advanced training courses for teaching staff in the field of rhetoric, speech culture and communication. The need to eliminate such factors as the failure of academic ethics, unfair treatment and excessive academic demands leads to improved management and quality control systems, as well as establishing feedback with students.
• Development of teaching staff’s methodological skills, in particular, improving learning materials to meet business requirements and eliminating discrepancies between learning materials and examination databases for providing better test preparation.
• Solving methodological problems related to the academic curriculum and class schedule. Figure 1 shows some issues related to Group 2 “Milieu”. For instance, the large amount of bureaucracy and useless work is due to insufficient planning. The group “Method” demonstrates almost the same issue caused by a large amount of students’ test work, calculation and graphic work during the semester. However, this factor is considered to be a subjective one.
• Providing activities in purchasing planning. It should be noted that academic and classroom facilities are of great importance. Both the lack of whiteboard markers or cleaners and poor classroom acoustics have a significant negative impact on educational effectiveness. Moreover, the problem of printing out learning materials is quite significant for students. An increased focus on the organization of efficient higher education supply systems in academic and classroom facilities can solve these problems.

The authors consider the important results of this study to be the identification of the specific unmet needs of students for improving the quality of academic training courses, upgrading classroom facilities and laboratory equipment, reducing the workload, improving methods of monitoring academic performance, etc. Meeting all these kinds of needs is a condition for improving the educational quality and effectiveness and, as a result, increasing a country’s economic prosperity. The results of this study can help the authorities of higher educational institutions to develop measurement systems for improving the quality and effectiveness of the educational process. A properly designed system for improving educational quality and effectiveness integrated into the academic curriculum will help to better meet the needs of a student as the main stakeholder of the educational process.

Our further research will be focused on the key factors having a sufficient impact on the effectiveness of the educational process in higher educational institutions from the teaching staff’s perspective.

5. Conclusions

The main goal of this study is to fill the lack of research aimed at assessing the educational effectiveness of higher educational institutions. To support this, this study assessed the internal educational process of a tertiary institute from the perspective of students.

As a result of this research, the problematic issues requiring priority solutions were identified, as well as the areas that students do not consider particularly important.

However, it should be emphasized that the research results are given from the perspective of students which may not give a complete idea of the tertiary education system’s shortcomings. For obtaining more precise results, another study should be carried out based on the most important factors of educational effectiveness in higher education institutions from the point of view of teaching staff. Conducting such a study will be the objective of our further research. The results of further research can help higher educational institution authorities to develop a measurement system for improving the quality and effectiveness of the educational process.