3.1. The Neural Network Models—Theoretical Grounds
In economics, neural networks (NN) have produced, especially since 1990s, a large body of research through the medium of which new models have become deep-rooted as a valid alternative to conventional statistical models. The comprehensive scope of applications has covered different areas such as macroeconomic forecasts, bankruptcy predictions, stock market dynamics, credit evaluations, marketing segmentation, decision-making processes, etc. [
35,
36]. Due to their huge potential to determine and replicate nonlinear and linear associations amongst various groups of variables, artificial neural networks have emerged as widespread and efficient models that can be applied for classification reasons, clustering issues, pattern detection, and forecasting operations in many other fields that are connected to business [
37].
The build-up process of neural networks originates from biology; thus, NN represent a simplified model of the human brain aimed at simulating the intelligence patterns of learning and storing large amounts of knowledge through tightly interconnected neurons [
38,
39]. Due to their flexibility, learning properties, and generalization power, an important number of NN architectures have been advanced in the literature. Among these categories, we selected a back-propagation learning algorithm with a feed-forward design. This model consists of artificial neurons organized in three different types of layers: the input layer, one or several hidden layers, and an output layer.
The neurons that make up a layer can be also called units. The units arranged in every single layer are linked to those operating in the neighboring layers (via respective weights and biases which are manifest for every single interrelation). Thus, the numerous weights that are contained within the network architecture correspond to the estimated coefficients of a classical regression model while the bias can simply be paralleled with the intercept term.
Figure 3 shows the classical architecture of a feed-forward NN with one hidden layer.
The training of a NN implies that a certain amount of information (which constitutes the training dataset) is supplied into the network through the input units; it moves across the hidden layers and moves to the output units. Each unit has to reckon the weighted average of its inputs; the outcome calculated as the sum of the weighted averages for every single input is produced through a non-linear function. The unit starts running when the total value of the determined sum is higher than a previously defined threshold and, consequently, it activates the adjacent units from the next layer. Through this mode of action, a NN can establish complex non-linear associations between explanatory and response variables.
Moreover, a feedback dimension is incorporated in the process of NN functioning. Thus, the results provided by the network are set in opposition with the factual value expected to be generated; the difference between the two values is used in order to adjust the weights corresponding to the associations between units. The machine learning algorithm described here is acknowledged as “back propagation” [
40].
The moment the learning process is completed, an unseen dataset known as the testing dataset is employed in order to assess the generalization power of the categorizer. Consequently, the validation dataset represents a group of data that is not at all involved in the process of NN training (or, in other words, the algorithm never “encounters” the validation data), but it acts as a marker that highlights the out-of-sample prediction rigorousness of the network.
Following each iteration performed in the assessment process, an out-of-sample prediction is drawn up by using the values that make up the data set, and the mean squared error (MSE) is determined. The network that minimizes MSE for the validation dataset indicates the most accurate alternative of the model. The process of training the neural network continues by repeatedly changing the neurons’ weights, and stops at the moment the output value reaches the expected target [
41].
In order to develop an NN algorithm, we must start from the basic process, which is known as “single perceptron”. In other words, a perceptron amounts to a unique McCulloch–Pitts neuron, which works with adjustable weights and bias. In an attempt to build a multilayer perceptron, the perceptron is adapted in a manner that allows it to incorporate a few layers of neurons together with nonlinear activation functions that ensures higher model potency since it can be applied in cases of nonlinear independent datasets [
42].
The following paragraph contains an in-depth presentation of the main stages included in our methodology, which were conceived in order to build a neural network for the purpose of classifying EU countries.
3.2. Research Design
As stated above, our analysis is aimed at taking into account the main tendencies of the teleworking upsurge within EU countries in the midst of the pandemic and to outline a few scenarios that could determine a significant increase in the implementation of these new work arrangements in Romania. In order to accomplish our goal, we have used data extracted from a survey on living and working issues during COVID-19, which were based on a questionnaire that was conducted online at the European Union level by Eurofound between 22 June–27 July 2020 (Round 2), with 24,123 complete responses provided by respondents aged 18+.
In the light of the above theoretical grounds, the methodology employed within our study entails the following stages (
Figure 4):
Stage 1: Collection of data. Based on a European-level questionnaire on various aspects of telework, we have defined ten distinctive variables regarding the level of home-based work adoption during the COVID 19 pandemic crisis. These variables were expressed as percentages in order to facilitate the comparison procedure between the data gathered from different EU countries.
Stage 2: Using the SAS Enterprise Guide software in order to acquire the initial classification of EU countries. The data matrix containing the values recorded for each variable and every single country was processed by using the SAS Enterprise Guide, version 8.3.
In order to emphasize the similitudes and the discrepancies between EU countries (in respect to their willingness to adopt teleworking) through the medium of a systematic analysis of ten relevant variables, in this preliminary stage, we have employed the hierarchical cluster method. In this case, the algorithm that was applied was adapted to the process of building the tree cluster structure (i.e., the dendogram).
Thus, Ward’s method was chosen in order to characterize the similitudes and the discrepancies highlighted between objects that form different clusters [
43]. The method involves the minimization of the error sum of squares (also known as inertia) for the objects in the clusters, the error being calculated as the distance between an object and the center of the class it belongs to.
Thus, the hierarchical clustering approach led us, after successive calculations of error sum of squares, to a first classification of countries into three different groups that differ significantly relative to the main issue addressed by our research but at the same time, gather homogeneous elements within each group. The output variable defined in this manner reflected the membership of each country to one class or another, and these classes were symbolized by Class1, Class2, or Class3. The neural network developed in Stage 4 was aimed at improving the first taxonomy generated by the cluster analysis by minimizing the probability of including a country into an inappropriate class.
Stage 3: The individualization of three datasets. As required by the neural network training process, we divided the input data into three categories: training data, validation data, and test data. The percentages for each type of data were to be estimated the moment that the model was defined: 60% from the input data used to train the neural network, 20% was used to validate the results, and 20% was used to test the neural network after training.
Stage 4: The training of the neural network. The neural network employed in order to classify the EU countries according to their willingness to embrace teleworking was a multilayer perceptron network using a group of ten variables on remote work (denoted by I1-I10) and an intermediate layer with six neurons as input data. The neural network was trained with the help of SAS Enterprise Miner, version 15.2.
The activation functions for the neural network were defined by the following formulae [
44]:
The activation function belongs to the hyperbolic tangent type, and it was applied to the input-to-hidden layer weights, which had and exponential distribution that was in a range between [−1, 1]. The formula applied for the activation function is presented below [
45]:
The rationale behind choosing the hyperbolic tangent function for our model was related to the fact that it leads to faster convergence in the optimization process, as compared to various logistic functions.
Because we only used one output variable—the assignment to one class of countries or another—there was no need for several hidden layers to be placed in the architecture of the neural network. The number of neurons in the hidden layer was determined experimentally, the appropriate results being obtained for the six-neuron alternative.
We chose the Levenberg–Marquardt algorithm to train the neural network because of its stability, celerity, and high level of convergence. The technique aims at minimizing the sum of the squared errors of the hyperbolic tangent function (Tanh) and incorporates both the steepest descendent algorithms and the Gauss–Newton method, as depicted in Equation (4) [
44]:
where:
J(n) is the Jacobian matrix of first derivations of the error function with respect to each weight estimate;
A(n) is the second derivative of the residual squared sums of the parameter estimates at the nth iteration;
r(n) is the vector of the residuals at the nth iteration.
Stage 5: Final classification of EU countries provided by the neural network.
From the literature, we know that neural networks have the potential to indicate object membership to a pertinent class more accurately then any of the traditional hierarchical clustering methods [
46]; the completion of this stage was aimed at improving the precedent systematization acquired in Stage 2.
In order to ensure the learning process, the neural network needs to be provided with several patterns, which arise from a previous classification. This previous systematization may be instituted either in an empirical manner or as the result of another analysis—in our case, cluster analysis.
By showing the number of classes that could be highlighted in any particular case, we assume a certain level of classification error. Through the training of the neural network, we seek to minimize this error by reaching a categorization that is as close as possible to the real situation while providing the values of the network transfer functions.
Consequently, a future classification would not imply additional research efforts in order to train the network, but the values associated with a specific country that did not take part in the initial training process could simply be added, and the country would be assigned to the proper group by taking the transfer functions into consideration.
In the specific case of the neural network developed in our research, the number of neurons included in the input, hidden, and output layers were 10, 6, and 3, respectively.
As described in
Section 3.1, the network that generated the lowest value for the mean squared error (MSE) in the validation dataset represented the optimum choice for our research, as it ensured the prediction rigorousness for the model. The training process continued until the network accomplished the closest output value to the sought-after output by continuously modifying the unit weights correspondingly [
47].
Stage 6: Following the successful training of the neural network, a few scenarios and practical implications were drawn in order to increase the level of telework adoption by Romanian employees, on the strengths of both the global tendencies highlighted in paragraph 2 and the unprecedented challenges of the pandemic.
3.3. Data and Measures
The answers provided by employees from various fields and that were registered between 22 June and 27 July 2020 were classified into the following categories: work–life balance, quality of work performed, work experience at a distance, and the health and safety of employees [
2]. For comparative purposes, the datasets involved in the research are percentages within the range 0 to 100 and reflect the share of respondents that totally agree or agree with a specific statement/question (included in the Eurofund questionnaire).
The respective shares assess employee willingness to adopt the new legal arrangements and procedures that facilitate telework. The variables extracted from the 2020 Eurofound survey that were assumed as highly relevant from the point of view of reflecting the degree at which employees adopted telework (
Table 1) were used in order to substantiate the architecture of our neural network.
The first variable, symbolized by I1—high and very high satisfaction on the amount of work submitted—was assessed based on the answers provided by the respondents to the following statement: “I am satisfied with the amount of work I have to do”, with the answer options: totally agree (1), agree (2), indifferent (3), disagree (4), totally disagree (5), do not know/do not answer (6). The response analysis illustrated a high percentage of respondents who agreed and totally agreed with the abovementioned statement; the sums of the percentages determined for each EU country are displayed in the second column of
Table 2.
The I2 variable extracted from the survey—high and very high satisfaction on the quality of work submitted—was estimated on the grounds of the statement “I am satisfied with the quality of work I have to do”, which had the same answer options as the previous question. The sum of the percentages provided by the answers “totally agree” and “agree” was also kept further among the datasets of our model.
The I3 variable—work in optimal conditions with the equipment from home—was assessed by virtue of the statement “With the equipment I had at home, I was able to carry out my work in optimal conditions”, with the same options as the previous questions that were used in the Eurofound research. The percentages provided by respondents who agreed and totally agreed with the statement are summarized in
Table 2.
The variable denoted by I4—satisfaction with the experience of working from home—follows the same construction reasoning as the above, cumulating the answers agree and totally agree with the statement “Overall, I am satisfied with the experience of working from home”.
The variable symbolized by I5—work normally involves physical contact with other people most of the time, quantifies the affirmative answers to the question: “In your activity, you are in direct contact with other people (colleagues, clients, passengers, students, patients and others)?” Other possible answers to this question were the following: sometimes, rarely, never, and do not answer.
The I6 variable—maintaining constant work performance—quantifies the share of respondents who have chosen the alternative “is kept constant” to the question “During the COVID-19 pandemic, how did the performance in the submitted work change?”, other possible options for answering the abovementioned question were: increased a lot; increased to a lesser extent; decreased a lot; decreased to a large extent; do not know/do not answer.
The I7 variable—the risk of SARS-CoV-2 transmission in the workplace quantifies the affirmative answers of the respondents with respect to their perception on the magnitude of the threat of becoming infected with the new coronavirus while accomplishing their work tasks.
The I8 variable—not accepting work from home before the pandemic—assesses the share of employees belonging to companies inside of which the organizational culture did not integrate remote work arrangements before the health crisis.
Keeping the same work schedule during the pandemic, I9, estimates the respondents’ opinions regarding the modification of their work schedule compared to status-quo before the forced adoption of remote work. Thus, the figures displayed in
Table 2 revealed that, in a very large proportion of people, the same work schedule as before the outbreak of the pandemic was maintained.
Finally, non-involvement of family issues and duties in performing work tasks, denoted by I10, shows that the majority the people who completed the questionnaire declared that family problems and responsibilities did not prevent them from successfully conducting the obligations imposed by their job.
The initial database employed in order to achieve the first classification of countries with the SAS Enterprise Guide is displayed in
Table 2.
Once the variables were defined and the values recorded for each variable were collected, the next stage of our research methodology implies the establishment of the initial classification of countries by virtue of the SAS Enterprise Guide. The results are presented in the paragraph below.