Criteria Weights in Hiring Decisions—A Conjoint Approach

Abstract

Understanding human behavior in the decision-making process represents a challenge for researchers in the socio-economic field. The complexity comes from multiple criteria acting simultaneously. Hiring decisions are made on a set of criteria representing the attributes of the applicants. This study’s main objective is to investigate Romanian employers’ behavior when recruiting for jobs targeting graduates from economic studies. The method used to identify the weights employers assign to different skills was based on an experimental technique-choice based conjoint. A survey experiment was conducted to produce causal conclusions about the recruiting process. The estimation was performed with a methodology based on machine learning, which allows to investigate interactions between subjects’ characteristics and conjoint criteria. The findings of our experiment align with other studies pointing to the increased relevance of non-cognitive skills for employability. Additionally, our results show that criteria weights in hiring decisions depend on company size, ownership, activity sector or personal characteristics of the recruiter. Our research provides a mechanism for understanding employers’ perspectives. This is valuable for informing job seekers to adjust their job search strategies and to invest in the skills offering hiring opportunities. Moreover, universities can use the results to adapt their educational programs to labor market needs.

1. Introduction

In the context of technological developments, the efficiency of the education system is imperative, and the match between educational supply and labor market requirements represents a key element within this process. This study’s main objective is to investigate Romanian employers’ behavior when recruiting for jobs targeting graduates from economic studies, and to identify those skills that education providers should focus on to facilitate the economists’ integration into the labor market. For fulfilling this objective, the utility employers assign to the characteristics of the graduates coming to the hiring interview is estimated through microeconometric techniques specific to consumers’ theory preferences.

Allocation on the labor market reflects a complex process of matching the demand and supply of labor. The demand is given by employers recruiting for job vacancies, while the supply is represented by job seekers applying for employment. While the job seeking has been extensively studied, employers’ hiring behavior received less attention from researchers. Employers are active agents whose hiring decisions are shaped and enabled by social, organizational, and institutional factors [1]. The main theories explaining how employers make hiring decisions can be grouped into competency-based, status-based, and social closure–based approaches [2]. In all these theoretical perspectives, employers are utility maximizers who make decisions thorough systematic and constrained assessments on workers’ skills and potential productivity. In more recent views, such decisions are also shaped by the emotions, biases, and identities of employers [2,3,4]. Employers’ hiring decisions are based on three main sources of information that are related to human, social, and cultural capital [1]. Previous studies show that employers value attributes that signal a high proficiency of job-relevant skills [5,6]. Among such attributes, education credentials are very important as they signal pre-school abilities. Moreover, previous studies found that educational credentials influence hiring decisions even after controlling for abilities, suggesting that they also signal other attributes, such as perseverance and motivation [7]. However, for employers who view education as a noisy signal, referrals of applicants are more important, confirming the role of social networks to compensate for poor signaling [8]. In addition, in contexts when the signals are not very clear, employers are likely to use alternative information such as socio-economic status [9] or the group identity of applicants [10]. Moreover, studies have shown that the importance of the attributes taking into consideration by employers varies in relation to the status of the target job [11].

Understanding human behavior in the decision-making process represents a challenge for researchers in the socio-economic field. The complexity comes from the collection of criteria acting simultaneously. In a traditional approach, decision-makers are asked directly which are the most important criteria influencing their decisions. The major drawback of this procedure is coming from the fact that it does not take into consideration the eventual trade-offs interfering when decisions have to be made on multiple criteria. In 1964, the psychologist Luce and statistician Tukey developed a new technique where the factors influencing a decision are considered jointly [12]. Thenceforth, this method, known as conjoint, became a vital marketing analysis tool employed for utility estimation. Recently, the technique was extended to understand decision-makers’ behavior in many other domains.

Conjoint analysis is based on the economic theory regarding preferences and utility [13]. Usually, it is used to measure consumers’ preferences and to estimate the probability that a new product will have success. The methodology behind this approach is based on the consumers’ evaluation of some product profiles obtained through variations of their characteristics such as dimension, price, brand, and cover. Respondents’ attitudes towards the generated profiles are modeled using statistics and econometrics to estimate how much each attribute weighs in the decision process [14].

In the past, the conjoint technique was successfully used in economics, psychology, health, and medicine to identify the best practices, products, and strategies. The results offered objective means for assessing and improving methods, products, or programs.

Even though conjoint analysis was developed more the 30 years ago, the tools employed in the estimation are still evolving and adapting [15]. Therefore, there are multiple solutions defined for the estimation stage: logit and probit models [16,17], the Bayesian approach [18,19], nonparametric regression [20,21], machine learning algorithms [22], and latent variables [23].

The conjoint method provides the framework for understanding the decision-making process when multiple attributes describe the feasible alternatives. Hence, during the past years, it proved its efficiency in many domains; the education and labor market are among them.

In the studies related to the labor market, this approach was used to measure employers’ preferences regarding the skills of their desired candidates. A study developed in the Netherlands showed that when hiring graduates in the public health system, the recruiters appreciate the general competencies instead the specific ones [24]. Another study investigating Dutch employers emphasized that age, gender, and origins are the most critical selection criteria [25].

A choice-based conjoint design related to the labor market access topic investigated the characteristics that foster immigrants’ integration in Germany. The findings revealed the features signaling employability [26].

A meticulous research based on two discrete choice conjoint experiments was developed in 2012 to depict those characteristics associated to labor market success for higher education graduates [27,28]. There are nine European countries included in these studies. The first experiment mimics the CV-selection process and the second the hiring process. The respondents are employers who evaluated hypothetical candidates’ profiles applying for jobs in the respondents’ companies. The estimation results underlined that teamwork and communication are the skills most wanted by employers. The relative importance of these attributes is similar to the one associated with professional expertise. These findings underlining the huge importance of interpersonal skills are also supported by the conjoint study developed in 2010 in SUA for the hospitality industry [29]. In this former study, the conjoint experiment is based on ranking hypothetical profiles instead of choosing the preferred one. Successively choosing one of two profiles is more efficient, positively affecting prediction quality as compared with this approach [30].

Romania was not included in the before mentioned study developed in nine European countries. Therefore, the research presented in this paper will identify the attributes higher education graduates should possess for successful integration into the Romanian labor market. The investigation undertaken focuses on a more homogeneous segment: higher education graduates from economics. According to Eurostat, in Romania, in 2020 tertiary education graduates from economics represented 2.4% of the total number of tertiary education graduates. This study’s results will support the institutions responsible for a better matching between labor supply and demand.

In the education field, a conjoint approach was used in a university in Poland to investigate students’ choices in terms of study domain [31]. The applicability of the conjoint method to evaluate teachers’ performance in higher education was tested at Belgrad University [32,33]. Data collection focused on students’ preferences for a set of factors considered to influence the quality of teaching. In this case, the main contribution of conjoint approach was related to restricting the input and output variables used in a DEA efficiency measurement.

The results returned by the conjoint method were used in Korea to set the weights for the criteria dictating the score for college admission. The criteria used to build the profiles were represented by the following: test, high school grades, essay, and interview. In this case, the evaluators were teachers, parents, and students, and they were tasked to sort out eight profiles [34].

Another study related to our topic identified that physical appearance affects the perception of capabilities. The attributes used in the experiment were: the color of the clothes, conservative versus trendy attire, professional versus casual attire, and body modification (including tattoos and piercings). Within this experiment, the profiles were presented using laminated photos. The results indicate that grooming and professional attire are the most relevant characteristics producing favorable perceptions [35].

In Romania, there is only one study developed in this field based on conjoint. It investigates the attitudes of higher education graduates concerning preferred job characteristics [36]. The questionnaire used in this study asked the respondents to sort out a list of jobs. Estimation results illustrated that net wage is the most important attribute of a job. The other attributes (type of work, the match with the education domain, the match with education level, promoting opportunities) have similar importance coefficients. The differences become more significant when comparing genders, education levels and occupation status. Other recent studies based on conjoint were developed in Northeastern China to investigate the career selection process. Unlike the previous work, relevant urban environmental attributes were chosen to describe potential jobs [37].

The examples presented in this section prove the utility of conjoint analysis for understanding the decision-making process in education or labor market.

2. Methodology and Data

The main concepts conjoint analysis operates with are as follows: attribute, profile, or alternative. The attributes are the characteristics defining a product. Each attribute has a set of possible states that we call levels. A certain combination of levels set for each attribute will reveal one profile. In conjoint analysis, a set of profiles is generated. The respondents will evaluate pairs of different profiles through several tasks. The estimation will be undertaken on a number of observations given by the sample size multiplied by the number of tasks per respondent. To estimate the value associated with each level, the conjoint approach uses the respondents’ reactions toward the evaluated profiles.

In a conjoint experiment, the decision-making behavior is observed by two means: sort or choice. In the first approach, the respondent’s task is to sort a set of profiles meanwhile the second one involves choosing the most preferred profile. It is considered that the second approach reflects a more accurate real-life decision process. Moreover, the task is less demanding given that the evaluation is performed on a reduced number of profiles at a time. In a choice-based conjoint, the respondent receives two or more profiles from which he/she has to select one. This task is repeated for multiple combinations of profiles.

Traditionally, to investigate how much each attribute weights in the decision process a conditional logistic regression was fitted.

The main estimation methodology behind multiple choice modeling based on conditional logit belongs to Mcfadden [16]. In a general framework, the choice behavior model is described by: the universe of possible alternatives, denoted by X and the universe of the vectors of the decision makers’ attributes, representing individual characteristics denoted by W.

We observe the choice behavior of an individual i given by the characteristics $w_i$ when the available alternatives are A = $\left\{a, a_2\dots a_J\right\}\subseteq X$.

Then, the conditional probability that this individual will choose alternative j is denoted by:

$$P(a_j|w_i,A),$$

(1)

The observed choice given the choice set A and the attributes $w_i$ is considered to be drawn from a multinomial distribution with selection probabilities $P(a_j|w_i,A)$.

Assuming that the utility provided by the jth profile has the following linear form:

$$U_{ij}=V\left(a_j,w_i\right)+\epsilon \left(a_j,w_i\right),$$

(2)

In the previous equation, V is linear function, nonstochastic, representing the population preferences and $\epsilon$ is a random variable reflecting idiosyncrasies of decision-maker i in tastes for the profile $a_j$.

The respondent will select the profile j when $U_{ij}=\underset{k\in J}{max}{U}_{ik}$. The statistical model built to estimate the influence of each criterion arises from the probability that alternative j is the most wanted:

$$P\left(U_{ij}>U_{ik}\right),\forall k\ne j,$$

(3)

Therefore, the probability that a respondent with attributes $w_i$ will select $a_j$ given the subset A becomes:

$$P\left(a_j|w_i,A\right)=P[\epsilon \left(a_k,w_i\right)-\epsilon \left(a_j,w_i\right)<V\left(a_j,w_i\right)-V\left(a_k,w_i\right) for all k\ne j,$$

(4)

Making assumptions about the distribution of the random variable $\epsilon$, [5,28] in terms of binary odd will obtain:

$$P\left(a_j|w_i,A\right)=\frac{e^{V\left(a_j,w_i\right)}}{{{\displaystyle \sum }}_{k=1}^J{e}^{V\left(a_k{w}_i\right)}}$$

(5)

This is the mathematical formalization of the conditional logit model [38,39].

The utility respondent i associates to profile j depends on the profile’s characteristics and the respondents’.

This traditional approach was recently developed to combine conjoint analysis with the potential outcomes framework, in the context of causal inference [40,41,42]. A new causal estimand was proposed namely the average marginal component effect (AMCE) that shows the marginal effect of a particular attribute over the joint distribution of the other attributes [40,41,42,43]. In this new methodology, no assumptions are imposed regarding the functional form of the behavioral model. Therefore, we have the following experiment: a sample of N respondents indexed by i receive R choice tasks (rounds). Each task simulates a decision-making process where respondents have to choose a preferred alternative from different options, where options vary across two or more attributes. The attributes are L categorical variables. Additionally, within the causal inference framework, a profile defined by the specific values assigned to these attributes it is considered a treatment with L components. Moreover, the potential outcome for a profile j evaluated by respondent i in round r is determined by a function [40,41,42,43,44]:

$$\begin{array}{l}{Y}_{i,j,r}\left(c_l,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)\\ =f\left(S_i\left(c_l,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right),R_{ir}\left(c_l,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right),P_{ijr}\left(c_l,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)\right)\end{array}$$

(6)

where

-

$c_l$ is the value of the lth attribute in the profile j evaluated be individual i in round r of the experiment

-

$C_{ijr\left[-l\right]}$ is a vector capturing the values encountered by the other attributes in the profile j

-

$C_{i\left[-j\right]r}$ is the set of the possible alternatives

Moreover, there are three random components of function f, namely:

-

$S_i$ is the random component at the respondent level

-

$R_{ir}$ is the random component at round level

-

$P_{ijr}$ is the random component at profile level

Using these concepts and notations, AMCE becomes [40,41,42,43,44]:

$${\tau }_l=E\left[Y_{ijr}\left(c_l=l_1,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)-Y_{ijr}\left(c_l=l_0,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)\right]$$

(7)

It measures an average behavior of the respondents concerning each attribute of the profiles. To understand the variations of these preferences depending on the subjects’ characteristics, AMCE has to be decomposed into lower-level effects, mainly individual level marginal effects (IMCE) [41,43,45]. The importance of moving from the average preference towards understanding the distribution of individual level predictions is underlined in recent researches [43,44,45]. This debate relates to the estimation of heterogeneity in conjoint experiments. IMCE measure reflects how the probability of selecting one profile changes for the respondent i. It is obtained by conditioning the AMCE estimand on the random component at respondent level $S_i$:

$${\tau }_{il}=E\left[Y_{ijr}\left(c_l=l_1,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)-Y_{ijr}\left(c_l=l_0,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)|S_i\right]$$

(8)

N-individual level effects are estimated by subsetting the data on the subject identifier. Therefore, this approach allows us to investigate the variation of IMCE with respect to the covariates.

Given that each individual makes choices across multiple rounds, the IMCE can be decomposed to extract the round level marginal component effect (RMCE). It will measure the effect of a component within a specific round r of the experiment for subject i:

$${\tau }_{irl}=E\left[Y_{ijr}\left(c_l=l_1,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)-Y_{ijr}\left(c_l=l_0,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)|S_i{R}_{ir}\right]$$

(9)

Furthermore, by conditioning on the profile level random component, the observation level marginal component effect (OMCE) could be estimated:

$${\tau }_{ijrl}=E\left[Y_{ijr}\left(c_l=l_1,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)-Y_{ijr}\left(c_l=l_0,C_{ijr\left[-l\right]},C_{i\left[-j\right]r}\right)|S_i{R}_{ir}{P}_{ijr}\right]$$

(10)

In order to estimate these quantities and understand heterogeneity in respondents’ behavior we used a new strategy based on machine learning tools [44]. It allows us to investigate interactions between subjects’ characteristics and conjoint attributes without imposing a functional form of these interactions. In a nutshell, following this strategy, the estimation of IMCE involves training a model where the outcome is a binary variable, expressing whether subject i selected profile j in round r. The attributes of the profiles and the respondents’ characteristics are the explanatory variables. Therefore, the trained model is used to predict counterfactual outcomes at the observation level giving the OMCE. Finally, by aggregating the OMCEs estimates, IMCE’s are computed. The main steps undertaken are as follows [44]:

Step1. Training a model to find an estimation $\widehat{f}$ of the function f capturing the true ddata-generating process:

$$P\left(Y_{ijr}=1|C_{ijr},W_i\right)=f\left(C_{ijr},W_i\right)$$

(11)

where $Y_{ijr}$ is the observed binary outcome, $C_{ijr}$ is a vector showing the attributes levels for profile j in round r and $W_i$ is a vector characterizing respondent i.

A nonparametric approach is used to estimate function f, namely Bayesian Additive Regression trees (BART) [44,46]. Comparative to other estimation methods, BART features offer the flexibility in capturing interactions and non-linearities. Moreover, the estimation approach distinguishes from other ensemble of tree models because it relies on a Bayesian procedure enabling uncertainty estimation.

In a general framework, if we need to make inference about the function f:

$$Y=f\left(x\right)+\epsilon , \epsilon ~N\left(0,{\sigma }^2\right),$$

(12)

a methodology based on BART will approximate it by a sum of m trees:

$$Y={\displaystyle \sum }_{t=1}^{m}g\left(x;T_t,M_t\right)+\epsilon$$

(13)

where

-

$M_t=\left\{{\mu }_{1t},{\mu }_{2t},\dots {\mu }_{bt}\right\}$ is the set of parameters values associated to the b terminal nodes of tree $T_t$.

-

$g\left(x;T,M\right)$ is the function which assigns a value $\mu$ to $x$.

Each sum of trees model is determined by $\left(T_1,M_1\right), \dots , \left(T_m,M_m\right),$ which includes the parameters of the terminal nodes, tree structures and splitting criteria and the error variance $\sigma$. A prior over all these parameters is imposed when the model is built. The aim of the prior is to provide regularization, diminishing the influence of any single regression tree [46].

For the investigation developed within this paper, an extension of BART is used to deal with a classification problem:

$$p\left(x\right)=P\left(Y=1|x\right)=\mathsf{\Phi }\left[G\left(x\right)\right]$$

(14)

where $G\left(x\right)={{\displaystyle \sum }}_{t=1}^{m}g\left(x;T_t,M_t\right) and \mathsf{\Phi }\left[·\right]$ is the cumulative density function of the standard normal distribution [46].

This represents a probit model setup. In this classification problem, the implicit assumption is that $\sigma =1$. The priors for the remaining parameters imposing the tree structure have to contribute to keeping individual tree effect to a reduced level. As demonstrated with many examples, the following default specifications are very effective [46]:

-

The probability that a node at depth d is nonterminal is given by the prior:

$$\alpha {\left(1+d\right)}^{-\beta }, \alpha \in \left(0,1\right), \beta \in \left[0,\infty \right)$$

(15)

The hyperparameters $\alpha$ and $\beta$ have the default values 0.95 and 2.

-

The prior for each leaf node s in tree t is defined as:

$${\mu }_{st}~N\left(0,{\sigma }_{\mu }^2\right), \mathrm{where} {\sigma }_{\mu }=\frac{3}{k\sqrt{m}}$$

(16)

The recommended default value for the hyperparameter k is 2.

-

Concerning the choice of m, in the scientific literature, it is underlined the importance of avoiding choosing m too small. The default value m = 200 produced strong predictive performance on a vast collection of conjoint experiments.

For the observed data Y, the Bayesian setup for the sum of trees model, defines the posterior distribution $P(\left(T_1,M_1\right), \dots , \left(T_m,M_m\right), {\sigma }^2|Y)$. A backfitting Markov Chain Monte Carlo algorithm is used to sample from this posterior. One iteration within this procedure involves that for every tree t in the model:

-

computes the partial residuals $R_{-t}$ showing the part of the variance left unexplained by the other m – 1 trees.

-

updates its structure to improve performance over $R_{-t}$.

One of the main features of the BART model is that each tree explains a small and different part of the outcome variance. Therefore, during training, the algorithm updates each decision tree by taking into account the performance of the others trees. At each step in every iteration, the tree t is updated by performing Metropololis- Hastings draws from the posterior of the tree distribution. This means that small changes are probabilistically introduced in its structure: splits a terminal node (p = 0.25), prunes a pair of terminal nodes (p = 0.25), changes a non-terminal splitting rule (p = 0.4), swaps split criteria between parent and child (p = 0.1) [44,46,47].

Over many MCMC iterations, trees are revised to minimize the errors between predictions and observed data.

Step2. The estimated function $\widehat{f}$ is used to compute the counterfactual outcome by changing the values of the attribute’s levels. In this step, observation-level marginal component effect (OMCE) is estimated by taking B draws from the training dataset, where each element in the column of attribute l, is set to $l_1$ and then to the reference category $(l_0)$. For each observation we obtain an approximation of the posterior distribution for $l_1$ and $l_0$. The parameters of the model are random variables. Therefore, the predictions will represent an average over B draws from the model posterior predicted values.

Where

$$\begin{gathered} {\widehat{y}}_i=\frac{1}{B}{\displaystyle \sum }_{b=1}^B{\widehat{y}}_i^{\left(b\right)} \\ {\widehat{y}}_i^{\left(b\right)}={\displaystyle \sum }_{t=1}^m{\widehat{g}}_t\left(x_i\right) \end{gathered}$$

(17)

The average of the B predictions will yield to the OMCE.

$$OMCE={\widehat{\tau }}_{ijrl}=\frac{1}{B}\left(\widehat{f}\left(C_{ijrl}=l_1,W_i\right)-\widehat{f}\left(C_{ijrl}=l_0,W_i\right)\right)$$

(18)

Step3. The estimation of IMCE is given by the average of the OMCEs for each individual i:

$$IMCE={\widehat{\tau }}_{il}=\frac{1}{R\times J}{\displaystyle \sum }_{r=1}^R{\displaystyle \sum }_{j=1}^J{\widehat{\tau }}_{ijrl}$$

(19)

As described in the next section, the experiment developed in our study, considers six tasks for each respondent, therefore the number of rounds R = 6. Also, each task presents two profiles, meaning that J = 2, built with six attributes (L = 6).

To conclude, in order to benefit from the advantages of the nonparametric approaches and the flexibility of machine learning algorithms, the estimation of the attributes’ influence in the decision process, is performed with the latest technique based on Bayesian Additive Regression Trees [44]. Additionally, this approach has a vast potential for further development.

Data collection was performed through a sociological investigation within 510 Romanian companies using a stratified sample design. The stratification variables are the size class and activity domain. The questionnaire simulates the recruiting process by giving the respondents the task of selecting from different candidates’ profiles the one they would hire. Data collection was undertaken online, the respondents being represented by managers or human resources staff. The survey experiment was conducted in Romania in 2014. Filtering questions were used to ensure that only companies that had been involved in recruiting a higher education graduate from economics in the past 3 years or planning to recruit in the next 3 years were eligible to participate in the experiment. Moreover, at the moment of the survey, all the companies had employees graduated from economic studies. All 42 Romanian counties were represented in the sample and 72.5% of the companies are private-sector companies with domestic capital.

The selection of the attributes used to define candidates’ profiles was based on the findings returned by a previous research project. Within this project, 400 employers were directly asked which are the key competencies they look for when hiring economic studies graduates. They had to evaluate 11 competencies: English communication, communication in other languages (except English), computer skills, communication skills, analytical and problem-solving skills, ability to adapt and act in new situations, decision-making, teamwork, planning and self-organization, initiative, and intercultural skills. For the conjoint experiment, we have kept the first most selected competencies by Romanian employers.

The key competencies used in the conjoint study are the following: computer skills, communication skills, analytic and problem-solving, decision-making skills, teamwork, and English communication.

For each attribute, we have considered two levels: average and above average. The arguments sustaining this decision are as follows:

-

To reduce the number of possible profiles as much as possible. In this case the total number of possible profiles is 64. Data quality could be affected if the respondents would receive too many evaluation tasks. Obviously, this issue is correlated to the number of levels per attribute [18].

-

To have the same number of levels for each attribute, to avoid the drawback of overestimated importance for the attributes with a higher number of levels [48,49].

-

It provides sufficient information to understand the trade-offs made by decision-makers.

To summarize, the investigation will reveal how much employers value cognitive skills like computer skills, analytical and problem-solving skills, and English communication. At the same time, we will find out how important some non-cognitive skills such as communication, decision-making, and teamwork, are.

Each respondent received six selection tasks. Each time they had to choose one candidate between two hypothetical profiles. To ensure sufficient variation in the alternative set, the sample was split into five subgroups of size 102, known as blocks. Each subset received a different questionnaire meaning the evaluated profiles were different. Hence from the 64 possible profiles, we randomly selected 30 which were randomly combined two by two to produce the tasks.

The tasks in the conjoint experiment are designed as suggested by the capture in Appendix A and Figure 1. The English translation of the conjoint vignette is the following: “Assuming that you have to recruit a young graduate of economics for the position you have mentioned before (in the questionnaire was a question regarding the position in the company occupied by economics graduates). Read the two profiles carefully and select the one you prefer”. Each profile is described by the levels of the six attributes. For an expressive visual representation, in the original vignette, the average levels are coloured orange meanwhile the above average are in red.

Each respondent receives six tasks of this kind. Obviously, the profiles are distinct each time. Moreover, the order of the attributes changes each time. As a mechanism to avoid automatic answers by keeping the respondent active, the profile presentation at each task is altered in the sense that the order of the attributes is modified.

The conjoint experiment developed within this paper will reveal the contribution of each of the six attributes previously identified to the employers’ final decision. To implement a solution to carry out this exercise we used the “support. CE” library in R [50,51].

The functions included in this package allowed us to undertake the main steps necessary in the development of a conjoint experiment:

-

Generating the complete set of profiles where each attribute can take two possible outcomes: “Average level” or “Above average level”. The size of the set is two raised to power six.

-

Randomly select 30 profiles from the set of 64 profiles. In this step the following restrictions were established: (i) the dominant or leading profile was excluded; (ii) the inferior/the dominated profile was excluded; and (iii) only the profiles having at least two attributes on the same level were kept. The superior or dominant profile is a profile where each attribute takes the level “Above average level”. This is the profile that will be preferred over all others. At the other extreme, the inferior profile is the one receiving “Average level” for each position. To explain the third restriction, let’s consider an example. If a profile is defined by the vector (Average Level, Average Level, Average Level, Average Level, Average Level, Above average) it has a high probability of being dominated by a profile having at least another position changed to “Above average”. Hence, the constraint imposing at least two elements of the vector with the same value reduces the chance of comparing one profile which definitely dominated the other.

-

Generating the choice experiment design consisting of five blocks each of them containing six tasks. The list consisting of the 30 selected profiles represents the argument of the function generating this matrix. There are two available generation methods implemented in “support.CE” library, namely: the rotation method and the mix and match method. When using the rotation method, the first alternative in each choice set is an orthogonal array randomly extracted and by adding one to each attribute level the second option is produced. This means that if the first alternative of a task is given by the vector (Average, Average, Average, Above Average, Above Average, Average) the second one will become (Above Average, Above average, Above average, Average, Average, Above Average). The mix-and-match method operates on a principle where each alternative is randomly extracted from two separate urns. The first urn contains the profiles representing the first alternative in the previous method. In a different urn are placed all the profiles generated with the rotation method. A choice set is designed by pulling out one alternative from each urn at random. In our study, the choice experiment design was created with the rotation method. We did not select the mix-and-match method because it has a higher probability of generating a choice set where one alternative dominates the other. This is a repercussion of the fact that this method could produce alternative profiles that differ only by the level encountered by one attribute. The main output in this stage is 30 choice sets, which were randomly divided among five blocks.

-

Transposing the choice design into a questionnaire. The matrix supporting the choice tasks in the conjoint experiment is presented in Figure 2. For each block, there are 12 lines because, for each of the six tasks, there are two vectors illustrating the choice set.

The first column indicates the block, the second column indicates the choice task inside the block. Within each block, there are six questions. Column 3 specifies the alternative number. The last six columns represent the levels defining the six criteria. Therefore, the first row embodies the following information: in the first block, the first question asks the respondent to evaluate the profile of a graduate having average English communication skills. Average IT skills, average communication skills, above-average analytical and problem-solving skills, above-average decision-making skills, and average level of teamwork skills. The respondents receiving the questions in block 1 will have to select between this profile and the one detailed in the second row.

The information presented in the previous matrix has to be matched with the responses data set (Figure 3) in order to perform conjoint analysis estimation.

In this matrix in each row are registered the choices made by each respondent. For example, the respondent with ID 1 selected the first profile in question 1.

Finally, the training data is given by the matrix resulting from combining the previous two. In the final training data set, a number of lines equal to the number of questions multiplied by the number of alternatives presented within each task are assigned to each respondent. Therefore, the final data set will contain 6120 lines. The columns of the matrix contain the following information: respondents ID, the attributes defining the profiles, the outcome taking a logical value indicating whether the respondent selected the option presented in that line or not, respondents’ characteristics. The last columns comprise: the size of the company, company ownership, company activity sector and the gender of the respondent.

3. Results

The data set described in the previous section comprising 6120 observations, was the training set, investigated with the machine learning methodology implemented in cjbart library in R [44]. In order to estimate the individual level marginal effects (IMCE), we used m = 200 trees, and B = 1000 draws. The estimated coefficients presented in

Table 1. Average marginal component effects.

Level	AMCE	Min.	Max.	Std.Dev
ENabove	−0.011	−0.101	0.063	0.026
ITabove	0.144	0.054	0.205	0.033
COMMabove	0.136	0.053	0.175	0.027
PSabove	0.281	0.229	0.316	0.014
DECabove	0.163	0.012	0.340	0.072
TEAMabove	0.157	0.095	0.212	0.033

show the results of the BART model in terms of average behavior.

Excepting the AMCE coefficient estimated for the communication in English criterion, all the others are assigned with a positive quantity. It seems that employers will accept an employee who does not communicate perfectly in English but who has other skills at an above-average level.

Average marginal components effect estimates yield the conclusion that analytical and problem-solving skills is the criterion to which employers attach the greatest importance in the recruitment process of economics graduates. The chances of being selected by a firm recruiting for positions requiring economics studies are higher for graduates who demonstrate above-average analytical and problem-solving skills. Understanding problems and identifying robust solutions for them are abilities that employers highly value. Analytical skills are also related to proficiency in using data analysis techniques [52]. Therefore, this criterion is a mixture between noncognitive skills and cognitive skills.

Strongly related to problem-solving are the decision-making and teamwork skills. Hence, these attributes are coming on the two subsequent positions relative to their impact on hiring decisions. Possessing decision-making skills above the average level of the population from which the recruitment is made increases the chances of being recruited. Additionally, the chances of being selected increase for graduates who demonstrate above-average teamwork skills. No matter the industry, honest and good collaboration with others is correlated with a robust structure.

Computer skills come in the fourth position after problem-solving, decision-making, and teamwork, revealing that soft skills are generally more valuable for Romanian employers. This could be explained by the fact that many employers offer on-the-job training for specific software applications, therefore basic computer skills are often acceptable.

The ability to give and receive information, captured by the communication skills attribute is valued almost as much as computer skills are. Communicating in a precise and efficient way is always a requested skill independent of the methods, field, or period.

However, the significant contribution BART methodology brings to our investigation has to do with the heterogeneity investigation. The main findings are presented in the next figures plotting the ordered distribution of the estimated IMCEs, colored by the characteristics of the companies or personal characteristics of the respondents. We have included only the plots presenting the attributes exhibiting non-random heterogeneity. Hence, Figure 4 showing point estimates for IMCEs with 95% Bayesian intervals underlines a different behavior of the large companies compared to small and medium ones concerning the recruiting process. Recruiters representing large companies appreciate more the analytical and problem-solving proficiency meanwhile, they value less communication and English skills.

Representation in Figure 5 reveals an interval heterogeneity by the company ownership for English communication attribute. Private owned companies with foreign capital need employees showing good English communication skills more than domestic companies. The IMCEs distribution in Figure 6 emphasizes that companies in the Services sector assign a higher importance to the communication skills.

As mentioned in the data description section, the training dataset also includes the gender of the respondents. The algorithm detected heterogeneity in IMCE by this covariate. Figure 7 illustrates how computer use and teamwork skills influence hiring decisions depending on recruiter’s gender.

4. Discussion and Conclusions

The efforts undertaken at the European level emphasize the need for information regarding the skills of young graduates from the employers’ perspective. This information is vital for the proper functioning of the labor market and for adequate policies. This study aims to support this initiative by investigating Romanian employers’ behavior when recruiting higher education graduates from economic studies. The originality of the approach mainly resides in adjusting quantitative techniques specific to marketing research to investigate labor market-related issues, namely conjoint experiment.

In the investigation of complex decision processes simultaneously involving multiple criteria, conjoint experiments findings are superior to those based on rank-ordering tasks or qualitative surveys because they simulate the real-world decision-making process by jointly considering the relevant factors or attributes. For example, in the hiring process, recruiters are forced to trade off some characteristics for others. Moreover, the specific data collection process increases the number of profiles included in the analysis compared to standard experiments since respondents in conjoint experiments evaluate multiple profiles with randomized attribute values. Additionally, this feature allows impact estimation at the individual level, which is relevant for heterogeneity investigation. Moreover, conjoint design reduces social desirability bias given the random generation of the vignettes [53]. Regarding the estimation method employed within this framework, BART approach brings the advantage of nonparametric estimation. This means that no functional form assumptions are required nor a behavioral model like the one maximizing utility is imposed. Hence, the bias produced by functional form misspecification is eliminated. The power of this method resides in combining machine learning features with the Bayesian inference. Thus, in the first place, BART is a sum of trees ensemble which enhances the ability to detect and model interactions and non-linear relationships. Second, the Bayesian approach brings a regularization prior is introduced to avoid overfitting. Within this framework, inference is acquired by successive iterations of a MCMC algorithm, known as backfitting algorithm [54]. The excellent practical performance of BART was demonstrated by comparing its estimation results to other machine learning algorithms such as random forest, neural networks, lasso, or boosting on a vast collection of data sets [44]. Another important positive aspect pf BART methodology, contrasting to other ML algorithms, is related to its stability over hyperparameter choices [55]. Finally, recent studies prove the utility of BART methodology within causal inference field where the observed outcome has to be compared with a counterfactual [55,56]. Furthermore, current research in conjoint analysis is concerned about its integration into the potential outcome estimation issue where the intervention or the treatment is represented by the attributes used in the design. All these arguments justify the suitability of BART estimation approach for conjoint data. To sum up, this method takes advantage of the data structure collected within a choice-based experiment, involving repeated observations across individuals, rounds, and profiles and captures heterogeneous treatment effects without dividing data into subgroups.

With respect to the limitations of our approach, even if BART methodology was previously used for modeling heterogeneous treatment effects in survey experiments [55,56], its implementation for conjoint designs is instead new [44,57]. Obviously, there are aspects that will be improved in the following years. For example, implementing random intercept modeling will enhance cluster-specific estimation needed for situations where respondents have the same characteristics and receive the same profiles for evaluation [44,58].

Related work in the field of investigating hiring preferences through the conjoint methodology was undertaken for: healthcare management field in the United States [59]; public health field in the Netherlands [24]; seven occupational fields (electro-technology, engineering, financial services, ICT, legal services, media and communication, and policy and organization) in nine European countries [27]; and in the hospitality industry in the United States [29]. Our investigation reveals findings extracted from the Romanian labor market in a different field. Specifically, this study shows employers’ perspectives when recruiting higher education graduates from economics. Compared to the before mentioned studies, our approach allowed the investigation of heterogeneity by employing machine learning tools in the estimation stage. Thus, our research contributes to a topical issue that emerged in conjoint analysis literature, namely the detection of heterogeneity, by testing a new strategy that disaggregates average level quantities to the individual level using machine learning methods [44]. A nonparametric estimator based on Bayesian Additive Regression Trees that yields outstanding predictive performance was used to understand preferences variation with respect to the respondents’ characteristics. Explicitly, by analyzing the distribution of the individual marginal effects we demonstrated that criteria weights in hiring decision depend on company size, ownership, activity sector or personal characteristics of the recruiter. Furthermore, our contribution in terms of the methodology consists in proposing a practical and simple framework that could be used to investigate the set of skills that makes a graduate more employable. Compared to similar studies undertaken in Europe [27,28], where respondents are presented with three full profile stimuli and a none option at a time, we focus on reducing the information processing burden on respondents by building choice sets with two alternatives. In the same direction of increasing the reliability of the results by reducing the fatigue effect on respondents, we have tested two solutions. The first one is based on the attributes being designed as two-level categorical variables. Regarding the second one, the choice experiment is divided into five blocks in order to cover a more comprehensive set of feasible profiles without increasing the number of tasks per respondent.

For the Romanian labor market, there is no previous research explaining employers’ preferences by the means of a conjoint experiment. An earlier work based on qualitative content analysis presents skill requirements in the Romanian business service industry [60]. Their findings are consistent with our heterogeneity results, showing the variation of skills needs across job categories stressing the need for research investigating specific job positions instead of a vast field.

Recently, employability skills topic is frequently approached through text-mining tools using data extracted from online recruiting platforms [61]. This represents a new valuable source of information offering objective and up-to-date intelligence. However, often job descriptions are rather exhaustive, presenting an extensive list of desired skills. Obviously, only some of them are critical. Given that the findings extracted using a conjoint approach reflect real-life situations where decision-making implies making concessions and trading off some aspects with others, this tool is suitable for understanding which criteria most affect individuals’ choices or decisions. Of course, the major drawback is related to the extra costs coming from data collection. For future improvements with respect to the attribute selection phase in a conjoint study, the results extracted from content analysis on the job description could be beneficial. Thus, future research will benefit from examining a narrow area by focusing on a specific industry and on particular company functions. The findings of our experiment align with other studies pointing to the increased relevance of non-cognitive skills for employability. According to previous studies, a combination of average digital skills and a high level of non-cognitive skills is believed to provide labor market success [27,61].

The outcome underlining the critical importance of soft skills is also confirmed by a study based on content analysis of job advertisements, and factor analysis on survey data in five European countries [62,63]. This cross-country analysis emphasizes that basic soft skills and analytical skills are considered the most important for marketing graduates. Still the results drawn within their research suggest that firms’ behavior varies with their digitalization level. Therefore, highly digitalized firms give more importance to digital and technical skills. This is an important aspect we should consider for our future work to understand how new advanced technology affects skills requirements in the Romanian labor market. As a matter of fact, this relates to another limitation of our study regarding the covariate information. Future work should incorporate other relevant characteristics of the respondents. Our current heterogeneity results motivate forthcoming research aiming to reveal other personal recruiter features or organization aspects that influence decision-making. Hence, even if our study reflects the perspective of employers in 2014, many opportunities for future work arise. Undoubtedly, it is always useful to compare the dynamics of the labor market requirements.

Another possible limitation of our research, which should be investigated in future studies, is related to the simulated design of the recruitment process. We have used the assumption that employers have a fair assessment of the graduates’ level of skills relative to others. This issue should be addressed by a proper investigation of the recruiting techniques.

The benefits of the discrete choice experiments for the labor market research topics related to recruitment decisions are proven by the following elements: takes into account the multidimensional structure of the process, and brings robust information from the demand side. The implications of the results gathered from using a multi-criteria decision-making method on the Romanian labor market could be transposed in terms of education policy. Hence, higher education institutions are acquainted about the requirements in the labor market and this is the central pillar in raising compatibility.