Measuring Learning Outcomes of Entrepreneurship Education Using Structural Equation Modeling

Abstract

This paper empirically substantiates a novel tripartite framework for measuring learning outcomes of entrepreneurship education (EE) by employing structural equation modeling. Three types of learning outcome are estimated—cognitive, skill-based, and affective—following Bloom’s (1956) taxonomy of educational objectives. The study is based on a sample of 249 imminent and recent Bachelor-level graduates from the leading universities of Estonia. The key fit, reliability, and validity indicators show statistically that the tested framework can serve as an instrument for measuring the learning outcomes of EE. This novel instrument may also serve as an alternative to entrepreneurial intention-based models very frequently used in EE to evaluate the learning outcomes. The studied interrelationships demonstrate that (1) the affective outcomes correlate significantly with the cognitive outcomes (r = 0.273, p < 0.001) and with the skill-based (r = 0.368, p < 0.001) outcomes; a correlation between the cognitive and skill-based outcomes is also significant and comparatively high (r = 0.602, p < 0.001); (2) the learning outcomes explain more variance in the cognitive and skill-based outcome constructs (44.7% and 81.0%, accordingly) than in the affective outcome construct (16.7%). Conclusions and implications for entrepreneurship educators and researchers are discussed.

1. Introduction

The impact of entrepreneurship education (EE) has been on the research agenda since entrepreneurship as a subject in higher education started spreading rapidly in the mid-1990s on both sides of the Atlantic (Katz 2003; Solomon 2007; Wilson 2008). Despite numerous papers published in this area, EE impact studies still have significant knowledge gaps in measuring learning outcomes (Longva and Foss 2018; Nabi et al. 2018, 2017). While the first evaluation approaches in EE adapted from higher education were somewhat formal, such as the ranking of study programs based on certain criteria (e.g., strategic and operational planning, educational and business process management, stakeholder satisfaction) (Vesper and Gartner 1997), the most widespread contemporary approaches tend to rely on psychology theories such as Ajzen’s (1987) theory of planned behavior. They focus on measuring changes in students’ and graduates’ entrepreneurial attitudes, self-efficacy, and intentions (Nabi et al. 2018, 2017; Bandura 1994; Boyd and Vozikis 1994; Krueger and Brazeal 1994). However, these measures cover only some aspects of learning, which can be referred to as affective and conative (Kyrö 2008) or solely affective (Lackéus 2014) in the EE literature.

An alternative approach to measuring learning outcomes originates from the science of education (Kennedy et al. 2007; Kraiger et al. 1993) and is known as a tripartite competence framework, where the learning outcomes are expressed as cognitive, skill-based, and affective—put another way, knowledge, skills, and attitudes (Bacigalupo et al. 2016, p. 20), including specific competences (Ferreras-Garcia et al. 2020). This framework arguably represents a more systemic view, considering various types of learning rather than focusing merely on the affective/conative side as the intention-based models do (e.g., Lackéus 2014; Krueger et al. 2000).

Fisher et al. (2008) for the first time used the tripartite framework in the EE context, expanding on constituents of the learning outcomes specific to EE. The research drew upon prior works of over 25 prominent scholars (e.g., Baron 2000; Rae 2000; Gibb 1993). Although this alternative approach to evaluating the outcomes of EE appears in research papers published by institutions (e.g., Bacigalupo et al. 2016; Heder et al. 2011) and individual authors (e.g., Lackéus 2014; Matlay 2008), statistical evidence that the framework works as an independent measurement instrument is still absent. Furthermore, prior studies do not yet highlight whether and to what extent the three types of outcome are indeed measuring the outcomes of entrepreneurial learning.

This paper aims to test the tripartite EE learning outcomes framework using structural equation modeling (SEM), which is a robust statistical method of model building. The ensuing tasks are to devise the cognitive, skill-based, and affective outcome constructs and to test relationships between them, with the assumption that they are positive and significant.

The research is based on a survey of 249 imminent and recent Bachelor-level graduates (Kozlinska and Mets 2013) who studied entrepreneurship in major Estonian universities. Estonia was chosen as the empirical context for testing the tripartite learning outcomes framework, since it is the only Baltic country that has been purposefully developing EE for over a decade. For example, the Ministry of Education and Science in 2013 officially integrated entrepreneurship into curricula of all disciplines at local universities (European Commission 2020).

The surveyed universities are the five biggest in the country, where entrepreneurship is taught; they cover 66% of total number university students, 58% of students of private universities, and 37% of students of applied universities (Estonian Ministry of Education and Research 2020). Universities not included in the sample had a smaller share of or missing EE. In addition to favoring external validity, this sampling method also ensured that the wide variety of teaching methodologies, from teacher-centered to learner-centered, were captured.

This study contributes to the literature by devising a novel instrument for measuring the learning outcomes of EE. It concurrently arrives at renewed conclusions pertaining to the relationships between the learning outcomes’ constructs, extending the findings of prior research (e.g., Sousa 2018; Lackéus 2014; Fisher et al. 2008). Given the rise of EE worldwide and increasing investments into its delivery, a validated approach to measuring its outcomes that is rooted in the science of education is becoming a necessity. The findings of this study highlight one of the possible statistically robust ways of conducting such measurement.

The following section elaborates on the theoretical background and hypotheses set for the study. The section after that presents the study methodology followed by an outline of the empirical findings. “Conclusions and Discussion” is the final section of the paper.

2. Theoretical Background

EE studies devoted to impact evaluation can be divided into two major groups by the types of measures used: (1) studies that use subjective measures, such as entrepreneurial intentions, self-efficacy, and competences (Fayolle and Gailly 2013; Ferreira et al. 2012; Graevenitz et al. 2010); (2) studies that use objective measures, such as nascent entrepreneurship and number of founded start-ups (Charney and Libecap 2000; Brown 1990). Although both groups of studies most often focus on short-term EE impact, subjective measures informed by psychology theories are the most widely encountered in the EE impact research (Nabi et al. 2017; Martin et al. 2013). Psychology theories penetrated EE research along with the field of entrepreneurship, which was starting to gain its legitimacy in the 1990s. Entrepreneurial self-efficacy, attitudes, and intentions became the most commonly used measures (e.g., Nabi et al. 2017). Later on, a competence-based approach coming from the science of education surfaced. This approach puts forward knowledge, skills, and attitudes towards entrepreneurship as key measures of EE outcomes (e.g., Lackéus 2014; Heder et al. 2011; Fisher et al. 2008; Kraiger et al. 1993).

This paper conceptualizes learning outcome measurement in EE based on education theories, as an alternative to psychology theories that employ entrepreneurial attitudes, self-efficacy, and intentions (Zhang et al. 2013; Krueger et al. 2000; Boyd and Vozikis 1994). It posits that the tripartite framework comprising cognitive, skill-based, and affective learning outcomes, jointly referred to as “entrepreneurial competences” (Bacigalupo et al. 2016), represents a more holistic approach to evaluating the learning outcomes of EE using subjective measures. Similar to the intention-based approach, the framework is relatable to a wide variety of entrepreneurship programs, from learner-centered and experiential to teacher-centered and traditional (Béchard and Toulouse 2013). In a way, any entrepreneurship course conveys the basics of theory like entrepreneurship process, business modeling, and, depending on the teaching methodology, connects this theory with practical experience to a greater or smaller extent. Therefore, the starting assumption when building upon the tripartite approach to competences is that heterogeneous data can fit such a framework (Iakovleva et al. 2011).

To date, numerous EE impact studies have already been conducted, with a variety of measures employed (Nabi et al. 2017). The need for improvement in this research area has been evidenced by systematic and meta-reviews (e.g., Martin et al. 2013; Rideout and Gray 2013), mainly in research methodology but also in the measures employed to better meet the labor market demands and policy expectations from EE (Bacigalupo et al. 2016).

2.1. Tripartite Learning Outcomes Framework

The tripartite framework for writing learning outcomes in general education originates from Benjamin Bloom’s (1956) taxonomy of educational objectives (Kennedy et al. 2007; Bloom et al. 1956). The taxonomy consists of three domains of learning: cognitive (dealing with knowledge, comprehension, and critical thinking), affective (concerning attitudes, emotions, and feelings), and psycho-motor (focusing on change/development in behavior/skills) (Kennedy et al. 2007; Krathwohl 2002). Kraiger et al.’s (1993) adaptation of the framework is a theoretically substantiated classification model that suggests a construct-oriented (i.e., multi-item) approach to developing training evaluation measures, although, in terms of constituents, it varies modestly from the original. For training evaluation purposes, it interprets cognitive outcomes as a class of variables related to verbal knowledge (declarative, procedural, and tacit), knowledge organization (mental models), and cognitive strategies (self-insight); skill-based outcomes as comprising variables related to composition (e.g., generalization, strengthening), proceduralization, and automaticity (e.g., attention, tuning); while affective outcomes entail attitudinal (e.g., strength, awareness) and motivational/volitional variables (e.g., self-efficacy, goal-setting). The adapted model also deals with quite generic components applicable to any discipline (Kozlinska 2016).

In the realm of EE, Fisher et al. (2008) were the first to adapt and develop this tripartite approach. The study analyzed works of dozens of experts in entrepreneurship research and categorized relevant constituents of the three types of outcomes. As a result, the basics of accounting, technology, marketing, knowledge of personal fit with entrepreneurship career, understanding risk, and other items made up the cognitive outcomes (Fisher et al. 2008). Recognizing and acting upon business opportunities, business planning, obtaining financing, managing people, setting priorities, and focusing on goals, etc., comprised the skill-based outcomes (Fisher et al. 2008). Finally, entrepreneurial spirit (measuring each participant’s attitude towards entrepreneurship), entrepreneurial learning (measuring attitude change to the entrepreneurial course), and self-esteem, among other multi-item constructs, represented the affective outcomes (Fisher et al. 2008).

The European Competence Framework (ECF) is another close adaptation of the tripartite framework, where cognitive outcomes stand for knowledge about entrepreneurship and affective and skill-based outcomes—for entrepreneurial attitude and skills, jointly referred to as competences (Bacigalupo et al. 2016; Heder et al. 2011). ECF is used in European policy-related documents (e.g., European Commission 2012; Heder et al. 2011) and in publications of individual authors (Lackéus 2014; Mets 2010; Matlay 2008; Gibb 1993). Knowledge about entrepreneurship refers to an understanding of business start-ups and procedures, entrepreneurship process and theory, the role of entrepreneurs in society and economy; skills refer to planning, organization, leadership, networking, risk-taking, team-work, lateral thinking, etc.; attitude refers to social confidence, need for achievement, self-efficacy, motivation, amongst other attitudinal and individual difference items. All the skill-based items in both sources closely overlap with Sousa’s (2018) suggested measures of entrepreneurship skills conceptualized as “capacities”.

2.2. Psychology Frameworks

Ajzen’s theory of planned behavior (1987) has been driving entrepreneurship research for over two decades. The theory maps entrepreneurial intentions and their antecedents to predict goal-oriented behavior. Perceived behavioral control (self-efficacy), attitude toward behavior, and subjective norms, in turn, proved to be consistent predictors of intentions (e.g., Krueger et al. 2000). While this theory was oriented towards predicting entrepreneurial behavior in the first place, not towards measuring the learning outcomes of EE, intentions and self-efficacy are by far the most popular subjective measures in the EE impact studies (Nabi et al. 2017; Martin et al. 2013).

Self-efficacy as a measure brought into EE originates from social cognitive theory (Bandura 1994). Self-efficacy is a person’s conviction that s/he is able to perform entrepreneurial tasks (Cooper and Lucas 2006). Initially introduced by Bandura (1994), application of the self-efficacy concept varies in the EE literature. For instance, Barakat et al. (2010) measured a mixture of cognitive and skill-based self-efficacy without distinguishing between the two, as well as entrepreneurial attitude and intent. Vestergaard et al. (2012) elicited five entrepreneurial self-efficacy constructs—creativity, planning, marshalling of resources, managing uncertainty, and financial knowledge—alongside measuring attitudes and intentions.

If compared to the self-efficacy measure used on its own and the models of entrepreneurial intentions commonly employed in EE impact studies, the tripartite framework rooted in Bloom’s taxonomy arguably presents a more holistic outlook of the learning outcomes. Intention-based models tend to be highly reliant on affective-conative learning (Lackéus 2014; Kyrö 2008), and self-efficacy measure on its own is closer linked to skill-based learning (Vestergaard et al. 2012). The tripartite competence framework, in turn, “balances recognition of multiple types of learning with parsimony” (Fisher et al. 2008, p. 474). To bridge the gap in the empirical testing of the model and enable potential future use of it, this study puts forward a number of falsifiable hypotheses.

2.3. Hypotheses

The overviewed research papers that rely on the tripartite learning outcomes framework in contributing to the EE impact measurement discussion tend to combine very diverse constituents of the cognitive, skill-based, and affective outcomes. Thus, there is no unified, statistically validated conclusion regarding which competences to measure within each type of outcome—business planning, managing people, understanding risk, attitudes, or other. Furthermore, prior EE studies (e.g., Lackéus 2014; Fisher et al. 2008) do not quantify whether and to what extent the three types of outcome are indeed measuring the outcomes of entrepreneurial learning. The hypotheses set out in this study are targeted at bridging these gaps.

Building upon Fisher et al.’s study (2008), as far as the measurement refinement is concerned, using a robust statistical method, the first three hypotheses focus on interrelationships between the learning outcome constructs.

Hypotheses 1–3 (H1–3).

There is a significant positive correlation between the cognitive, skill-based, and affective outcomes.

In order to test whether employing the tripartite framework can be validated statistically, i.e., to substantiate measurement of the learning outcomes using this framework and, in doing so, to complement prior studies employing it, the second set of hypotheses suggests that:

Hypotheses 4–6 (H4–6).

The learning outcomes (as a latent variable) possess strong explanatory power over the cognitive, skill-based, and affective outcome constructs (as dependent variables).

Figure 1 visualizes the conceptual framework of the study.

The next section sheds light on descriptive information about the sample (N = 249), research method, and measures employed in the study.

3. Data and Research Methods

3.1. Sample

The research data (Kozlinska and Mets 2013) were collected from an online questionnaire-based survey conducted from May to November 2013 in the leading Estonian universities: University of Tartu, Tallinn University of Technology, Estonian Business School, Tallinn University of Applied Sciences, and Estonian Entrepreneurship University of Applied Sciences. These are not only leading but also the biggest universities in the country in general and in the EE context specifically (Estonian Ministry of Education and Research 2020).

The survey targeted imminent and recent Bachelor-level graduates (i.e., those who were in their final year of study or were within two years after graduation) from the business- and entrepreneurship-related programs of the selected universities. All the respondents had educational intervention in entrepreneurship, 3 to 6 ECTS (points according to the European Credit Transfer System, 1 point usually corresponds to 25–30 h of study), in their curricula. In total, 89.6% of the respondents studied full-time, 9.6% studied part-time, and 0.8% studied extramurally.

Contacts established in the contributing institutions facilitated distribution of 2487 e-mail invitations that resulted in 271 complete responses, translating to a 10.9% total response rate. Few respondents (N = 5) marked studying in “Other universities”. After further data cleaning that included removing observations of respondents who did not study entrepreneurship, the final sample size used in the analysis amounted to 249.

Table 1. Descriptive information about the sample.

University		Bachelor’s Degree		Total
		Yes	No
University of Tartu	N	38	41	79
	%	48.1%	51.9%	100.0%
Tallinn University of Technology	N	12	13	25
	%	48.0%	52.0%	100.0%
Estonian Business School	N	17	18	35
	%	48.6%	51.4%	100.0%
Tallinn University of Applied Sciences	N	18	11	29
	%	62.1%	37.9%	100.0%
Estonian Entrepreneurship University	N	45	31	76
	%	59.2%	40.8%	100.0%
Other universities	N	3	2	5
	%	60.0%	40.0%	100.0%
Total	N	133	116	249
	%	53.4%	46.6%	100.0%

provides descriptive information about the sample. While the number of the recent graduates prevails slightly (53.4%), 40% of the students who filled in the questionnaire expected to graduate with Bachelor’s degrees in 2013 (i.e., shortly after the survey was conducted) and in 2014.

At the time, study programs that focused solely on entrepreneurship were a rare occasion in Estonia. For this reason, most of the respondents in the sample studied in business-related programs, such as “Business Administration” or “Management”; a few others were representatives of non-business programs, e.g., “Computer Sciences”, “Mechanical Engineering”. Estonian Entrepreneurship University of Applied Sciences and Estonian Business School, having a stronger focus on entrepreneurship historically, did offer “pure” entrepreneurship programs. EE courses in these programs varied in their aims and methods of teaching, providing a useful heterogeneous context for the analysis. For instance, Estonian Business School offered “Business Basics” and “Student Enterprises” (3 ECTS each), which were designed in such a way that the former taught the basics of entrepreneurship theory and the latter made students work on real-life projects, go outside the classroom, and gain practical experience of the entrepreneurial process. Tallinn University of Technology’s “Entrepreneurship Basics” (3 ECTS) and “Entrepreneurship” (4 ECTS) courses were more theoretical, aimed at conveying the knowledge necessary for setting up and managing one’s own enterprise and at helping students to make well-grounded career choices. Students of these courses attended traditional lectures and seminars and worked on business plans and simulations but with a stronger emphasis on theory. In a similar way, other universities in the sample provided theoretically or experientially oriented or mixed teaching methodology courses

Testing the conceptual framework in a heterogeneous context (with different teaching approaches and students’ backgrounds) can be viewed as an even more challenging empirical setting. That is, on one hand, the model might be harder to estimate, and, on the other hand, if it works, this increases the likelihood that the model will work in any context (Iakovleva et al. 2011).

In total, 60% of respondents in the sample were aged between 20 and 26, and 21% were aged between 27 and 30, generally confirming that the survey reached its target audience. More responses were received from females than males—59% compared to 41% of the total. This ratio reflected the nation-wide distribution in Estonian HEIs at the time, where females accounted for 58%, as well as the distribution of students studying in economics and business-related disciplines, where females accounted for 63% (Estonian Ministry of Education and Research 2020).

The potential research contributors were asked to be as sincere as possible when answering the survey questions. They were incentivized with a lottery of electronic gadgets, which was drawn in December 2013. Prizes varied from navigation systems to MP3 players.

3.2. Method

Following recommendations of Coltman et al. (2008) to assess research models, the proposed EE learning outcomes measurement model falls into a reflective type characterized by existing latent constructs independent of the used measures, items manifested by the constructs, causality from the construct to items, identifiable error term, amongst other features (cited in Kozlinska 2016). With this type of model, for instance, the ordinary least squares regression method does not seem appropriate, since, applied to a single equation from the proposed system, it would produce biased parameter estimates as a consequence of ignoring simultaneity (Asteriou and Hall 2011). Furthermore, single equation models neglect the structure of co-variance in data and thus can produce less stringent or even misleading results (Florit and Lladosa 2007; Schreiber et al. 2006).

To test the formulated hypotheses, this study employed structural equation modeling (SEM), using the analysis of moment structures (AMOS) add-on in the SPSS software (Kozlinska 2016). SEM allowed the estimation of a series of separate, though interconnected, equations for modeling the students’ and graduates’ learning outcomes, as well as considering complex links among them (Schreiber et al. 2006). Since the devised conceptual framework already prescribed indicators, or observed variables, for the underlying latent variables, i.e., cognitive, skill-based, and affective outcomes, confirmatory factor analysis (CFA) with AMOS as the first step of SEM was employed to examine whether the collected data fit the framework and to create three independent outcome constructs (Obadia and Vida 2011). For the second step, the measurement (correlation relationship) and structural (dependence relationship) models were devised, using the estimated learning outcome constructs (Hair et al. 2010). The first step helped to unveil correlations between the created constructs, while the second step regressed the learning outcome variables of the cognitive, skill-based, and affective outcome constructs to identify how much variance the learning outcomes actually explain in the constructs comprising them (Obadia and Vida 2011). Maximum likelihood method was employed to perform estimations. The minimum recommended sample size of 200 for using SEM was reached (Jackson et al. 2013; Westland 2010). Potential limitations (number of observations per estimated parameter) are acknowledged in the final section of the paper.

Following examples of earlier studies (Obadia and Vida 2011; Zampetakis et al. 2011; Hooper et al. 2008; Carson et al. 2006), the model’s fitness, reliability, and validity were assessed using the following set of indicators:

-

Chi-square (χ²)

-

Probability level (p)

-

Chi-square/degrees of freedom (χ²/DF), around 2

-

Goodness-of-fit index (GFI), >0.9

-

Normed fit index (NFI), >0.9

-

Tucker-Lewis index (TLI), >0.95

-

Comparative fit index (CFI), >0.95

-

Root-mean-square error of approximation (RMSEA), <0.06

-

Average shared variance (AVE), >0.5

-

Reliability (ρ), similar to Chronbach’s alpha, >0.8.

As part of the second step of SEM—construction of the measurement and structural models—the discriminant validity was also tested (Farrell and Rudd 2009; Fornell and Larker 1981), which requires each latent variable to have more variance in common with its indicators (AVE) than with any other latent variable.

3.3. Measures

The survey questions were designed by relying on the established conceptual framework and were derived from previous works by Fisher et al. (2008); Lackéus (2014); Liñán and Chen (2009), as well as the GUESS survey (Sieger et al. 2014). Thus, the questionnaire was a cumulative effort informed by numerous research papers to test as many relevant variables as possible prior to creating the final constructs. The wording of questions was based entirely on Fisher et al. (2008), while the items were informed by all the aforementioned sources. In the SEM terms, the cognitive, skill-based, and affective outcomes were treated as independent, or latent, variables, while respective constituents of those outcomes were treated as dependent, or observed, variables in the models. In the structural model; however, the cognitive, skill-based, and affective outcomes represent dependent variables or regressands, while the learning outcomes represent independent variables or regressors.

The online questionnaire elaborated for this study pursued the self-reported perceptions approach, widely used in EE research (Zampetakis et al. 2011; Fisher et al. 2008; Krueger et al. 2000, to name a few) and even considered as more important than any objective evaluation (Schaper and Casimir 2007). In the case of the cognitive and skill-based outcomes, the respondents were asked to self-assess change in their knowledge and skills, intending to overcome limitations of the post-hoc measurement in the cross-sectional study design. A different solution was applied to the affective outcomes though, due to a stronger bias that could arise from self-evaluation of softer, affection-driven categories, with a reference to two to three years in the respondents’ past.

Cognitive outcomes, or knowledge about entrepreneurship, were measured based on the following question: “During your entrepreneurship course or program, have you learnt new information that you did not know at the beginning of the course?” The list of topics to be assessed using a 5-point Likert scale, coded as 0—learned nothing, 1—was exposed to the topic, 2—learned some basic facts about it, 3—learned a moderate amount, 4—gained extensive knowledge of the topic, entailed the following: business modeling, evaluation of business opportunities, team management, entrepreneurship process, and other themes that could potentially act as measures relying on the previous studies (in particular, Fisher et al. 2008; cited in Kozlinska 2016).

In the same way, to measure the skill-based outcomes of the respondents, they were asked “Can you do things now that you could not do at the beginning of the course? Please select the response that best describes your level of improvement in each of the skills listed, if 0—no improvement, 1—made one or two minor improvements, 2—made some improvements, 3—made substantial improvements, 4—can now perform very well”. They were provided with a list of skills comprising the following: “Lead a team”, “Organize and control ongoing projects”, “Negotiate deals”, and “Solve creative business problems”, among others (Fisher et al. 2008).

The third group, measures of affective outcomes, comprised statements relating to entrepreneurship career proclivity, self-efficacy, passion, desirability, etc., having a lot in common with the theory of planned behavior’s informed measures (Krueger et al. 2000). The respondents were asked to indicate the extent to which they agreed with the statements, using the 5-point Likert scale, from 0—strongly disagree to 4—strongly agree.

Despite the fact that some of these measures were sourced from studies dating back to the 2000s, the onset of EE surge, the core themes of opportunity recognition, business modeling, etc., in entrepreneurship theory remained topical, as evidenced by the recent systematic literature review by Nabi et al. (2017) and joint research efforts of numerous EE experts in Bacigalupo et al. (2016). Theoretical and contextual advancement is further addressed in the final section of the paper.

4. Results

Table 2. Results of the confirmatory factor analysis (CFA) for the learning outcome constructs.

Scales Properties and Items (All Items Measured with Five-Point Likert Scales)	Indicators’ Standardized Loadings	Squared Loadings (R2)
Cognitive outcomes (F1), 7 items; AVE = 0.543, ρ = 0.892 Fit indices: χ2 = 23.191, d.f. = 14, p = 0.057, χ2/d.f. = 1.656, GFI = 0.975, NFI = 0.973, TLI = 0.983, CFI = 0.989, RMSEA = 0.051
opportunity recognition	0.837	0.701
evaluation of business opportunities	0.769	0.591
attraction of financing	0.765	0.585
business modeling	0.739	0.546
lean start-ups	0.727	0.529
development of new products and services	0.682	0.465
business communication	0.621	0.386
Skill-based outcomes (F2), 11 items; AVE = 0.626, ρ = 0.948 Fit indices: χ2 = 119.424, d.f. = 43, p = 0.000, χ2/d.f. = 2.777, GFI = 0.913, NFI = 0.944, TLI = 0.953, CFI = 0.963, RMSEA = 0.085
set priorities and focus on realistic goals	0.842	0.709
negotiate deals with other businesses	0.838	0.702
solve creative business problems	0.831	0.691
build up professional networks	0.826	0.682
organise and control on-going projects	0.819	0.671
identify unmet needs of people	0.778	0.605
develop innovative working environment	0.776	0.602
deal with uncertainty, adapt to new and uncertain situations	0.776	0.602
lead a team	0.767	0.588
keep good interpersonal relations	0.731	0.534
resolve conflicts	0.704	0.496
Affective outcomes (F3), 7 items; AVE = 0.670, ρ = 0.934 Fit indices: χ2 = 37.505, d.f. = 13, p = 0.000, χ2/d.f. = 2.885, GFI = 0.959, NFI = 0.973, TLI = 0.971, CFI = 0.982, RMSEA = 0.087
To be an entrepreneur and have own company is my true passion	0.879	0.773
My professional goal is to be an entrepreneur	0.879	0.773
A career as an entrepreneur suits me well—it gives me freedom and autonomy	0.847	0.717
I will make every effort to start a new enterprise when aspired	0.842	0.709
Among various options I would rather be an entrepreneur	0.811	0.658
I want to start a new company	0.764	0.584
I am confident in my ability to start an enterprise	0.692	0.479

shows the results of the first step of SEM–CFA for the learning outcome models. Following purification of the models from the items with low standardized regression weights (in this case, under 0.62), which did not represent the latent variables well enough, the models’ fit indexes were compared with the thresholds commonly accepted in SEM (χ²; χ²/d.f.; GFI, goodness of fit index; NFI, normed fit index; TLI, Tucker Lewis index; CFI, comparative fit index; RMSEA, root mean square error of approximation). Squared multiple correlations (R²) show the proportion of variance of the items that the latent variables explain. In total 8, 13, and 12 items were eliminated, respectively, in the cognitive, skill-based, and affective outcomes models to reach the acceptable fit indicators (see Appendix A). Afterwards, the reliability (ρ) and validity (AVE, average variance extracted) indicators were computed. The models’ fit indexes and other key indicators either met or exceeded the criteria. In the case of the skill-based construct, the model fit and RMSEA were slightly higher than the norm but acceptable (Hooper et al. 2008; Barrett 2007). The high enough validity indicators confirmed that the observed variables selected were measuring the respective latent constructs, while the latter could explain over 50% of variance in the observed variables (Farrell and Rudd 2009). The reliability indicator showed that the observed variables belonging to one construct were highly interrelated with each other.

While the main assumptions of SEM (Kline 2012) were met to a large extent (no correlation among error terms and latent variables, no correlation of error terms, no cross-loadings of the items, i.e., one item should measure only one latent variable), the skill-based and affective outcome models had one pair of the error terms each where co-variances were quite high: respectively, these were “Resolve conflicts” with “Have good interpersonal relations” and “A career as an entrepreneur suits me well” with “I am confident in my ability to start an enterprise”. Being justifiable theoretically, the respective correlations were freed (co-variances allowed) inside one particular construct, as recommended by Schreiber et al. (2006).

Table 3. Results of the measurement and structural models.

Hypothesis	Link	Estimate/Correlation	S.E.	|t| Value	Two-Tailed p	Result
H1	Cognitive <-> Skill-based	0.602	0.046	13.115	0.001	Supported
H2	Skill-based <-> Affective	0.368	0.059	6.274	0.001	Supported
H3	Affective <-> Cognitive	0.273	0.064	4.246	0.001	Supported
Fit indexes: χ2 = 462.305, df = 270, p = 0.000; CMIN/DF = 1.712; GFI = 0.865; NFI = 0.903; TLI = 0.952; CFI = 0.957; RMSEA = 0.054
Hypothesis	Link	Standardized Coefficient	R2	|t| Value	Two-Tailed p	Result
H4	Learning outcomes -> Cognitive	0.669	0.447	6.835	0.001	Supported
H5	Learning outcomes -> Skill-based	0.900	0.810	7.535	0.001	Supported
H6	Learning outcomes -> Affective	0.409	0.167	5.226	0.001	Supported

illustrates the results of the second step of SEM, highlighting correlation relationships between the learning outcome constructs (the measurement model) as well as dependence relationships between the learning outcomes and the three types in question (the structural model). The first finding suggests that the affective outcomes have significant correlations with the cognitive (r = 0.273, p < 0.001, two-tailed) and skill-based (r = 0.368, p < 0.001, two-tailed) outcomes. The correlation between the cognitive and skill-based outcomes is comparatively high and significant (r = 0.602, p < 0.001, two-tailed). The structural model signposts that the latent variable “EE learning outcomes” possesses greater explanatory power over changes in the cognitive and skill-based outcome constructs (44.7% and 81.0%, accordingly) than over changes in the affective outcome construct (16.7%).

These results mean that all the study hypotheses received support. However, the results also imply that over a half of the variance in the cognitive and affective learning outcomes of the surveyed graduates from the business- and entrepreneurship-related programs who studied entrepreneurship is seemingly attributed to other factors than the EE intervention.

To back up the findings, the discriminant validity test was performed, according to which the squared correlation estimate (e.g., Cognitive<->Skill-based, 36.2%) must be lower than the AVE of each of the learning outcome constructs (e.g., Skill-based, 62.6%; Cognitive, 54.3%).

Table 4. The discriminant validity test.

	Correlation	Squared Correlation	AVE
Cognitive <-> Skill-based	0.602	0.362	0.543 (Cognitive)
Skill-based <-> Affective	0.368	0.135	0.626 (Skill-based)
Affective <-> Cognitive	0.273	0.075	0.670 (Affective)

shows the lowest AVEs of the models for each pair of outcomes. The test means that the latent variables are able to account for more variance in their observed variables (items) than measurement error or other constructs within the conceptual model (Farrell and Rudd 2009; cited in Kozlinska 2016).

In the next section, possible reasons for and implications of the obtained results are discussed.

5. Conclusions and Discussion

The conducted quantitative analysis yielded three major findings addressing the research aim and tasks. First of all, the learning outcome constructs were estimated using structural equation modeling (SEM), as a result of which many items, or components, of the outcomes present in the earlier studies applying the tripartite framework (e.g., Lackéus 2014; Fisher et al. 2008) had to be eliminated during the purification process for the constructs to form. Secondly, the created measurement and structural models indicate that all the learning outcome constructs (cognitive, skill-based, and affective) are significantly intercorrelated. Thirdly, all the pathways in the resulting structural model (learning outcomes as an independent variable explaining variance in the learning outcome types) were highly significant. Therefore, all the hypotheses set for the study were supported, leading to the conclusion that the tripartite competence framework can serve the purpose of measuring the learning outcomes of EE. The results also demonstrate that the measurement instrument developed using SEM is consistent, functioning properly, and can be applied in further studies. Composition of the final constructs reconfirmed the importance of such core themes as business modeling, opportunity recognition, product development, network building, and dealing with uncertainty, among others, which form a contemporary theoretical body of knowledge in entrepreneurship research. The instrument thus advances prior efforts made in creating and applying measures of EE outcomes (Nabi et al. 2017; Sousa 2018; Bacigalupo et al. 2016; Fisher et al. 2008).

It is noticeable that the items which formed the final cognitive outcomes construct find sound support from entrepreneurship theory (e.g., business modeling, development of new products and services), while those that were driven by business management had low loadings (e.g., business planning, financial accounting, project management, business communication) and thus did not represent the construct well enough and had to be eliminated. The second, skill-based construct largely entailed interpersonal/personal content, not business-specific skills (e.g., lead a team, negotiate deals with other businesses), yet “organization and control of ongoing projects” as well as “solving creative business problems” demonstrated some of the highest loadings. Similar to the affective outcomes, it had over 10 items eliminated. The items forming this construct can be viewed as quite generic to business management education (e.g., set priorities, resolve conflicts) and as very specific to EE (e.g., deal with uncertainty, identify unmet needs of people) at the same time. One of the possible conclusions from this finding is that EE develops both entrepreneurship and business management skills. Affective outcomes, in turn, comprised the items relating to entrepreneurial passion and career preferences as well as self-efficacy, all representing strong, emotionally laden statements. Other items like attitude to failure and self-reliance in uncertain situations, while being related to entrepreneurship, did not statistically belong to the affective outcome construct1.

Based on the estimation of the learning outcomes measurement and structural models, and the variance explained in each construct (45% in cognitive outcomes; 81% in skill-based outcomes, and 17% in affective outcomes), it is likely that the unexplained variance is attributable to other factors than the EE intervention, while the share of EE influence remains highly significant. For instance, in the case of the affective outcomes, the respondents’ level of agreement with the statements offered in the questionnaire could be predicated upon the already formed career preferences and entrepreneurial self-efficacy in starting up a new enterprise. Similarly, in the case of the cognitive outcomes, the respondents would not indicate significant improvement in business modeling, lean start-ups, product development, etc., if they already possessed substantial prior knowledge about these topics.

Results of the SEM-based analysis confirm the significance of the studied relationships. It might well be that, depending on the type of intervention and the study program content, as well as the individual-level characteristics of students, the correlation coefficients will vary alongside the share of variance in the outcomes that EE explain. It should also be denoted that the SEM approach is one of the most demanding in the statistical estimation of multi-item constructs. At the same time, SEM estimation does not allow the authors to draw conclusions about the causal inferences. What is proven and clear is that the collected data fit the established conceptual framework, while the causal connection between educational intervention and the learning outcomes, skill-based, and cognitive outcomes is taken as an assumption based on the literature (Fisher et al. 2008; Kraiger et al. 1993), as it is often the case in behavioral sciences (Kline 2012).

At a more practical level, the findings may suggest that entrepreneurship curricula developers should target educational activities either in terms of the skill-based and cognitive learning or affective learning. As the cognitive and skill-based constructs share less variance with the affective construct, it makes it reasonable to target them separately with specific sets of educational activities. For example, willingness to pursue entrepreneurship as a career path can be induced with such methods as guest lectures by entrepreneurs (most often, success stories), job shadowing, and company visits, while particular knowledge and skills can be developed by engaging students in real-life projects with companies, business model competitions, business games, etc. Whether to start the teaching process with affective learning or cognitive/skill-based learning remains with educators to decide. Some scholars, such as Kyrö (2008), view affection as the solid foundation for building up other entrepreneurial competences, in contrast to the original Bloom’s taxonomy and its later adaptation by Kraiger et al. (1993), which both commence with the cognitive component.

Another practical implication for entrepreneurship curriculum development that the obtained results suggest is the need for holistic learning, where students are exposed to diverse educational activities and develop all three types of learning, making up the consistent structural model.

Limitations and Further Research

This study is not devoid of limitations. One of them, “inherited” from a previous study (Fisher et al. 2008) relates to the wording of the questions that allowed estimation of the final constructs. For the cognitive and skill-based outcome estimation, the respondents were asked to evaluate a change in their knowledge of entrepreneurship topics and skills that occurred throughout the study course. In the case of the affective outcomes, the respondents indicated the extent to which they agreed with presented statements. The resulting seven indicators (items) in the latter type of the outcomes carry a rather strong degree of affect, which can be regarded as an explicit EE outcome only in a limited number of entrepreneurship programs targeted at venture creation specifically (like at Chalmers University, Babson College, Aalto University). It might be the case that launching the same survey in another country or a purely constructivist entrepreneurship study program could bring slightly, but not critically, different results.

When building the learning outcome SEM construct, some co-variances of the error terms were freed. Indeed, this is a justifiable practice, but, according to Farrell and Rudd (2009), it might indicate that other items exist that could be included in the respective constructs. Further comparative analysis between countries with application of the measurement instrument could help to shed light on changes in the composition of the learning outcome constructs in different regional contexts.

The sample size of 249 was enough for estimating the knowledge, skills, and attitude constructs separately, implying that there were 10 to 16 observations per estimated parameter in the measurement models. Ten observations per parameter in some sources is viewed as a rule of thumb; in others, this is more relaxed (Jackson et al. 2013; Westland 2010; Schreiber et al. 2006). The bottom line is that there is no right number of observations by default, but it depends on the model (number of factors, level of loadings, number of indicators per factor, etc.). The tendency to have a smaller sample size than needed is a characteristic of over 50% of studies that employ SEM (Westland 2012). Although our sample size could have been bigger, the structural model contained three factors featuring high loadings, with many indicators under each factor, meaning that the sample size did not necessarily have to be bigger (Jackson et al. 2013). The measurement and structural models had ideal or acceptable fit indicators; hence, the SEM results can be considered reliable.

Finally, the study is based on data collected several years ago, which might be viewed as a limitation. However, since the main aim of the study was to come up with the measurement approach, and the competences measured are still the ones present in EE, required to be developed and targeted in many study programs (e.g., Bacigalupo et al. 2016), the time lag does not seem to be a significant factor affecting the results and contributions. Such competences as dealing with uncertainty, resolving conflicts, opportunity recognition, etc., are highlighted in the EntreComp framework (Bacigalupo et al. 2016), for instance, and are in even higher demand in the current socio-economic conditions. The same competences are viewed as transferrable, i.e., universal, across disciplines. That said, the tripartite framework does not cover more nuanced affective measures such as inspiration, as called for in the recent study by Nabi et al. (2017). Further research may address this deficiency.

In summary, the results of the SEM analysis confirmed that the tripartite competence framework could alternate or supplement existing approaches to measuring EE outcomes, entrepreneurial self-efficacy, and intentions (Zhang et al. 2013; Barakat et al. 2010), in particular. Despite multiple EE impact studies having been conducted and published in the last couple of decades (Nabi et al. 2017; Martin et al. 2013), the tripartite framework had limited coverage even in the newest studies, focusing on some competences but not the full model. A further step in addition to a possible replication study could be a comparison of the learning outcomes estimated using SEM depending on the type of educational intervention, e.g., behavioristic or constructivist, traditional or experiential, which has been lacking in EE impact research.