How to Assess Generic Competencies: From Sustainable Development Needs among Engineering Graduates in Industry

Round 1

Reviewer 1 Report

Achieving the UN's 17 Sustainable Development Goals requires the contribution of qualified engineers. However, there is a mismatch between the sustainable competencies that engineering graduates possess and those demanded by the industry. This gap requires a reliable method of establishing which competencies are most important to engineering professionals. In this research, a generic engineering competency model is presented. Using exploratory factor analysis (EFA): (1) leadership, (2) engineering design, (3) professionalism, (4) problem solving, (5) lifelong learning, (6) technical theory, and (7) ) Communication. The paper is correctly structured and presents chapters on research methods and a literature review. The topic is relevant and the authors are to be congratulated.

Author Response

Thank you very much!

Reviewer 2 Report

Performed methodology is not accurate, because:

1. Characteristics of sampling distribution are not presented: skew, kurtosis, cv...

2. Assumption of the normal distribution is not tested?? 

3. EFA-  what about criterium of stopping the extraction of factors (components); is not clarified- Parallel forms Monte Carlo simulation (permutation) mandatory

4.  EFA -  why rotation not proceeded?? - The oblique but not orthogonal 

5.  CFA in AMOS -  vaguely; I want to see a model; standardized matrix of covariances; before and after rejecting Item(empirical and saturated model; common method bias like a Hermans single factor has to be performed... good advice is Common latent factor to be used....

 

Author Response

Response 1: The normality of the distributions of all the item was acceptable (skewness ranging from −1.91 to −0.72; kurtosis ranging from −0.05 to 2.83). See p6.

Response 2: The article assume the normal distribution of the data (skewness ranging from −1.91 to −0.72; kurtosis ranging from −0.05 to 2.83). See p6.

Response 3: Items were retained based on eigenvalues of greater than 1. See p9.

Response 4: As factors were hypothesized to correlate, oblique rotation (promax) was used. See p5.

Response 5: After rejecting the item "breadth of knowledge", the model fits better. See the table below.

(After deleting the item of "breadth of knowledge"，the model fit is more better. See the below table)

Index	CMIN/DF	IFI	TLI	CFI	RMSEA
Before deleting	5.024	0.896	0.89	0.896	0.058
After deleting	4.447	0.913	0.908	0.913	0.054

 

The modified model based on CFA is shown in the following figure. (in  the word）. Here do not copy Figure.

The second-order model is shown in the following figure.(in  the word）. Here do not copy Figure.

Standardized Regression Weights Based on CFA is as follows. See table 6.

			Estimate
VI21	<---	LLL	.811
VI22	<---	LLL	.835
VI23	<---	LLL	.859
VI24	<---	LLL	.875
VI25	<---	LLL	.840
VI01	<---	EK	.681
VI02	<---	EK	.732
VI03	<---	EK	.767
VI04	<---	EK	.754
VI05	<---	EK	.806
VI27	<---	Co	.832
VI28	<---	Co	.842
VI29	<---	Co	.817
VI30	<---	Co	.733
VI31	<---	Co	.774
VI20	<---	EL	.697
VI19	<---	EL	.766
VI18	<---	EL	.665
VI17	<---	EL	.820
VI16	<---	EL	.808
VI15	<---	EL	.814
VI14	<---	EL	.815
VI13	<---	EL	.826
VI12	<---	EL	.784
VI10	<---	EL	.702
VI09	<---	EL	.735
VI08	<---	EL	.654
VI07	<---	EL	.692
VI61	<---	ED	.828
VI60	<---	ED	.785
VI59	<---	ED	.811
VI58	<---	ED	.842
VI57	<---	ED	.818
VI56	<---	ED	.811
VI55	<---	ED	.657
VI54	<---	ED	.738
VI53	<---	ED	.804
VI52	<---	ED	.793
VI40	<---	Pro	.746
VI42	<---	Pro	.755
VI43	<---	Pro	.798
VI44	<---	Pro	.825
VI45	<---	Pro	.758
VI46	<---	Pro	.852
VI47	<---	Pro	.824
VI48	<---	Pro	.802
VI50	<---	Pro	.784
VI33	<---	PS	.796
VI34	<---	PS	.797
VI35	<---	PS	.829
VI36	<---	PS	.819
VI37	<---	PS	.847
VI38	<---	PS	.824
VI39	<---	PS	.827
VI32	<---	PS	.784

Self-reported data in surveys could contain a common method bias. We evaluated the potential presence of this bias by calculating the common latent factor. Using this method, two different models were tested: The first model was the proposed model, and in the second model, a method factor variable was introduced. The resulting path coefficients were not substantially different between these two models: ∆GFI=0，∆IFI = 0. 002，∆NFI = 0. 001，∆TLI=0.006，∆CFI=0.002，∆RMSEA =0.002，RMR=0.001. GFI, IFI, NFI, TLI and CFI are did not exceed .1, and RMSEA and RMR did not exceed .05, which indicated that there was no significant bias related to common methods. See p13.

 

Finally, we invited professional services agencies to polish the articles.

 

 

 

Author Response File: Author Response.pdf

Reviewer 3 Report

 The authors studied on the generic engineering competencies that must be developed by engineering graduates in China. The findings reported are explained to be useful to prepare a framework to drive the engineering educational objectives in higher learning institutions in China.

 

Overall, the study is sound. I would recommend the paper be considered for publication pending revision as given below.

 

Comment #1:

I believe that the referencing format is inaccurate. Square parentheses are missing. Please update accordingly.

 

Comment #2:

Can the authors explain why electrical, chemical, compute and materials engineers are selected to optimize content validity of the questionnaire development?

 

As from the engineering education and lecturers’ aspect, what are their backgrounds? Which engineering areas are they educating in?

 

Comment #3:

What are the requirements set by the authors when defining the term “engineer”? Are they classified based on or recognized by an engineering regulatory body?

 

Comment #4:

It will be useful to also analyze the data based on working experience. Will the priority of the different factors vary with years of working experience? For example, more priority will be on leadership at a later part of the career?

 

The current analysis seems to have averaged everything all together, which might not reflect the year of working experience bias on the factors considered.

 

Comment #5:

Typically, references should be omitted in the conclusions.

 

Author Response

Response 1:

We invited professional services agencies to polish the articles, including the format.

Response 2:

This is a good question. The reason for choosing electric, chemical, compute and materials engineers is electrical, chemical, compute and materials engineers occupy a high proportion in China and have certain representativeness. Second, in terms of the current conditions of our team, sampling engineers in these industries is relatively more feasible. But in order to increase the wide applicability of the content, the future sampling of engineers for content validity verification will also be extended to mechanical, aerospace, nuclear and other fields. See p17.

The engineering lecturers who participated in our discussion were all Ph.D.s from materials, environment, computer science and technology, and naval architecture. See p4.

Response 3:

China has established a registered engineer system in the fields of structure, civil engineering, electrical, chemical industry and other fields, and implemented qualification management for technical personnel engaged in engineering. However, existing regulations on the qualification of engineers emphasize professional and technical capabilities and rarely involve cross-disciplinary general capabilities. As the importance of generic competencies increased, the generic competencies required by practical engineers gained attention, and a number of related classification studies emerged.  See p2.

Response 4:

This is a very good recommendation. But, the purpose of this article is to take a holistic approach and develop a generic competency needs assessment tool for engineers. For the impact of the working experience on the general capabilities of engineering, see our team's forthcoming article "Investigating the Generic Competencies Mismatch of Engineering Graduates Using IPA", which will be published in Research in Higher Education of Engineering.

Response 5:

The article had been modified. See p17.

 

Round 2

Reviewer 2 Report

revised version is ok