Living Labs for AI-Enabled Public Services: Functional Determinants, User Satisfaction, and Continued Use

Round 1

Reviewer 1 Report

Thank you for asking me to review this exciting paper. I compliment the authors on exploring such an important topic and hope to read more work in this area.
After critically reviewing the article, I think this manuscript has a lot of potential and definitely covers an exciting area in research and practice.
1. The contribution of your study is missing in the introduction section. Could you explain it at the end of the introduction?
2. Improve your discussion part a bit.
3. Explain your methodology part a bit more including the participant's demographical information and study area.4. Explain the practical implication of the study.
Good luck to the authors

Author Response

Responses to Reviewer 1

 

1. The contribution of your study is missing in the introduction section. Could you explain it at the end of the introduction?

>> Our response:

Thank you for your suggestion to include the research contribution. We have added the contribution of our study at the end of the introduction section to emphasize its significance in the field of ICTs in government.

 

2. Improve your discussion part a bit.

>> Our response:

We have made significant improvements by incorporating additional explanations and discussions. These enhancements help us to deepen insights provided and offer a more comprehensive understanding of our findings. Additionally, we have restructured the last section, dividing it into two sections: discussions and conclusions. This separation allows for a more organized presentation of our research outcomes and their implications.

 

3. Explain your methodology part a bit more including the participant's demographical information and study area.4. Explain the practical implication of the study.
   >> Our response:

In accordance with the suggestion provided, we have included additional details regarding the respondents' demographic information in a newly added table.

 

Author Response File: Author Response.docx

Reviewer 2 Report

Full Title: Living Labs for AI-Enabled Public Services: Functional Determinants, User Satisfaction, and the Continued Use

Objective: The present study aimed to determine the influence of six functional factors (i.e., usefulness, ease of use, service reliability, service quality, responsiveness, and security) on the continued use of AI-enabled public services through the mediating role of user satisfaction.

 

The manuscript needs more work and effort in each section:

1. The author(s) should check English since there are (much too) many instances of typos and mistakes throughout the manuscript (e.g., responsevness (on p. 4) should be responsiveness).

2.      Providing a conceptual model showing the relationships between study variables is very nice. But indicate all hypotheses over the research model (Figure 1).

3. Provide more information about the profile of the respondents (age, gender, etc.). Provide more demographic data such as age (mean, range), gender (frequency/percentage), and usage experience (hours/day, year) in a table or text.
4. Best practices require the list of items used in the questionnaires, accompanied by variables/dimensions, item codes, and sources. Please, reassess.

5.      The instrument is formed by 32 Likert-type items. Before the main analysis, a Confirmatory Factor Analysis (CFA) should be conducted:

a.       Convergent and discriminant validity should be tested.

b.      CR and AVE values along with HTMT results should also be reported in the matrix to test the convergent and discriminant validity.

 

6. In this cross-sectional study, several instruments were applied at the same time. Therefore, Common method bias should be tested. One of the simplest ways to test if CMB is of concern in your study is to use Harman’s single-factor score, in which all items (measuring latent variables) are loaded into one common factor. If the total variance for a single factor is less than 50%, it suggests that CMB does not affect your data, hence the results.
7. As for the mediation analysis, Preacher and Hayes (2008) suggest bootstrapping the indirect effect to test the mediation effect of the study. LL and UL do not straddle a 0 in between, indicating a mediation effect between the independent variable and dependent variable (Preacher & Hayes, 2008).
8. Discussion & Conclusion should be enlarged, including a better interpretation of results and a better justification of the conclusions. Referring to the following recent studies may contribute to the study: “Understanding the impact of knowledge management factors on the sustainable use of AI-based chatbots for educational purposes using a hybrid SEM-ANN approach,” “An empirical examination of continuous intention to use m-learning: An integrated model,” “Examining the impact of psychological, social, and quality factors on the continuous intention to use virtual meeting platforms during and beyond COVID-19 pandemic: A hybrid SEM-ANN approach,” and “The impact of preservice teachers’ cognitive and technological perceptions on their continuous intention to use flipped classroom”. The author (s) need to modify the discussion by considering the above-mentioned points and modifying the results.

1. The author(s) should check English since there are (much too) many instances of typos and mistakes throughout the manuscript (e.g., responsevness (on p. 4) should be responsiveness).

Author Response

Responses to Reviewer 2

1. The author(s) should check English since there are (much too) many instances of typos and mistakes throughout the manuscript (e.g., responsevness (on p. 4) should be responsiveness).

>> Our response:

Thank you for noting these typos issues. We have thoroughly reviewed and corrected all these errors.

 

2. Providing a conceptual model showing the relationships between study variables is very nice. But indicate all hypotheses over the research model (Figure 1).

>> Our response:

Thank you for your valuable suggestions regarding the inclusion of all hypotheses in Figure 1. Initially, our intention was to present the hypothesized relationships in Figure 1. However, incorporating all these hypotheses in Figure 1 would result in a complex depiction, particularly with the inclusion of mediating paths for all variables, considering that we have not applied structural equation modeling (SEM) in our study. By doing so, we aim to preserve an intuitive logic for conducting a causal mediation analysis that is not reliant on SEM.

3. Provide more information about the profile of the respondents (age, gender, etc.). Provide more demographic data such as age (mean, range), gender (frequency/percentage), and usage experience (hours/day, year) in a table or text.

>> Our response:

We have included a new paragraph in section 3.1. Data, which describes the demographic characteristics of respondents. Additionally, we have created a new table to present this information in a concise manner.

4. Best practices require the list of items used in the questionnaires, accompanied by variables/dimensions, item codes, and sources. Please, reassess.

>> Our response:

Thank you for the valuable suggestions on how to list the items used in the                questionnaires. We would like to clarify that our preference is to reference all items in the body text within the measurement section, including indications of all reliability tests conducted. However, we have included the item codes in the body text as suggested.

1. The instrument is formed by 32 Likert-type items. Before the main analysis, a Confirmatory Factor Analysis (CFA) should be conducted:
2. Convergent and discriminant validity should be tested.

>> Our response:

In our field, internal consistency reliability, assessed through Cronbach's alpha, is the commonly used method to measure reliability. Before conducting the Cronbach’s alpha test, we also performed the Kaiser-Meyer-Olkin (KMO) test to evaluate the suitability of our data for factor analysis. The KMO test reported that most items for each variable in our survey ranged between 0.7 and 0.8, indicating adequate sampling. In this study, we did not employ structural equation modeling (SEM) for causal mediation analysis, we relied on the KMO and Cronbach’s alpha tests to validate the conceptual construction of our variables using items in the questionnaire, which was suggested by Kline’s (2014) work, “An Easy Guide to Factor Analysis.”

Additionally, we conducted an assessment of the average inter-item covariance to evaluate the internal consistency reliability. The optimal range of the average inter-item covariance is 0.15 to 0.50. If a score exceeds 0.50, it indicates that the items are highly similar and may exhibit redundancy. In our assessment, all scores remained below 0.50, indicating that each item possesses discriminant power in constructing to the measure.

 

We added the KMO and the average inter-item covariance tests in Table 3.

 

Furthermore, we have already taken the scores of the principal-component factors into consideration before constructing the measures. As a result, when plotting the items, they are appropriately grouped into each measure based on their clustering presented in Figures 1-3.

Figure 1. Component loadings for all variables

 

 

Figure 2. Score plot with principal-component factor for the dependent variable

 

 

Figure 3. Score plot with principal-component factor for the independent variable

 

1. CR and AVE values along with HTMT results should also be reported in the matrix to test the convergent and discriminant validity.

>> Our response:

We employed the casual mediation analysis technique following the recommendations of Imai et al. (2010) and Tingley et al. (2014). It is important to note that the calculation of composite reliability (CR) and average variance extracted (AVE) values are typically conducted within the framework of SEM since it involves calculating the differences between individual items. In our analysis, SEM was not utilized, thus CR and AVE values are not provided.

 

Heterotrait-monotrait (HTMT) ratio is a technique commonly employed after conducting SEM. As SEM was not performed in our analysis, the HTMT ratio was not applied in our analysis.

 

6. In this cross-sectional study, several instruments were applied at the same time. Therefore, Common method bias should be tested. One of the simplest ways to test if CMB is of concern in your study is to use Harman’s single-factor score, in which all items (measuring latent variables) are loaded into one common factor. If the total variance for a single factor is less than 50%, it suggests that CMB does not affect your data, hence the results.

>> Our response:

We acknowledged the potential common method bias (CMB) issue, as all data were collected using the same method, following the concerns raised by Podskoff and Organ (1986). However, it is important to note that our survey questionnaire was thoughtfully designed to capture specific concepts accurately to minimize CMB. For example, we detailed survey purpose and instructions given to respondents as participants' motivation to provide accurate information tends to increase when they are aware of how the gathered information will be utilized and how it will benefit either themselves or the organization involved (Podskoff et al., 2013).

 

We also conducted several reliability tests to ensure reliability of conceptual construction. Although we could not conduct Harman’s single-factor score, we stated this issue as one of this study’s limitations.

7. As for the mediation analysis, Preacher and Hayes (2008) suggest bootstrapping the indirect effect to test the mediation effect of the study. LL and UL do not straddle a 0 in between, indicating a mediation effect between the independent variable and dependent variable (Preacher & Hayes, 2008).

>> Our response:

Since we employed the casual mediation analysis technique in accordance with the guidance provided by Imai et al. (2010) and Tingley et al. (2014), it may not be necessary to employ a resampling distribution through bootstrapping.

 

8. Discussion & Conclusion should be enlarged, including a better interpretation of results and a better justification of the conclusions. Referring to the following recent studies may contribute to the study: “Understanding the impact of knowledge management factors on the sustainable use of AI-based chatbots for educational purposes using a hybrid SEM-ANN approach,” “An empirical examination of continuous intention to use m-learning: An integrated model,” “Examining the impact of psychological, social, and quality factors on the continuous intention to use virtual meeting platforms during and beyond COVID-19 pandemic: A hybrid SEM-ANN approach,” and “The impact of preservice teachers’ cognitive and technological perceptions on their continuous intention to use flipped classroom”. The author (s) need to modify the discussion by considering the above-mentioned points and modifying the results.

>> Our response:

We greatly value the suggestions provided to enhance the interpretation of our results and strengthen the justifications for our conclusions. The incorporation of more comprehensive insights derived from the suggested recent studies would substantially enhance the value of our research. Although our discussions were primarily grounded in the technology acceptance model (TAM) to investigate citizens' inclination to continue utilizing AI-enabled public services, integrating discussions on other approaches (e.g., expectation-confirmation model, artificial neural network, theory of planned behavior) could provide an alternative perspective for interpreting the hypothesized relationships examined in this study.

 

In the discussion and conclusions section, we have included additional explanations and discussions to further enrich the depth of the findings provided in our research.

9. The author(s) should check English since there are (much too) many instances of typos and mistakes throughout the manuscript (e.g., responsevness (on p. 4) should be responsiveness).

>> Our response:

Again, we have thoroughly reviewed and edited all these errors and issues.

 

Author Response File: Author Response.docx

Reviewer 3 Report

While the study provides insights into the functional factors that influence citizens' adoption and intention to continue using AI-enabled public services, there are several areas where the study can be improved.

- The authors use a cross-sectional online survey to collect data, which may not capture the full range of experiences and perspectives of citizens using AI-enabled public services. It would be beneficial to use a mixed-methods approach that includes qualitative data collection methods such as interviews or focus groups to gather more in-depth insights into users' experiences and perceptions of AI-enabled public services.

- The study focuses on only six functional factors and may not capture other important factors that could impact users' adoption and intention to continue using AI-enabled public services, e.g., factors such as trust in government and privacy concerns may also play a role in users' perceptions of AI-enabled public services.

- The study only focuses on AI-enabled public services in Korea, which limits the generalizability of the findings. It would be beneficial to conduct similar studies in other countries to explore the cultural and contextual factors that may influence users' perceptions of AI-enabled public services.

- The study does not provide any suggestions for improving the identified functional factors that influence users' adoption and intention to continue using AI-enabled public services. Future research could explore strategies for improving these factors and enhancing users' satisfaction with AI-enabled public services, e.g., user-centered design approaches and personalized feedback mechanisms could be explored to enhance users' satisfaction and improve the continued adoption of AI-enabled public services. Suggest adding: 1) Understanding Use Intention of mHealth Applications Based on the Unified Theory of Acceptance and Use of Technology 2 (UTAUT-2) Model in China. International Journal of Environmental Research and Public Health, 20(4), 3139. 2) From user-centered to participatory design approaches. In Design and the social sciences (pp. 18-25). CRC Press.

NA

Author Response

Responses to Reviewer 3

1. The authors use a cross-sectional online survey to collect data, which may not capture citizens' full range of experiences and perspectives using AI-enabled public services. Using a mixed-methods approach that includes qualitative data collection methods such as interviews or focus groups to gather in-depth insights into users' experiences and perceptions of AI-enabled public services would be beneficial.

>> Our response:

We appreciate the valuable suggestion regarding implementing a mixed-methods approach to enrich our study. While our primary focus was empirically testing hypothesized relationships, we recognize the potential benefits of collecting and incorporating qualitative data concerning users' experiences and perceptions of AI-enabled public services. Thus, we include this aspect as a topic for future research in the Discussion and Conclusions section.

 

2. The study focuses on only six functional factors and may not capture other important factors that could impact users' adoption and intention to continue using AI-enabled public services, e.g., factors such as trust in government and privacy concerns may also play a role in users' perceptions of AI-enabled public services.

>> Our response:

We highly value the suggestion regarding including additional influential factors on top of our six independent variables. We fully acknowledge the significance of trust in government and privacy concerns as critical predictors of the continued use of AI-enabled public services, as highlighted by the reviewer. However, it is important to note that the survey in this study did not include specific questions addressing these additional explanatory factors. This decision was made to maintain a concise survey length to ensure a high response rate and to minimize survey complexity for the respondents.

 

We have added this concern as an area of potential future research in the Discussion and Conclusions section.

3. The study only focuses on AI-enabled public services in Korea, which limits the generalizability of the findings. It would be beneficial to conduct similar studies in other countries to explore the cultural and contextual factors that may influence users' perceptions of AI-enabled public services.

>> Our response:

Thank you for your valuable feedback and for raising the concern regarding generalization based on our use of survey data from Korea. We want to ensure that our data quality for the empirical analysis is rigorously tested, ensuring its legitimacy within statistical and sampling boundaries. This validation process lends credibility to examining hypotheses based on the collected data.

We acknowledge the limitations of utilizing data from a single country, so we have added the implications of our findings within the specific context of Korea while also emphasizing the necessity for further research to establish broader generalizations in the Discussion and Conclusions section.

 

It is worth noting that our research serves as a foundation for future studies exploring cross-country comparisons and identifying commonalities or differences in various regions. We recognize the potential for our work to stimulate and inform future comparative research efforts.

 

4. The study does not provide suggestions for improving the identified functional factors that influence users' adoption and intention to continue using AI-enabled public services. Future research could explore strategies for improving these factors and enhancing users' satisfaction with AI-enabled public services, e.g., user-centered design approaches and personalized feedback mechanisms could be explored to enhance users' satisfaction and improve the continued adoption of AI-enabled public services. Suggest adding: 1) Understanding Use Intention of mHealth Applications Based on the Unified Theory of Acceptance and Use of Technology 2 (UTAUT-2) Model in China. International Journal of Environmental Research and Public Health, 20(4), 3139. 2) From user-centered to participatory design approaches. In Design and the social sciences (pp. 18-25). CRC Press.

>> Our response:

Thank you for pointing this out. We added suggestions and implications of our findings and how identified functional dimensions should be improved to increase citizen adoption and intention for continued use in the conclusion section. We appreciate the two sources the reviewer identified – we cited and included them in the discussion.

Author Response File: Author Response.docx

Reviewer 4 Report

Thank you for the opportunity to review a scientific study, which is especially relevant in today's digital age of INDUSTRY 5.0.

In general, the study can be evaluated very positively.

However, I recommend it to authors: 

1. supplement the abstract with at least one sentence describing the scientific research methods used,

2. supplement chapter 2 with a look at such possibilities as the electronicization of public administration, public participation in decision-making on important matters of self-government, since the realization of effective public administration is connected with the correct management approach, as done by: Srebalová, Mária, and Tomáš Peráček. 2022. "Effective Public Administration as a Tool for Building Smart Cities: The Experience of the Slovak Republic" Laws 11, no. 5: 67. https://doi.org/10.3390/laws11050067 as well as Funta, Rastislav, Králiková, Kristína. 2022. Obligation of the European Commission to review national civil court judgments? Tribuna Juridica, 12 (2), pp. 215-226, doi: 10.24818/TBJ/2022/12/2.04

3. supplement Chapter 5 Discussion and Conclusion with a few sentences about their future research.

Author Response

Responses to Reviewer 4

1. supplement the abstract with at least one sentence describing the scientific research methods used.

>> Our response:

As suggested, we added the name of the analytical approach in the abstract.

 

2. Supplement Chapter 2 with a look at such possibilities as the electronification of public administration and public participation in decision-making on important matters of self-government since the realization of effective public administration is connected with the correct management approach, as done by: Srebalová, Mária, and Tomáš Peráček. 2022. "Effective Public Administration as a Tool for Building Smart Cities: The Experience of the Slovak Republic" Laws 11, no. 5: 67. https://doi.org/10.3390/laws11050067 as well as Funta, Rastislav, Králiková, Kristína. 2022. The obligation of the European Commission to review national civil court judgments? Tribuna Juridica, 12 (2), pp. 215-226, doi: 10.24818/TBJ/2022/12/2.04

>> Our response:

We appreciate the valuable suggestions provided to enhance the discussions in our work. As suggested, we reviewed and incorporated further talks from the suggested references in Chapter 2.

 

3. Supplement Chapter 5 Discussion and Conclusion with a few sentences about their future research.

>> Our response:

In the Discussion and Conclusions section, we have previously highlighted potential ideas for future research that align with our ongoing investigations. Building upon your suggestions, we have expanded the discussion of future research within the section.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The authors have addresses all of my concerns. This study and its presentation were significantly improved by the adjustments made by the authors in the revised version.