IS-DT: A New Feature Selection Method for Determining the Important Features in Programmatic Buying

Round 1

Reviewer 1 Report

I read the work with interest. I liked the writing style, the text is unitary and logical.

I appreciate the results obtained after applying the questionnaire and their way of interpretation through the new proposed model. I think it is correct to say that these obtained characteristics are very important for m-shopping providers.

 However, I asked myself the following questions as I read:

- In the applied questionnaire, the respondents were asked if they accessed a website/completed a purchase, based on the information provided by the displayed advertisements? ("programmatic buying" (PB), which generally refers to the use of computer programs to allow the right people to see the right advertising information at the right time to complete the purchase, [...]).

- How can PB providers use these features? In this case, is the sample of respondents used representative?

 Recommendations:

- Considering that the title of the paper refers to PB, the results obtained and the synthesized conclusions do not make the connection between how those who manage PB can actually use characteristic results for customer loyalty, I propose rethinking the title of the paper or adding a paragraph to the Conclusions that would the connection between the obtained research results and the title of the paper.

Thank you very much!

Author Response

Responses to Reviewer #1

I read the work with interest. I liked the writing style, the text is unitary and logical. I appreciate the results obtained after applying the questionnaire and their way of interpretation through the new proposed model. I think it is correct to say that these obtained characteristics are very important for m-shopping providers. However, I asked myself the following questions as I read:

Response:

Thank you for your comments regarding the submitted paper. The following corrections have been made according to your suggestions. Please check the revised manuscript.

Comment #1

In the applied questionnaire, the respondents were asked if they accessed a website/completed a purchase, based on the information provided by the displayed advertisements? ("programmatic buying" (PB), which generally refers to the use of computer programs to allow the right people to see the right advertising information at the right time to complete the purchase, [...]). 

Response:

Thank you very much for your valuable comments and suggestions. We modified it based on your suggestions. Please confirm it on Page 1.

Comment #2

How can PB providers use these features? In this case, is the sample of respondents used representative?

Response:

According to your suggestion, we have added suggestions for PB providers in 2_~4 paragraphs of conclusion section. About samples, the main purpose of this study is to propose a feature selection method that integrates domain knowledge and apply it to PB. We recommended to include a larger sample size in future studies to draw general conclusions. Please check the modifications in conclusion section.

Recommendation:

Considering that the title of the paper refers to PB, the results obtained and the synthesized conclusions do not make the connection between how those who manage PB can actually use characteristic results for customer loyalty, I propose rethinking the title of the paper or adding a paragraph to the Conclusions that would the connection between the obtained research results and the title of the paper.

Response:

According to your suggestion, we have added suggestions for PB providers in 2_~4 paragraphs of conclusion section.

Reviewer 2 Report

Dear Authors,

I am very pleased to be able to review your scientific article. It touches upon the important from the point of view of, inter alia, implementers of big data solutions in e-commerce entities (but indirectly also for e-consumers themselves) an issue that gives new light for predicting consumer behavior, also related to their retention, loyalty and trust. I am also pleased with the thourough level of knowledge demonstrated by the authors in constructing the new approach and in the cited literature sources (however: in the following comments I also suggest reading other items, but this is a strong recommendation, not a thrust nor pressure). Although the approach presented by the authors is innovative, I have some remarks that I recommend that you implement in the final version of the manuscript:

1. In my opinion, it is the issue of user retention that should be raised (at least briefly). Retention is directly (behaviorally) related to user / consumer loyalty. I suggest reading this article to see this relationship between these two constructs: https://doi.org/10.1080/10196780802265843 - also citation [37] seems to be appropriate to use in this context (it's also up-to-date).

2. Sources are missing in the "introduction" section on the first page. When the authors expand on the definition of "programmatic buying" (PB), they are not citing any literature. Moreover, the sentence "According to statistics, the global PB market reached $129 billion in 2020, with an annual growth rate of 14.3%" also does not mention any report - the general statement "according to statistics" without mentioning the source is not scientific and inadmissible. This should be corrected, even if the citation [2] has to be repeated (unless the authors cite a different source).

3. Sources are also missing in the next paragraph. "It is an emerging and rapidly developing IT phenomenon that uses huge amounts of data (big data) to disseminate deeply personalized marketing themes to target audiences" - who indicates (or what scientific material) the authors refer to? Moreover, evaluative statements like "programmatic advertising is very important" lacks of quality and scientific soundness. Authors should refer to the source of the importance of the notion of programmatic advertising.

4. I am also not delighted with the direct transition to the description of the importance of m-commerce spheres without first announcing the term e-commerce and its relevance in the context used by the authors. I understand that the emphasis is on m-commerce, but without this transition ("from e-commerce to m-commerce"), the introductory part of the article is too "rugged" and loses fluidity/coherence significantly.

5. As you publish an article in the journal about particular emphasis on big data, I also missed the importance of big data mining mechanisms (or broadly speaking: big data analytics) in the context of the process of achieving comprehensive and in-depth customer insight (in particular in the context of satisfaction and loyalty). You also describe the importance of timeliness as an aspect influencing the processes related to PB. I suggest you read these articles regarding aforementioned issues:
a) https://doi.org/10.3390/app11156993
b) https://doi.org/10.1016/j.ijinfomgt.2019.11.002
c) https://doi.org/10.1016/j.ipm.2018.01.010

6. A small note about the application nature of ML approaches. Maybe it is also worth considering the use of ML-based tools in the case of shaping the demand? I would also pay attention to the growing trend of building proprietary algorithms using ML in the context of gaining customer insight in e-commerce (also their engagement/satisfaction/trust) and the programmatic buying itself. I recommend that you familiarize yourself with the following literature items:
a) https://annals-csis.org/proceedings/2022/pliks/256.pdf
b) https://doi.org/10.1177/1096348017753521
c) https://doi.org/10.1080/09540091.2021.1912710

7. It is true that there are no references that would indicate factors affecting the level of consumer loyalty / satisfaction in programmatic buying. However, I did come across a very recent article (published in July 2022) which, while admittedly revolves around a relatively narrow area in this regard, may have some content in it regarding percieved usefulness and annoyance that are worth considering in own work.  https://doi.org/10.1108/CCIJ-03-2022-0033

8. A very apt approach presented by you is to illustrate concepts (like SERVQUAL / e-SERVQUAL, LASSO, SVM-RFE etc.). In the context of m-commerce, when you mention the TAM approach, it should be noted that it is a relatively old concept and its further advancements are also used today. Such are, for example, TPB, UTAUT or TAM-TPB. I am able to present you many articles that deal with these issues, but I do not want to overburden you too much with the excess of sources. However, I suggest extending the spectrum of your studies by signaling these concepts in the context of the high application character in the m-commerce industry.

9. Section 2.4 - I suggest reducing the distance between the reference to Figure 1 and the figure itself.

10. Page 9 - Figure 3 is incorrectly labeled "Figure 2".

11. I suggest working on the quality of the figures (currently they are low resolution, and quite uneven, with chaotic formatting).

12. Table 3 is too "tight" in my opinion. Personally, I think that a better solution would be to use two columns: Variable and Distribution, rather than stacking them next to each other. A similar note to Tables 5 and 7.

13. I have no methodological objections. The solution presented by you is innovative, clever and concise. The description of the methodology is at a high level, and the verification is also carried out in accordance with the art of science.

14. Note on the "conclusions" section. In my opinion it is not very clear as it is and I would improve its readability. Adding a subsection with working names "further research" and "limitations" would definitely improve the reception.

15. Moreover, I suggest checking the text in terms of language in order to eliminate numerous stylistic and grammatical errors that are detrimental to the reader's perception of the content.

Summing up, I believe that this work contributes a lot to the scientific sphere and provides an appropriate level of scientific soundness. As this work is methodologically high, I suggest that you follow my amendments - mainly those related to improving the overall quality of the presentation and extending the literature database. After applying the corrections suggested by me, I believe that this article is amenable to publication. Thank you for your cooperation.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

The article proposes a feature selection method based on Importance-Satisfaction (IS) and Decision Tree (DT) algorithm to identify service quality factors in programmatic buying.

In my opinion, The proposed method has some serious issues listed below:

- The dataset used (133 responses to an online questionnaire) is pretty small for effective conclusions.

- Most service quality factors are already stated in literature (24 in literature + 2 proposed by the authors).

- The total score of the IS-DT method for each method is just the product (Why the product) of IS and DT scores.

- The authors claimed to perform 5-fold cross-validation, then stated that they selected one tree with the best classification performance. (Section 3.2, Page9). This is not cross-validation, as cross-validation means taking the average results across the 5-folds.

Author Response

Responses to Reviewer #3

The article proposes a feature selection method based on Importance-Satisfaction (IS) and Decision Tree (DT) algorithm to identify service quality factors in programmatic buying. In my opinion, the proposed method has some serious issues listed below:

Response:

Thank you for your comments regarding the submitted paper. The following corrections have been made according to your suggestions. Please check the revised manuscript.

Comment #1

The dataset used (133 responses to an online questionnaire) is pretty small for effective conclusions.

Response:

The main purpose of this study is to propose a feature selection method that integrates domain knowledge and apply it to PB. Regarding the insufficient data size, we have add it in the limitation. And we recommended to include a larger sample size in future studies to draw general conclusions.

Comment #2

Most service quality factors are already stated in literature (24 in literature + 2 proposed by the authors).

Response:

From the existing literature, the published literatures only discuss some of the service quality factors in different issues/topics. No single one paper considers all possible variables and attempts to find the crucial ones. In addition, this study proposed a new feature selection method. For such feature selection method, we must include all possible relevant candidate factors to select important attributes from these candidate attributes. Therefore, we collected 24 potential candidate factors from available literature and 2 presented by this work for our experiment.

Comment #3

The total score of the IS-DT method for each method is just the product (Why the product) of IS and DT scores.

Response:

Many thanks for your suggestion. We have included this point in research limitations. In future research directions, we have suggested to try different combinations except product, such as the sum of the two. Please check the last paragraph of conclusion section in page 20. They also could be found as below.

“Besides, we used the total score of the IS-DT method for each method is just the product of IS and DT scores. Future researchers can try different combinations rather than product, such as the sum of the two.”

Comment #4

The authors claimed to perform 5-fold cross-validation, then stated that they selected one tree with the best classification performance. (Section 3.2, Page9). This is not cross-validation, as cross-validation means taking the average results across the 5-folds.

Response:

In our algorithm, step 6.1~6.3 are shown as below.

Step 6.1: Use a 5-fold cross validation experiment and build a DT for each fold of data. In other words, the data set was divided into 5 equal sized sets and each set was then in turn used as the test set.

Step 6.2: Compute the occurrence frequency of features in nodes.

Step 6.3: Pick a tree whose performance is the best and rank features by its attribute usage.

We indeed implement 5-fold cross validation experiment to build 5 trees. But, unlike other works, we only pick the tree with the best performance for further feature selection. But, according to your suggestion, we have added mean and standard deviation of 5 folds in table 6 (Page 17).

Reviewer 4 Report

Dear Authors.
After reading and evaluating your research, I believe that the work done is of a very high standard, with a rigorous scientific character, an exemplary structure, and an excellent presentation.

I have only two suggestions that could use another technique in future work, namely the first is the use of structural equation models, using the SmartPLS software. The second is to include other intermediate constructs, such as price, customer experience, customer engagement, etc. These constructs are often used in marketing and are constructs that precede satisfaction and loyalty.
Congratulations on your work.

Author Response

Responses to Reviewer #4

Comment #1.

After reading and evaluating your research, I believe that the work done is of a very high standard, with a rigorous scientific character, an exemplary structure, and an excellent presentation. I have only two suggestions that could use another technique in future work, namely the first is the use of structural equation models, using the SmartPLS software. The second is to include other intermediate constructs, such as price, customer experience, customer engagement, etc. These constructs are often used in marketing and are constructs that precede satisfaction and loyalty. Congratulations on your work.

Response:

We are grateful for your valuable suggestions. We have added related descriptions in the potential directions of future works in conclusion section. Please check the last paragraph in page 20. The added part can also be found below.

“By the way, to utilize structural equation models (SEM) by using the SmartPLS software, and to include other intermediate constructs, such as price, customer experience, customer engagement, and so on are also one of directions of future works.”

Round 2

Reviewer 3 Report

I take this opportunity to congratulate the author(s) for the effort they demonstrated in revising the paper.

The authors clarified the most critical issues, and the limitations of the work are now added to the article and are planned for future research.