Personality or Value: A Comparative Study of Psychographic Segmentation Based on an Online Review Enhanced Recommender System

Round 1

Reviewer 1 Report

The paper investigates whether psychographic segmentation achieved using natural language could be useful in the case of e-commerce websites. The topic is interesting and definitely worth investigating. 

The "Introduction" section provides an adequate amount of information regarding the topic. The authors start from the observation that psychological variables such as the personality of the potential customer can be observed using Natural Language Processing techniques. This creates the opportunity to segment the customers by analyzing the text of their reviews.

The "Literature Review and Research Design" section includes a comprehensive overview of the related works and clearly states the contributions of the paper by pointing out the main research questions.

The "Methodologies" section adequately describes the approach taken for building the lexicon used for psychographic recognition as well as the chosen machine learning algorithms.

The "Experiment" section sufficiently describes on the experimental procedure and the Amazon Review dataset.
The "Results Analysis" section discusses the results achieved by applying the proposed approach in a comprehensive manner.
In the last section of the paper, the authors provide an in-depth discussion. The conclusions are well supported by the ideas discussed throughout the paper.

The paper is well written and can be accepted for publication. Please find below my recommendations.

# Recommendations

- the authors are advised to better highlight whether the proposed psychographic segmentation approach can be used together with other well-known customer segmentation approaches (ex: demographic segmentation, behavioral segmentation, geographic segmentation, etc.).

- it would be advisable to also better highlight the situations in which the proposed psychographic segmentation cannot be applied successfully: limited number of reviews for a user / limited number of words in the reviews written by a user. 

- line 466: The authors mention that "K-core 12 is capable for users SVS and BFF recognition compared to 25 tweets". The meaning of the phrase as well as the purpose of mentioning the number of tweets is not entirely clear.

- in order to facilitate the reproducibility of the results, the authors might consider uploading the lexicon used for psychographic recognition as supplementary material  

# Minor mistakes

- line 26: "includes" should be "include"
- line 70: The word "The" should not be capitalized.
- line 252: "What is he" should be "What is the"

Author Response

Point 1.1

# Recommendations

- the authors are advised to better highlight whether the proposed psychographic segmentation approach can be used together with other well-known customer segmentation approaches (ex: demographic segmentation, behavioral segmentation, geographic segmentation, etc.).

RESPONSE:

Sure, we think these psychographic segmentations can be used together with other consumer segmentation approaches, especially behavioral segmentation. Indeed, segmentation is shifting away from being something monolithic and turning toward a state where many different types of segmentation are able to coexist simultaneously (for example, Bayer,2010; Wen and Run, 2013). By the way, we are working on it. We have made it our future research in paper. See p25 for details.

Bayer J. Customer segmentation in the telecommunications industry[J]. Journal of Database marketing & customer strategy management, 2010, 17(3-4): 247-256.

Teck Weng J, Cyril de Run E. Consumers' personal values and sales promotion preferences effect on behavioural intention and purchase satisfaction for consumer product[J]. Asia Pacific Journal of Marketing and Logistics, 2013, 25(1): 70-101.

Point 1.2

- it would be advisable to also better highlight the situations in which the proposed psychographic segmentation cannot be applied successfully: limited number of reviews for a user / limited number of words in the reviews written by a user. 

RESPONSE:

We adopted your advice, and make it our future work. See p25 for details.

Point 1.3

- line 466: The authors mention that "K-core 12 is capable for users SVS and BFF recognition compared to 25 tweets". The meaning of the phrase as well as the purpose of mentioning the number of tweets is not entirely clear.

RESPONSE:

We want to avoid the potential bias caused by limited number of reviews for a user / limited number of words in the reviews written by a user, by giving an example of how many words are needed to assess consumer psychographic. The average length of review in our work is 189 words, basically we got consumer psychographic form 1890 word with 10 reviews. Considering the length of tweet are shorter than the reviews we got, we think 10 reviews and about 1890 words is reasonable.  

Point 1.4

- in order to facilitate the reproducibility of the results, the authors might consider uploading the lexicon used for psychographic recognition as supplementary material  

RESPONSE:

Thanks for your suggestions, we would like to provide all 4 lexicons we got for the reproducibility of our works.

Point 1.5

# Minor mistakes

- line 26: "includes" should be "include"
- line 70: The word "The" should not be capitalized. 
- line 252: "What is he" should be "What is the"

RESPONSE:

Thanks for your comments, we have revised these Minor mist

Author Response File: Author Response.docx

Reviewer 2 Report

This is an interesting paper with a promising methodology and results. The authors analyze the use of lexicon-based measurements of SVS and BFF scores in order to psychographically segment users in order to better predict their product preferences (I.e., scores given for specific products in Amazon reviews). DBSCAN is used to cluster users and the best predictions are achieved using a 2-layer DNN. In the end, the authors make a good case that the linguistically inferred traits are related to consumer behavior. Overall, this is a nice paper, but I am left with some outstanding questions that should be addressed before I can confidently recommend the paper for acceptance.

In section 3.1.1., I’m a bit confused about exactly what words are used as seed terms. Is it the category names (e.g., “Empathy”), or the words belonging to those categories (e.g., “People, Treat, Respect, Kindness”)? If it’s the latter, did you only use the example words listed in the Boyd et al. paper, or did you obtain the full list of words belonging to each category? If it’s the former, how were things like “Faith (Positive)” and “Faith (Negative)” differentiated and looked up in wordnet? Also, these themes are only loosely related to the SVS, and in fact, one of the central claims of Boyd et al. is that these value themes are much more predictive of behaviors than the SVS dimensions themselves. Why not just use the 15 or so value-related themes that Boyd et al. propose directly, instead of trying to relate them back to the SVS via some (arguably) weak correlations? Perhaps this will lead to even better segmentation.

In section 3.1.2., when you find the most similar 10 words to the seed terms, is this the 10 most similar vectors to *each* seed term or to *the average* of all seed terms for a given lexical category? What is the “thesaurus” being used here? 

Equation 2 doesn’t quite make sense to me. Why is w_seed1 a parameter to the equation if it is never actually used on the right-hand side? Why is there no similar equation for BFF scores?

In equation 3, what exactly is w_ij? Where would I obtain this number?

At the end of section 3.1, more details are needed about the prediction experiments performed. How were the 1000 users selected? How was this dataset obtained? How was the text data preprocessed? Were the exact scores being predicted for each user, or were users sorted into classes? Was a normalization or mapping done to ensure that the value or personality scores were in the same range as the survey results? What would be the MAE of predicting the average scores across the 1000 users, or of randomly guessing? It’s very difficult for me to understand the meaning of the reported MAE scores without more details on the setup.

In the description of SVM, what is w? What is epsilon? What is C? Why include the mathematical notation if it isn’t defined or referenced later?

In the description of DNNs, the loss function given is one example, but it wouldn’t quite be correct to say that a DNN minimizes (y-y_hat)^2 as there are many other objective functions that can be used. It might be better to say that “in this case, we use the following loss function”, or something along those lines. When you say that the network isn’t randomly initialized, where are the initial weights coming from? Is it a pertained network of some kind?

I believe the Amazon datasets used in 4.1 are originally from J. J. McAuley, C. Targett, Q. Shi, and A. van den Hengel 2015, not He and McAuley 2016, as the authors write. 

In section 4.2, the authors mention that they perform stemming, but wouldn’t this make it difficult to match words correctly with those in the lexicons? Word stems are not necessarily real words, and depending on the stemming algorithm used (which should also be noted here), things like “happiness” will be converted to “happi” which won’t match “happy” if it’s in the lexicon. Related: which stop word list was used? Which tokenizer? Which normalization map?

In the prediction experiments, how was the validation set selected? Was this part of the training set for each fold or a separate held-out dataset?

What does “Rond” mean in tables 4,5,6? (Sorry if I’m missing something obvious here).

Line plots don’t really make sense for figure 5 since moving across the x-axis has no meaning as it is categorical. Bar/column charts would be better here.

In these experiments, is there any way to estimate what level of RMSE would actually constitute a useful model for marketing professionals? Do any previous study suggest how good (in RMSE) predictive models can do on similar tasks using text features (e.g., all word embeddings from the product reviews) alone, rather than mapping everything to SVS/BFF scores as the independent variables?

In the discussion section, I’d avoid using “sentiment analysis” to refer to what was done, and instead say something like “lexicon-based analysis”, since “sentiment analysis” has a very specific meaning in the computational linguistics literature.

There are wording and grammatical issues throughout the paper that should be addressed. I would specifically suggest that the authors re-examine the writing in sections 3.1 onward, as there are numerous issues to be addressed, and there are relatively few mistakes up to that point (though some do exist). Finally, the axis labels and titles in figures 6 and 7 need to be enlarged.

Author Response

The point-by-point response to the reviewer’s comments was uploaded as a Word file below.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Thanks to the authors for carefully considering my concerns from before, and I believe that they have been addressed in the author response and are also reflected in changes in the paper. The text, equations, and figures are much more clear and are described adequately in this new version.

My main remaining concern is that there are still a fair number of English language issues that should be resolved before publication. Some of these were already pointed out in my first review (see that review for more examples), such as:

In figure 2, “Clusteing” should be “Clustering

“What is the predicting and explanatory power…” might be better worded as “What is the predictive and explanatory power…”

“semantic knowledge and online corpus can” should be “semantic knowledge and online corpora can” or “semantic knowledge and an online corpus can”

...

 Some additional examples include:

 "(details words see Boyd..." should be "(for details regarding the set of words, see Boyd..."

The text that originally was "Rond" was changed to "Rondom" instead of "Random" as the authors claimed to have done in the author response.

"...their synonyms got by WordNet." should just be "...their synonyms from WordNet."

...

Again, these lists are not exhaustive, and the English issues throughout the paper should be addressed before publication.

Author Response

We tried our best to improve the manuscript and made some changes in the manuscript. These changes will not influence the content and framework of the paper. And here we list the changes briefly which marked in the “track of change “part of revised manuscript. We appreciate for Editors/Reviewers’ warm work earnestly, and hope that the correction will meet with approval.

Revised list:

1.     We sincerely thank the reviewer for careful reading. As suggested by the reviewer, we have corrected the “predicting power”, “Clusteing”, “Rondom”, “corpus” and “WordNet” part.

2.     We split the psychographic segmentation to two part: psychographic variables and segmentation method, to make our findings clearer. The same changes have been made throughout the paper.

3.     I basically changed the tense of our article to the present tense; the past tense was used only when discussing previous work. The present perfect tense is used when summarizing or when mentioning content of the previous chapter when introducing a section.

4.     Titles of tables and figures. we add a legend to Figure 4 to discriminate train and test line.