Explainable Neural Tensor Factorization for Commercial Alley Revenues Prediction

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1-Some sentences are lengthy and complex. Breaking them into shorter, more manageable sentences would improve readability

2-There are minor grammatical errors throughout the paper. A thorough proofread would help in polishing the manuscript

3-Provide more details on the preprocessing and hyperparameter tuning steps.

 

Author Response

Comments 1: Some sentences are lengthy and complex. Breaking them into shorter, more manageable sentences would improve readability

Response 1: Following your advice, I have broken the sentences into shorter, more manageable sentences to improve the readability of the sentences throughout the paper.

Comments 2: There are minor grammatical errors throughout the paper. A thorough proofread would help in polishing the manuscript

Response 2: Thank you for your feedback. We've fixed more than 20 minor grammatical errors.

Comment 3: Provide more details on the preprocessing and hyperparameter tuning steps.

Response 3: We describe our data and preprocessing methods in the Dataset section, while the hyperparameters are covered in detail in the Hyperparameter Study section for a reproducible level of detail

Reviewer 2 Report

Comments and Suggestions for Authors

This paper introduces a new deep learning based recommendation model. It essentially combines a feedforward network to a tensor factorization (TF), which is comparable to a higher-order matrix factorization (MF). The author also proposes an explainable variant of this model. After introducing the method, the authors present an experiment on the Seoul Commercial Alley dataset.

The method is of high relevance to the community. The paper is clear and well explained. However my only concern is related to the experiment, which is not sufficiently compelling. I believe this paper can be accepted after the authors provide more experimental results. Those should include a comparison against SoTa recommender models, an ablation study where the effect of different components of the method is assessed (e.g. remove the loss of Eq. 11), and possibly application of the system to another dataset (since the method appears very generic, this should be doable).

Author Response

Comments 1: The method is of high relevance to the community. The paper is clear and well explained. However my only concern is related to the experiment, which is not sufficiently compelling. I believe this paper can be accepted after the authors provide more experimental results. Those should include a comparison against SoTa recommender models, an ablation study where the effect of different components of the method is assessed (e.g. remove the loss of Eq. 11), and possibly application of the system to another dataset (since the method appears very generic, this should be doable).

Response 1: We added two state-of-the-art models, Adaptive Factorization Network (AFN) and EulerNet, into our experiments. Despite this, our xNeuralTF continues to demonstrate the best performance, achieving the lowest MAE. Furthermore, xNeuralTF shows the best WEP and WER in the explainability comparison. AFN exhibited low accuracy due to its constraint on embedding vectors, which requires all values to be non-zero positives, potentially hindering the training process.

xNeuralTF is specifically designed for handling 3D tensors. However, since most public recommendation datasets are 2D and primarily focus on user-item interactions, applying xNeuralTF to other datasets is challenging. Generalizing xNeuralTF to arbitrary dimensional tensors is our future work.

 

Table 1. Performance comparison of different models.

	RMSE	MAE
Linear	0.764038(±0.039668)	0.351001(±0.009508)
TF	0.220472(±0.046086)	0.093706(±0.002539)
FM	0.269507(±0.045068)	0.115827(±0.002810)
MLP	0.205352(±0.028200)	0.088146(±0.002657)
Wide & Deep	0.205623(±0.026090)	0.088506(±0.002419)
DeepFM	0.202959(±0.028913)	0.087444(±0.002301)
AFN	0.360200(+-0.072595)	0.157997(+-0.019907)
EulerNet	0.207433(+-0.023316)	0.103130(+-0.004636)
NeuralTF	0.204925(±0.027364)	0.087600(±0.002782)
XNeuralTF(alley)	0.207087(±0.028343)	0.086300(±0.002460)
XNeuralTF(service)	0.204597(±0.027268)	0.085614(±0.001445)

 

Table 2. Explainability comparison of different models.

	WEP	WER
TF	0.423714	0.00022
FM	0.425143	0.000204
MLP	0.624571	0.000269
DeepFM	0.597714	0.000263
AFN	0.408286	0.000185
EulerNet	0.508857	0.000227
NeuralTF	0.650000	0.000280
XNeuralTF	0.654286	0.000282

Reviewer 3 Report

Comments and Suggestions for Authors

The current work describes two different new neural tensor factorization models, NeuralTF and XNeuralTF, and applies it to the problem of predicting commercial ally revenues. The performance of the proposed models exceeds those of existing models. The following minor comments and suggestions should be addressed during revision:

1. Page 5, figure 2, and related text. The authors chose to use addition to combine the results from the TF and MLP components. How was this determined? Other data combination method, such as dot product (e.g., https://doi.org/10.1016/j.cma.2023.116277) and a combination of arithmetic operations (e.g., https://doi.org/10.1016/j.cma.2023.116681) have been used and seen good success. The authors should discuss how this data combination method will potentially affect the prediction accuracy of the model.

2. During the hyperparameter optimization, the models were trained for 1000 epochs. How many epochs were the models trained for after they have been optimized? 

3. Were the results in Sections 4.3 and 4.4 generated with the optimized hyperparameter set? Since Figure 8 shows a generally increasing trend for error vs. explainability, what Explainability Strength value was used to generate the results in Section 4.4? The authors should also provide some practical guidance on how to select this parameter to balance accuracy and explainability.

Author Response

Comments 1: Page 5, figure 2, and related text. The authors chose to use addition to combine the results from the TF and MLP components. How was this determined? Other data combination method, such as dot product (e.g., https://doi.org/10.1016/j.cma.2023.116277) and a combination of arithmetic operations (e.g., https://doi.org/10.1016/j.cma.2023.116681) have been used and seen good success. The authors should discuss how this data combination method will potentially affect the prediction accuracy of the model.

Response 1: Combining explicit and implicit components is a popular strategy for improving performance in recommender systems. We chose the tensor factorization component to make xNeuralTF specialized for 3D tensors and the MLP component to capture complex non-linear patterns. In our experiments, we compared different combinations of components, including Wide & Deep and DeepFM. xNeuralTF achieved the lowest MAE among these models.

Comments 2: During the hyperparameter optimization, the models were trained for 1000 epochs. How many epochs were the models trained for after they have been optimized? 

Response 2: We also trained the models with 1000 epochs and selected the best validation model (i.e., the lowest validation loss) to measure their performance.

Comments 3: Were the results in Sections 4.3 and 4.4 generated with the optimized hyperparameter set? Since Figure 8 shows a generally increasing trend for error vs. explainability, what Explainability Strength value was used to generate the results in Section 4.4? The authors should also provide some practical guidance on how to select this parameter to balance accuracy and explainability.

Response 3: In our experiments, we set the explainability strength λ₂ to 0.001. Based on the results of our hyperparameter study, we recommend setting λ₂ to either 0.001 or 0.005 to achieve a balance between accuracy and explainability.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

My previous comments have been addressed and I believe the paper is now suitable for publication.