A Machine-Learning-Inspired Opinion Extraction Mechanism for Classifying Customer Reviews on Social Media

Abstract

Machine learning frameworks categorizing customer reviews on online products have significantly improved sales and product quality for major manufacturers. Manually scrutinizing extensive customer reviews is imprecise and time-consuming. Current product research techniques rely on text mining, neglecting audio, and image components, resulting in less productive outcomes for researchers and developers. AI-based machine learning frameworks that consider social media and online buyer reviews are essential for accurate recommendations in online e-commerce shops. This research paper proposes a novel machine-learning-based framework for categorizing customer reviews that uses a bag-of-features approach for feature extraction and a hybrid DNN framework for robust classification. We assess the performance of our machine learning framework using AliExpress and Amazon e-commerce product review data provided by customers, and we have achieved a classification accuracy of 91.5% with only 8.46% fallout. Moreover, when compared with state-of-the-art models, our proposed model shows superior performance in terms of sensitivity, specificity, precision, fallout, and accuracy.

1. Introduction

Recently, due to the COVID-19 pandemic, customers’ intention has shifted towards social media and online e-commerce platforms. They express and post their opinions about products and services, shaping brand perception [1,2]. The large-scale manufacturer relies on customer reviews, ratings, and feedback related to their product design and quality. However, the effectiveness of customer reviews is largely impeded by fake reviews [3]. Moreover, the manual analysis of these reviews is time-consuming, thus further reducing effectiveness. To address this, automated opinion extraction mechanisms use machine learning algorithms to classify customer reviews as positive, negative, or neutral. Social media platforms, including Facebook, Twitter, Instagram, YouTube, WhatsApp, and TikTok, have billions of users sharing their content, feedback, and reviews related to products, etc. Product reviews on these platforms exist in a textual, visual, and vocal form, which requires robust feature extraction techniques to improve classification performance [4]. Smart cities equipped with an intelligent navigation and positioning system can provide an intelligent recommendation system based on consumer reviews on social media platforms. The recommendation mechanisms become more sophisticated when users’ verbal, visual, and textual content is classified through machine learning and deep learning models. In the online e-commerce market, product suggestions can be made intelligently by utilizing social media reviews posted by online customers for DNN model training and optimization. The consumer textual sentimental opinions related to products require natural language processing (NLP) techniques to identify key phrases and terms that are associated with particular features [5], whereas the visual text in the shape of an image needs optical character recognition (OCR) followed by NLP [6]. The vocal feedback and reviews require voice recognition based on 1D CNN models and handcrafted models [7].

Recently developed AI frameworks have been broadly classified into two categories known as machine learning and non-machine-learning frameworks [8]. Machine learning methods require a robust and distinctive set of features extracted from the training database, whereas non-machine-learning frameworks do not require feature extraction mechanisms. The non-machine-learning techniques categorize the input files by their signature data. In the static and dynamic analysis methods, feature values are extracted in the idle and the program execution stage. Non-machine-learning methods solely depend on the developer’s expertise and do not consider prior examples of malware information. Moreover, non-machine-learning methods do not require special-purpose software and hardware resources. The machine learning models are most often employed in combination with isolated classifiers that can learn the pre-existing training data [9]. Generally, ML methods, including reinforcement learning, have become a common practice in cybersecurity and healthcare systems [10]. The machine-learning-based identification is carried out in two stages, which are the feature extraction and classification stages. Two broad categories of feature extraction are employed, consisting of handcrafted statistical methods and deep neural networks. Both extract feature values based on combinations of data values in their neighborhoods.

Depending on customer reviews, the existing feature extraction models can broadly be classified into three classes: visual-image-based representation, voice-based features, and textual features. The visual features require image descriptors to extract language characters as a pre-processing stage of the NLP methods [11]. The existing visual object segmentation and representation both are handcrafted as well as DNN-based [12]. Images with diverse set textures, shapes, and colors require a hybrid features model to include textured features, such as overlapped multi-oriented tri-scale local binary pattern (OMTLBP) [13], robustness-driven hybrid descriptor (RDHD) [14], and discriminative local features of overlapping stripes (DLFOS) [15]. Deep neural networks developed to detect textual characters of different languages include BLPnet [16], HMM-DNN [17], distance-based edge linking (DEL) [18], and KDANet [19]. The DNN model recognizes the character of specific language depending on text mining methods to bag-of-words [20,21].

The visual and vocal reviews of the customer transformed into textual shape require a pre-processing stage of voice and text character recognition. The images and voice pre-processing sub-stages are included for the precise transformation of data into textual shape [22]. Visual review can be based on either video or image data. In the case of video, vocal analysis can be used for classification, while, in the case of an image, review classification requires a distinctive feature set that remains robust and invariant to variations in view angle, scale, and illumination [23]. Text images, which consist of characters with various shapes and textures, necessitate texture and shape representation models [23]. Due to intra-class variations in color, shape, and illumination, image identification becomes challenging, and, therefore, stable features are considered. Various types of neural networks, such as AlexNet, ResNet, DagNet, Vgg-16, InceptionV3, GoogleNet, and Yolo, have been developed for the classification of the ImagNet dataset [24]. These models have achieved promising performance in image classification, but they require extensive data for fine-tuning new categories. Therefore, handcrafted statistical frameworks are associated with a pre-processing step for feature extraction. Textured images can be represented using local binary pattern (LBP) and gray-level co-occurrence matrix (GLCM) [25]. LBP and GLCM are chosen because of their noise robustness and photometric invariance. To represent the input image as a whole for object recognition, we use the concatenated shape of all three features—color, texture, and shape.

The robust features require distinctiveness and a stable classification approach for precise categorization of customer reviews. Researchers have employed various isolated classifiers, including multi-class SVM, discriminant analysis, ensemble, decision tree, and KNN, to obtain the estimated label of the test image [26]. Kernel improves the classification performance of existing classifiers by mapping non-linear data into high-dimensional space [27]. The kernel function estimates the similarity between pairs of samples in the transformed spaces. Classifier kernel functions include linear, polynomial, radial basis function (RBF), and sigmoid kernels [28]. Careful selection of kernels enhances the categorization precision of the model; the RBF kernel outperforms other kernel types in many classification problems, whereas the polynomial kernel can perform well in image recognition tasks [29]. The performance of the classification models has been validated by comparing the estimated label with the true label of the test data. The label represents the class name of the customer review. Regarding the set of true labels and estimated labels, when compared, the classification performance is evaluated and presented in the shape of a confusion matrix. The true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) obtained from the set of true and predicted labels are used to validate the performance of the image recognition frameworks.

The industrial sectors and product manufacturers encourage constructive feedback from reviewers; however, initially, when purchasing a product, consumers can hardly provide valuable feedback related to the specific product. However, after using the product for a specific time period, reviewers can initiate effective feedback. In the case of online shops and social media platforms, customers can post comments/reviews instantly without any hurdles or visit the product manufacturing and seller’s shop. This enables online sellers to collect and provide valuable information to the manufacturer. Customer feedback, in most cases, is extensive, containing thousands of reviews and redundancies. Therefore, an AI-based algorithm is desired to automatically and precisely classify customer reviews and ratings into respective classes, helping the manufacturer to improve product quality. Existing customer reviews classification approaches have low accuracy due to the complex nature of human language. In our paper, we propose a new opinion extraction mechanism that effectively classifies customer reviews on social media using natural language processing techniques and a deep learning neural network. We evaluate our approach on a real-world dataset and show that it outperforms existing approaches, achieving high accuracy in classifying reviews. To validate our approach, we compare our results with those obtained by human evaluators.

The main contribution of this work is listed below:

• Bag-of-words hybrid approach: The proposed article employs the bag-of-words model to cluster the textual reviews and create a histogram feature. The 14-layer DNN is also employed to transform raw features into invariant BoW features.
• DNN-based optimization: The 14-layer DNN model consisting of 1D CNN, LSTM, and BiLSTM optimizes the features, and softmax along with the classification layer is introduced to classify the features into their respective categories.

The remainder of the paper is organized such that Section 2 demonstrates comprehensive details of the related work on opinion extraction and sentiment analysis. Section 3 explains the proposed approach in deep detail. Section 4 analyzes the experimental results and compares them to recently reported works. Finally, Section 5 concludes the paper and discusses future research directions.

2. Related Works

Opinion extraction based on costumers’ social media posted reviews is a crucial task with complex challenges, including language variability, syntax errors, vocal tone variations, format variability, and script variability. Opinion extraction in natural language processing (NLP) aims to identify and extract opinions from text data. Several studies have been conducted to explore the different techniques and approaches used for opinion extraction, with machine learning being a popular choice due to its ability to learn patterns and classify data. Machine learning frameworks require robust features from the input data of uniform shape and a distinctive classifier. Various features extraction methods have been adopted by researchers in their frameworks for feature extraction and classification.

2.1. Feature Extraction

Feature extraction is the process of calculating stable attributes from data that have less intra-class variation and more inter-class variation. It is the process of transforming raw data into a discrete finite set of values with a uniform and lower dimension. Feature extraction is crucial for the precise classification of data into their respective classes. Moreover, feature extraction represents the input file with a selected set of values, excluding redundant information. Researchers have developed their robust and distinctive frameworks for text feature extraction and employed them in text mining and sentiment classification applications [30,31]. Various frameworks for extracting robust and invariant features from text data have been developed by researchers, which can be broadly categorized into two classes: handcrafted and deep neural networks. The handcrafted descriptors are developed through feature engineering and mostly consist of statistical models. The researcher adopts various statistical analyses to represent the distribution of words in the textual data for precise categorization. The frameworks of shallow learning (SL) enormously comprise machine learning methods with handcrafted feature representation methods. Shallow learning requires a combination of heavy dimensional features provided in [32]. However, the SL techniques are still applicable to textual data classifications. In [33], the bag-of-n-grams and parallel convolution neural networks (CNNs) frameworks are developed to detect and categorize fake and real reviews. In [34], costumer reviews have been classified by employing the eigen-space graph generation and then classifying the graphs by KNN and SVM classification methods. In [35], the authors have developed a deep learning framework for review sentiment classification. This approach incorporates rating information to capture the overall sentiment distribution of sentences. It involves adding a classification layer over the embedding layer and fine-tuning using labeled sentences for supervised learning. The classification models of random forest, KNN, and SVM obtained an F1 score of 95%. The PDF format adopts to visualize various content, such as images, text, and sequential files. The methods showed powerful real-time classification of costumer feedback compared to previously reported work. In [36], gradient-boosted decision tree classification of the log-loss function is developed and achieves an 82% reduction in false positives.

2.2. Deep Neural Networks

Deep neural networks (DNN) are utilized to learn features from the input training data for customer review classification, which improves the accuracy and reliability of recognition and categorization techniques. Recently, deep learning methods have been heavily relied upon in a vast number of review classification frameworks developed for malware identification. The proposed MXNet approach in [36] consists of an auto-encoder and multi-layer RMBs that utilize unsupervised feature learning and supervised fine-tuning to categorize costumer reviews. In [37], a long-term memory (LSTM) network-based costumer feedback categorization mechanism is developed to resolve the multi-class imbalance issue by combining binary and multi-class classification models and converting them into values. The LSTM algorithm is optimized to improve memory and macro accuracy by 7% compared to other methods, which positively impacts malware detection. In [37], LSTMs are used for text sentiment in customer reviews classification from a different perspective in e-commerce and IoT applications. As IoT devices are becoming more widely used, they have become a prime selection of the costumer, and online shops target costumers on online social media platforms for their product sale. In [38], a recurrent neural network (RNN) is used to analyze the opcodes of ARM-based IoT devices, and three LSTM frameworks are used to analyze the trained model, with the LSTM structure having the highest accuracy of 98.18% in reviews classification. In [39], a new graph network (GCN)-based customer review classification mechanism is introduced, achieving an accuracy of 98.32%, superior to other available options. In [40], a process is used that is known as multi-label text classification, where the authors developed new data on the behavior of text functions learning models for automatic feature extraction based on convolutional neural network and hybrid bidirectional gated recurrent unit (CNN-BiGRU) that identifies the advanced features for API calls for customer rating and reviews classification.

Most deep learning models described so far are quite large and intricate, making the process of retraining them a time-consuming and resource-intensive task. Moreover, as training these models involves a great deal of repetition, achieving the desired level of accuracy can take even longer. However, as customer reviews are ubiquitous on electronic devices and platforms, it is crucial to tackle this issue by developing lightweight solutions that can handle complex control systems without requiring excessive computing power.

3. Proposed Methodology

The framework proposed in this study comprises a handcrafted bag-of-features (BoF) feature extraction model, which is followed by a deep neural network (DNN) classification model. The BoF model is cascaded to extract features, which are then further refined and classified using the proposed DNN model. To pre-process the data, we first identify repeated keywords and remove artifacts, such as articles, prepositions, adverbs, and helping verbs. The performance of the proposed model was evaluated using a dataset of 3512 reviews, collected and labeled based on customer ratings posted in the review section of an e-commerce website. The performance metrics of the proposed model are presented in the Results section. Figure 1 provides a comprehensive overview of the detailed architecture of the proposed framework, while Figure 2 presents the layered details of the DNN model.

The BoF clusters the keywords in k different sets to generate a vocabulary of features through k-means algorithm. The keywords are categorized into k classes on the basis of their similarity measures. The iterative process is adopted to optimize the clusters given in Equations (1) and (2). The total of k cluster centroids are initially selected randomly, represented with ${\mu }_j$, with j ranging from 1 to k.

$${\mu }_j=\frac{{\sum }_{i=1}^m({\chi }_j^i)}{m}$$

(1)

$${\mu }_j=argmi{n}_u\left|\right|{\chi }^i-{\mu }_j{\left|\right|}^2$$

(2)

where ${\chi }_j^i$ denotes the i_th keywords belonging to cluster j, while the ${\mu }_j$ denotes the centroid of cluster j. The ${\chi }^i$ in Equation (1) denotes the i_th keyword belonging to cluster j. The symbol m in the denominator of Equation (1) denotes the cardinality of cluster j. The difference between successive centroids converges at i_th and $i-1$ iteration iteratively. The cluster centroid collectively represents the vocabulary set. The histogram of the vocabulary centroid in the raw feature data represents the final feature vector.

$$\begin{array}{c}\hfill {\zeta }_v=\{\begin{array}{c}P+0,\mathrm{if}\psi ({\chi }^n,{\mu }_j)\ge \rho \hfill \\ P+1,\mathrm{otherwise}\hfill \end{array}\end{array}$$

(3)

where ${\zeta }_v$ denotes v_th feature value of the histogram feature vector. The ${\chi }^n$ denotes the n_th keywords of the review data. The function $\psi$ calculates the distance of the m_th keywords and the n_th centroid. The symbol $\rho$ in Equation (3) denotes the minimum difference value of the v_th centroid and the keywords. The final BoF feature $\zeta$ is then fed to the proposed DNN for classification. The DNN model initial layer consists of 64 hidden neurons, which convolve with the input features vector ${\mathit{F}}_v$. The $C_k^1$ in Equation (4) represents the k_th hidden convoluional kernel, while the $b_k$ denotes the bias vector of the respective k_th kernel.

$${\mathit{F}}_v={\zeta }_v⊛C_k^1+b_k$$

(4)

$$B_v^k=(\frac{{\mathit{F}}_v-\lambda }{\sqrt{{ϵ}^2}})\ast \gamma +\beta$$

(5)

where $B_v^k$ in Equation (5) denotes the batch normalization vector resulted by k_th kernel output of the v_th feature input. The $\gamma$ denotes the scale operator and the $\beta$ represents bias variable. The batch normalization layer is followed by the max pooling layer with stride 2.

$$M_v^k=max\left\{B_v^k(x,y)\right\}$$

(6)

where $M_v^k$ is the max pooling result of the layer, where from each stride a maximum value is selected only. The LSTM with 64 hidden units is followed by the dropout layer in the proposed DNN architecture.

$$\varphi =tanh({\tau }^{\varphi }+(M_t)({\vartheta }^{\varphi })+(h_{t-1}){(\eta )}^{\varphi })$$

(7)

where $M_t$ in Equation (7) denotes the current input to the LSTM, $h_{t-1}$ denotes the previous cell output. The $({\vartheta }^{\varphi })$ denotes weights for input and the ${(\eta )}^{\varphi })$ weights for previous cell output. The ${\tau }^{\varphi }$ is the input bias, and the $\varphi$ indicates input weight and bias values. The LSTM is followed by ReLU layer, which removes non-linearity and vanishes the negative values in the data.

$$Y_r^1=max(0,h_t).$$

(8)

where the $Y_r^1$ in Equation (8) is the output produced by ReLU layer 1. The ReLU converts all negative values to zero and preserves the positive output. The second dropout of the network is followed by ReLU layer dropping 20% of connections to reduce complexity of the model and avoid the model over-fitting. The BiLSTM is the next layer, which propagates bidirectionally to identify the temporal connection between data. The BiLSTM with 16 hidden units is employed in the proposed model. The $\acute{h}$ in Equation (9) denotes the output to BiLSTM layer. The second ReLU layer removes non-linearity added by BiLSTM layer in the data. The $Y_r^2$ in Equation (10) shows the output of the second ReLU function.

$$\acute{h}=BiLSTM(Y_r^1).$$

(9)

$$Y_r^2=max(0,{\acute{h}}_t).$$

(10)

$$\widehat{Y}={\left[Y_r^{2,i}\right]}_{i=1}^M$$

(11)

The $\widehat{Y}$ in Equation (11) is the output of the fully connected layer, which has vectorized and concatenated the output of the precious layers’ ranges from 1 to M.

$$\sigma {(\widehat{Y})}_i=\frac{e^{{\widehat{Y}}_i}}{{\sum }_{j=1}^k(e^{{\widehat{Y}}_j})}$$

(12)

The $\sigma$ in Equation (12) denotes the softmax function, while $\widehat{Y}$ is the input and k denotes the number of output classes.

The proposed DNN model is optimized and trained for a total of 800 epoch steps, with a learning rate of 0.001. The batch size of 128 is chosen on the ADAM optimizer option selected for training. The proposed DNN model results in the estimated class label at the output of the final classification layer. The model requires performance evaluation, which has been carried out by utilizing the true positives, true negatives, false positives, and false negatives of the model. To measure classification performance, the specificity, sensitivity, precision, accuracy, discovery rate, fallout, and other parameters have been calculated with expression given here.

$$TPR=\frac{TP}{TP+FN}$$

(13)

$$TNR=\frac{TN}{TN+FP}$$

(14)

$$PPV=\frac{TP}{TP+FP}$$

(15)

$$FPR=\frac{FP}{FP+TN}$$

(16)

$$FNR=\frac{FN}{TP+FN}$$

(17)

$$FDR=\frac{FP}{TP+FP}$$

(18)

$$ACC=\frac{TP+TN}{TP+FP+FN+TN}$$

(19)

where, in Equations (13)–(19), the terms TP, TN, FP, and FN refer to true positives, true negatives, false positives, and false negatives, respectively. Furthermore, the terms TPR, TNR, PPV, FPR, FNR, FDR, and ACC refer to true positive rate, true negative rate, positive predictive value, false positive rate, false negative rate, false discovery rate, and accuracy, respectively.

4. Results

The proposed framework consists of two cascaded models that utilize both handcrafted techniques and the optimization and classification capabilities of deep neural networks (DNNs). When assessed on textual data from customers on an online e-commerce platform, the framework exhibited favorable performance in contrast to alternative methodologies. The performance of the proposed model was assessed on a labeled data section and subsequently tested to validate its classification performance. During the training process, bag-of-features (BoF) features are extracted and used to optimize the DNN layers’ weight and bias variables through an iterative process. The model’s hyperparameters, including the size of the bag-of-features (BoF) clusters, the configuration of the deep neural network (DNN) layers, the number of hidden units, the dropout percentages, and the epoch size, are determined through experimentation to optimize performance. To validate the efficacy of the proposed model, its performance metrics were compared against state-of-the-art benchmarks. The intel^® Core ${}^{\mathrm{TM}}$ i7-1255UL processor with 8 GB RAM and NVIDIA GeForce GTX 1650 is used for processing the data; moreover, the Matlab 2021a release is employed for the simulation setup.

4.1. Dataset Description

The performance of the proposed model was assessed using a dataset obtained from the AliExpress product Insta360 x3 action camera, which has an estimated value of 470 USD. The review section yielded a total of 3512 customer reviews, each consisting of a textual review and a rating. Out of the 3512 reviews, 1802 were categorized as positive reviews, while 1710 were identified as negative reviews. The second dataset utilized was collected from the Amazon ecommerce website for a similar product, with a total of 1552 reviews. Among these, 890 were positive reviews, and the remaining 662 were negative reviews. The rating system used a 5-star scale to label each textual review. Reviews with a rating above 3 stars were classified as positive, while reviews with a rating below 3 stars were labeled as negative. Reviews with a 3-star rating were excluded as they were deemed irrelevant. To evaluate the proposed framework for classifying customer reviews, reviews were selected from three different vendors who sold the same product. The dataset was divided into three sets for evaluation purposes: a training set comprising 70% of the data, a validation set comprising 15%, and a test set also consisting of 15%. The performance ROC curves in Figure 3 display the superiority of the proposed method compared to various state-of-the-art classification approaches.

4.2. Performance Evaluation

A novel method for feature extraction and classification using a bag-of-features (BoF) followed by deep neural network (DNN) framework is proposed in this study. The method is evaluated on a dataset, achieving a classification accuracy of 91.54%. When compared to standalone classifiers, the bidirectional long short-term memory (BiLSTM) achieved the second highest accuracy at 89.72%, while the simple LSTM achieved 89.55%. The support vector machine (SVM), k-nearest neighbor (KNN), and decision tree (DT) classifiers yielded classification accuracies of 85.02%, 84.85%, and 84.94%, respectively. The confusion matrix generated by each classification method is illustrated in Figure 4. The proposed DNN model, consisting of a layered architecture of LSTM and BiLSTM, achieved a classification accuracy that exceeded that of the standalone LSTM and BiLSTM classifiers by 0.94% and 0.%, respectively. The model demonstrated superior sensitivity, specificity, and precision, achieving a rate of 91.54% when compared to the other methods listed in

Table 1. Performance comparison with related work on the AliExpress and Amazon datasets.

Dataset	Performance Metric	Second Name	DT	SVM	KNN	BiLSTM	LSTM	Proposed
AliExpress	True positive rate (TPR)	Sensitivity	84.94	85.02	84.85	89.72	89.55	91.54
	True negative rate (TNR)	Specificity	84.94	85.02	84.85	89.72	89.55	91.54
	Positive predictive value (PPV)	Precision	84.94	85.02	84.85	89.72	89.55	91.54
	Negative predictive value (NPV)	-	84.94	85.02	84.85	89.72	89.55	91.54
	False positive rate (FPR)	Fallout	15.06	14.98	15.15	10.28	10.45	08.46
	False negative rate (FNR)	-	15.06	14.98	15.15	10.28	10.45	08.46
	False discovery rate FDR)	-	15.06	14.98	15.15	10.28	10.45	08.46
	Overall accuracy (ACC)	-	84.94	85.02	84.85	89.72	89.55	91.54
Amazon	True positive rate (TPR)	Sensitivity	83.80	84.63	83.76	89.44	89.83	90.21
	True negative rate (TNR)	Specificity	83.80	84.63	83.76	89.44	89.83	90.21
	Positive predictive value (PPV)	Precision	83.80	84.63	83.76	89.44	89.83	90.21
	Negative predictive value (NPV)	-	83.80	84.63	83.76	89.44	89.83	90.21
	False positive rate (FPR)	Fallout	16.02	13.56	14.27	09.56	11.38	07.89
	False negative rate (FNR)	-	16.02	13.56	14.27	09.56	11.38	07.89
	False discovery rate FDR)	-	16.02	13.56	14.27	09.56	11.38	07.89
	Overall accuracy (ACC)	-	83.49	83.14	83.69	87.80	87.66	90.42

. Additionally, the proposed method resulted in a lower fallout rate compared to other classifiers, indicating its ability to accurately detect and classify instances of interest.

In Figure 5, the performance metrics of sensitivity, fallout, and accuracy were compared between decision tree, support vector machine, k-nearest neighbor, BiLSTM, LSTM, and the proposed method. The results demonstrated that the proposed method outperformed all the other models in all performance metrics. This suggests that the proposed model may be the most effective option for the task at hand.

In

Table 2. Performance of the proposed framework on AliExpress and Amazon datasets with various training and testing data ratios.

Database	Train/Test Ratio	50/50	60/40	70/30	80/20
AliExpress	Overall accuracy (ACC)	74.65	83.98	87.20	91.54
AliExpress	Overall accuracy (ACC)	70.01	81.49	85.71	90.42

, different ratios of the training and testing data size have been used to investigate the performance of the proposed model. The proposed model, when evaluated on both AliExpress and Amazon data with 50% train and 50% test data, provided an accuracy of 74.65% and 70.01%. The accuracy increased to 91.54% and 90.42% when the training data size was increased to 80%, while 20% is utilized for testing.

5. Conclusions

In this research, we employ a hybrid framework that combines the handcrafted bag-of-features (BoF) method for feature extraction with a fourteen-layer deep neural network (DNN). This combination optimizes and effectively categorizes customer feedback related to online products. The BoF, along with the DNN, enhances the features by adding invariance and distinctiveness to the feature selection and classification models. Extensive experimentation was conducted to determine the optimal cluster dimension in BoF and the hyperparameters in the DNN. The proposed model was tested using customer feedback data collected from the AliExpress e-commerce website and Amazon product listings. Specifically, we selected the InstaX3 camera listing pages from three different sellers on both platforms. This dataset comprised a total of 3512 reviews, including 1802 positive and 1710 negative reviews from AliExpress, as well as 1552 reviews from Amazon, with 890 being positive and 662 negative. The classification model achieved a classification accuracy of 91.54% and 90.42% on the AliExpress and Amazon data, respectively. These results demonstrate the superior performance of the proposed model in categorizing customer feedback effectively.