Customer Profiling Using Internet of Things Based Recommendations

Abstract

The digital revolution caused major changes in the world because not only are people increasingly connected, but companies are also turning more to the use of intelligent systems. The large amount of information about each product provided by the e-commerce websites may confuse the customers in their choices. The recommendations system and Internet of Things (IoT) are being used by an increasing number of e-commerce websites to help customers find products that fit their profile and to purchase what they had already chosen. This paper proposes a novel IoT based system that would serve as the foundation for creating a profile, which will store all the contextual data, personalize the content, and create a personal profile for each user. In addition, customer segmentation is used to determine which items the client wants. Next, statistical analysis is performed on the extracted data, where feelings, state of mind, and categorization play a critical role in forecasting what customers think about products, services, and so on. We will assess the accuracy of the forecasts to identify the most appropriate products based on the multi-source data thanks to the IoT, which assigns a digital footprint linking customers, processes, and things through identity-based information and recommendations, which is applied by using Raspberry Pi and other sensors such as the camera. Moreover, we perform experiments on the recommendation system to gauge the precision in predictions and recommendations.

1. Introduction

The Internet of Things (IoT) has revolutionized our society in general and industries specifically. IoT refers to the “things” we use and that connect to the internet, empowering us to receive and exploit data about that the “thing” from our personal computers or smartphones [1]. IoT devices provide insights into the inner workings of companies, e.g., machine performance to supply processes, logistic operations, and, more importantly, customer behavior and possible assumptions about specific products [2]. Therefore, it helps save time, and money, and even reduce worries and hesitations, especially for customers whose needs change constantly. In electronic commerce (e-commerce), it is vital to identify customer needs correctly to be able to satisfy those needs. Moreover, IoT assists us in reimagining and designing new strategies for marketers, customers, and product relationships. The data we sense and collect from the IoT infrastructure enable precise customer segmentation by broadening our understanding of how customers choose and use their goods [3].

Customers’ purchasing intentions have become a major debate point in assessing the sales and growth of a firm due to the growing number of items with even the smallest variances. On e-commerce platforms, there is a wealth of information that can be evaluated to create marketing plans. Utilizing the available data and machine learning approaches, users are able to provide outcomes and forecast client intent. Machine learning is a concept that uses algorithms and current data to forecast potential outcomes. Consumer reviews and ratings have a significant impact in predicting customer intention. This paper demonstrates how an IoT system (comprising a camera, image processing, etc.) can anticipate customer preferences and recommend products in stores. Customers who are unsure about the items they want to buy will browse and get feedback from shelf displays and salespeople, which is an attractive feature of shops. The system recommendations to online customers are based on previous purchases of in-store products. The model employs a customized matching product creation algorithm and fine-tunes the results according to user preferences. The application of recommendation models is not new in e-commerce [4,5]. Based on experiences, a customized recommendation algorithm looks to identify and offer products that are most pertinent to a particular customer [6].

E-commerce personalization is the process of dynamically displaying content, product suggestions, and specific deals on e-commerce sites based on past actions, browsing habits, purchasing history, demographics, and other personal information [7,8]. However, to choose the best algorithm from a group of candidates, it is essential to evaluate recommendation algorithms concerning various properties [9]. Online reviews are unusual since they enable direct evaluation of system goals, e.g., long-term benefits and user retention. Tailored product reviews on the site or in the product description tab, cart abandonment marketing email messages, and onboarding tests provide customers with a personalized showroom of items [10,11]. The unfortunate news is that there are so many of these tools, each with a varying degree of personalization, making it challenging to know what it means to create individualized user experiences in e-commerce and how to execute them [12,13]. However, combining various technologies to generate a holistic intelligence around user intent is the way forward to get visitors to their current destination; one must first understand them and customize the experience to meet their demands [14]. Work focuses on the behaviors customer that may prevent accurate estimation of an assessment measure, and therefore the selected metrics may not accurately reflect the effect of the recommendation model on the visitor [15,16]. The effectiveness of the recommendation model might be diminished by behavior measured in terms of click-through rate or buy-through rate [17]. Data from business-to-business (B to B) customers, outliers, new users on an e-commerce website, and the possibility of gathering duplicate information are additional challenges [18]. Companies may deliver extremely contextualized messages to individual clients at the right time and location, via the right medium, using hyper-personalization [19]. This technique improves customer satisfaction while increasing brand loyalty, ability to buy, and overall marketing efficacy [20]. While more businesses have embraced personalization, from product creation to outreach and from the user experience to competitive pricing, the customer path has instilled in customers a need for customized engagement. The days are gone when mass media was effectively engaging diverse clients by broadcasting general advertisements to all prospective customers. Every contact is now new and customers demand a personal relationship with the businesses with which they do business [21].

Following are the contributions of our suggested system:

• Develop an e-commerce system that uses recommendation algorithms and IoT (camera sensors) to suggest products to clients based on their unique requirements.
• The similarity between items and consumers is assessed using two types of textual feature extraction algorithms.
• A web application system that employs a customized comparable product creation algorithm and fine-tuning based on previous purchases, products, and preferences.
• This approach aims to minimize consumer search expenses while making it simpler to find what they are looking for. Customers’ feedback and purchase habits are used by our recommender system to infer customer preferences and provide automated product recommendations. In the vast majority of cases, these systems rely on collaborative and content-based filtering algorithms. The majority of recommender systems employ agents to gather transactional data from the user’s profile in order to determine the user’s preferences and interests.

The rest of this article is divided into 7 sections. Section 2 covers the relevant work on e-commerce and recommendation systems based on IoT. Section 3 and Section 4 outlines the techniques employed by the recommendation system framework with the essential justification of the newly suggested recommendation system. The experimental findings of the proposed IoT-based face recognition recommendation system are presented in Section 5. In Section 6 outlines, discussion. The conclusions and future work are presented in Section 7.

2. Related Work

This section reviews studies that explore the use of recommendation systems in IoT-based e-commerce applications. A few of these are discussed in this essay.

In [22], the authors proposed a new method, titled Personalized Recommendation Systems and the Impact of the COVID-19 Pandemic on Recommendation Systems, which focuses on new and atypical users and describes how online machine-learning algorithms can detect and adapt to changes in consumer behavior. They claim that user profiles can be utilized to provide accurate products as per the user’s expectations. The primary concept behind the proposed solution, is to support users in narrowing down their selections by learning their preferences and customizing their experiences.

The authors in [23] proposed an IoT-based smart e-marketing system. Their aim is to help marketers address the needs of customers for selecting goods and services as per their needs that would lead to customer satisfaction. It also helps marketers obtain access to new data sources and develop a better understanding of their customers’ interests and behaviors. Therefore, marketers are better prepared to satisfy their clients’ requirements. The customer data stores in a cloud database and includes information such as client position purchase date, sales list and customer profiles. Marketers may extract and analysis consumer preferences and behaviors using preserved customer data, allowing them to develop more tempting marketing approaches. In order to minimize misconceptions and to preserve the personal and privacy rights of the consumers, the authors propose that the advertisers should warn customers about the user of their personal information for commercial purposes. Authors claim that businesses may benefit from a high level of transparency that will lead to a reduction in consumer dissatisfaction.

In yet another study [24], authors propose a practical collaborative filtering suggestion system that was employed by a sizable Korean fashion retailer that sells clothing both online and off-line. First, the Internet and physical stores of the corporation both sell the same things. Second, shoppers’ choices change with the seasons since fashion products are often seasonal. Finally, purchasers generally purchase products to replace or complete previously acquired items. A product list, product information, online purchase history data, and click history data are all produced by the suggested suggestion system (online). The system first pulls product metadata from the online shopping mall database, which contains a product code (individual and group code), gender type (male, female, unisex), status (active and inactive), sale type (generic and set sale), product kind (clothing, shoes, etc.), and production year. Second, utilizing the offline shopping mall database, the algorithm creates a dataset of purchases. The general structure of the suggested model. Because metadata is crucial to the process, any product that does not activate metadata is disqualified.

The authors in [25] explain a method for assessing customer expectations for e-commerce recommendation systems based on many sources of data. The weights of suggestions are determined via an extended radial basis function (RBF) network, multi-source data is fused using an updated Dempster–Shafer theory, and mobile e-commerce capabilities are merged. The fusion results are then managed and decision-making is aided by the power-spectrum estimation. Even though the study suggests employing multi-source information fusion technology to improve personalized recommendation prediction accuracy in mobile e-commerce terminals, there are a few issues that need to be brought up. First, the experimental results of the study demonstrate that the suggested approach requires more time to compute than the conventional method. However, the suggested technique offers no advantages in terms of speed. Future studies should thus concentrate on accelerating the suggested technique. Second, while creating a recommendation system, authors put the user’s convenience first. This is done by accurately recognizing the products that the user is interested in. As part of the study, the authors go into great depth on the steps of a multi-source information fusion-based product recommendation approach, including data sources, the weight of the recommendation evidence, the decision-making process, and the result.

In [26], the authors present a new method for evaluating IoT products. Their proposed method recommends high-quality solutions to prospective customers. The authors have noticed that new IoT consumers confront a variety of obstacles when it comes to selecting smart gadgets that best meet their needs. The proposed method is based on recommendation system techniques, in which many factors are integrated to produce useful recommendations. The authors concentrated on using hybrid strategies because they allow for data mining while also avoiding shallowness. Several levels of recommendations are used to meet customer requirements, including filtering items based on initial search parameters, filtering items based on consumer preferences, filtering items based on consumer limitations, and utility estimates for recommended items.

A comparison of the relevant works and the system that has been presented based on the recommendation system is shown in

Table 1. Analyze the relevant works in comparison to the suggested system.

Paper	Diversity of Recommendation	Customer’s Satisfaction	Time Needed for Sale	Accuracy of Recommendation	Customer Preference Analysis	Similarity Measure Efficiency	Data Source
[22]	Yes (CB, CF, Hybrid	Yes	No	Yes	No	Yes	Amazon
[23]	Yes (CB technique)	No	No	Yes	Yes	No	application in App Store and Google Play by Registration of the customer)
[24]	Yes (CF technique)	No	Yes	NO	No	Yes	Website online
[25]	No	No	No	Yes	No	Yes	Taobao.com Selected 100 customers
[26]	Yes (CB and CF techniques)	Yes	Yes	Yes	No	Yes	2500 customers.
Our Proposed System	Yes (CB and CF techniques)	Yes	Yes	Yes	Yes	Yes	Website local In tunisia 1000 customers

. In this table, “Yes” stands for solved problems while “No” indicates the problems that are not solved.

This previously mentioned approach places focus on user’s interest without keeping in mind accuracy and satisfaction of customer. Therefore, there were no advantages in terms of satisfaction of purchaser. These drawbacks will be solved in the proposed method below. In fact, we will focus on precision, accuracy, and the satisfaction of the customer. Despite the effectiveness of the recommendation systems provided, there are a number of significant obstacles, including data variety, predictable suggestions, and the problem of age, accuracy, purchasing capability, satisfaction, and content inclusion. This system attempts to reduce customer search costs while also making it easier to identify desired items including data variety, predictable suggestions age, accuracy, purchasing capability, satisfaction. Customers’ feedback and purchasing behavior information are used by recommender systems to infer customer preferences and create automatic product recommendations.

3. Materials and Methods

3.1. Recommendation Systems in E-Commerce

Recommendation systems have become increasingly widespread and play an essential part in today’s information systems and websites. This paper exhibits the usefulness of an IoT-based personalized recommender system in e-commerce. A recommender system strives to predict which items would stimulate user interest based on their behavior on a specific web site. In our research, we employ two types of approaches, namely collaborative filtering (CF) and content-based filtering (CBF), both of which are well-known methods for creating a recommender system [27,28,29].

3.1.1. Collaborative-Filtering (CF) Based Recommendation Systems

The method of suggestion that is most frequently employed is collaborative filtering (CF). The two main forms of CF are memory-based and model-based. While model-based strategies utilize machine learning and deep learning algorithms to provide recommendations, memory-based techniques leverage the ratings that other users have given to comparable goods and services. Most memory-based techniques rely on the Bag of Words (BOW), which is simple to use and yields excellent results. This class of techniques is based on the premise that users who have previously favored particular goods are more likely to like similar products in the future. The collaborative aspect of the CF refers to how numerous people engage with the same items and share the same similarities [30,31].

Figure 1 shows Collaborative filtering (CF) is a strategy built on consumers exchanging viewpoints. The underlying premise of this approach is that it is possible to reasonably forecast an active user’s preference for a product that he has not yet assessed by using the views of other users. These techniques make the assumption that if customers have the same preferences for one group of products, they will likely have the same preferences for a different group of products that they have not yet assessed. Therefore, collaborative filtering approaches suggest to the present user the products liked by users who have similar likes to him, i.e., comparable users.

3.1.2. Cosine Similarity

Before we can build a collaborative filtering-based product recommendation system, we need to build a user-to-item matrix. This matrix is used to calculate the similarities between various users. Cosine similarity is a popular metric for determining similarity. The cosine similarity of two users is calculated using Equation (1).

$$\mathrm{Similarity} \left({\mathrm{U}}_1,{\mathrm{U}}_2\right)=\cos \left({\mathrm{U}}_1,{\mathrm{U}}_2\right)=\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{P}}_{1\mathrm{i}}{}_{ }{\mathrm{P}}_{2\mathrm{i}}}{\sqrt{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{P}}_{1\mathrm{i}}^2 {{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{P}}_{2\mathrm{i}}^2}}$$

(1)

where U₁ and U₂ represent User 1 and User 2, respectively. P_1i and P_2i refer to each product that U₁ and U₂ have purchased.

3.1.3. Content-Based Filtering (CB) Recommendation Systems

CBF is another common recommendation approach (Figure 2). It places a greater emphasis on the dataset description and its characteristics than on previous user interactions and preferences. Another benefit of CBF is that it is easier to apply than CF because it relies on patterns in characteristics rather than inferring hidden user preferences [32].

4. Proposed Methodology

In the start of this section, we focus on consumer profiling activities with the following goals in mind:

• Extraction of a customer profile for the store
• Profile integration based on the extracted data and patterns
• Discovering customer interests using accessible data to address the issue of consumer profiling, we suggest a new architecture.

The second part of the section presents our proposed architecture for products recommendation. Our proposed approach is composed of three key components i.e., profile extraction, integration, and consumer interest analysis. Extracting user profiles is the primary goal of the first component, the second component integrates profile data from the Internet, and the third component highlights the process of evaluating customer interests.

4.1. Customer Dataset Collection

The dataset that was used to test our model originates from an e-commerce website of a real business. We spent approximately one year collecting and processing the required data. The collected data included customers from various countries, that helped us learn the difference in preferences of customers coming from diverse backgrounds.

We will be able to access a range of brands, as well as information about transactional history of diverse customers. In addition, the individual dataset for each client comprises a unique ID, gender, age, number of transactions, purchasing capability, spending money, rating, brands, and so on. We used factors, such as product reviews, items, buying patterns, watching patterns, and time-based segments, to segment the data. We analyzed preferences of nearly 1000 customers from the dataset we obtained from a selected shopping store and website. Data collection began in 2002. The dataset was located in Tunisia and all access needed was allowed by the market website. We have developed a review customer web application with a form where the customer may provide feedback on the products and his needs in order to gather the dataset. Different customers from various countries will give their opinions when this application is hosted, and as a result, this enhances the dataset, as shown in Section 5.3.

Table 2. Example from the Raw Dataset.

Custmer ID	Gender	Age	Nub_Tr	Pur_Ca	S-Money
00124	Male	19	1	2$/day	20
00125	Female	20	1	3$/day	30
00126	Male	15	0	1$/day	10
00127	Female	25	2	5$/day	35
00128	Male	40	5	10$/day	250
00129	Female	35	5	8$/day	200
00130	Male	18	1	2$/day	25
00131	Male	22	4	4$/day	35
00132	Male	13	0	0$/day	10
00133	Female	28	2	5$/day	40
00134	Female	50	5	12$/day	400
00135	Female	42	4	10$/day	300
00136	Male	55	6	13$/day	350
00137	Male	60	7	15$/day	450

gives us a glimpse to customers’ shopping preferences. The dataset is obtained from an online internet store. Pre-processing is performed on the provided dataset to eliminate any missing items. The dataset comprises unique values that will be used in the research for prediction purposes.

4.2. Customer Profile Extraction

Extracting valuable information about a consumer from various sources is known as profile extraction. Both the store and the customer must be profiled in our application. As a result, we have broken down our framework into two simple profile development steps, including store profiling and customer profiling.

4.2.1. Store Profiling Phase

Different characteristics must be identified for store profiling.

Figure 3 depicts the store schema that will be generated after the store profile has been extracted. During the pre-processing stage, we divide the text into tokens and assign possible tags to each token. In contrast to tokens, which are crucial units in the tagging problem, pages are collections of units or a tree structure of units. In tagging, given a set of units or a tree structure of units, we apply a trained tagging model to discover the most probable linked tags. Each property has a tag associated with it.

4.2.2. Customer Profiling Phase

Customer profile extraction is the method of gathering information about customers as soon as they register in our system. We define various customer characteristics in order to profile them. To enable implicit profiling, we gather information about clients from various social media sites and profile them. Figure 4 shows a sample of a customer profile.

After profiling the store and the customer at the same time, we can say that a person logs into the home page. He/she finds the store profile and customer’s profile. As such, he visualizes the customer’s profile on a web page and gets access to it. In fact, this web page includes basic information about the person (name, family name, address, gender, email, phone number, level of education…etc.). Hence, the customer inputs his or her personal information’s, and the system creates a customer’s profile according to the provided data. Moreover, the system predicts the customer’s purchasing preferences after diagnosing and analyzing his or her surfing on that page. Things, such as interests in clothing, drinks, sweets, home appliance, electrical goods, etc., are considered. In other words, this profile suggests some questions in relation to the customer’s taste and preferred products. In this way, we will learn about the customer’s interests thanks to his own logging in that ‘customer profile’. For instance, the user looks at some introduced and suggested photos and data in the page, and answers questions concerning his or her likes and dislikes (for example, furniture with different articles, sizes and colors). Consequently, we will be able to identify his or her taste in relation to color, size, and type, etc. Indeed, the customer will be asked if he or she likes to receive notifications concerning new products or a reduction in prices. After entering and storing all the information about his or her personal likes and dislikes, the customer will press the “Send” button.

4.2.3. Profile Integration

The store data were crawled from existing online data sources. Profile integration is used to combine data derived from multiple sources. The method will invariably have a name ambiguity problem. To integrate the data gathered in the previous phase, we must combine data from many sources into a single format so that becomes consistent across all data collected from our store and website; in other words this data is well checked. In this stage, we construct a dataset that is used in subsequent operations such as interest discovery. We delete missing values from different attributes and customize the data to meet our needs. For example, instead of holding the exact value, we may form data value ranges. This aids us in our search for new areas of interest.

4.2.4. Interest Discovery

We have a simple customer profile that comprises of a set of profile features and a collection of documents for each client after extracting and combining the customer profiles. Based on the consumer profile and pertinent data, we now conduct user interest analysis. We want to identify the latent topic distribution related to each client based on the idea of consumer interest. The knowledge required to discover the distribution of the latent product varies depending on the application. Store location, store control, number of customers in the e-commerce system who recently visited the store, store services, and other relevant details about the store are all available in our system. In addition, the profile extracted in the previous steps contains details about the customer’s history, such as his or her location, visited locations, and so on.

4.2.5. Customer Segmentation

Customer segmentation is the process of dividing your existing customer base into smaller groups. Based on data from current statistics, each of these smaller groupings has distinct features. Marketing and CRM technologies are used to collect data on a regular basis. Consumer profile and segmentation, in a nutshell, assist businesses in determining who they should market to. Businesses may make smarter judgments, provide more customized customer service, and enhance customer loyalty by integrating the two tactics. Customer segmentation profiling enables a company to determine which group of the viewers has the biggest affinity for a certain product. The info should pass the data preparation stage before it is utilized as a dataset. While most people understand the need for a client profile and segmentation, many people fail to do it successfully. Because they spend most of their time planning and producing the product, therefore many business executives are out of touch with their audience. Hence, they do not grasp their customers’ wants and demands, and they struggle with marketing. As a result, there is a sense of disconnection between the two parties. Because businesses are unable to form a link with their customers, the latter will look for other ways to meet their demands.

Table 3. Description of related columns.

Column Name	Description
Customer ID	It represents the identify of the customer.
Gender	It represents the gender of the customer.
Age	This is the age of the customer.
Number of transactions (Per week)	It is the number of transactions per week by the customer.
Purchasing capability	It refers to the purchasing capability for each customer.
Money spending (Score 10–1000)	This refers to the client monthly spending, where the score is assigned between 10 and 1000.

below describes the dataset for segmentation.

4.3. The Architecture of the Proposed Recommendation System

In this section, we explain the approach that highlights the proposed system’s high-level functional architecture, as shown in Figure 5.

The proposed system is made up of four major parts, as indicated in Figure 5. The amount and price of previous purchases are identified via the previous purchase’s module. The customer feedback module extracts the target consumer information. The MD (manager decision) decision module examines the present context’s impact and makes a final suggestion to the customers. The recommendation approach module checks customers’ reminders.

Following are the main objectives of our proposed recommendation system:

• To provide consumers with suggestions based on their present and previous background information and their contextual preferences.
• To offer an enhanced relational profile paradigm for companies to manage service user expectations.
• To demonstrate how trust can be incorporated into context-aware tailored advice.

4.4. Flowchart for Suggested Architecture

The Figure 6 below shows the step-by-step process related to the purchasing activity performed in the store by the customer starting from entering the store, to the detection through cameras sensors until buying something or not. Each time a customer enters the store, a camera identifies him or her. If the customer already has a profile, i.e., he or she is not a new customer then the application searches his or her profile to find his or her favorite products and preferences from the store’s database. The customer is then informed about the availability of his or her preferred products. If the customer is new, then the application creates a new profile (Profile 0), including his or her photo and the list of products that may interest him or her. Then, the application displays the products accordingly.

5. Results and Discussion

This section presents the main findings as well as the experimental data used to evaluate the proposed product recommendation system.

5.1. Experimental Results

This section includes “Distribution age of purchasing capability” and “comparative the Age to Spending money”, “Purchasing capability to Spending money”.

5.1.1. Distribution Age of Purchasing Capability

The above chart represents the purchasing capability of each customer by age. The amount spent (in thousands of dollars) is shown to the customer per day in the store. The question that arises concerns the age distribution by purchasing capability of the customers. The majority of the customer in this dataset is between the ages of 35 and 60. One of the most crucial aspects of market segmentation is this. In terms of purchasing patterns and how people respond to marketing, that age group differs from the younger generation. The younger generation spends more time on social media and platforms like Facebook, whereas the elder generation frequents retail websites. In order to design marketing strategies, we must consider this.

Figure 7 shows the purchasing capability by the age of each gender, where we see that that the customer older than 35 had greater purchasing capability and the customer who younger than 35 had a weak purchasing capability When men and women are compared, women have a somewhat larger association between age and expenditure score, as shown in Figure 8 below.

5.1.2. Distribution Age of Purchasing Capability

The graph belowdepicts the absence of relationship between age and spending money. It is not a significant link, but the older a person becomes, the less money they spend. This graph depicts the average amount spent in the shop by consumers based on the number of transactions they made, and we can see that the older customers spend more than the younger ones. This information is critical for marketing, and we will see if there is a link between spending money and purchasing power in Figure 9.

Figure 9 show us the spending money for each age of the genders. We conclude that the people older than 30 had considerably more spending money and the people younger than 30 had much less spending money, so we deduce that the age has a very important role in affecting the amount of spending money.

5.1.3. Purchasing Capability to Spending Money

Figure 10 shows us the score of spending money in a month of each gender by the purchasing capability for one day, where we see that if the purchasing capability for each customer increases, the spending money increases in in the same way.

5.1.4. Customer Satisfaction Rate

Figure 11 shows us the customer satisfaction rate by sending a consumer a survey shortly after they make a purchase or contact with customer care, or by looking at product reviews. Customers rate your company from “not at all satisfied” to “very satisfy”. Customer satisfaction may also be measured by how much money a customer spends, i.e., if he spends a lot of money, he is “satisfied” and if the degree of spending is weak, he is “dissatisfied”.

5.2. System Application

This system is recommended to consumers who visited the store then checked its website. It employs a customized comparable product creation algorithm and fine-tuning based on previous purchases, products, and preferences, as shown in Figure 12 below, which describes the prototype system.

5.3. Product Recommendation System in Ecommerce through Face Recognition Using YOLOv5 and DeepSORT Algorithm

Our proposed recommendation system uses cameras, which have been carefully placed to cover the major area of a given store. The movement of the clients is tracked to figure out his or her interests. By observing the amount of time the client spends in front of a particular product and by utilizing facial recognition techniques to observe the behavior of the customer, we try to compile a list of recommended products for the client. This list is stored in the database so that when the same client visits the online store, then he or she is presented with the compiled list of recommended products on the homepage. In other words, the system focuses on the customer’s behavior. It measures the amount of time a person spends in front of a certain item or product. Then, it predicts or prognosticates the user’s preferences, needs and priorities. This will help him or her make instant online purchase of the products of interest. This technique of providing recommended lists of products as per the priorities and interests of the client is called targeted marketing. Our proposed system makes use of the YOLOv5 model and deep sort algorithm.

This study proposes a unique recommendation system that incorporates face recognition techniques and consumer feedback, as well as the amount of time spent in front of each product. The system uses identification and product recommendation algorithms to target items the proper customers by observing customers’ behavior from a live camera feed. It also employs an object detection technology in order to obtain input on the suggested product. The proposed approach increases user engagement and may be utilized in shopping websites and other locations that require effective product recommendation systems. It also aids in more effective and user-friendly marketing of items.

In Figure 13, we display how YOLOv5 can be deployed to detect various products placed at different racks of a store. With the help of this detection model, we can identify the products of interest to a particular client.

Figure 14 shows an example scenario where the deep learning capabilities of the DeepSORT algorithm are deployed to track and identify the customer in order to figure out his or her interests in a particular product.

The following Algorithm 1 can be used to represent the suggested system:

Algorithm 1. DeepSORT: Deep Learning to track custom objects.

# Run tracking

model.warmup(imgsz = (1 if pt else nr_sources, 3, *imgsz)) # warmup

dt, seen = [0.0, 0.0, 0.0, 0.0], 0

id_ = {}

chocolate = []

ice_cream = []

if save_txt:

# to MOT format

bbox_left = output [0]

bbox_top = output [1]

bbox_w = output [2] − output [0]

bbox_h = output [3] − output [1]

x_center = (output [2] − output [0])/2 + output [0]

dimensions = im0.shape

res = x_center/dimensions [1]

if res > 0.4:

ice_cream.append(int(id))

else:

chocolate.append(int(id))

if str(int(id)) not in id_.keys():

id_[str(int(id))] = int(id)

# Print results

# speeds per imaget = tuple(x/seen × 1E3 for x in dt)

for x in id_:

ch = (chocolate.count(int(x)))/30

ic = (ice_cream.count(int(x)))/30

writer_sum.writerow([x, str(round(ch, 3)) + ‘ Sec’, str(round(ic, 3)) + ‘ Sec’])

The graphs of the metrics curves as training advances are shown in Figure 15. After validation for the YOLOv5 model were obtained, a recall score of 0.95, and mAP values of 0.95 and 0.5.

Customers are asked to fill in this form when they log in to their customer profile. In fact, they must introduce their personal information and answer some questions concerning their tastes, likes and dislikes as it is shown in Figure 16. To increase consumer product suggestions in the system, between content-based and collaborative filtering that is based on user ratings, there is a hybridity. When a user logs in, he is sent to a dashboard page where he can access his “customer profile”, which contains his personal information, and “the shop profile”, which allows him to select from pre-existing product categories as well as those developed by the page’s administrator. He can also add some suggested items, goods, or products. He can specify the requirements and rate every product. He can, indeed, mention his likes and dislikes. The is all found in the customer profile who logs in his profiles with see all this. All users can add products, goods categories, preferable items and ratings. However, it is possible only for the administration to delete any information. In an event scheduler part concerning advertisement about new products, users can specify all the event details (the date, time, location, subject) and all those relevant details. They can also invite guests to the stores. Here, every user who has registered a profile on the system can be viewed and invited. Finally, the products and things that will be delivered may be selected from the proposed list, which uses CF and CBF to provide choices on the basis of user ratings. Their preferences or dislike of a certain product makes up the content-based part of the recommendation system. The collaborative-based section computes the similarity of users to one another using the cosine similarity method and works on the assumption that if a certain user preferred or disliked a certain item, a “similar” user would have the same preferences and dislikes and recommends products on this basis. Invited users (in an event for example)/or customers are notified on the system via an electronic visit, concerning new products for instance. They must respond by either accepting or declining the invitation.

After we have personalized the consumer profile, we give a rating to each product from one to five stars according to his purchase list and his desires to help him find the suitable product his looking for. This will make the buying process much easier and faster for each customer, as shown in Figure 17. In fact, after narrowing down the list of options for our customers, they will no longer be perplexed or hesitating. Rather, they will browse and get feedback according to their own likes. Hence, the system helps us infer their preferences and deliver them easily. As such, customers are guided to their preferable or desired destination, which, indeed, saves times and facilitates life.

6. Discussion

The suggested system is adapted for the use of electronic trade websites for the particular purpose of selling products. Now, we can say that issues of ascending and dissimilarity are solved. In electronic trade, a field which includes a variety of categories, these problems can be resolved by utilizing IoT to learn more specs based on the sorts of items. By using a specific algorithm named “Deep Sort” and a model named “YOLOv5”, these specifications are used in the suggested program to support the selling process.

Contrarily, for customers who do not have accounts and are regarded as visitors, we have no knowledge of their preferences. The recommendation list is compiled in accordance with the product preference matrix, which compiles data on the most popular products based on the varied website activities of various customers who have subscribed to the service. The consumers’ actions are often used to extract information and return their true trends. According to our findings, the recommended program performs better than the traditional way when accuracy, time, decision-making, and satisfaction are used to gauge system performance.

7. Conclusions and Future Work

Our proposed recommendation system presents comprehensive recommendation metrics and notable improvements in performance when checked against the existing recommendation systems. In this article, we have attempted to reduce customer search costs while also making it easier to identify desired items, including data variety, predictable suggestions age, accuracy, purchasing capability, and satisfaction. We have used customers’ feedback and purchasing behavior information by recommender systems to infer customer preferences and create automatic product recommendations.

Our work is based on enhancing the existing recommendation methodologies in e-commerce using the Internet of Things (IoT), as well implementing additional strategies to gain superior performance. We have provided a short summary of existing researches on the use of recommendation systems based on IoT in e-commerce. Moreover, we have discussed the facets of personalization as a general strategy for tailoring services as per the customer needs. Companies will use the acquired and stored data to provide real-time personalized product suggestions to customers, resulting in a one-of-a-kind, user-centric shopping experience. By incorporating the IoT into the user interface, users may get real-time assistance that would otherwise be impossible to offer manually. In comparison to existing systems, the experimental findings revealed that our proposed recommendation system provided greater performance.

In future, we will aim to add and test reinforcement-learning strategies adopting an agent approach to improve the user-personalized recommendations. This will help to handle situations when there is not enough historical customer data. We will explore various strategies, such as questionnaires, to collect required consumer feedback after they purchase a product, which would eventually help to enhance the website’s functionality and offer useful input for the recommendation system. Moreover, we aim to consider ontology-based recommender systems and to assess their impact on performance.