Big Data and AI-Driven Product Design: A Survey

Abstract

As living standards improve, modern products need to meet increasingly diversified and personalized user requirements. Traditional product design methods fall short due to their strong subjectivity, limited survey scope, lack of real-time data, and poor visual display. However, recent progress in big data and artificial intelligence (AI) are bringing a transformative big data and AI-driven product design methodology with a significant impact on many industries. Big data in the product lifecycle contains valuable information, such as customer preferences, market demands, product evaluation, and visual display: online product reviews reflect customer evaluations and requirements, while product images contain shape, color, and texture information that can inspire designers to quickly generate initial design schemes or even new product images. This survey provides a comprehensive review of big data and AI-driven product design, focusing on how big data of various modalities can be processed, analyzed, and exploited to aid product design using AI algorithms. It identifies the limitations of traditional product design methods and shows how textual, image, audio, and video data in product design cycles can be utilized to achieve much more intelligent product design. We finally discuss the major deficiencies of existing data-driven product design studies and outline promising future research directions and opportunities, aiming to draw increasing attention to modern AI-driven product design.

1. Introduction

Developing competitive products that exceed consumers’ expectations plays a central role in enterprise activities. Successful products can enhance user satisfaction, stimulate purchase desire, increase sales, and promote the completion of the ‘product–money–new product’ cycle, enabling enterprises to thrive in the competitive market [1]. Over the past few decades, product design has undergone significant changes in design concepts and methods, driven by advancements in manufacturing and technology. Design concepts have evolved from ‘form follows function’, ‘technology first’, and ‘product-centric’ to ‘form follows emotion’ and ‘user-centric’ [2]. Design methods have shifted from experiential and fuzzy design to intelligent design, computer-aided design, and multi-domain joint design [3,4,5,6]. Despite this progress, the fierce market competition calls for next-generation product innovation and design methodologies based on the significant progress of big data, deep learning, and other AI algorithms, which have brought a transformative revolution in the understanding, pattern recognition, and generative design and synthesis of text, image, video, audio, etc. [7,8].

Currently, customers are increasingly focused on their diversified and personalized spiritual and emotional needs. They pay more and more attention to product appearance [9,10,11,12,13] in addition to its functions. Traditional product design methods, such as the theory of inventive problem solving (TRIZ), quality function deployment (QFD), KANO model, Kansei engineering, and axiomatic design (AD), have been well-established [2,14,15,16]. However, capturing user requirements, ensuring visibility, and evaluating products remain significant challenges in the field of product design. The fundamental aspect of capturing user requirements and evaluating products is data acquisition. Traditional methods rely on manual surveys [10,17,18,19], which are time-consuming, labor-intensive, and often result in uneven data distribution. Additionally, these data are one-time and cannot be updated, which is a major obstacle in the fast-paced era we live in [20]. Alternatively, relying on designers’ or experts’ experience and intuition is highly subjective, uninterpretable, and risky [21]. As we will discuss in detail in Section 3, traditional methods also have several other drawbacks. These deficiencies led to lengthy product development cycles, poor predictability, and low success rates, which are crucial to enterprises. However, recent studies have found that big data and AI algorithms can alleviate these problems in an automated and intelligent manner [20,22]. Especially, the AI-Generated Content (AIGC) can bring disruptive breakthroughs to product design. Big data and AI algorithms have been increasingly applied in the field of product design [18,23].

With the rapid development of the Internet, loT, and communication technologies, a large amount of data has accumulated in the product lifecycle (Figure 1), which is expanding exponentially every day [24,25,26]. The product lifecycle contains a lot of product feedback information, such as user preferences, market demands, and visual displays [27,28,29]. This information is valuable for guiding product design and has sparked increasing interest in both product design and big data fields [30,31]. On one hand, processing and analyzing such a massive amount of data presents new challenges that need to be appropriately addressed. On the other hand, successful analysis can lead to better products. Thus, how to extract valuable information from big data and apply it to design remains the primary difficulty and focal point of current research [22,27,31].

Big data in the product lifecycle are characterized by multiple data types, a large volume, low-value density, and fast update, making them challenging to handle with conventional techniques and algorithms. However, AI algorithms have strong capabilities in processing big data, including convolutional neural networks (CNNs), generative adversarial networks (GANs), natural language processing (NLP), neural style transfer, motion detection, speech recognition, video summarization, and emotion recognition. Figure 2 provides an overview of mining product-related information from big data. This paper focuses on analyzing big data with AI algorithms and applying the findings to product design.

In recent years, big data and AI-driven product design have become a significant research hotspot. However, to the best of our knowledge, before this work, there has been almost no comprehensive summary of the applications of big data in product design. Therefore, a thorough literature review is necessary to provide theoretical foundations that can be utilized to develop scientific insights in this area. Furthermore, it can help enterprises to reduce the time and cost of product development, enhance user satisfaction, and promote the advancement of product design towards automation and intelligence. This paper introduces both the traditional product design method and the big data-based method, highlights their limitations and challenges, and focuses on the application, algorithm, and process flow of structured, textual, image, audio, and video data. The research framework is shown in Figure 3.

Figure 3 shows the structure of the paper. Section 2 discusses the fundamental tasks in product design, while Section 3 provides an overview of several widely used traditional product design methods along with their limitations. Section 4 reviews the application of big data and AI algorithms in product design, including structured, textual, image, audio, and video data. Moreover, potential future studies and current limitations are discussed in this section. In Section 5, we summarize the paper and outline the future direction of the product design field.

2. Key Tasks in Product Design

Figure 4 summarizes the process of product design. It includes nine key tasks: product vision definition, market research, competitive research, user research, idea generation, feasibility analysis, sketching, prototyping, and scheme evaluation. The product vision, which clarifies the overall goal of the product, is typically defined before the design process begins. Market research is carried out to understand the development trend, user demand, and purchasing power. Competitive analysis compares existing products from multiple dimensions and derives their advantages and deficiencies.

User research is supported by data. Traditional methods of data collection, such as observation, experiment, interview, and survey, are often insufficient in terms of the quality and quantity of data, when compared to big data-driven methods. To extract user requirements, preferences, and satisfaction, traditional methods typically utilize Kansei engineering, QFD, the KANO model, AD, and affective design [14,16,32,33]. In contrast, big data-driven methods widely use technologies such as NLP, speech recognition, emotion recognition, and intelligent video analysis. Depending on different object-oriented, product design methods can be broadly categorized into product-centric and user-centric. The product-centric approach focuses on improving performance and emphasizes that users passively adapt to the product. In contrast, the user-centric approach prioritizes satisfying users’ spirit requirements and emphasizes that products must actively adapt to users. As users’ requirements have become more and more diversified and personalized, the user-centric approach has become mainstream. Kansei engineering and the KANO model are typical user-centric methods, and QFD also involves user requirements [14,16,33].

Idea generation can be classified into two types based on different innovative thinking: logical thinking and intuitive thinking [34]. Logical thinking focuses on detailed analysis and decomposition of problems, such as TRIZ, universal design, and AD [15,32,35]. In contrast, intuitive thinking aims to inspire designers and includes brainstorming, bionics, analogy, combination, and deformation [1,2,36]. For instance, Nathalie and John [37] utilized brainstorming to stimulate inspiration, while Youn et al. [38] found that combining ideas was a significant driver of invention by analyzing a vast number of patents. Similarly, Lai et al. [9] created new products by merging various product forms and colors, and Zarraonandia et al. [39] used the combinatorial creativity in digital game design. After generating ideas, the next task is feasibility analysis. This involves technical, economic, security, infringement, and environmental analysis. Notably, technical analysis examines whether there are any inconsistencies among various technical attributes and resolves them.

Product display is important for enabling users, designers, and experts to intuitively comprehend the designed product. Sketches and prototypes serve as visualizations of ideas. However, traditional methods often require designers to process drawing skills, such as hand drawing or 3D modeling. In contrast, big data and AI algorithm driven methods offer simpler operations and do not require drawing skills. Furthermore, traditional display methods often involve images or 3D models, whereas big data and AI algorithm driven methods can incorporate videos or interactions, resulting in more intuitive and engaging displays.

Evaluation is the final step. Once the product is designed, it will be presented to users, experts, designers, or decision makers for feedback. Based on this feedback, the product can be improved and verified to ensure it aligns with the original vision. For instance, Mattias and Tomohiko [40] proposed a novel evaluation method that considers the importance of customer values, the contribution of each offering to the value, and the customer’s budget. They successfully applied this method to a real-life case at an investment machine manufacturer. User evaluation can also be regarded as a part of the user research task.

3. Traditional Product Design Methods

3.1. Kansei Engineering

Kansei engineering is a widely used user-centric method that is commonly utilized in the user research task [41]. In the 1970s, Nagamechi noted that, in Japan, where material wealth was already abundant, the consumption trend had shifted from functionality to sensibility, with sensibility becoming the core of product design [42]. Nissan, Mitsubishi, Honda, and Mazda used Kansei engineering to improve car positioning, shape, color, dashboard, and more, resulting in significant success for the Japanese automobile industry [43,44,45]. In the late 1990s, Kansei engineering expanded to Europe. Schütte proposed a modification strategy to simplify the approach for European culture [46,47,48,49]. In addition, Schütte discussed samples selection [50] and visualized Kansei engineering steps [51]. Nagamechi laid the foundation of Kansei engineering and continued to explore it further. In his latest research [52], he suggested the introduction of AI in Kansei engineering. The framework of Kansei engineering is shown in Figure 5.

The Kansei engineering process comprises four key stages: (i) Kansei words collection involves gathering words from various sources, such as magazines, documents, manuals, experts, interviewees, product catalogs, and e-mails. However, collected words may lack pertinence and applicability due to their diverse fields of origin; (ii) Kansei words selection can be done manually, but this may lead to subjective results. To address this issue, some researchers have used techniques like principal component analysis (PCA), factor analysis, hierarchical clustering analysis, and K-means for word selection. For instance, Djatna et al. [53] adopted the Term Frequency-Inverse Document Frequency (TF-IDF) approach to select high-frequency Kansei words for their tea powder packaging design, while Shieh and Yeh [10] used cluster analysis to select four sets of Kansei words out of a hundred; (iii) For Kansei evaluation, most studies use semantic differential (SD) scales or Likert scales to design questionnaires and obtain Kansei evaluations from survey results. However, some researchers have paid attention to physiological responses [11,54] as they believe these signals are more reliable than the defined score. For example, Kemal et al. [12] used the eye-tracker to obtain objective data on the ship’s appearance, including the area of interest (AOI), scan path, and heat maps. Nevertheless, relying on a single physiological signal can lead to one-sided results, and thus, Xiao and Cheng [55] designed several experiments involving eye-tracking, skin conductance, heart rate, and electroencephalography. We notice that physiological signals are more related to the intensity of Kansei than the content; (iv) The final step is constructing the relationship model that links user affections to product parameters. Various methods like quantification theory, support vector machine, neural networks, etc., can be employed to achieve this.

Although Kansei engineering has achieved significant success in product design, there are still several deficiencies that need to be addressed. (i) The collected Kansei words are often inadequate in terms of their pertinence and number; (ii) The small number of samples and subjects; (iii) Both questionnaires and physiological signals are susceptible to the effects of time, environment, and concentrated subjects, which can reduce the authenticity and objectivity of the results; (iv) Limited by life background and work experience, subjects’ evaluations in the questionnaire vary greatly; (v) The survey scope is often small, leading to uneven data distribution. These deficiencies mainly arise from data acquisition, especially the collection of Kansei words and Kansei evaluation. The quantity and quality of survey data can directly affect the constructing of the relational model, thus, making data acquisition an urgent issue that need to be addressed.

3.2. KANO Model

The KANO model is utilized in user research tasks to classify user requirements by revealing the nonlinear relationship between user satisfaction and product performance. For a long time, it was believed that user satisfaction is directly proportional to product performance. However, it has been shown that fulfilling individual product performance to a great extent does not necessarily lead to high user satisfaction, and not all performance factors are equally important.

In the 1980s, Noriaki Kano conducted a detailed study on user satisfaction and proposed the KANO model [56], which divides user requirements into five types (Figure 6). Must-be quality refers to requirements that users take for granted. When fulfilled, users just feel normal; otherwise, they will be incredibly dissatisfied, such as the call function for mobile phones. One-dimensional quality is proportional to user satisfaction, such as the battery life for mobile phones. Attractive quality increases satisfaction when fulfilled but does not cause dissatisfaction when unfulfilled, such as the temperature display for cups. Indifferent quality has no impact on user satisfaction, such as the built-in pocket for coats. Reverse quality results in dissatisfaction when fulfilled, such as pearls on men’s wear. Researchers have utilized the KANO model for various applications, including Yao et al. [19], who categorized twelve features of mobile security applications through a structured KANO questionnaire, and Avikal et al.’s [13] examination of customer satisfaction based on aesthetic sentiments by integrating the KANO model with QFD.

The KANO model is an essential tool for understanding user satisfaction and prioritizing product development efforts [16,57]. According to the priority assigned by the KANO model, enterprises must ensure the must-be quality reach the threshold, and any additional investment is a waste; invest in one-dimensional quality as much as possible; prioritize attractive quality when the budget permits; avoid reverse quality and never waste resources on indifferent quality. However, not all customer requirements are equal, even within the same category. As the KANO model cannot distinguish the differences among requirements within the same category [58], Lina et al. [59] proposed the IF-KANO model. This model adopts logical KANO classification criteria to categorize requirements. They take elevators as an example; results showed that both load capacity and operation stability belong to a one-dimensional quality but the priority of operation stability is slightly higher than load capacity.

The KANO model relies on the data acquisition of the KANO questionnaire, which traditionally provides a single option or options within a given range, failing to fully capture the ambiguity and complexity of customers’ preferences. As a result, numerous scholars have attempted to enhance the KANO questionnaire’s options [60,61]. For instance, Mazaher et al. [62] proposed the fuzzy KANO questionnaire, which uses percentages instead of fixed options and allows participants to make multiple choices. Chen and Chuang [63] introduced the Likert scale to gauge the degree of satisfaction or dissatisfaction. Cigdem and Amitava [64] combined the Servqual scale and the KANO model in a complementary way. While some researchers have recognized that text can better express opinions than scale, they also note that text is challenging for traditional statistical methods [65]. In addition to improving questionnaire options, some studies have focused on questioning skills to enhance the KANO questionnaire. For example, Bellandi et al. [66] suggested that questions should avoid polar wording.

While improvements have been made to the KANO questionnaire, as mentioned in Section 3.1, it is important to acknowledge its limitations. In addition, the KANO model itself also has some drawbacks. (i) The model relies on pre-existing requirements and lacks research on the acquisition; (ii) The KANO model only focuses on the categorization and rating of customer requirements, and its application is relatively simplistic.

3.3. QFD

QFD was first proposed by Japanese scholars Yoji Akao and Shigeru Mizuno in the 1970s, and aims to translate customer requirements into technical attributes, parts characteristics, key process operations, and production requirements [15,33,67]. The house of quality (HoQ) is the core of QFD, as depicted in Figure 7. QFD can be used for multiple tasks, such as generating ideas, analyzing competition, and assessing technical feasibility.

Starting from meeting market demands and driven by customer requirements, QFD explicitly translates requirements information into specific information that is directly used by design, production, and sales departments, ensuring that the resulting products meet customer needs and expectations [68]. For instance, Tomohiko [21] applied QFD to hair dryer design, using it to decompose customer requirements into characteristics and building a HoQ to guide the design process. Yan et al. [69] applied QFD for competitive analysis, generating insights into product improvement strategies. Noting the inherently vague and ambiguous nature of customer requirements [17,70], Cengiz et al. [71] proposed fuzzy QFD. Additionally, QFD can translate requirements from various stakeholders such as recyclers, production engineers, and customers. Considering that the technical characteristics in the HoQ may be contradictory, Wang et al. [72] combined QFD with TRIZ and used the contradiction matrix to derive a solution. However, QFD’s reliance on experts to rank customer requirements leads to intense subjectivity. To mitigate this issue, scholars have introduced the Multi-Criteria Decision-Making (MCDM) methods [73,74,75,76]. Although QFD, similar to Kansei engineering, employs a definite score to express evaluation, ignoring the uncertainty of user and expert scoring, some studies have introduced the rough theory, interval-valued fuzzy-rough sets, and grey relational analysis to address this limitation [77,78,79,80].

Although QFD has made some progress in product design, it still has limitations. (i) The relationship between customer requirements and characteristics is determined by experts manually, which heavily relies on their expertise; (ii) The evaluation of satisfaction is often completed by a small number of subjects, leading to potential bias; (iii) Customer requirements are directly given by experts, customers, designers, or summarized by scholars, lacking data acquisition.

3.4. TRIZ

TRIZ is an approach that helps to of generate inventive solutions by identifying and resolving contradictions. It can be utilized for both idea generation and technical analysis tasks. Genrich Altshuller, a Soviet inventor, and his colleagues began developing TRIZ in 1946 [81]. By analyzing over 400,000 invention patents, Altshuller developed the technical contradiction, the concept of ideality of a system, contradiction matrix, 40 principles of invention (

Table 1. The 40 inventive principles of TRIZ [2].

No.	Inventive Principle	No.	Inventive Principle
1	Segmentation	21	Skipping
2	Taking out	22	Blessing in disguise
3	Local quality	23	Feedback
4	Asymmetry	24	Intermediary
5	Merging	25	Self-service
6	Universabrationlity	26	Copying
7	Nested dol1	27	Cheap short-living
8	Anti-weight	28	Mechanics substitution
9	Preliminary anti-action	29	Pneumatics and hydraulics
10	Preliminary action	30	Flexible shells and thin films
11	Beforehand cushioning	31	Porous materials
12	Equipotentiality	32	Colours changes
13	The other way around	33	Homogeneity
14	Spheroidality	34	Discarding and recovering
15	Dynamics	35	Parameter changes
16	Partial or excessive actions	36	Phase transitions
17	Another dimension	37	Thermal expansion
18	Mechanical vibration	38	Strong oxidants
19	Periodic action	39	Inert atmosphere
20	Continuity of useful action	40	Composite material film

), and 39 engineering parameters (

Table 2. The 39 engineering parameters [2].

No.	Engineering Parameter	No.	Engineering Parameter
1	Weight of moving object	21	Power
2	Weight of nonmoving object	22	Waste of energy
3	Length of moving object	23	Waste of substance
4	Length of nonmoving object	24	Loss of information
5	Area of moving object	25	Waste of time
6	Area of nonmoving object	26	Amount of substance
7	Volume of moving object	27	Reliability
8	Volume of nonmoving object	28	Accuracy of measurement
9	Speed	29	Accuracy of manufacturing
10	Force	30	Harmful factors acting on object
11	Tension, pressure	31	Harmful side effects
12	Shape	32	Manufacturability
13	Stability of object	33	Convenience of use
14	Strength	34	Reparability
15	Durability of moving object	35	Adaptability
16	Durability of nonmoving object	36	Complexity of device
17	Temperature	37	Complexity of control
18	Brightness	38	Level of automation
19	Energy spent by moving object	39	Productivity
20	Energy spent by nonmoving object

). In addition, Altshuller observed smart and creative individuals, discovered patterns in their thinking, and developed thinking tools and techniques to model this “talented thinking”.

The TRIZ methodology involves four main steps: (i) defining the specific problem; (ii) abstracting the problem to a more general level; (iii) mapping potential solutions to address the general problem; and (iv) concretizing the general solution to fit the specific problem. For instance, Wang [2] employed principles 1 (segmentation), 5 (combining), and 28 (replacement of a mechanical system) from Table 1 to design phone cameras. Yamashina et al. [15] combined QFD and TRIZ to perform washing machine design. Additionally, Ai et al. [82] designed low-carbon products by considering both technical system and human use, and they used TRIZ to identify measures for improving energy efficiency.

3.5. Summary of Limitations

Among the traditional methods mentioned above, QFD focuses on constructing the HoQ to map customer requirements to product parameters. The KANO model mainly emphasizes the classification and rating of customer requirements. Both QFD and KANO adopt the pre-given requirements and lack research on acquisition. Kansei engineering is a comprehensive method that involves the acquisition, expression, and mapping of customer requirements. However, Kansei engineering still has limitations in the data collection process. To capture customers’ affective responses towards different products, various traditional methods are widely used, such as user interviews, questionnaires, focus groups, experiments, etc. [10,11,41,55]. We have summarized their advantages and disadvantages in

Table 3. Comparison of user requirement mining methods.

Methods	Description	Advantages	Disadvantages
User interview	The interviewer talks directly with the subject	Detailed; easy to implement	Time-consuming; one-time; subjective; labor-intensive; small survey scope
Questionnaire	Record subjects’ opinions on specific questions	Easy to implement	Subjective; one-time; time-consuming; labor-intensive; centralized in time and place; small survey scope
Web-based questionnaire	Distribute questionnaires on the Internet	Decentralized in time and place; large survey scope	Subjective; one-time
Focus-group	Observe the opinions and behaviors of a group on the subject	Real data; low cost; in-depth questions	Time-consuming; labor-intensive; one-time; complex; small survey scope
Usability testing	Subjects test the product and give feedback	Detailed; high reliability	One-time; labor-intensive; small survey scope; time-consuming
Experience	Analyze the data generated by consumers during usage	Easy to implement	Time-consuming; slow; inefficiency; subjective; small survey scope
Experimental	Record the psychological and physiological data of the subject	Detailed; high reliability	Expensive; one-time; time-consuming; labor-intensive; complex; small survey scope

.

Overall, traditional product design methods suffer from several significant disadvantages. (i) It is difficult to capture accurate customer requirements due to the increasing diversification and complexity of customer needs, making it challenging for enterprises to determine product positioning; (ii) Various shortcomings in the process of data acquisition, such as being manual, time-consuming, labor-intensive, hard to update, limited in scope, and subject to time and place. This results in small sample sizes, poor real-time data, and limited quality and quantity of samples and data. In addition, traditional data collection is time-consuming and quickly becomes outdated; (iii) Survey results are susceptible to time and environment, and there may be differences between subjects and real users, which can impact the validity of the data; (iv) Heavy reliance on expert can increase workload and prolong the product development cycle, while different experts may have varying backgrounds and experiences, introducing uncertainty and subjectivity in designs; (v) Traditional methods lacking an intuitive approach to inspiring designers, and visualization plays a crucial role in it; (vi) The absence of visual display in early design schemes can make it difficult for enterprises to hold product development direction, as decision-makers rely on imagination alone and cannot see designs. This increases the probability of failure resulting in wasted resources and potentially leading to enterprise bankruptcy. These deficiencies have significantly hindered the development of product design.

The advent of the big data era has brought forth innovative ideas and technologies that can overcome the shortcomings of traditional product design methods and enhance innovation capabilities. By leveraging big data and AI algorithms, we can reduce subjectivity, expand the scope of survey, and automate data processing (including data acquisition, updating and analysis), to accurately acquire user requirements and present them in an intuitive visual way. To illustrate this point, we can consider product evaluation as an example. We can collect customer reviews from e-commerce platforms, social media, and review websites worldwide using web crawlers. With the help of NLP technology, we can automatically extract information, such as product attributes, opinion words, and sentiment orientations, in customer evaluations of products. Furthermore, as customer evaluations are dynamic, we can easily obtain real-time evaluation by simply adding a piece of updated code. In the next section, we will delve deeper into the new-generation data and AI algorithm driven product design methods, which are the primary focus of this article.

4. Product Design Based on Big Data and AI

4.1. Product Design Based on Structured Data

Big data in the product lifecycle can be categorized into structured, semi-structured, and unstructured [30]. Structured data separate the structure and content, whereas semi-structured data mix them. Unstructured data, such as audio, video, images, text, and location, have no fixed structure.

Structured data refer to information that has been formatted and transformed into a predefined data model, which provides regularity and strict formatting. However, structured data can suffer from poor scalability and limited flexibility. Semi-structured data can be considered as an extension of structured data, that offers greater flexibility and extensibility. For this reason, in this paper, we will treat structured data and semi-structured as the same and introduce their applications in design together.

To address the limitations of traditional questionnaire surveys, including small survey scope, difficulty in updating, time-consuming, and labor-intensive, Li et al. [83] proposed a machine learning-based affective design dynamic mapping approach (MLADM). The approach involves collecting Kansei words from literature, manually clustering them, obtaining product features and images from shopping websites, and generating online questionnaires. Four machine learning algorithms are used to construct the relationship model. However, despite MLADM being able to predict product feeling, it still heavily relies on the expert, and the online questionnaire data are also highly subjective. To overcome these limitations, some studies explore objective data for design knowledge. For instance, Jiao et al. [84] established a database to record affective needs, design elements, and Kansei words from past sales records and previous product specifications. They applied association rule mining to construct the relationship model and used it as the product design inference pattern.

Limited by data sources’ openness, it is not easy for designers to obtain structured data, which can impede their research in product design. In contrast, unstructured data holds advantages in terms of volume and accessibility, accounting for nearly 95% of big data [85]. Despite its amorphous and complex nature, unstructured data are still of great interest to scholars because of their implicit information and value. In Section 4.3, Section 4.4, Section 4.5 and Section 4.6, we will, respectively, introduce the application of textual, image, audio, and video data in product design, as well as the AI algorithms involved.

4.2. Product Design Based on Textual Data

The text exists in every phase of a product’s lifecycle. Among them, user experience feedback after purchasing the product is particularly valuable for product design. In recent years, more and more users have shared their opinions on Twitter, Facebook, microblogs, blogs, and e-commerce websites [86,87,88,89,90,91,92]. As online reviews on e-commerce websites are provided by consumers who have purchased the product, they are considered highly reliable and authentic [93]. Research shows that 90 % of customers use online reviews as a reference, with 81% finding them helpful for purchase decisions, and 45% consulting them even while shopping in physical stores [94,95]. Consequently, online reviews have become an important influence factor in consumer behavior and a reference for enterprises to use to improve their products [22,96].

Online reviews offer many benefits, including understanding customer satisfaction [97,98,99,100,101], capturing user requirements [102,103,104], finding product deficiencies [105,106,107], proposing improvement strategies [65], comparing competitive products [108], and providing product recommendations [109,110,111,112,113,114]. For instance, in order to identify innovation sentences from online reviews, Zhang et al. [4] proposed a deep learning-based approach. Jin et al. [22,108,115,116] focused on filtering out helpful reviews. Wang et al. [117] presented a heuristic deep learning method to extract opinions and classified them into seven pairs of affective attributes, namely “like-dislike”, “aesthetic-inaesthetic”, “soft-hard”, “small-big”, “useful-useless”, “reliable-unreliable”, and “recommended-not recommended”. Kumar et al. [118] combined reviews and electroencephalogram signals to predict product ratings, and Xiao et al. [94] proposed a marginal effect-based KANO model (MEKM) to categorize customer requirements. Simon et al. [119] explored product customization based on online reviews. Figure 8 summarizes the typical processing flow of textual data.

The rapid development of NLP is the key to product design based on textual data, which including topic extraction, opinion mining, text classification, sentiment analysis, and text clustering.

Product attributes extraction. In the field of product design, topic extraction can be used to extract product attributes (e.g., the screen, battery, weight, and camera of a smartphone) and opinions [120]. As each product has multiple attributes, and consumers have varying preferences and evaluations for each attribute, extracting attributes from online reviews becomes essential [110]. Typically, product attributes are nouns or noun phrases in online review sentences [121]. To extract product attributes, most studies utilize the part-of-speech (POS) tag to generate tags identifying whether a word is a noun, adjective, adverb, etc. and consider all nouns and noun phrases as attribute candidates. These candidates are then pruned using techniques such as term-frequency (TF) [116,122], TF-IDF [123,124], dictionary [125], manual definition [99], and clustering [96,126]. Moreover, since product attributes are domain-sensitive, some studies consider it a domain-specific entity recognition problem. For instance, Putthividhya and Hu [127] used named entity recognition (NER) to extract product attributes.

The methods discussed above are limited to extracting explicit attributes and are not suitable for implicit attributes. The distinction between explicit and implicit attributes lies in whether they are explicitly mentioned or not. For example, in the sentence “laptop battery is durable, the laptop is expensive”. The “battery” is explicitly mentioned as an explicit attribute, while “expensive” is related to price and is an implicit attribute. Explicit attributes can be easily identified using rule-based or machine learning-based methods, implicit attributes require more sophisticated techniques [126]. Various unsupervised, semi-supervised, and supervised methods have been developed for implicit attribute extraction [128]. For instance, Xu et al. [129] used an implicit topic model that incorporated pre-existing knowledge to select training attributes and developed an implicit attribute classifier based on SVM. To overcome the lack of a training corpus, they annotated a large number of online reviews. Meanwhile, Kang et al. [130] proposed an unsupervised rule-based method that can extract both subjective and objective features, including implicit attributes, from customer reviews. Additionally, employing synonym dictionaries is a viable method. For example, Jin et al. [108] combined WordNet and manually defined synonyms to extract attributes of mobile phones. WordNet is an English lexical database that organizes words based on synonyms and antonyms. By analyzing online comments, designers can gain a more detailed understanding of users’ attention to each product attribute at a finer granularity, enabling a more precise analysis.

Opinion mining. In product design, opinion mining aims to extract descriptive words that express subjective opinions, commonly known as opinion words. There are two primary approaches to this task: co-occurrence analysis and syntactic analysis. The co-occurrence approach identifies opinion words by analyzing the adjectives that appear in proximity to product attributes. Meanwhile, the syntactic approach relies on analyzing the structure and dependency of review sentences. For example, Hu and Liu [131] categorized product attributes as either frequent or infrequent, with the former identified through POS and association mining, and nearby adjectives deemed opinion words. The latter were identified through a reverse search for opinion words.

In addition to the two types of approaches discussed above, topic modeling has also shown promising results in opinion word extraction. For instance, Bi et al. [97] used latent Dirichlet allocation (LDA) to extract customer satisfaction dimensions from online reviews and proposed the effect-based KANO model (EKM). LDA is a topic model that can group synonyms into the same topic and obtain its probability distribution. Wang et al. [132] applied the long short-term memory (LSTM) model to extract opinion words from raw online reviews and mapped customer opinions to design parameters through deep learning. However, it is worth noting that the topic model ignores the fine-grained aspects.

Several scholars have studied the expression of opinion words since different reviewers may share the same opinions but use different words to express them. In our previous research [133], we clustered similar opinion words based on word2vec. Additionally, Wang et al. [134] restructured raw sentences to extract attribute-opinions from online reviews, following grammatical rules. However, different languages have different sentence structure rules, so it is hard to expand to other languages.

Sentiment analysis. In product design, sentiment analysis plays a crucial role in identifying customers’ emotional attitudes towards a product and classifying them as positive, negative, or neutral. Sentiment analysis can be conducted at three different levels [135,136,137]: (i) document-level, which is a coarse-grained analysis of all reviews in a document; (ii) sentence-level, which is a medium-grained analysis of individual sentences; and (iii) aspect (attribute)-level, which is a fine-grained analysis of specific product attributes.

In addition, sentiment analysis methods are categorized into three categories: machine learning methods, lexicon-based methods, and hybrid methods [117,118,138,139]. Machine learning methods regard sentiment analysis as a classification problem, using sentiment polarities as labels and employing various techniques, such as Recurrent Neural Network (RNN) [4], support vector machines [140,141], conditional random field (CRF) [135], and neural networks [136], to construct classifiers with text features. In contrast, lexicon-based methods identify sentiment orientations by referring to pre-defined lexicons like LIWC, HowNet, and WordNet [141,142]. These lexicons contain sentiment-related terms and their corresponding polarity. However, the quality of these lexicons is critical, and scholars are working towards improving the coverage, domain adaptation, and continuous updating of these resources. For instance, Cho et al. [143] constructed a comprehensive lexicon by merging ten lexicons. Araque et al. [144] proposed a sentiment classification model that uses semantic similarity measures and embedding representations, rather than keyword matching, to compute the semantic distance between the input word and lexicon word rather. Marquez et al. [145] analyzed several lexicons and how they complement each other.

Lexicon collection can be divided into three categories: manual, dictionary-based, and corpus-based. The manual method is time-consuming and labor-intensive. The dictionary-based method uses a set of opinion words as seeds to search in existing dictionaries. The corpus-based method expands lexicons based on information from the corpus, resulting in domain-specific lexicons [146].

Machine learning methods offer higher accuracy, while lexicon-based methods are more general. Hybrid methods combine both. For instance, Dang et al. [147] proposed a lexicon-enhanced approach for sentiment classification that combines machine learning and semantic-orientation methods. Their findings indicate that the hybrid method significantly improves sentiment classification performance.

By reviewing literature related to product design based on textual data, we have found that it is an important research topic that has been addressed by many scholars in recent years. NLP technology is used to extract product attributes and sentiment features from text, which helps to understand user requirements, evaluate products, and grasp product development trends. While requirements and evaluates are the core foundation, it is important to note that the rest of the design process is equally crucial. Unfortunately, most existing research has overlooked this point, resulting in designs that are more of an improvement rather than being original and innovation. In other words, product design based on text still has vast potential for development. Undoubtedly, the emergence of textual data has great significance for product. Compared to traditional methods, textual data provides richer information of higher quality and can be acquired and processed much more quickly.

4.3. Product Design Based on Image Data

Product images serve as intuitive representations of products that convey essential information on color, texture, and shape, playing a crucial role in the design process. Images are an example of “what you see is what you get”, as our eyes can easily interpret the information, constituting a significant advantage. As image data keep growing, it has evolved into a vital information carrier alongside textual data. In recent years, CNN has made significant breakthroughs in image recognition, classification, and segmentation [148,149,150]. Its powerful ability to learn robust feature has attracted attention across various fields. Figure 9 shows the structure of CNN. In product design, image data plays two key roles: inspiration and generation. The former inspires designers by existing product images, while the latter directly generate new product images based on large amounts of existing images.

Spark creative inspiration. Existing product images are helpful to inspire designers to come out with new ideas and initial design schemes more efficiently [151,152]. To achieve this goal, image retrieval, matching, and recommendation play crucial roles [3,150,153,154]. Designers and users can express their requirements through images and text and search for related product images from databases or e-commerce websites. The matched images can be recommended as design references. The retrieval input can be either text, images, or both [155,156,157,158].

However, product retrieval is more complicated than simple image retrieval due to the different shooting angles, conditions, backgrounds, or postures of the images [159,160,161,162,163]. For instance, clothing images taken on the street or in a store with a phone may differ from those in databases and e-commerce websites. Liu et al. [164] used a human detector to locate 30 human parts, and then utilized a sparsely coded transfer matrix to establish a mapping that ensure the two distributions dose not compromise the retrieval quality. On the other hand, free-hand sketches are even more abstract. Yu et al. [165] introduced two new datasets with dense annotation and built a deep network trained with triplet annotations to enable retrieval across the sketch/image gap. Ullah et al. [166] used a 16-layer CNN model (VGG16) to extract product features from images and measure their similarities using the Euclidean distance.

In addition, some studies have explored the use of both image and text features for product matching and categorization. For instance, Ristoski et al. [125] used CNN to extract image features and match them with text embedding features to improve product matching and categorization performance. Liang et al. [167] proposed a joint image segmentation and labeling framework to retrieve clothing. They grouped superpixels into regions, trained an E-SVM classifier using confident foreground regions, and propagated segmentations by applying the E-SVM template to the entire image.

In addition to serving as a reference, image data can be leveraged to track fashion trends [168,169], identify user preferences [168,170], provide product matching recommendations [171,172,173,174], and prevent infringement [151]. For instance, Hu et al. [169] established a furniture visual classification model that contains 16 styles, such as Gothic, Modernist, and Rococo style. The model combines image features extracted by CNN with handcrafted features and can help users understand their preferred styles. Most products do not stand alone, and their compatibility with other products must be considered during the design process. To address this, Aggarwal et al. [175] used Siamese networks to assess the style compatibility between pairs of furniture images. They also proposed a joint visual-text embedding model for recommendation, based on furniture type, color, and material. Laura et al. [176] built a graph neural network (GNN) model to account for multiple items’ interactions, instead of just pairwise compatibility. Their GNN model comprised a deep CNN to extract image features and enabled evaluating the compatibility of multiple furniture items based on style, color, material, and overall appearance. Moreover, they applied the GNN model to solve the fill-in-the-blank task, for example, recommending the most suitable bed from multiple alternatives based on a given desk, cabinet, chair, and mirror.

Generate new product images. In the field of product design, two popular models are used to generate new product images: GAN and neural style transfer. Both of these models take images as input and produce output images. GAN is an unsupervised model that consists of two neural networks, namely the generator and discriminator [177]. The generator creates images, while the discriminator evaluates the generated images against real images. Both networks are trained simultaneously and are improve through competition. Figure 10 shows the structure of GAN. We summarize the contribution of GAN to product design as follows: schemes generation [151,178,179,180], text-to-image synthesis [181], generative transformation [182], collocation generation [183,184,185], sketch acquisition [152,186], colorization [187,188,189,190], and virtual display [28,191,192]. Some examples of new product generation based on image data are shown in Figure 11. In Figure 11a, a new image of handbag is generated given an image from the shoe. In Figure 11b, the edge image is input, new shoes are generated in the second and fourth columns, and the rest are ground truth. In Figure 11c, a product image and style image are inputted, then a new product will be generated.

The automatic generation of product schemes is made possible by using a large number of existing product images. GAN can learn the distribution of product features from these images and generate new designs through the gradual optimization of a generator by a discriminator. For example, Li et al. [151] used GAN to create new smartwatches schemes by training their model with 5459 smartwatch images. They also compared the results of GAN with various extensions, including deep convolution GAN (DCGAN), least-squares GAN (LSGAN), and Wasserstein GAN (WGAN). In another example, to produce collocation clothing images, Liu et al. [194] proposed Attribute-GAN, which comprises a generator and two discriminators. However, GAN only generate one view of the product, which is insufficient for design purposes. To address this limitation, Chen et al. [191] proposed conditional variational GAN (CVGAN), which can synthesize arbitrary product views. This is useful for virtual display, especially for designs that require high shape and appearance standards, such as clothing design where it’s necessary to show the effect of try-on. Moreover, GAN and its extended models are also applicable to layout generation, such as interior, websites, and advertisements [6,195,196].

Text-to-image synthesis is a promising technique for product design, although current research in this field is scarce. One notable study by Kenan et al. [181] proposed an enhanced attentional GAN (e-AttnGAN) for generating fashion images from text descriptions, such as “long sleeve shirt with red check pattern”. Most existing studies aim to improve the synthesis technology [197,198,199,200], and the potential of text-to-image synthesis has yet to be fully realized by designers. It is worth noting that a lot of commercial software has been developed for text-to-image synthesis, including Midjourney, DALL-E2, Imagen, Stable Diffusion, Novel AI, Mimic, Dream by WOMBO, wenxinyige, Tiamat, and 6pen art. These platforms allow users to express their requirements in text and their desired image will be automatically generated. To demonstrate the effectiveness of this technique, we also made some attempts, and the resulting product images are displayed in Figure 12. We generated plates, vases, mugs, butterfly necklaces, crystal balls, and phone cases with wenxinyige. We believe that text-to-image will become an important direction for product design in the future. Users will only need to type out their requirements, without having to master complex skills such as hand-drawing, sketching, or 3D modeling. In addition to text-to-image, progress has also been made in text-to-3D [201,202]. For instance, Wang et al. [203] proposed the Variational Score Distillation (VSD), which models 3D parameters as a probability distribution and optimizes the distance between the distribution of rendered 2D images and the distribution of a pre-trained 2D diffusion model. This approach generates high-quality textured 3D meshes based on the given text.

Generative transformation and colorization can be seen as a type of image-to-image problem, where an input image is transformed into a modified output image [190,204,205]. Generative transformation aims to convert one product image into another. This process generates a sequence of intermediate images that may be used as new design ideas. Zhu et al. [206,207,208] developed a product design assistance system that leverages GAN to achieve three applications: (i) change product shape and color by manipulating an underlying generative model; (ii) generate new images from user scribbles; and (iii) performing generative transformation of one product picture to another product. For example, a short black boot can be converted into a long brown boot, and intermediate shoe images are displayed. Nikolay et al. [192] proposed the conditional analogy GAN (CAGAN) to swap clothing on models automatically. Pan et al. [209] proposed DragGAN to control the pose, expression, and layout of products in the image.

Since the sketch already contains the structural and functional features of the product, once it is colored, a preliminary design scheme is completed. Compared to other product features, color has an intuitive influence, and users have different preferences for it. GAN can automatically color the sketch and quickly generate multiple alternatives. For example, Liu et al. [190] established an end-to-end GAN model for ethnic costume sketch colorization, demonstrating its excellent ability to learn the color rules of ethnic costumes. Additionally, Sreedhar et al. [180] proposed a car design system based on GAN that supports single-color, dual-color, and multiple-color coloring from a single sketch.

GAN and its extended models have made significant contributions to product design but the generated images often suffer from blurriness, lack of detail, and low quality. To address these problems, Lang et al. [184] proposed Design-GAN, which introduces a texture similarity constraint mechanism. Similarly, Oh et al. [5] combined GAN with topology optimization to enhance the quality of two-dimensional wheel images. By limiting the wheel image to a non-design domain, a pre-defined domain, and a design domain for topology optimization, they achieved results that were significantly different from the initial design.

Neural style transfer is a deep generative model that enables the generation of images by separating and reconstructing their content and style. Nevertheless, neural style transfer is not simply a matter of overlapping a content image with a style image. Its implementation relies to the features learned by CNN. Figure 13 shows the idea of neural style transfer.

In 2015, Gatys et al. [210] found that the image style and content could be separated in CNN and manipulated independently. Building upon this finding, they proposed a neural style transfer algorithm to transfer the style of famous artworks. Later, Huang and Serge [211] introduced an adaptive instance normalization (AdaIN) layer, which aligns the mean and variance of content features with style features to realize arbitrary style transfer. Inspired by their method, our previous research [212] combined Kansei engineering with neural style transfer to generate product design schemes (Figure 11c), resulting in an enhanced semantic of the generated product. Additionally, we developed a real-time design system that allowed users to input images and receive product schemes as output [193]. In a similar vein, Wu et al. [213] combined GAN with neural style transfer to generate fashionable Dunhuang clothes, using GAN to generate clothing shapes and neural style transfer to add Dunhuang elements. Neural style transfer is performed over the entire image, limiting its flexibility. By using masks, it is possible to transfer different styles to different parts of a product [211]. For example, using masks, three style images can be transferred to the collar, pocket, and sleeve of a coat, respectively. Overall, neural style transfer still offers tremendous untapped potential for product design.

Although GAN and its extended models show some limitations in generating high-quality and controllable designs due to their strong randomness, recent work by Sohn et al. [214] have demonstrated that AI and GAN can improve customer satisfaction and product design freshness, indicating their potential in generating innovative and attractive design proposals. On the other hand, neural style transfer is a technique used for generating product schemes with good image quality, strong interpretability, and controllability. While it may lack the innovation ability of GAN, it can be regarded as a combination of design elements such as product shape, texture, and color. Generation using these models promises to be a cost-effective way to generate product designs and support designers in creating design schemes quickly. However, further research is necessary to enhance the controllability and quality of GAN-based design generation and explore the potential of combining these techniques to generate innovative and high-quality design schemes.

Generally, image data and AIGC have broken old thinking of traditional product design. Despite the emergence of generative models in product design, the literature on this topic remains relatively sparse. Additionally, most of the existing literature primarily focuses on technical aspects and lacks product design knowledge [215,216]. Simply replacing different training image data is not a viable solution for achieving design goals [217,218], as product design requires specific knowledge and considerations. Traditional product design methods and knowledge cannot be abandoned, but rather should be combined with big data and AI algorithms. It is possible that this integration is the future direction of product design.

4.4. Product Design Based on Audio Data

Big audio data [219] refer to information in the form of sound or voice. The customer center is a valuable resource for collecting users’ complaints, inquiries, and suggestions throughout the product service cycle [220]. Audio feedback can be utilized to collect user requirements, provide recommendations, evaluate products, and improve product design. However, there is currently a lack of literature on product design based on big audio data. Therefore, we will explore several key AI technologies that could be employed in the future, such as speech recognition, speaker identification, and emotion recognition.

Some studies have opted to manually record telephone complaints as text to avoid direct processing of audio signals [221,222]. However, this approach heavily relies on the recorder and is a labor-intensive and time-consuming process. In comparison, speech recognition can accomplish this task automatically [223]. Speech recognition, also referred to as automatic speech recognition (ASR), computer speech recognition, and speech-to-text, involves converting human speech into computer-readable information, typically in the form of text, although it may also be binary codes or character sequences [224]. The process of speech recognition is shown in Figure 14. Speech recognition is widely used in mobile communication, search engines, human-computer interaction, among other applications [225].

Speaker identification, also referred to as voiceprint recognition, is a technology that identifies individuals based on their speech. Each person’s voice has unique characteristics, which are determined by two factors: the size of the sound cavity and how the vocal organs are manipulated. Like fingerprints, voiceprints need to be collected and stored in a database prior to analysis. The spectrogram is generated by the amplitude of the short-time Fourier transform (STFT) of the audio signal (Figure 15) [226]. Voiceprint recognition is achieved by extracting speaker parameters, such as pitch frequency and formant, and using machine learning methods and AI algorithms. Voiceprint recognition has been widely applied in various fields, including biometric authentication, crime forensics, mobile payment, and social security. However, the variability of voiceprint, the differences in audio acquisition equipment, and environmental noise interference pose significant challenges for the voiceprint recognition task.

As a vital division of audio processing and emotion computing, speech emotion recognition aims to identify the emotions expressed by speaker [227], including but not limited to, anger, sadness, surprise, pleasure, and panic. This process can be regarded as a classification problem, and selecting the appropriate emotion feature is essential for its success [228,229]. We have summarized commonly used acoustic parameters in

Table 4. Acoustic parameters.

Category	Parameter
Prosody parameter	Duration
	Pitch
	Energy
	Intensity
Spectral parameter	Linear predictor coefficient (LPC)
	One-sided autocorrelation linear predictor coefficient(OSALPC)
	Log-frequency power coefficient(LFPC)
	Linear predictor cepstral coefficient(LPCC)
	Cepstral-based OSALPC(OSALPCC)
	Mel-frequency cepstral coefficient(MFCC)
Sound quality parameter	Format frequency and bandwidth
	Jitter and shimmer
	Glottal parameter

. Moreover, since spectrograms are images, CNN can be used to automatically learn features for speech recognition, speaker identification, and emotion recognition [230].

As AI products gain popularity, there has been increasing focus on audio signal processing algorithm, particularly in the areas of speech recognition and emotion recognition. These algorithms directly impact the user experience and play a crucial role in customer decision-making. Despite the enormous potential of audio data in product design, practical applications remain limited by processing algorithms and data quality, creating a substantial between theory and practice applications.

4.5. Product Design Based on Video Data

Videos showcasing the usage of products are also valuable source of information for product design. They can be viewed repeatedly without restriction, making it easier to obtain hard-to-find but essential information. For instance, by watching the cooking videos of homemakers, Nagamechi found that standing up while taking food made them feel more comfortable than bending over. Additionally, Nagamechi found that the refrigerator compartment was used more frequently than the freezer compartment. Based on these insights, he improved the traditional refrigerator structure by changing the upper layer to a refrigerator compartment and the lower layer to a freezer compartment [231], which is still in use today.

Although video data is useful for product design, manual viewing can be time-consuming, and is only suitable for small data volume scenes. In comparison, intelligent video analysis supports motion detection, video summarization, video retrieval, color detection, object detection, emotion recognition, and more [219,232,233,234]. This capability makes it possible to apply big video data to user requirement acquisition, user behavior observation, experience improvement, and product virtual display. Intelligent video analysis establishes a mapping relationship between images and their descriptions, allowing computers to understand video through digital image analysis.

Product detection. Compared to single image, object detection in the video has a temporal context, which helps to solve the redundancy between consecutive frames, motion blur, unfocused image, partial occlusion, singular postures, etc. Li et al. [235] proposed a novel method for annotating products in videos by identifying keyframes and extracting SIFT features to generate BOVW histograms, which were then compared with the visual signature of products for annotation. Meanwhile, Zhang et al. [236] developed a framework for identifying clothes worn by celebrities in videos, which involved utilizing DCNN for tasks such as human body detection, human posture selection, human pose estimation, face verification, and clothing detection. Additionally, Zhang et al. [237] linked clothes worn by stars with online shops to provide clothing recommendations. To improve the matching results, Chen et al. [238] used image feature network (IFN) and video feature network (VFN) to generate deep visual features for shopping images and clothing trajectories in videos.

User behavior observation. Understanding user behavior is crucial in designing products and improving user experience. To explore user behaviors in VR spherical video streaming, Wu et al. [239] collected a head tracking dataset. Additionally, Babak et al. [240] evaluated product usability through video analysis. They performed temporal segmentation of video featuring human–product interaction, automatically identifying time segments where users encountered difficulties. They took water faucet design as an example and used optical flow for motion detection. Optical flow, temporal difference, and background subtraction are commonly used for motion detection in videos [241].

Product virtual display. Compared to images, video-based virtual displays demand higher spatiotemporal consistency, but they offer a more comprehensive display for customers. Various AI algorithms have been proposed to enhance the customer experience. For instance, Liu et al. [242] presented a parsing model to predict human poses in the video, while Dong et al. [243] proposed a flow-navigated warping GAN (FW-GAN) to generate try-on videos conditioned on person, clothing images, and a series of target poses. Interactive displays not only promote product understanding but also evoke enjoyment during the experience [244]. To this end, An et al. [245] designed a video composition system that displays products on mobile phones in an interactive and 3D-like manner. This system can automatically perform rotation direction estimation, video object segmentation, motion adjustment, and color adjustment.

Current intelligent video analysis technology has limitations in analyzing complex activities, as it can only identify simple ones. Structuralization is a vital obstacle to video analysis, which identifies features in the video through background modeling, video segmentation, and target tracking operation. In addition, complete video data acquisition requires the cooperation of multiple cameras, but achieving consistent installation conditions for each camera is difficult. It is also challenging to perform continuous and consistent visual analysis of moving targets in multiple videos. Moreover, the existing motion detection technology is not yet mature, as it can only detect target entry, departure, appearance, disappearance, wandering, trailing, etc. The product use process is complicated, and the information extracted at present is of little help to product design. Despite audio data holding great potential in product design, its development and application are restricted by processing technology and data quality.

4.6. Summary of Opportunities

Big data and AI algorithms hold enormous potential in modern product design. There are two types of data: structured and unstructured. Structured data offer strong pertinence and high-value density, but they are limited in openness and volume. Obtaining large-scale structured data is challenging for individuals, and even enterprises—it requires substantial workforce and resources for manual collection, sorting, summarizing, and storage in standard databases. On the other hand, unstructured data enjoy high openness and volume compared to structured data. It has great advantages in terms of veracity, velocity, value, and variety, all may strongly contribute to modern product design.

We have detailed the common data types in the product lifecycle, including text, image, audio, and video. Among these, text and image data are widely used in product design due to their ease of acquisition, mature processing algorithms, and high data quality. Textual data can overcome the limitations of traditional survey methods, such as small sample size, limited survey scope, and high labor-intensive, while also guaranteeing data authenticity, reliability, and timeliness. Image data, on the other hand, is useful for surveys and excels in visual inspiration and schemes display. The color, texture, sharp information in images can inspire designers to create preliminary schemes. Furthermore, generative models can generate new product designs from image data with varying colors, structures, textures, etc. AIGC is a revolutionary advancement in product design, which was previously unattainable through traditional methods.

Video data are also a visual type of information, which have high value for user behavior observation and product virtual display. Compared to images, video can provide customers with a comprehensive understanding and allowing them to interact with products virtually. However, both video and audio data are still in their early stages, with many technological and data quality challenges to overcome before they can be fully utilized in product design. Despite the development of technologies for audio and video processing, few researchers have employed them in product design. However, these limited studies warrant a discussion to motivate further research in the field.

Big data have brought about numerous benefits for product design; however, current research still has its limitations and requires further investigation. There are two research directions that are especially promising: (i) the fusion of different types of data. Existing research is confined to only one type of data (i.e., text, image, video, audio, and location), despite these data types co-existing at various stages of the product lifecycle. Furthermore, multi-modal data fusion could enhance the persuasiveness and efficacy of the extracted information; (ii) The synergistic exploitation of big data techniques and design domain knowledge. While previous efforts have focused on technological breakthroughs, especially for data lacking advanced processing technologies (e.g., audio and video), domain knowledge in the field of product design should be ignored. The knowledge accumulated in traditional product design is precious and helpful. Big data and traditional product design methods are not contradictory but complementary. Only by combining them can we better capture user requirements and improve the success rate of design. By addressing these research directions, we can further unlock the potential of big data and AI-driven product design, leading to more intelligent, personalized, and successful products across various industries.

5. Conclusions

In the era of the knowledge-driven economy, customer demands are more diversified and personalized, and the lifecycles of products are becoming shorter, especially in the usage phase. Therefore, successful product innovation now requires the assistance of interdisciplinary knowledge, the support of powerful techniques, and the guidance of innovation theories that break conventional wisdom.

Currently, big data are one of the resources with the most potential for promoting innovation throughout the whole product lifecycle. In this survey, we presented a comprehensive overview of existing studies on big data and AI-driven product design, aiming to help researchers and practitioners understand the latest developments and opportunities in this exciting field. Firstly, we visualized the product design process and introduced the key tasks. Secondly, we introduced several representative traditional product design methods, including their functionalities, applications, advantages, and disadvantages. Based on this, we summarized seven common shortcomings of traditional methods. Thirdly, we illustrated how big data and related AI algorithms can help solve challenges in modern product design, especially in user requirements acquisition, product evaluation, and visual display. We offered a detailed analysis of the current and potential application of AI techniques in product design, utilizing the textual, image, audio, and video data. For textual data, NLP techniques can be used to extract product attributes and sentiment features, aiding in understanding user requirements, evaluating products, and grasping product development trends. For images, neural style transfer, CNN, and GAN and its extended models can be used to spark creative inspiration and generate new product images. Audio can be used to capture user requirements, provide recommendations, evaluate products, and improve product design. Video can be used to observe user behavior and display product. Since audio data are still in an early stage, we focused on possible processing technologies and workflow that may be applied in the future. Finally, we summarized the deficiencies of existing data-driven product design studies and provided future research directions, especially for synergistic methods that combine big data-driven approaches and traditional methods for product design.

Product design based on big data and AI is typically performed with vast amounts of real-world data. This approach provides unprecedented opportunities to harness the collective intelligence of consumers, suppliers, designers, enterprises, etc. With the aid of advanced big data processing and AI technologies, it is possible to more accurately acquire user requirements, product evaluations, and virtual displays, leading to higher success rates in developing competitive products while saving time, effort, and development costs. We hope this survey will bring greater attention to the role of big data and cutting-edge AI technologies in the modern product design field. By exploiting the power of deep learning-based NLP, speech recognition, and generative AI techniques (such as GAN) in product design, it is believed that product innovation can reach an unprecedented level with high intelligence and automation.