1. Introduction
The field of engineering design, especially in product design, service system design, and complex system design, is being reshaped by two major trends: on the one hand, with the rise of large language models, generic-field generative intelligent design is gradually becoming mainstream [
1]. On the other hand, driven by the Internet, human designers have formed new design models, such as crowdsourcing design and group intelligence innovative design, through collaborative organization [
2,
3,
4]. The core goal of these development trends is to address the growing personalized requirements and achieve rapid iterative design of products and user-participation design.
In engineering design, the expression of personalized requirements has significant semi-professional characteristics, which will be elaborated in detail in
Section 3 “Nature of the Problem”. This semi-professional feature provides feasibility for directly extracting requirement elements from user descriptions, but also increases the importance and difficulty of implicit requirement mining. In the field of generative intelligent design, a major challenge is how to make machines empathetic [
5], so that after users input personalized requirements, they can empathize with the design and product usage context, automate the completion of users’ prompts, mine design problems that users have not anticipated or are difficult to express clearly, and improve the output quality of intelligent generative models. In terms of group intelligence innovation design, there are limitations in users’ understanding of products, which is not conducive to the emergence of innovation, but also becomes a fundamental problem that leads to the design process being repetitive, chaotic, and of low design quality [
2,
3,
4]. Therefore, accurately mining and supplementing implicit needs that have not been clearly expressed in personalized requirements is of great significance for improving the innovation of engineering design, the completeness of product functions, and the feasibility of solutions.
Many studies have analyzed the design requirements of [
6,
7,
8] through data-driven technologies such as machine learning. Although these studies extensively use data derived from user reviews on the Internet, user-generated content (UGC) has inherent limitations in complementing user needs [
9,
10]. In addition, it is difficult for the review data in the field of engineering design to meet the research needs in terms of quantity and quality, and the timeliness of the data is not in line with the development trend of engineering design. To overcome these limitations, this study introduced the engineering design knowledge contained in the patents.
This research presents a method to identify implicit personalized requirements using knowledge graphs. First, this study provides an in-depth analysis of semi-professional, task-oriented personalized requirements from the perspective of design science and engineering theory to reveal the characteristics of implicit requirements. Then, it builds a domain knowledge graph by gathering product design knowledge from patents. By optimizing the ontology layer and stacking the pre-training model, the comprehensiveness and practicability of the knowledge graph are enhanced while improving its construction efficiency. Finally, taking the user’s personalized description as input, the explicit requirement elements are extracted and the corresponding knowledge entities are matched, and the mining of multiple personalized implicit requirements is realized based on the methods of structural similarity and link prediction.
This study makes several key contributions:
- -
It performs a detailed analysis of the user requirements of the front-end design stage, clarifies the characteristics, forms, and types of the personalized implicit requirements of users that drive product innovation, and enriches the theoretical basis of the related research of the implicit requirement mining for innovative design.
- -
It enriches the domain patent knowledge graph ontology layer, improves the comprehensiveness and practicability of the graph, and improves the accuracy and efficiency of the knowledge graph construction by optimizing the pre-training model of the entity layer.
- -
Aiming at the different characteristics of personalized implicit requirements driving product innovation, this paper puts forward a series of targeted implicit requirement mining method systems based on the domain knowledge graph.
In the upcoming sections, we systematically explore and analyze personalized implicit requirement mining methods.
Section 2 reviews in detail the literature related to the topic of this study, providing readers with the research background and theoretical basis in this field.
Section 3 deeply analyzes the unique nature of the research object and problem, and clarifies this research’s boundaries and purpose. In
Section 4, we propose a technical solution for building a domain knowledge graph.
Section 5 elaborates on the personalized implicit requirement mining methodology.
Section 6 demonstrates the platform we developed, verifies the effectiveness and practicality of our method through case studies, and discusses the performance of the algorithmic model proposed during this study. Finally, in
Section 7, we comprehensively summarize the entire research and propose prospects for future research directions.
3. Nature of the Problem and Research Framework
In order to clarify the essence of personalized implicit requirement mining, according to the classical theory of engineering design [
41], the classification and representation of elements in personalized requirements are defined, and the differences between general user requirements and personalized requirements are compared, so as to further analyze the characteristics and categories of implicit personalized requirements.
3.1. Classification and Characterization of Elements of Personalized Requirements
In engineering design, the expression of personalized requirements often has a certain degree of professionalism, and can clearly describe the core function and structure of the product. This feature provides the feasibility of directly extracting requirement elements from user description. Before information extraction, we refine the requirement elements into two categories: functional elements and structural elements based on axiomatic design theory. Further, according to the behavior theory, we subdivide the functional elements into “function operation” and “function object” to facilitate the subsequent matching and supplement process.
In most descriptions of personalized requirements, users usually describe the specific functions and partial structures of the product. Using natural language processing (NLP) technology, we can identify, classify, and extract the functional and structural requirements from the original text. Additionally, the functional elements can be further divided into FA (verb) and FO (noun). For example, in a real user requirement case shown in
Figure 1, we successfully identified functional requirements such as “loading and unloading” (FA) and “paper box” (FO), as well as structural requirements such as “six-axis robotic arm”.
3.2. Analysis of the Characteristics of Personalized Implicit Requirements
Based on the large amount of user design requirement data accumulated during the previous research and design practice, this study summarizes, extracts, and compares the characteristics of general user requirements and personalized requirements from the aspects of sources, targeted products, user characteristics, requirement characteristics, and contents. The symbolic characteristics of personalized requirements are studied.
It can be seen from
Table 1 that the biggest characteristic difference between general user requirements and personalized requirements is that the former is fuzzy and emotional, while the latter is relatively more professional. Therefore, the general user requirements are more vulnerable to the user’s incomplete expression, so the implicit requirements are represented by the lack of standard function operation words, typical structure words, and standard engineering parameters. However, personalized requirements are subject to the subjective limitations of users. The expressed requirements are often limited to specific functions or structural words. Implicit requirements are mostly actions related to the time series of FAs, sub/parent, or very similar structures of FOs/structures. Based on classical theories in requirement engineering, FBS and other design science fields, and massive design requirement data, the characteristics, typical performance, and requirement examples of the above implicit requirements are presented in
Table 2.
Therefore, the goal of this study is to mine the personalized implicit requirements of sub-actions of FAs, pre-/post-operations of FAs, sub/parent structures of FOs/structures, pre-/post-actions of sub-actions of FAs, and very similar structures of FOs/structures. Through the mining of these requirements, a more accurate personalized requirement completion model was constructed to improve the innovation, feasibility, and interaction efficiency of generative design or swarm intelligence design.
3.3. Research Framework
For the nature of the problem in this study, combined with the patent knowledge in the field, the research framework of this paper is proposed in
Figure 2:
4. Patent-Based Domain Knowledge Graph Construction Method
The process of constructing a domain knowledge graph based on patent text is illustrated in
Figure 3. It consists of four main stages: ontology layer construction, data pre-processing, model pre-training, and data layer construction. In order to enhance the entity recognition and sentence sequence classification performance of the model in the field of patent text, this study conducts incremental pre-training on a large number of collected domain-related patent text data, and constructs the domain knowledge pre-training models BERT-base-patent-fusion and BERT-wwm-ext-patent-fusion based on two base models, BERT-base and BERT-wwm-ext [
42], respectively. The pre-trained models marked with * in the figure are the models proposed in this study. The numbers and letters in brackets in the data layer construction represent the data sets and training model numbers referenced in the construction process.
4.1. Ontology Layer Construction
This paper uses ontology to standardize entities, relationships, and the types and properties of entities. In the knowledge graph ontology constructed in this paper, the patent attribute part is consistent with most related studies, and the ontology is constructed based on the content contained in the patent text. The schema of the ontology is shown in
Figure 4. The ontology of the design knowledge element contained in the patent (
Class) is designed by the authors for the research object of this study, based on classic theories in the field of design science, including FBS, axiomatic design, and design standard word basis. The ontology layer schema of the domain knowledge graph can be expressed as
<Class, Property, Relation>, where:
Class = {classi}, which specifies the node classes in the network and the hierarchical relationships between classes. The domain knowledge graph defines three types of entities, namely, functional entities, structural entities, and patent entities; a total of 11 node classes are specified, among which function-action class and function-object class are subclasses of the function class, and power component, transport component, control component, etc. are subclasses of the structure class.
Property = {<class, property_type, value>} specifies the attributes that each entity in the knowledge graph should have and the range of its property value.
Relation = {classA, rel_type, classB} specifies the relationships between various entities. Relationships can be divided into three categories: patent has functions, patent has structures, and related relationships between patents due to similar functions or structures.
4.2. Data Pre-Processing
This paper focuses on Chinese patents as the research subject, and first of all, it needs to complete the structured parsing and storage of patent data. In addition to the patent data set, for the construction of the pre-training model, three data sets are extracted and labeled, including the patent data set with domain-related labels, the data set with function and structural entity labels, and the data set with structural entity category labels.
The patent data set with domain-related labels is an annotation data set used to determine whether a patent is domain-related. The domain-related classification of patents is a sentence classification task. The domain features of the title and rights statement are significant. As the input data of the model, the patent data with domain-related labels is annotated to form the training and verification sets of the model.
The data set with function and structural entity labels is constructed to fine-tune the named entity recognition model of patent text. The BIO labeling method is used to label the functional and structural entities in the patent title and claim text for the model pre-training of extracting functional and structural entities.
The data set with structural entity category labels is used to build a classification model for structural entities, subdividing structural entities into subcategories such as power and transportation components.
4.3. Model Pre-Training
Patent text classification, information extraction, and entity semantic similarity calculation all need NLP support. At present, the best SOTA (state-of-the-art) model adopts the “pre train + fine tune” framework. Based on this framework, this paper superimposes patent text data for the pre-training process to enhance the model’s ability to recognize and understand patent text, so as to obtain better results in patent text-oriented NLP tasks. When Google released the BERT model in 2018, it included a Chinese model trained on the Chinese Wikipedia corpus. In 2019, the BERT-wwm-ext model was proposed by Harbin CuiY et al. [
43]. On the basis of BERT, the masked language model at the character level was improved to encompass the whole word mask, and the corpus used for pre-training was expanded, which achieved significant optimization results in multiple tasks.
To improve the model’s ability to recognize entities and classify sentence sequences in domain patent texts, this study conducted incremental pre-training using a large number of domain-related patent text data with two base models: BERT-base and BERT-wwm-ext. This led to the creation of the pre-training models BERT-base-patent-fusion and BERT-wwm-ext-patent-fusion, which superimposed domain knowledge. For fine-tuning in subsequent tasks, it will be trained based on four models: BERT-base, BERT-base-knowl-fusion, BERT-wwm-ext, and BERT-wwm-ext-knowl-fusion.
4.4. Data Layer Construction
The data layer construction of the domain knowledge graph can be divided into five stages: domain patent screening and information extraction, functional and structural entity extraction, classification of structural entities, structural entity resolution and fusion, and relationship generation in the graph.
4.4.1. Domain Patent Screening and Information Extraction
In order to improve the information purity of domain knowledge graph, it is necessary to screen all patents related to the domain in all patent data. Since there are patents related to the domain under all IPC primary partitions, it is not possible to directly screen through the IPC partition information of patents. Therefore, through the filter based on the NLP algorithm, this paper selects the domain patents for graph construction.
Based on the NLP algorithm and data set with domain-related labels, this paper fine-tunes four pre-training models to construct the domain-related patent filtering model.
As shown in
Figure 5, in the patent title and claim text, the last hidden layer vector
C∈RH output after the information extraction of the Transformers module in BERT [
44] is used as the representation vector of the sentence, where H is the number of neurons in the last hidden layer. Finally, the classification layer weight
W ∈ RK×H is adjusted, where
K = 2 is the number of labels, and the model training can be completed by calculating the classification error based on the SoftMax result output by the following formula:
Domain-related patents and their attribute information, such as patent numbers, titles, and IPC, are imported into the knowledge graph to complete the construction of patent entities and extract their attribute information. An example is shown in
Figure 6.
4.4.2. Functional and Structural Entity Extraction
Extracting functional and structural entities from unstructured patent text is a text sequence annotation problem [
45]. Specifically, given a text sequence of length N,
where
wi represents a single word or token. The named entity recognition model returns a series of tuples
<Is, Ie,
t> (
Is ∈ [1,
N],
Ie ∈ [1,
N]), where
Is and
Ie represent the start and end positions of a named entity, and
t represents the entity category, which should be one of the entity categories predefined according to the actual problem model. The function and structure entity extraction model has three types of named entities, namely FA (function-action), FO (function-object) and S (structure), so
t ∈ {
FA,
FO,
S}.
The patent title mainly includes the function of the patent and its overall structure. For example, “A kind of stone grinding machine” clearly states that its function is “grinding stone”, where the function-action entity is “grinding”, the function-object entity is “stone”, and its overall structure is “stone grinding machine”, which is a structural entity. The claim of the rights mainly contains the technical characteristics of the patent and expresses the scope of protection requested. The structural entity can be obtained using named entity recognition technology from the claim of rights.
The functional and structural entity extraction model based on the patent title, and the structural entity extraction model based on the patent rights claim, are constructed. Based on the data set with patent functional and structural entity labels, we fine-tuned four pre-trained models, selected the best iteration rounds and models, and constructed three entity extraction models.
4.4.3. Classification of Structural Entities
In order to provide more detailed entity information, the structural entities are divided into eight categories: power component, transport component, measuring component, adjusting component, raw materials, mechanical basic part or other component, electronic components, and functional modules, as shown in
Table 3. In the process of named entity recognition, it is inevitable that some wrong recognition entities such as non-related fields are introduced, so invalid entity is also regarded as a category. Similarly, based on the data set with a structural entity category label, the structural entity classification model is constructed by fine-tuning and testing the four pre-training models.
4.4.4. Structural Entity Resolution and Fusion
Entity resolution is mainly used to solve the problem of multiple references actually pointing to the same entity. In the domain knowledge graph, since the product described in the patent may have multiple identical structures, they will be distinguished by “Xth structure”, “Structure X”, etc. Therefore, it is necessary to perform co-referential resolution analysis on the structural entities and fuse the entities that point to the same entity.
A rule-based method is used for the structural entity resolution: first, regular expressions are used to determine whether the structural entity is expressed in the form of “Xth structure” or “Structure X”. If so, the “Xth” prefix or “X” suffix is deleted to determine whether the new entity formed exists. If it exists, the entity is merged with the existing entity; otherwise, a new entity is created.
4.4.5. Relationship Generation in the Domain Knowledge Graph
There are three types of relationships in the domain knowledge graph: “Has_Function”, “Has_Structure”, and “Is_Related”. Among them, the “Has” type relationship is between the parent entity and the function and structure entities. In extracting the function and structure entities, this type of relationship has been obtained and can be directly constructed in the graph. The “Is_Related” is constructed based on the network structure similarity between entities. Based on the highly interpretable Jaccard similarity measure, the structural similarity between patent entities with more identical functional and structural entities is higher, and the “Is_Related” is constructed between patent entities above the similarity threshold.
5. Multi-Category Personalized Implicit Requirement Mining Method Based on the Knowledge Graph
In view of the different categories of personalized requirements mentioned in
Table 2, this paper implements entity-layer co-occurrence implicit requirement mining based on structural similarity, and constructs a co-reference embedding layer to realize non-co-occurrence implicit requirement mining based on link prediction, as summarized in
Table 4:
Therefore, the process of mining personalized implicit requirements based on a domain knowledge graph mainly includes three stages: explicit requirement element identification and entity matching, co-occurrence implicit requirement mining of the entity layer, and non-co-occurrence implicit requirement mining of the co-reference embedding layer. As shown in the following formula, for the expression of explicit requirement elements in the domain knowledge graph, that is, each element in the knowledge entity set
: If there is a knowledge entity of co-occurrence relationship, the structural similarity algorithm is used to measure its relativity; otherwise, its relativity with will be measured based on the entity embedding and link prediction algorithm. Based on the correlation
, the two types of entities are sorted to realize the personalized implicit requirement mining.
5.1. Explicit Requirement Element Identification and Entity Matching
The model fine-tuned based on the four pre-training models is used as the identification model for personalized explicit requirement elements, completing the mapping of unstructured RD (requirement description) to ERE (explicit requirement entity), as shown in the following equation, where
represent the explicit FA, FO, and structural elements expressed by the user:
The matching process between explicit requirement elements and entities in the knowledge graph is completed based on the cosine similarity calculation of word embedding vectors. Each explicit requirement entity is input into the BERT model to obtain its semantic vector representation
. Similarly, the semantic vector representation of functional and structural entities in the domain knowledge graph is obtained, that is, the matrix
, where
N is the number of functional and structural entities in the network. Then, based on the cosine similarity, the cosine similarity measures of
M are obtained to complete the matching of demand elements and knowledge entities, as follows:
Based on the matching process between requirement elements and knowledge entities, the mapping process from the explicit requirement element set ERE to the entity KE (knowledge entities) set in the knowledge graph is completed, as shown in the following formula:
5.2. Co-Occurrence Implicit Requirement Mining of the Entity Layer
For implicit requirement elements that co-occur with explicit requirements, the structural similarity algorithm can be used to mine them. With explicit requirement entities as input, possible implicit requirement entities and their ranking are output, as shown in
Figure 6:
The mining of co-occurrence-related entities with the same neighbors can be completed using the node similarity calculation method based on network theory. The node structure similarity calculation algorithms based on network theory mainly include Jaccard similarity, Sorensen–Dice similarity, Hub Promoted similarity, and Hub Depressed similarity, which are all based on the neighbor node set calculation. This paper uses Jaccard similarity to calculate node similarity.
In the network
G = (
V,
E),
V is the vertices set and
E is the edges set. For a vertex
x ∈ V,
N(x) represents the set of its neighbors. Then, for
x,
y ∈ V, the Jaccard similarity is:
The output of the above formula is sorted, and the co-occurrence layer implicit requirement entities are mined and recommended.
5.3. Non-Co-Occurrence Implicit Requirement Mining of the Co-Reference Embedding Layer
Some personalized implicit requirements do not have a co-occurrence relationship with the explicit requirement elements in the graph. This study uses network node embedding and link prediction algorithms to mine these implicit requirements. Additionally, the explicit requirement entities are taken as the input, and the possible non-co-occurrence implicit requirement entities and their ranking are output. Taking the requirement “transportation” as an example, examples of non-co-occurrence implicit requirements, such as “Preparation according to order” and “delivery”, are shown in
Figure 7 (in addition to the “secondary co-occurrence” implicit requirements in the figure, our method can also mine the entities with other types of network structural similarity):
Link prediction is an algorithm that predicts the possibility of a new connection between two nodes in a given network. With the continuous advancement of complex network modeling technology, researchers have proposed many link prediction algorithms for different types of networks, including simple similarity-based, probability theory and maximum likelihood-based, and dimension reduction-based. According to the research of Kumar A et al. [
46], based on the AUPR (area under the precision–recall curve), it was found that the network embedding algorithm Node2Vec based on the dimension reduction had the best performance in the test data set. Therefore, this paper uses the Node2Vec algorithm to complete the link prediction task of the knowledge graph to assist in the discovery of non-co-occurring-related entities and support the mining of implicit requirements. The process of implicit requirement mining based on the Node2Vec algorithm includes the following three steps:
5.3.1. Build Co-Occurrence Networks of Functional and Structural Entities
If the sequence is collected directly based on the knowledge graph, it can only start from the patent entities, and the sequence length just can be 2, such as
. Therefore, we construct an entity co-occurrence network that can generate a more extended node sequence, such as
, which is conducive to capturing the relationship between entities. Based on the co-occurrence of functional and structural entities, the two co-occurrence networks are constructed respectively. The network nodes are still functional and structural entities in the knowledge graph, and the edge weight is the number of co-occurrences. This process is shown in
Figure 8:
5.3.2. Construct Node Sequences by Biased Random Walk
As shown in
Figure 9, the Node2Vec algorithm draws on the method of generating word vectors. It first starts from any point in the network based on a biased random walk and transfers between connected nodes. In this way, a limited nodes sequence can be collected, and the SkipGram model [
47] is used to train the embedding vector E based on these sequences.
The “Biased Random Walk” method proposed by Node2Vec uses two parameters to adjust the random walk process. As shown in
Figure 10, nodes V
5 and V
1 belong to the same node community, showing the homogeneity of nodes; while V
5 and V
8 belong to two different communities, but have the same structure (or role) in these two communities, showing the structural equivalence of nodes. In order to adjust the weights of these two structural characteristics in the algorithm, Node2Vec generalizes DeepWalk [
48]. By setting parameters, the bias of the breadth-first sampling strategy and the depth-first sampling strategy in the node sequence generation process can be selected to support the adjustment of the homogeneity and structural equivalence information reflected in the node embedding vector.
Specifically, Node2Vec defines a random walk with parameters. Suppose a random walk has just walked through the edge (step ① of
Figure 10), and the current position is at node
. The number on the arrow represents the step number during the search. Next, the random walk needs to make the next edge by evaluating the transition probability
. Let the transition probability be
where
is the edge weight, and
represents the shortest distance between
and
. The parameters
p and
q can control the strategy of random walk exploration, where
p controls the possibility of a walk returning to the previously reached node. Setting it to a large value ensures a more breadth-first walk, and a smaller
q value encourages the depth-first strategy.
5.3.3. Train the SkipGram Model to Obtain the Entity Embedding Vector, and Sort the Entities by Relevance
Based on the collected node sequence set, a node is regarded as a word, and a node sequence is regarded as a sentence. Based on the SkipGram method of the Word2Vec model, each node is represented as a low-dimensional dense vector
.
When mining non-co-occurring implicit requirements, the formula can be used to calculate and sort the relevance of all entities, giving possible implicit requirement entities.
7. Conclusions
The Internet’s rapid development has led to a surge in the requirement for personalized product design and development. However, meeting implicit requirements that drive product innovation remains a challenge. To address this, this study proposed a method for mining personalized implicit requirements based on a domain knowledge graph. This method involves analyzing personalized requirements and knowledge from domain patents. A patent knowledge ontology layer representation method was proposed, leading to the construction of a domain knowledge graph. This expands the information contained in the knowledge graph. The two NLP models proposed during the study successfully improve the efficiency of graph construction. Two implicit requirement mining methods are proposed: one based on structural similarity and the other based on link prediction. These methods aim to overcome subjective professional limitations and broaden the requirement space for product innovation design.
The method we propose and the platform we develop have good application prospects in the following contexts: First, in the context of generative design driven by large language models, it can be applied to prompt engineering, mining real and deep requirements that users find difficult to express, improving the model’s understanding and empathy abilities, and further enhancing the feasibility and innovation of product development. Second, in the context of crowd innovation, it is possible to break through the subjective limitations of user requirements, improve communication efficiency between users and designers, reduce design iterations, and accelerate innovation convergence.
However, this study has identified some limitations that require further exploration. On the one hand, the requirement expression paradigm represents the functions a product or component should achieve under what context. The requirement analysis method described in this paper realizes the extraction and mining of product functional requirements and structural requirements, but the requirement context is not considered enough. On the other hand, the study selects the patent as a field of knowledge network data and the knowledge source; in actual engineering applications, different enterprises have different types of databases and knowledge bases, and have less storage and a limited technical field. The research method proposed in the application of this kind of knowledge base effect may thus have certain limitations.
Future research could focus on combining the usage context during personalized requirement analysis. Secondly, a general requirements graph can be constructed based on the method proposed in this study with comments data, and the implicit requirement mining methods can also act on general requirements, so as to realize the mining methodology that drives design innovation and is coupled with personalized requirements. Additionally, in practical engineering applications, determining how to further expand the knowledge source and build a knowledge graph with both breadth and depth under knowledge integration remains to be further studied.