A Knowledge Graph-Based Approach for Assembly Sequence Recommendations for Wind Turbines

Abstract

There are various forms of assembly data sources for wind turbines, which contributes to the lack of a unified and standardized expression. Moreover, the reusability of historical assembly data is low, which leads to the poor reasoning ability of a new product assembly sequence. In this paper, we propose a knowledge graph-based approach for assembly sequence recommendations for wind turbines. First, for the multimodal data (text in process manual, image of tooling, and three-dimensional (3D) model) of assembly, a multi-process assembly information representation model is established to express assembly elements in a unified way. In addition, knowledge extraction methods for different modal data are designed to construct a multimodal knowledge graph for wind turbine assembly. Further, the retrieval of similar assembly process items based on the bidirectional encoder representation from transformers joint graph-matching network (BERT-GMN) is proposed to predict the assembly sequence subgraphs. Also, a Semantic Web Rule Language (SWRL)-based assembly process items inference method is proposed to automatically generate subassembly sequences by combining component assembly relationships. Then, a multi-objective sequence optimization algorithm for the final assembly is designed to output the optimal assembly sequences. Finally, taking the VEU-15 wind turbine as the object, the effectiveness of the assembly process information modeling and part multi-source information representation is verified. Sequence recommendation results are better quality compared to traditional assembly sequence planning algorithms. It provides a feasible solution for wind turbine assembly to be optimized from multiple objectives simultaneously.

1. Introduction

With the rapid development of intelligent advanced technologies such as big data and artificial intelligence, traditional manufacturing is gradually transforming into intelligent manufacturing [1,2]. The concept of Industry 4.0 provides an opportunity for manufacturing enterprises to integrate data at all stages of the product life cycle, so as to better meet the needs of users’ personalized and customized products [3]. As a customized electromechanical product, the manufacturing process of large wind turbines can be divided into three major parts, including manufacturing process, logistics process, and on-site process. Among them, assembly as the core link of the on-site process is the last step of wind turbine installation. The assembly quality directly affects the reliability and working performance of the wind turbine. In addition, the assembly workload of each component has always occupied a high ratio of the development workload of the whole product. The assembly time accounts for more than 40% of the manufacturing time of the whole wind turbine [4]. The development of artificial intelligence provides opportunities for assembly semantic processing and assembly sequence recommendations, which focus on analyzing and planning assembly sequences more effectively [5,6].

Assembly knowledge is constantly evolving in practice, providing a key reference for the design and manufacturing of complex products [7,8,9]. Manufacturing products are constantly iterated and upgraded. Thus, a large amount of historical data is generated. However, the lack of effective organization of these heterogeneous data makes it difficult to provide technicians with convenient knowledge services. Therefore, the search for efficient data organization and convenient knowledge acquisition methods has become an urgent problem for a long time. In addition, in traditional assembly sequence planning, the selection of assembly solutions based on different assembly requirements is often realized semi-automatically by assembly personnel based on manual experience or computer-aided tools. How to obtain the assembly object, assembly process, and other related information based on the existing assembly knowledge with a sequence planning solution is also a major difficulty in current intelligent planning.

Researchers have applied knowledge graphs to build structured semantic networks in manufacturing, especially in assembly, and have demonstrated its application value in guiding assembly work in real industrial scenarios [10,11,12]. This is because knowledge graphs can describe assembly objective concepts and their interrelationships in a structured form [13], where the subordination between assembly entities and assembly attributes can be visually expressed. It provides powerful support for the sharing, querying, and derivation of assembly knowledge.

However, since the planning of assembly sequences for complex wind turbine products is a combinatorial optimization problem, the results generated by the sequence may be unreasonable. In addition, the actual assembly process is also limited by factors such as assembly stability and assembly reorientation. Therefore, the sequences obtained by traditional manual experience and similarity matching methods are usually not the optimal solutions. It is necessary to combine the wind turbine-specific assembly requirements and the optimization degree of several assembly evaluation indexes to obtain the optimal or near-optimal feasible assembly sequences.

Based on the problems emphasized above, in this paper, a knowledge graph-based approach for assembly sequence recommendations for wind turbines is proposed. Firstly, the assembly information representation model is developed to link knowledge from the assembly process text, on-site installation images, and 3D part models. Also, knowledge extraction methods for different modal data are designed to construct a knowledge graph for wind turbine assembly. Then, a graph-matching network (GMN) is designed for the retrieval of similar assembly process items. Further, the Semantic Web Rule Language (SWRL) is used for assembly semantic rule reasoning to mine the implicit relationships between total assemblies and parts. Then, a multi-objective sequence optimization algorithm for the final assembly is designed to output the optimal assembly sequences. Finally, the VEU-15 wind turbine generator is used as a case object to demonstrate the effectiveness of the proposed method for the recommended design of assembly sequences with multi-configuration requirements.

The innovations of this paper are as follows:

(1)

The assembly information representation model is designed to describe the assembly information data from multiple sources in a unified way. Then, multiple modal assembly entity joint extraction methods are proposed to construct a multimodal knowledge graph for wind turbine assembly.

(2)

A knowledge graph-based optimal assembly sequence recommendation is proposed. The retrieval of similar assembly process items based on BERT-GMN is proposed to predict the assembly sequence subgraphs. Also, a SWRL-based assembly process item inference method is proposed to automatically generate subassembly sequences by combining component assembly relationships. Then, a multi-objective sequence optimization algorithm for the final assembly is designed to output the optimal assembly sequences.

The rest of this paper is organized as follows: Section 2 introduces the literature review of assembly domain knowledge graph construction and assembly sequence planning. Section 3 proposes the knowledge graph-based assembly sequence recommendations for wind turbines in detail. In Section 4, the VEU-15 wind turbine generator is taken as an example to verify the effectiveness of the proposed method in this paper. Finally, Section 5 concludes and outlines future work.

2. Related Work

2.1. Knowledge Graph Construction in Assembly

Most of the knowledge graphs in the assembly use a top-down approach to build ontologies. Chen et al. [10] proposed a semantic recognition method for the assembly process based on a Long Short-Term Memory (LSTM) network to automatically recognize the assembly semantics in the assembly process document to construct an assembly process knowledge graph. Shi et al. [11] proposed knowledge reuse of assembly resources in a complex product assembly process based on knowledge graphs. In multi-source heterogeneous data, it is becoming more common to use text as a data source to construct knowledge graphs [14].

However, multimodal knowledge graphs present a big step forward in graph construction [15]. The multimodal data are not only limited to text and images, but 3D part models [13] are also an important source of information in the assembly. More types of data have been considered in manufacturing to build multimodal knowledge graphs. Yang et al. [16] proposed a knowledge-based multimodal knowledge extraction method for visual question and answer, which associates visual objects and factual answers with implicit relations. Zhou et al. [17] proposed an end-to-end tabular and textual information extraction model. The causality event evolutionary knowledge graph of tabular and text is realized. Liu et al. [18] designed a corresponding ontology design and joint knowledge extraction model to realize the end-to-end automatic construction of knowledge graphs. The experiment confirmed the advantages of this model in the field of aerospace assembly. Wen et al. [19] established an interpretable multimodal knowledge graph answer prediction model and effectively extracted text information from images through a multimodal fine-grained entity extraction method, which achieved a better performance in multimodal link prediction.

Inspired by the previous studies related to knowledge graph construction in assembly, it provides a viable solution that multimodal knowledge graphs can integrate and utilize the multimodal data (text in process manual, image of tooling, and 3D model) of assembly for wind turbines. In this way, the effective organization of these heterogeneous data makes it possible to provide technicians with convenient knowledge services. Thus, it is considered that link knowledge from the assembly process text, on-site installation images, and 3D models can be used in a unified way to construct a knowledge graph for wind turbine assembly.

2.2. Assembly Sequence Planning

To shorten the assembly sequence planning time and reduce the assembly difficulty, machine learning and deep learning are used to predict the optimal assembly sequence [20]. Therefore, this section discusses the assembly sequence information model and optimization algorithm.

(1)

Assembly sequence information model

Currently, scholars propose a variety of assembly information modeling approaches. In terms of product functional assembly information modeling [21], Anthony [22] and Kopena et al. [23] adopted the SBF (structure-behavior-function) model to develop a conceptual understanding and prototyping environment to capture the assembly functional information of CAD (computer aided design) artifacts. However, the SBF model is relatively simple in describing the relationships between upper-level products and lower-level components. In the research based on product structural assembly information [24], Chen et al. [25] proposed a structural assembly design model to support a top-down product design. It utilizes a multilevel assembly model to capture the product information, which fills the gap of a hierarchical feature information model [26]. In terms of product process assembly information [27], Duan et al. [28] presented a pooling framework for RPA (relative position accuracy) measurements based on MBD (model based definition) datasets, which meets the needs of integration and efficiency in large component assembly.

In addition, some researchers have combined the new generation of information technology to develop assembly sequence information descriptions. Zhou et al. [12] utilized a graph neural network to propose knowledge graph-driven assembly process generation and evaluation for complex components. Xu et al. [29] proposed a top-N recommendation method named the collaborative knowledge-aware graph attention network (CKGAT) to accurately capture users’ potential interests.

(2)

Assembly optimization algorithm

In recent years, there have been various assembly sequence optimization algorithms proposed. To improve assembly accuracy and service safety, Champatiray et al. [30] proposed modified cat swarm optimization for optimal assembly sequence planning problems. Shen et al. [31] proposed intelligent material distribution and optimization in the assembly process of large offshore crane lifting equipment. On the other hand, Han et al. [32] presented a new multidimensional-based clustering and retrieval method for CAD assembly models, which comprehensively evaluates the similarity between assembly models by considering both part information and assembly relationship information. In addition, Cong et al. [33] introduced a simulated annealing program module into the existing genetic algorithm to form a new simulated annealing genetic algorithm, which searches for the globally optimal assembly sequence scheme more accurately and efficiently. Also, Xie et al. [34] designed an improved multi-pheromone ant colony optimization algorithm, which provides a new strategy for improving operator matching. Li et al. [35] developed an assembly sequence planning method based on the ACO (ant colony optimization) algorithm, which obtains the optimal assembly sequence by assisting the search for the optimal solution of the ACO path. In addition, some scholars have presented new insights in optimization algorithms. Chaudhari et al. [36] compared the NSGA-III (non-dominated sorting genetic algorithm III) with NSGA-II (non-dominated sorting genetic algorithm II) for the multi-objective optimization of an adiabatic styrene reactor. Ehsaeyan et al. [37] introduced a novel meta-heuristic algorithm called the fireworks optimization algorithm (FOA) with few control parameters for discrete and continuous optimization problems. Fountas et al. [38] proposed a virus-evolutionary genetic algorithm for the optimizing of selective laser sintering/melting operations.

Based on the above study, it is clear that if we obtain a better assembly sequence, it is not only necessary to model and associate the knowledge of multiple sources for wind turbine assembly, but also to design an optimization algorithm. Therefore, it is promising and valuable to explore a knowledge graph-driven approach for assembly sequence recommendations and reasoning for wind turbine assembly.

3. Knowledge Graph-Based Assembly Sequence Recommendations for Wind Turbines

This section is divided into multimodal knowledge graph construction and knowledge graph-based assembly sequence recommendations for wind turbine assembly.

3.1. Multimodal Knowledge Graph Construction for Wind Turbine Assembly

3.1.1. Multi-Process Assembly Information Modeling

Based on the assembly process information in the design stage, the text in the process manual, image of tooling, and 3D model files are used as the information source of the actual assembly process. Thus, the assembly information representation model based on knowledge graphs is constructed to provide a standardized expression for the assembly process design. The specific structure is shown in Figure 1. It consists of the final assembly element layer, the subassembly relationship layer, and the part property data layer, respectively. The information includes part feature classification information, assembly relationship information, and part multi-source data information, which is the meta-structure for constructing the domain knowledge graph schema layer.

Constructing the assembly information representation model starts with the abstraction of the assembly concept class. The process information extracted from the assembly process text and the model assembly tree can be categorized into three main classes, including assembly feature (AF) information, assembly semantic (AS) information, and assembly property (AP) information. The definition of the ontology schema layer is shown in

Table 1. Assembly element process information.

Assembly Category	Scope	Range	Interpretation of Nouns	Functional Description
AF	topology	$\{F_A,S_A,P\}$	{Total assembly, sub-assembly, parts}	Define component structure relations
	Geometric solid	{FaceTorch, EdgeCurve, VertexPoint}	{face, edge, point}	Describe the form of contact of parts
	Assembly operations	{ConnectionAssembly, AdhesiveAssembly, WeldAssembly}	{coupling, bonding, welding}	Determine the type of component connection
	Assembly Tools	$\left\{T_i\right\}$	{Assembly Tools}	Auxiliary parts assembly process
AS	Localization semantic relation	$\{G_{FD},G_{MO}\}$	{Degrees of Freedom, Mobility}	Parts Positioning
	Connection semantic information	{HasSub, Subof, HasPart, Partof, HasGFD, HasBegin, HasEnd, HasGMO}	{Collection of connection relations}	Parts Connection Relation
AP	Data property	{Name, RelativeMatrix, Direction, Weight, Num}	{Collection of Part Properties}	Retention of parts’ own properties
	Constraint property	{hasGF, hasGMO}	{Degree-of-freedom relation, Constraint relation}	Constraint information for parts
	$B_{SP}$	{TRUE, FALSE}		Determine whether to start parts
	$S_{AP}$	$\{S_{AM},S_{PN},K\}$	{Auxiliary parts collection, Collection of Connectors}	Auxiliary and connecting parts required before installation
	$S_{CR}$	$\left\{S_{PL}\right\}$	{Parts collection}	Collection of parts and components that are coupled

.

(1)

Assembly feature information

It is refined into assembly topology information (AT), assembly geometric information (AG), assembly operation information (AO), and assembly requirement information (AR).

$$AF=\{AT,AG,AO,AR\}$$

(1)

AT is the specific expression of the product process structure. It is the structural relationship between the final assembly F_A and the subassembly S_A, as well as between the subassembly and the parts S_P. Of these, $FA=\{SAi,SPi\},SA=\{SAi,SPi\}(i=\mathrm{1,2},3,\dots ,n;j=\mathrm{1,2},3,\dots ,m)$.

(2)

Assembly semantic information

It is the data and information used to describe the relationships between parts, components, and subsystems during the manufacturing and assembly process. G_FD = {FD1, FD2, FD3, FD4, FD5, FD6}, FD1, FD2, FD3 denote translation along the X-axis, Y-axis, and Z-axis, respectively. FD4, FD5, FD6 denote rotation around the X-axis, Y-axis, and Z-axis, respectively. G_MO = {MO1, MO2, MO3, MO4, MO5, MO6}. MO1, MO2, MO3 represent the matching constraint of Z-plane, Y-plane, and X-plane, respectively. MO4, MO5, MO6 represent the matching constraint of the X-axis, Y-axis, and Z-axis, respectively.

(3)

Assembly property information

It refers to a set of data and information used to describe the relationship between assembly parts in the process of product manufacturing and assembly. Assembly attribute information mainly contains data attributes and constraint attributes, of which the data properties are retained by the parts of their properties, including the weight and size of the parts, positioning direction, material number, as shown in Table 1.

3.1.2. Knowledge Extraction for Multimodal Data

In wind turbine assembly, an overall framework for extracting knowledge from multimodal data is shown in Figure 2. Multimodal data includes text in the process manual, the image of tooling, and the 3D model. Also, information extraction methods are divided into three types, including textual data knowledge extraction, knowledge extraction for image data, and information extraction for the 3D model, respectively. The implementation details of the framework are discussed below in three parts.

• Textual Data Knowledge Extraction

The text in the process manual is divided into general assembly text data and BOM (bill of materials) for assembly. On the one hand, the general assembly text data are extracted by designing the BERT-BiLSTM-CRF model, as shown in Figure 3. First, the BERT model is used to obtain word vectors for each input character from the annotated corpus. Next, a sequence of these output word vectors is passed into the BiLSTM layer for semantic encoding to obtain global sequence features. Based on this, the CRF (conditional random fields) layer decodes the output of the BiLSTM layer to produce the predicted label sequences. Then, entities are extracted and classified from the label sequences to obtain the final entity recognition results. Taking the text “nacelle gearbox…” in the assembly manual as an example, after labeling the text by the BIESO mode, the entity is labeled as {B-UNIT, I-UNIT, I-UNIT, I-UNIT, E-UNIT}, where a unit represents the assembly unit.

On the other hand, BOM for assembly shows the relationships of dependencies between wind turbine components. The BOM data in the ERP system can be accessed by calling the API (application programming interface) via python. Furthermore, the knowledge graph can be directly generated by incorporating the inherent dependencies from the BOM.

• Knowledge Extraction for Image Data

The semantic labels of the assembly image are extracted using an encoder-decoder architecture as shown in Figure 4.

Specifically, the image is firstly fed into the encoder part. Then, the feature map is generated by the convolutional neural network (CNN) layer, which can extract the features at m position of the image. The height and width of the feature map are set to 14, the number of channels is 256, the feature vector dimension D = 256, and m = 14 × 14 = 196. The ATT mechanism is added to decode the feature results.

$$Zt={\displaystyle {\sum }_{m=1}^n{\alpha }_{t,m}}{x}_m$$

(2)

α_t,m denotes the weight share of the image features at the mth position at stage t of the encoding. The feature result is obtained as Z_t. It is used as an input to the LSTM to generate hidden variables. In turn, the output of the model is y_t. Considering the issue of semantic labeling output integrity, a manual calibration step needs to be added, including expert experience combined with the model generation. Finally, the output result is p.

• Information extraction for 3D models

The structure information of the 3D model of the wind turbine consists of <ProductStructure> as the root node, including the node’s ID, pointer to Reference3D, and relative matrix position. Its leaf nodes are divided into four categories, including <Reference3D>, <Instance3D>, <ReferenceRep>, and <InstanceRep>. Reference3D nodes and Instance3D nodes have no obvious affiliation. The total assembly, as an assembly structure of multiple subassemblies, is identified as Reference3D, but a subassembly can also be an assembly of multiple parts, so its identity can be either Instance3D or Reference3D. In addition, information related to a part or assembly is stored in the ReferenceRep to indicate a reference to the part or assembly. It usually contains a unique ID value for identification and referencing in the file. Its data structure is shown in Figure 5.

3.2. Knowledge Graph-Based Assembly Sequence Retrieval and Reasoning for Wind Turbine

In this section, an assembly sequence retrieval and reasoning method for wind turbines based on an assembly process knowledge graph is proposed. Its design steps for the assembly sequence generation are accomplished, including assembly process items based on BERT-GMN, graph similarity calculation based on SWRL, assembly semantic rule generation, and automatic generation of subassembly sequences. It is discussed in detail below.

3.2.1. Retrieval of Similar Assembly Process Items Based on BERT-GMN

In particular, for the retrieval of similar assembly process items, a graph neural network is utilized to compute the similarity between assembly nodes and relationships to determine whether two graphs or local subgraphs are similar. Combined with the BERT in Section 3.1.2, BERT-GMN is proposed to retrieve similar assembly process items, as shown in Figure 6. The modules included from bottom to top in the figure are context node embedding, graph-matching neural network, and relationship classifier, respectively, where the BERT encodes the assembly text.

• Context Node Embedding Module

For any node v_i in the graph, there is a difference between the embedding of the initial nodes and the embedding of the intermediate context nodes. Specifically, the initial nodes need to first concatenate the assembly statements to form a new sequence $S=\left\{\right[CLS],{ca}_1,\dots ,c_{ta}^a,[SEP],c_1^b,\dots ,c_{tb}^b,[SEP\left]\right\}$. Its output will be used as input to the BERT model, resulting in a context-embedded representation of each character as $c^{CLS},c_1^a,\dots ,c_{ta}^a,c^{SEP},c_1^b,\dots ,c_{tb}^b,,c^{SEP}$. Assuming that node v_i consists of n_i consecutive character symbols $\{c_{si},c_{si+1},\dots ,c_{si+ni+1}{\}}^1$, for each character $c_{si+k}$$c_{si+k}$, a feature score vector ${\widehat{u}}_{si+k}$ is computed using a feed-forward network. Then, softmax is used for normalization. The corresponding character embedding $c_{si+k}$ is weighted to obtain the initial node embedding $Vi={\sum }_{k=0}^{n-1}{u}_{si+k}\odot c_{si+k}$ where $\odot$ denotes the product of the elements of two vectors.

• Graph Matching Neural Network Module

The context node is embedded in $v_i$ as the initial representation of the node $h_i^0$. Then, its representation from one layer to the next is updated and organized into two steps.

Message passing: each node $v_i$ in $G^a$ aggregates messages $m_i^{fw}$ and $m_i^{bw}$ from neighboring nodes.

$$\begin{array}{l}{m}_i^{fw}={\displaystyle \sum _{v_j\in N_{fw}(v_i)}{a}_{ij}(W^{fw}{h}_j^{l-1})},\\ m_i^{bw}={\displaystyle \sum _{v_k\in N_{bw}(v_i)}{a}_{ik}(W^{bw}{h}_k^{l-1})}\end{array}$$

(3)

Also, it aggregates all the node information $m_i^{b1}$ and $m_i^{b2}$ in $G^b$.

$$\begin{array}{l}{m}_i^{b1}={\displaystyle \sum _{v_m\in V^b}{a}_{im}(W^{fw}{h}_m^{l-1})},\\ m_i^{b2}={\displaystyle \sum _{vq\in V^b}{a}_{iq}(W^{bw}{h}_q^{l-1})}\end{array}$$

(4)

${\alpha }_{ij},{\alpha }_{ik},{\alpha }_{im},{\alpha }_{iq}$ are the attention parameters. When the match value exceeds the similarity threshold, the $m_i^{self}\approx m_i^{cross}$. The reason for the two assembly nodes not being exactly equal is that node $v_i$ can only aggregate messages from its neighboring nodes in $G^a$, while it can aggregate messages from all nodes in $G^b$.

Representing updates: after aggregating the messages, each node $v_i$ updates its representation from $h_i^{l-1}$ to $h_i^l$, which can be first compared to $m_i^{self}$ and $m_i^{cross}$ using the multi-view cosine distance.

$$d_k=\cos ine(w_k^{\cos }\odot m_i^{self},w_k^{\cos }\odot m_i^{cross})$$

(5)

$$h_i^l=FEN(\left[m_i^{self},d_i\right])$$

(6)

where $k\in \{\mathrm{1,2},\dots ,P\}$. $w_k^{\cos }$ is a parameter vector. FEN denotes a two-layer feedforward network. After the step of updating the node representation L, a graph-aware representation $h_i^L$ of each node $v_i$ will be obtained.

• Relational Classifier Module

A similarity between assembly clauses or assembly sequence subgraphs can be predicted using two graph-level representations, $G^a$ and $G^b$.

$$p=FEN(\left[g^a,g^b,g^a\odot g^b,|g^a-g^b|\right])$$

(7)

where $p\in \left[\mathrm{0,1}\right]$, the training objective is to minimize the binary cross-entropy loss.

3.2.2. Reasoning about Instance Items of Assembly Process Based on SWRL

Not all assembly units or assembly types can find similar process items in the assembly library. For a new assembly, designers are required to redesign the assembly process. Therefore, the design is based on the SWRL to construct the inference rules for the semantics of assembly operations and generate the entities describing the process of executing assembly operations, the assembly resource entities, and the relationships between them to complete the design of process steps for a new assembly.

• Assembly Semantic Rule Generation

The assembly relationship defines the freedom degree of assembly between parts. This expresses the restriction of various positional relationships to different components so as to judge the disassembly level of the components. A series of assembly specifications, such as assembly degrees of freedom GF and cooperative relationship CR, are obtained based on the positional and cooperative relationships between the assembled components.

Taking the cooperative relationship of the YZ plane and the positional relationship of the Y-axis direction as an example, the corresponding assembly specification obtained by the combination of other cooperative and positional relationships can also be defined as follows, $R1:\left(\forall p\right)\left(\forall r\right)\left(\exists f\right)\left(p\in FA\right)\wedge r\in CR\wedge p.hasCR\left(r\right)\wedge f\in GF\wedge r.hasGF\left(f\right)\to p.hasGF\left(f\right).$ In R1, the freedom degree of limitation relationships of the mutually supported parts in the SWRL are described. If $p\in FA$ and there is a restricted freedom degree of relation between CR(r) and GF(f), then FA(p), GF(f), and hasGF(p, f) are held by the OWL (ontology wed language) assertion formula rule.

Assembly coupling rules are used to assist in improving the stability and feasibility of the installation process. To ensure a reasonable assembly sequence, assembly coupling rules R2~R4 are given to generate a collection of couplings.

$$R2:(\forall p1)(\exists p2)(p1\in SPN\wedge p2\in SAandP\wedge p1.Next(p2)\to p1.SCR(p2\left)\right)$$

$$R3:(\forall p1)(\exists p2)(p1\in SPN\wedge p2\in SAandP\wedge p1.Next(p2)\to p1.SAP(p2\left)\right)$$

$$R4:(\forall p1)(\exists p2)(p1\in SPN\wedge p2\in SAandP\wedge p1.Next(p2)\to p1.SCP(p1\left)\right)$$

R2 − R4 describes the coupling relationship between parts. If $SPN\left(p1\right)$ and $SAandP$ have a coupling relationship, then the OWL assertion formula holds for $SAandP$ and $SCR\left(p2\right)$, $SAP\left(p2\right)$, and $SCP\left(p1\right)$.

The assembly interference relationship reflects the mutual interference status of two parts in the spatial coordinate system during the assembly process. As a pre-step of subassembly sequence generation, it is necessary to accurately obtain the interference information between the parts. There are the following OWL rules to define the interference relationship.

$$R5:\left(\forall p\right)\left(\forall p1\right)\left(\exists p2\right)\left(\exists p3\right)\left(\exists f\right)\left(p\in FA\wedge p1\in SAandP\wedge p2\in SAandP\wedge p3\in SAandP\wedge p.hasBegin\left(p2\right)\wedge p.hasEnd\left(p3\right)\wedge p1=p2\wedge f\in GMOi\wedge p.hasGMO\left(f\right)\to p1.S_{IR}i\left(p3\right)\right)$$

$$R6:(\forall p)(\forall p1)(\exists p2)(\exists p3)(\exists f)(p\in FA\wedge p1\in SAandP\wedge p2\in SAandP\wedge p3\in SAandP\wedge p.hasBegin(p2)\wedge p.hasEnd(p3)\wedge p1=p3\wedge f\in GMOi\wedge p.hasGMO(f)\to p1.S_{IR}o(p2\left)\right)$$

where both R5 and R6 describe the interference of the combination of parts. In the formula, considering the idea of detachable and mountable, the interference direction is also divided into positive and negative where i represents the positive direction and o represents the negative direction. Taking R5 as an example, if there is interference between $FA\left(p\right)$ and $GMOi$, the parts $p$ belonging to the subassembly $FA$ are used as the reference parts for judging the interference location and, then, the set of $FA\left(p\right)$, $SAandp,$ and $SIRo\left(p,p\ast \right)$ hold by the assertion formula rule.

• Automatic generation of subassembly sequences

In the case of similar assemblies, partial modifications are made based on existing sequences. If it is a new product, the relationships between the different concepts in an assembly information representation model are used. Also, the data are transferred to the subassembly sequence generation algorithm through a rule-based reasoning method. If the local planning results meet the assembly constraint rules, the subassembly sequences are filtered out to find a reasonable sequence of eligible subassemblies. Finally, for the demand of assembly optimization, the global optimal result is output. The steps and flow chart are shown in Figure 7.

$Step1$: Import the initial parameters of the algorithm, including multiple assembly constraint relationships based on the ontology representation.

$Step2$: The initial parameters of the algorithm are set based on the rules defined in the assembly process knowledge base, such as the maximum number of iterations N_max, the part class Parts, and the container class $P_{SA}$ retains the subassembly sequence.

$Step3$: Build an ontology model of the assembly.

$Step4$: Iterate through the Parts class.

$Step5$: Iterate through $P_{SA}$ and add the queue information from the Parts class to the sub-planning collection.

$Step6$: If the conditional judgment function is met, a queue of subassemblies of the part class Parts is created. If not, the next part in the subassembly is traversed.

$Step7$: Initial generation of localized planning results.

$Step8$: Query from the ontology model whether the current local sequence planning result satisfies the assembly constraint rules, including $S_{IR}$, $S_{PR}$, $S_{CR}$. If not, go back to the sub-planning set traversal.

$Step9$: Outputs the results of the calculations for the sequence planning of this subassembly.

• Final Assembly Sequence Prioritization Calculation

After obtaining the subassembly outputs, the overall assembly planning needs to ensure the geometric feasibility of the assembly sequence, including the geometric feasibility of the final assembly sequence, the stability of the assembly operations, the cohesiveness of the assembly process, and the number of assembly reorientations.

(1)

The geometric feasibility of the final assembly sequence

When a final assembly consists of n subassemblies, it refers to the case where a subassembly $S_{Ai}\left(0<i<n\right)$ in the final assembly translates to infinity along a certain assembly direction and interferes with other subassemblies.

(2)

The stability of the assembly operations

With all known assembly sequences being feasible, both the stability of the connection relationships between parts and the stability of the gravitational direction affecting the assembly process needs to be considered. The stability of the connection relationship is expressed:

$$S_c={\displaystyle {\sum }_{i=1}^n{c}_{i,j}}$$

(8)

$c_{i,j}$ denotes the number of unstable connections between two components. The gravity direction stability calculation is expressed as follows.

$$S_s={\displaystyle {\sum }_{i=1}^n{s}_{i,j}}$$

(9)

In summary, the stability of the assembly operation of the general assembly sequence can be obtained as follows.

$$f_s=\frac{1}{{\displaystyle {\sum }_{i=1}^n(c_{i,j}+s_{i,j})+1}}=\frac{1}{N_S+1}$$

(10)

(3)

The cohesiveness of the assembly process

It is the utilization of the same set of assembly process equipment in the assembly process. The number of changes of tools used in the process of successive completion of the coupling of products is $N_i(0\le N_i\le n-1)$, provided that the assembly sequence is determined to be feasible. Based on the principle that the lower the number of changes in the assembly tool, the better the assembly performance. The aggregation affiliation in the assembly process is expressed as follows.

$$f_i=\frac{1}{N_i+1}$$

(11)

(4)

The number of assembly reorientations

The number of assembly reorientations is the sum of the number of changes in the three axes. The number of assembly reorientations may vary due to sequential differences in the assembly order of different feasible subassemblies. The affiliation number of assembly reorientations in the final assembly sequence can be obtained based on the total number of changes $N_d(0\le N_d\le n-2)$ of the final assembly on the three axes.

$$f_o=\frac{1}{N_d+1}$$

(12)

• Multi-objective sequence optimization algorithm

Through the analysis of the above multi-objective comprehensive optimization method for wind turbines, the local subassembly sorting results based on the output of the rule generation layer are combined. Then, the final assembly sequence generation algorithm is shown in Algorithm 1.

Algorithm 1 Assembly sequence generation algorithm

Input: list i//Serial information for subassemblies in the list Output: list AS//Final assembly results after multi-objective sequential optimization Algorithm GenerateAssemblySequences(List list) 1: list AP = createArrayList() 2: list a = createArrayList() 3: list b = createArrayList() 4: FOR each list Pj, in the list i.P 5: IF list Pj.Bsp = TRUE 6: list b.add(list Pj.ID) 7:    list AS. add(list b) 8:  IF list AS=NULL 9:   FOR each list Pj in the list i.P 10:    IF list Pj.NPR = maxof(list Pj)//Find the maximum number of locational relations 11:       list b. add(listPj.ID) 12:       list AS.add(listb)//Base parts for assembly sequences 13:    WHILE//Iterate over all subassembly sequences 14:      FOR each list b in the list AS 15:      IF list Pi.SA(fg) ≠ NULL 16:      IF list Pi.SA(fs) ≠ NULL 17:      IF list Pi.SA(fp) ≠ NULL 18:      IF list Pi.SA(fo) ≠ NULL 19:      IF list Pi.SA(fr) ≠ NULL 20:    list Pj.add(list b*) 21:  return

4. Case Study

4.1. Case Illustration

In this section, the example of generating the assembly sequence of a VEU-15 wind turbine (from Vestas Production Site in Tianjin, China) is used to validate the effectiveness of the proposed knowledge graph-based approach for the wind turbine assembly sequence. The wind turbine is shown in Figure 8. The part number is shown in

Table 2. Part number.

	$S_{A1}$	1	WP15 01 01 000	Wind turbine base parts		1
		2	WP15 01 02 000	plunger		1
		3	WP15 01 03 000	Control tower		1
		4	WP15 01 04 000	Complete shells		1
		5	WP15 01 05 000	alternators		1
		6	WP15 01 06 000	nacelle		1
	$S_{A2}$	7	WP15 02 01 000	Rotor plug-in		1
		8	WP15 02 02 000	Blade steering transmission device		1
		9	WP15 02 03 000	Pod steering transmission device		1
		10	WP15 02 04 000	Wind wheel		1
		11	WP15 02 05 000	Rotor stopper		1
		12	WP15 02 06 000	brake mechanism		1
	$S_{A3}$	13	WP15 03 01 000	blade		3
		14	WP15 03 02 000	steering		2
	$S_{A4}$	15	WP15 04 01 000	Brake mechanism terminal switch group		1
		16	WP15 04 02 000	Pod slewing transmission terminal switch group		1
		17	WP15 04 03 000	Blade steering transmission terminal switch group		1
		18	WP15 04 04 000	Current limiter drive terminal switch group		1
	$S_{A5}$	19	WP15 05 01 000	Tower shell		1
					WP15 01 00 000
file	subassembly	serial No.	attachment	date

.

4.2. Knowledge Graph Construction and Wind Turbine Assembly Sequence Generation

• Statistics of assembly process information

The assembly information is obtained from the assembly process text, on-site installation images, and 3D models. Then, the assembly knowledge entities and relationships are extracted as the basis for knowledge reasoning.

Table 3. Statistics for the wind turbine assembly process information.

Feature Type	Number of Instances	Feature Type	Number of Instances
subassembly/SA	5	SubassemblyOf	196
FaceTorch	82	HasPart	205
EdgeCurve	503	PartOf	205
VertexPoint	1052	GF	550
ConnectionAssembly	847	GMO	275
AdhesiveAssembly	48	RelativeMatrix	46
HasSubassembly	196	Direction	3411

shows the statistics of the wind turbine’s assembly information.

• Knowledge extraction and knowledge graph generation from assembly information

As the BOM of an assembly contains the relationships of dependencies between the components of a wind turbine, it can be achieved by accessing the BOM data in the ERP system via an API to build the knowledge graph directly. Therefore, only general assembly texts are discussed in this section. For textual entity recognition and relationship extraction models, the labeled corpus is divided into training, validation, and test sets in the ratio of 8:1:1. The entities of other modalities are extracted directly: 10% of them are taken and aggregated with the text entity test data to form the total entities. Meanwhile, in this paper, 10% of the cross-modal relations extracted are manually verified. These results are aggregated with the textual relations test data to become total relationships. To evaluate the accuracy of the extraction results, the F_1-score is used for both entities and relationships.

$$F_{1-entity}=\frac{2\times P_{entity}\times R_{entity}}{P_{entity}+R_{entity}}$$

(13)

$$F_{1-link}=\frac{2\times P_{link}\times R_{link}}{P_{link}+R_{link}}$$

(14)

Precision on P_entity, P_link denotes the ratio of correct entities to recognized entities. Recall on R_entity, R_link denotes the ratio of the number of correctly recognized entities to the total number of entities.

Table 4. Evaluation of the quality of multimodal knowledge graph generation.

Evaluation Indicators	Entity	Link
$P$	0.85	0.81
$R$	0.86	0.83
$F_1$	0.85	0.82

gives the overall evaluation of wind turbine multimodal knowledge extraction. It shows that each index reaches more than 0.8, proving that the method is reliable due to its consistency.

The visualization of the assembly knowledge graph of the wind turbine is shown in Figure 9. The relationships include mainly bolted, screwed, hasPart, partOf, HasSubassembly, and SubassemblyOf.

The visualization of component assembly relationships is an important part of knowledge reuse. It ensures that the assemblers select the required knowledge services to accomplish the target planning according to their assembly requirements. The assembly relationship retrieval is shown in Figure 10. In the process of wind turbine installation, the final assembly can be divided into seven parts of subassemblies, which are wind turbine foundation parts, electrical platform at the bottom of the tower, tower hoisting, nacelle hoisting, blade assembly, wind turbine installation, and others. Different subassemblies are indicated by different colors, which represent different labels in Neo4j. These colors can be used to determine which subassemblies belong to an assembly entity node. Taking the assembly part “Spiral buckle bolt” as an example, there are two ways to obtain the assembly process information of this part in the lifting of tower shell. The first one is to input the part to be retrieved in the search box. If the entity of this part exists in the database, the assembly process chain can be obtained successfully. The other method is to obtain the required assembly knowledge by clicking on the part name directly below the search box. After the system matches the filters, the knowledge graph automatically associates the entity nodes and their relationships. As can be seen from Figure 10, the relationship between the “Spiral buckle bolt” and the “Wire rope fixation” is fixed. The relationship between the “Spiral buckle bolt” and the “Wire rope type fall arrest system” is hasPart. The connection method between the “Spiral buckle bolt” and the “Lock bolt buckle” is boltConnection.

4.3. Knowledge Graph-Based Assembly Sequence Recommendation for Wind Turbine

• Component assembly relationship rule generation and knowledge reasoning

Firstly, the API functions of the CAD system are utilized to extract the assembly constraint relationships. After extracting the results, the Jena API is used to input the information into the OWL file to realize the instantiation of the assembly information ontology. Through the instantiated fit relationship assertion formulas, the part assembly rules under different preconditions can be obtained.

Next, the extracted assembly information ontology is instantiated to execute the reasoning based on the defined rules. Ontology construction and instantiation reasoning are performed using Protégé, running the algorithms in a Java development environment. Under the premise of determining the geometric feasibility of the assembly, the parameter settings are adjusted according to the assembly requirements for stability affiliation, aggregation affiliation, and reorientation affiliation. Then, the weights are set for the priority relationship levels. Finally, the subassembly sequences planning for the final assembly are obtained. Its generation process is shown in Figure 11. It is shown that three feasible global sequences can be generated. Firstly, sequence 1 is Blade−Tower Shell−Electrical Platform−Nacelle−Wind Wheel. Secondly, sequence 2 is an Electrical Platform−Tower Shell−Nacelle−Blade−Wind Wheel. Thirdly, sequence 3 is a Nacelle−Tower Shell−−Electrical Platform−Blade−Wind Wheel.

By analyzing the sequences of the three wind turbine products obtained above, it can be found that each sequence can meet the assembly requirements of the products. However, from the consideration of comprehensive indexes at the part level, the best assembly sequence is selected as sequence 2, which starts from the electrical platform S_A3. The subsequent parts take this as the center to determine their respective assembly sequences. The global sequence representation based on the knowledge graph is completed by combining subassembly planning, whereby the model information and attribute information of the parts are visualized together, as shown in Figure 12. In general, combining Figure 11 and Figure 12, the proposed methods provide a chance to integrate and utilize data at the assembly stages of the life cycle in wind turbines, so as to better meet the needs of users’ personalized and customized assembly requirements.

• Optimization of assembly sequences for wind turbines

Under the premise of determining the geometric feasibility of the assembly sequence, it is necessary to further use the optimization algorithm proposed in this paper to determine the advantages and disadvantages of the assembly sequence in terms of assembly reorientation, assembly aggregation, and assembly stability. Then, the classical optimization algorithms, including the Pareto ant colony algorithm [39] and the NSGA-II algorithm [40], are selected as the baseline because the Pareto ant-colony algorithm ensures the rationality and feasibility of the assembly sequence by constructing the feasible solutions of the components in a step-by-step manner. Secondly, the Pareto ant colony will be affected by the heuristic information in the process of selecting the next assembly part, so it will have a greater probability to select those parts to be assembled with a greater value of heuristic information, which is unique to this local search strategy and is conducive to the planning of high-quality assembly sequences. Also, NSGA-II is a classic algorithm in the field of multi-objective optimization. It is renowned for its powerful search capabilities and diverse maintenance, making it an excellent choice for addressing multi-objective problems. In this paper, taking the wind turbine assembly sequence planning as an example, the nacelle, the part with the largest size, is selected as the priority assembly part. In this case, the algorithm’s operating parameters are taken from reference [39,40].

Finally, the TOPSIS (technique for order of preference by similarity to ideal solution) method is introduced to rank the optimization methods for assembly sequences because TOPSIS is a multi-criteria decision-making method that compares the Euclidean distance between three experimental algorithms and ideal or anti-ideal solutions. Also, it determines the relative merits of each algorithm by calculating the relative closeness of each assembly sequence’s solution to the ideal sequence, taking the first three near-optimal solutions from each of the three algorithms, including the proposed algorithm, the Pareto ant colony algorithm, and the NSGA-II algorithm. Thus, the TOPSIS method is used to rank these methods. The actual assembly sequences for each method are shown in

Table 5. Assembly sequences are generated by the three methods.

Methods	Approximate Optimal Solution	Global Assembly Sequences
Pareto ant colony algorithm	p1	P2 → (P5, −Z) → (P4, +X) → (P3, − X) → (P13, +Y) → (P14, +Y) → (P6, −Z) → (P7, +X) → (P8, +X) → (P9, +X) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P12, −Z) → (P11, +X) → (P10, +X) → (P1, −Z) → (P19, −Z)
	p2	P2 → (P3, −Z) → (P4, +X) → (P5, −X) → (P13, +Y) → (P14, +Y) → (P9, −Z) → (P8, +X) → (P7, +X) → (P6, +X) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P10, −Z) → (P12, +X) → (P11, +X) → (P1, −Z) → (P19, −Z)
	p3	P2 → (P5, −Z) → (P4, +X) → (P3, −X) → (P14, −X) → (P13, +Y) → (P7, −Z) → (P8, +X) → (P6, +X) → (P9, +X) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P11, −Z) → (P12, +X) → (P10, +X) → (P1, −Z) → (P19, −Z)
NSGA-II algorithm	n1	P2 → (P12, −Z) → (P10, +X) → (P11, −X) → (P14, −X) → (P13, +Y) → (P8, −Z) → (P9, +X) → (P7, +X) → (P6, +X) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P4, −Z) → (P3, +X) → (P5, −X) → (P1, −Z) → (P19, −Z)
	n2	P2 → (P11, −Z) → (P10, +X) → (P12, −X) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P6, −Z) → (P9, +X) → (P7, +X) → (P8, +X) → (P14, −X) → (P13, +Y) → (P4, −Z) → (P3, +X) → (P5, −X) → (P1, −Z) → (P19, −Z)
	n3	P2 → (P12, −Z) → (P11, +X) → (P10, −X) → (P8, −Z) → (P7, +X) → (P6, +X) → (P9, +X) → (P5, −Z) → (P3, +X) → (P4, −X) → (P13, +Y) → (P14, +Y) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P1, −Z) → (P19, −Z)
KG-driven rule-based reasoning algorithm	k1	P2 → (P8, −Z) → (P9, +X) → (P7, +X) → (P6, −Z) → (P13, +Y) → (P14, +Y) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P4, +X) → (P3, +X) → (P5, +X) → (P12, −Z) → (P11, +X) → (P10, +X) → (P1, −Z) → (P19, −Z)
	k2	P2 → (P8, −Z) → (P9, +X) → (P6, +X) → (P7, −Z) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P10, +X) → (P11, +X) → (P12, +X) → (P4, −Z) → (P3, +X) → (P5, +X) → (P13, +Y) → (P14, +Y) → (P1, −Z) → (P19, −Z)
	k3	P2 → (P8, −Z) → (P6, +X) → (P9, +X) → (P7, +X) → (P13, +Y) → (P14, +Y) → (P15, +X) → (P18, −X) → (P16, +Y) → (P17, +Y) → (P12, +X) → (P11, +X) → (P10, +X) → (P4, +X) → (P5, +X) → (P3, +X) → (P1, −Z) → (P19, −Z)

. The results of the TOPSIS ranking are shown in

Table 6. Calculation results of the relative closeness of the assembly solution.

Serial No.	Stability Affiliation Degree	Reorientation Affiliation Degree	Polymerization Affiliation Degree	Relative Closeness
p1	0.77	0.41	0.28	0.37
p2	0.78	0.46	0.31	0.41
p3	0.82	0.48	0.34	0.49
n1	0.75	0.82	0.40	0.47
n2	0.78	0.86	0.43	0.48
n3	0.81	0.88	0.48	0.54
k1	0.74	0.89	0.45	0.63
k2	0.79	0.92	0.51	0.65
k3	0.82	0.95	0.51	0.76

.

Each solution in the computational results represents a feasible assembly scheme. The optimal solution k3 of this paper’s method among these assembly solutions has an assembly reorientation affiliation of 0.95, corresponding to two reorientations. The assembly aggregation affiliation feedback is the number of tool changes in the execution of assembly steps. The optimal solution of this paper’s method aggregation affiliation is 0.51, which corresponds to the number of tool changes in the assembly process of six times. This is the least among the assembly schemes. From the point of view of the actual assembly of the wind turbine, even if different assembly programs are selected, the connection between the parts will not be easily changed. Therefore, the performance of the stability affiliation of the three algorithms is in proximity. It can be seen that all three methods can provide a wider decision space for decision makers. They can solve different practical problems according to the users’ assembly needs. The final result of solving the multi-objective assembly sequence planning problem is a set of approximate optimal solutions. In practice, it is necessary to choose a solution that makes each optimization objective as optimal as possible. The method in this paper achieves optimization in the TOPSIS multiple assembly scheme ranking.

Overall, in the manner of the new generation of information technologies and methods, including the construction of knowledge graphs in wind turbine assembly and knowledge-driven assembly sequence recommendations for the wind turbine, the assembly workload of each component will be reduced in the development workload of the whole wind turbine. It can be noticed that various assembly data related to wind turbine assembly scattered in the assembly workshop are effectively integrated and utilized by multi-process assembly information modeling and knowledge extraction. It provides powerful support for workers to share, query, and trace the assembly knowledge of wind turbines. In addition, the sequences obtained by knowledge-driven assembly sequences are superior and more efficient than traditional manual experience and similarity matching methods, by combining the wind turbine-specific assembly requirements and the optimization degree of several assembly evaluation indexes. It provides a viable means for assisting workers in designing assembly sequences with multi-configuration requirements.

5. Conclusions and Future Work

A knowledge graph-based approach for assembly sequence recommendations for wind turbine is proposed in this paper. Firstly, an assembly information representation model and knowledge extraction methods for different modal data are designed to construct a knowledge graph for wind turbine assembly. Secondly, a knowledge graph-based reasoning method for wind turbine assembly sequences is proposed. For the existing assembly knowledge items in the assembly knowledge base, similar assembly process item retrieval is realized based on the BERT-GMN model. For new assemblies without assembly experience support, the reasoning rules of assembly process instance items are proposed to analyze the set of assertion formulas for the support relationship, coupling relationship, and interference relationship. Considering the multi-objective evaluation index optimization, a multi-objective sequence optimization algorithm is designed. Finally, the comprehensive evaluation metrics for both entities and links in knowledge extraction tasks reach above 0.8, proving that the consistency of different modal knowledge extraction methods is reliable. The effectiveness of the multi-source information representation of parts and optimal sequence reasoning is verified using the VEU-15 wind turbine as a case. In the comparison of the three methods, the proposed method achieves better results when assembly sequence reasoning under assembly reorientation, assembly aggregation, and assembly stability are considered simultaneously.

Although the proposed method shows superior results for assembly sequence recommendations for wind turbines, it still has some limitations. It is possible to extend the assembly information representation model, which currently focuses on geometric constraints. In fact, in complicated assembly scenarios, process constraints also form various limitations on the manufacturing process, including the manufacturing process, machining accuracy, and material selection related to PFEP (plan for every part). Therefore, by considering more constraints, the extended assembly information representation model may be made more comprehensive and accurate to adapt to more complicated products. Also, there is insufficient consideration of the dynamics and complexity of factors of production. The introduction of temporal information in subsequent assembly sequence recommendations is worth investigating, such as the rotation of assembly personnel and the updating of assembly equipment, to meet needs in complicated assembly situations.