An Artificial Intelligence-Based Model for Knowledge Evaluation and Integration in Public Organizations

Abstract

In the construction of knowledge bases, it is very important to evaluate the quality of the knowledge entered into them. This is exacerbated in public administrations, where knowledge should be oriented towards public services. In this study, an artificial intelligence-based method for the evaluation of knowledge is described. This method takes advantage of the structure and contents of the knowledge representation schemas (representing the knowledge of the corresponding experts) to carry out knowledge evaluation. More precisely, the method allows the various comparisons between the schemas to be integrated and the overall schema to evaluate the contribution of each schema.

1. Introduction

Knowledge in organizations is a valuable organizational asset. Organizational Knowledge Management (KM) is the process of identifying, acquiring, evaluating, and disseminating knowledge into the organization aiming to make smart decisions. Knowledge has become the distinctive element for the competitiveness of organizations, so KM is one of the key factors in achieving organizational goals.

KM makes use of a wide repertory of procedures, techniques, and tools, including Artificial Intelligence (AI). Ontologies, which are the standard method of knowledge representation in AI, have been used consistently in KM for a variety of organizational endeavors and application domains, including education [1].

Various types of knowledge can be allocated in different functional units of an organizational structure. At the same time, [2] pointed out that knowledge integration at an organizational level enables the organization to carry out better innovation processes.

Knowledge integration implies collaboration in organizations. This requirement is hard to meet sometimes, especially when the size of the organization is small since the reduced organizational size usually conveys the limitation of resources at all human capital, material, technological, and temporal levels [3].

Knowledge integration is an important factor for innovation processes since collaboration makes it easier to carry out experiments [4], increase cohesion within collaborative teams [5], or generate new solutions to challenging issues [6].

Sustainable practices in agriculture and farming are gaining momentum worldwide. However, such practices usually involve all environmental, economic, and social parameters, which makes such practices complex. This also has an influence on the management of crops incorporating such practices so that often different crop management views can emerge that can be complementary to each other. In fact, there is a demand for integrating different views addressing crop management in farming organizations that can tackle the increasingly complex administrative processes necessary to meet the regulations to receive subventions from the EC. This has been exacerbated after the decision of the EU to align itself with the 2030 Agenda of the United Nations (UN) and, therefore, with the UN-defined Sustainable Development Goals.

In this article, a method to carry out evaluation and integration of (different) views on a given topic is introduced. The method is based on comparisons between the knowledge representation schema responding to each view and the integration of the resulting knowledge representation schema.

The structure of this article is as follows. Section 2 describes the state of the art of knowledge integration in the context of knowledge management. Section 3 describes the knowledge representation model on which the approach presented here is based. In Section 4, a method for knowledge assessment and integration is described. On this basis, Section 5 details the general method of the knowledge evaluation process. In Section 6, a case study is described that shows the application of this approach in a sustainable agriculture setting. Section 7 discusses the benefits and limitations of the research approach as well as a comparison of the results of this method with the results of other approaches. Finally, in Section 8, some conclusions are presented.

2. State-of-the-Art

Knowledge in organizations is a valuable organizational asset. Hence, it should be managed adequately. Organizational Knowledge Management (KM) can be defined as the process of identifying, acquiring, evaluating, and disseminating knowledge into the organization aiming to make smart decisions. In recent years, knowledge has become the distinctive element for the competitiveness of organizations, so KM is one of the key factors in achieving corporate goals. According to [7], the KM objectives in an organization are to promote growth, communication, and the preservation of knowledge.

Knowledge Management Systems (KMS) allow access, exchange, and update of organizational business knowledge. Organizations that recognize the relevance of KM usually make use of available capabilities in the organization or create new ones with the purpose of investing in new solutions demanded by the market. Overall, a good KM allows organizations to increase their effectiveness and efficiency [8].

KM makes use of an ample range of procedures, techniques, and tools, including Artificial Intelligence (AI). Ontologies, which can be said to be the standard method of knowledge representation in AI, have been used consistently in KM for a variety of organizational endeavors and application domains, including education [1].

According to some studies [9], different types of knowledge can be allocated in distinct functional units of an organizational structure. At the same time, in [2], it is pointed out that knowledge integration at the organizational level enables the organization to carry out better innovation processes. This has been claimed in recent research studies [10].

Knowledge integration within the context of problem-solving at the organizational level requires collaboration. This requirement is difficult to meet sometimes, especially when the size of the organization is small, since the reduced organizational size usually conveys a limitation of resources at all human capital, material, technological, and temporal levels [3]. However, at the same time, the need for collaboration in small corporations is high as they need to create and manage adequate knowledge for realizing innovations [11].

Based on another research line, it has been put forward that independently of the viability of collaboration within organizations, collaboration can be very positive for organizations [12]. Thus, for instance, it assists in overcoming sole restrictions related to (human) resource availability and in finding synergies amongst actors.

Organizational members may overcome collaborative impediments or obstacles by making use of knowledge sharing that allows them to benefit from complementary views on a certain issue of interest at the organizational level [11]. More recently, the authors in [13] determined that knowledge integration has a tremendous influence on companies’ performance.

In general, knowledge integration is also a key factor for innovation processes since collaboration makes it easier to carry out experiments [4], increase cohesion within collaborative teams [5], or generate new solutions to challenging issues [6]. However, in the case of small organizations, a few individuals must address unknown parts of the innovation process using their few resources [14]. Furthermore, this issue jeopardizes the finding of solutions to complex problems requiring collaboration [4].

From another perspective, other authors have researched possible solutions or factors to be considered to overcome such limitations. Thus, [15] considers that knowledge integration is affected by several factors related to psychological profiles and the possession of common resources, such as shared interests, meanings, or lexicons. Moreover, following the research results described in [16], experts’ collaboration in organizations minimizes the effects of their (human capital, cognitive, material) resource limitations considered at an individual scale. In other words, collaboration increases the organization’s capabilities to deal with complex issues.

To achieve adequate knowledge integration in organizations, Corporate Memory (CM) systems, which can be defined as knowledge repositories and know–how in a group of individuals who work in a firm, have played an essential role [17]. Within the CM system construction process, it is paramount to evaluate knowledge before including it in the CM system.

On the other hand, by placing the citizen at the center of the advancement of public administrations, it is necessary to bet on KM mechanisms that allow the design and evaluation of public policies in sectors still with low rates of digitalization, such as farming. In other words, it is necessary to analyze, design, and implement public policies by adopting a governmental–social co-production and co-authorship approach that allows “the Government of society to become Government with society” [18]. In this process, the function of the Public Administration, in collaboration with the Government, should become a “dynamic resource.” Institutional strength should not be confused with firmness and inability to introduce resources and processes that serve to adapt to changes. In turn, the involvement of other social agents in collaboration with public administrations would expand the capacity to develop the necessary institutional and professional skills within public services. Thus, “the State cannot be understood as a monolithic actor, but rather as an entity in which an endless number of mechanisms and interests operate in multiple logics” [19].

Elsewhere, it has been argued that the State is not the only institution capable of carrying out the task of proposing and implementing collective goals in an effective and efficient manner. In this sense, the State has the perfect mechanisms to carry out the decision-making process. So, it could be said that the State is the only locus available when it comes to carrying out a legitimate collective action. However, it must also be capable of delegating functions to other public structures for the development of mechanisms for citizen participation in the formulation of new goals; in other words: “in the contemporary world, responsibility for government actions can be a necessary substitute for other forms of democracy” [20].

According to [21], Collective Intelligence (CI) results from long processes and constitutes the social capacity within the framework of a territory. In this way, when speaking of an intelligent territory, it is not enough that the institutions dedicated to knowledge exist. It is necessary to generate relational dynamics that cause changes in the structures, processes, and collective rules [22]. The use of the CI is a two-way road. It is believed that public administrations should bet more and more on this type of method. The reason for this is that while public or private institutions incorporate the knowledge of the communities through crowdsourcing methodologies, these same institutions enrich their communities by facilitating access to the generated knowledge. This feedback has been carried out so far through different methods: in the form of open-access documents, with structured data sets for subsequent statistical exploitation by the user, or through improvements to the software on which this knowledge exchange between institutions and communities is based [23].

Management of knowledge on sustainable practices in agriculture and farming is gaining momentum worldwide. However, such practices usually involve all environmental, economic, and social parameters, which makes such practices complex. This also has an influence on the management of crops incorporating such practices so that often different crop management views can emerge that can be complementary to each other. In fact, from a knowledge management perspective, there is a demand for integrating different views addressing crop management in farming organizations that can tackle the increasing complexity of the administrative processes necessary to meet the regulations to receive subventions from the EC. This has been exacerbated after the decision of the EU to align itself with the 2030 Agenda of the United Nations (UN) and, therefore, with the UN-defined Sustainable Development Goals.

Ontologies play a fundamental role in CM construction and management processes. In the CI context, the conceptualization of the domain forms a conceptually coherent basis on which future extensions can be made or mapped to ontologies in other domains to enable interoperability [24,25,26,27]. To achieve this, it is necessary to carry out a fusion process involving different ontologies proposed by human experts. Such fusion must be conducted using manually created ontologies. Another approach to knowledge capture is learning ontologies [28,29,30] by automating their construction from texts. This often requires carrying out several processes, such as the identification of relevant terms [31,32,33,34], determination of concepts [35], obtaining of taxonomic relations through semantic similarity [36], or obtaining of partonomic relations. These processes frequently rely on natural language processing and the support of lexical resources.

3. Knowledge Representation Model

Knowledge representation is one of the most important areas in Artificial Intelligence. Although several methods for knowledge representation can be found in the literature, ontologies are the most popular knowledge representation schema. An ontology can be defined as a shared and common understanding of some domain that can be communicated across people and computers [24]. One of the most important types of ontologies is domain ontology, which can be defined as the specification of a conceptualization of domain knowledge [5].

In this study, ontologies are modeled using Multiple Hierarchical Restricted Domains (MHRD), where an MHRD can be understood as a finite set of concepts such that:

• The set contains at least two concepts. Formally, Cardinal (MHRD) ≥ 2.
• Every concept is defined through a finite, non-empty set of attributes. Formally, if AT(c) stands for the set of attributes defining a concept c ∈ MHRD, this condition states that Cardinal (AT(c)) ≥ 1. Furthermore, the relationship between a given concept c_j of an MHRD and a given attribute a_k, written HAS_AT_i(c_j, a_k), is defined as a logical function whose evaluation is true if a_k ∈ AT(c_j).
• Among a given pair of concepts, there may be either a taxonomic relationship so that (multiple) inheritance of attributes is possible or a partonomic one. Formally, a taxonomic relationship between two concepts, c_i and c_j, of an MHRD, written IS-A (c_i, c_j), is defined as a logical function whose evaluation is true if c_i is a semantic subcategory of c_j. A partonomic relationship between two concepts c_i and c_j of an MHRD, written PART-OF (c_i, c_j), is defined as a logical function whose evaluation is true if and only if c_i is a semantic element/component of c_j.
• Taxonomic relationships are irreflexive, antisymmetric, and transitive, while partonomic relationships are irreflexive and antisymmetric but not transitive [4].
• The set of concepts, together with the relationships between those, can be expressed through graphs, where the nodes represent the concepts (including their attributes), and the arrows account for the relationships so that every concept is linked (through a taxonomic or a partonomic relationship) to n concepts, n ≥ 1.

Some definitions that consider the two above-referenced types of semantic relationships embracing subsets of concepts in an MHRD are introduced next.

Definition 1. 

Taxonomic MHRD.

A taxonomic MHRD, written TMHRD, is an MHRDR subset where all its concepts are related to each other only through taxonomic relationships. Formally, let M and T be two MHRDs such that T ⊆ M. T is said to be a taxonomic MHRD of M, written TMHRD if ∀c_i ∈ T, ∃ c_j ∈ T such that [IS-A (c_i, c_j) ∨ IS-A (c_j, c_i)] ^ ¬ [PART-OF (c_i, c_j) ∨ PART-OF (c_j, c_i)].

Definition 2. 

Partonomic MHRD.

A partonomic MHRD, written PMHRD, is an MHRD subset where all its concepts are related to each other only through partonomic relationships. Formally, let M and P be two MHRDs such that P ⊆ M. P is said to be a partonomic MHRD of M, written PMHRD, if ∀c_i ∈ P, ∃ c_j ∈ P such that [PART-OF (c_i, c_j) ∨ PART-OF (c_j, c_i)] ^ ¬ [IS-A (c_i, c_j) ∨ IS-A (c_j, c_i)].

Definition 3. 

Uniform MHRD.

Let M and U be two MHRDs such that U ⊆ M. U is said to be a Uniform Multiple Hierarchical Restricted Domain, written UMHRD, of M if U ∈ {T, P}, where T is a TMHRD of M, and P is a PMHRD of M.

Based on the previous definitions, an immediate result is stated in the following corollary.

Corollary. 

Let M be an MHRD and let U be a UMHRD of M. Then, the following inequality holds 2 ≤ Cardinal (U) ≤ Cardinal (M).

Proof. 

Based on the definition of UMHRD, U is an MHRD. Based on the definition of MHRD, Cardinal (U) ≥ 2. On the other hand, based on the definition of UMHRD, U ⊆ M, which implies that Cardinal (U) ≤ Cardinal (M). □

4. Knowledge Evaluation and Integration Method

In general, the knowledge to be integrated can come from different views/sources. In this study, it is assumed that such knowledge can be represented using ontologies (one ontology per view), which in turn will be modeled through MHRDs, as indicated above. Moreover, each view is supposed to represent a person’s vision of the domain area, while (s)he will probably have more expertise in some knowledge area(s) than other persons [6].

The evaluation and integration process should be conducted without duplicating concepts or containing inconsistencies. In addition, if no additional information on the level of experience of the knowledge source is provided, the knowledge (represented through the corresponding ontologies) coming from the greatest number of experts can be considered the best. In this way, the knowledge contained in each ontology can be evaluated according to the support of each of its concepts in relation to the rest of the ontologies.

The evaluation and integration method presented in this study takes only the information contained in the input ontologies, allowing for the knowledge evaluation and integration to be automatic. Moreover, the relationships and concepts expressed in each ontology are evaluated independently. Thus, the value of each ontological element (e.g., concept) of one input ontology is based on the support that the element has in the other input ontologies.

As the input ontologies are supposed to come from the corresponding experts, in this study, it is assumed that they are really experts in scientific terms so that the knowledge contained in every input ontology is correct and is not inconsistent with the content in the other input ontologies. In other words, input ontologies will be assumed to be not inconsistent with each other and to use the same terms to describe the same concepts and attributes.

In the following subsection, the methods to evaluate and integrate semantic relationships are introduced.

4.1. Evaluation of Semantic Relationships

The proposed method to evaluate and integrate semantic relationships takes into account the overall concepts and semantic relationships included in all the (input) ontologies to be evaluated. Thus, a list of concepts is built as the union set of the concepts in all the input ontologies. Then, a similar procedure is followed regarding the taxonomic and the partonomic relationships in all the input ontologies. Subsequently, for each input ontology, the following is performed.

For each type of semantic relationship (i.e., for taxonomic and partonomic relationships), an evaluation matrix is generated, having the concepts of all input ontologies as rows and columns. Then, each cell of the matrix associated with an input ontology is marked with the value 1 if there exists a semantic relationship in the input ontology under question between the involved concepts in the corresponding row and column. Otherwise, it is marked with the value 0.

Formally, the semantic relationships evaluation method can be stated in an algorithmic fashion as follows. Let C be ⋃_n UMHRD_i, where UMHRD_i stands for a UMHRD of the MHRD corresponding to the i-th input ontology, i = 1,…, n; let TM_i(c_j, c_k) be the value of the cell formed by the concepts c_j and c_k in the evaluation matrix of the taxonomic relationships for the i-th input ontology; and let PM_i(c_j, c_k) be the value of the cell formed by the concepts c_j and c_k in the evaluation matrix of the partonomic relationships for the i-th input ontology. Then, the proposed method can be stated as follows (Algorithm 1):

Algorithm 1: Method for assessing semantic relations.

For i = 1 to n  For j = 1 to Cardinal (C)    For k = 1 to Cardinal (C)     If (cj ∈ UMHRDi) ^ (ck ∈ UMHRDi) ^ [IS-A (cj, ck) ∨ IS-A (ck, cj)] then TMi(cj, ck) = 1       else TMi(cj, ck) = 0;    If (cj ∈ UMHRDi) ^ (ck ∈ UMHRDi) ^ [PART-OF (cj, ck) ∨ PART-OF (ck, cj)] then PMi (cj, ck) = 1       else PMi(cj, ck) = 0;

To calculate the support for a taxonomic relationship, the following is carried out. For every input ontology, the support for a certain semantic relationship involving the concepts of that ontology is obtained by summing the values in the cells of the corresponding matrix for that relationship and the counterparts of the rest of the input ontologies when these take the value 1. Formally, the taxonomic support for the i-th ontology, written TSⁱ, is defined using the following equation:

$${\mathrm{T}\mathrm{S}}^i={\sum }_{j,k}^N{TM}_i(c_j,c_k)+{\sum }_{\begin{array}{c}l=1\\ l\ne i\end{array}}^n{\sum }_{j,k}^N{{TM}_i\left(c_j,c_k\right)TM}_l\left(c_j,c_k\right)$$

(1)

where N = Cardinal (C); i = 1, 2 ,..., n; n = number of input ontologies.

Proceeding in an analogous manner, the partonomic support for the i-th ontology, written PSⁱ, may be defined using the following equation:

$${PS}^i={\sum }_{j,k}^N{PM}_i(c_j,c_k)+{\sum }_{\begin{array}{c}l=1\\ l\ne i\end{array}}^n{\sum }_{j,k}^N{{PM}_i\left(c_j,c_k\right)PM}_l\left(c_j,c_k\right)$$

(2)

Finally, the relational support for the i-th ontology, written RSⁱ, is defined using Equation (3) below.

RSⁱ = TSⁱ + PSⁱ

(3)

4.2. Evaluation of Concepts

The evaluation of concepts is performed by proceeding similarly to the evaluation of relationships. Thus, the proposed method to evaluate concepts takes into account the overall concepts and their corresponding attributes included in all the (input) ontologies to be evaluated. Then, in addition to creating a list of concepts built as the union set of the concepts in all the input ontologies, another list of attributes is created as the union set of the concepts of all concepts involved in all the input ontologies.

For each input ontology, an evaluation matrix is generated, having the concepts of the input as rows and the attributes of such concepts as columns. Then, each cell of the matrix associated with an input ontology is marked with the value 1 if the concept associated with the corresponding row contains the attribute indicated in the column. Otherwise, it is marked with the value 0.

Formally, the concept evaluation method can be stated in an algorithmic fashion as follows. Let C be $\bigcup _{\mathrm{i}=1}^{\mathrm{n}}\mathrm{U}\mathrm{M}\mathrm{H}\mathrm{R}{\mathrm{D}}_{\mathrm{i}}$, where UMHRD_i stands for a UMHRD of the MHRD corresponding to the i-th input ontology, i = 1,…, n; let AT be $\bigcup _{\mathrm{j}=1}^{\mathrm{m}}\mathrm{A}\mathrm{T}{(\mathrm{c}}_{\mathrm{j}}$), where AT(c_j) stands for the set of attributes of the concept c_j, and m = Cardinal (C); and let CONCEPT_AT_i(c_j, a_k) be the value of the cell formed by the concept c_j and the attribute a_k in the evaluation matrix of the concept for the i-th input ontology. Then, the proposed method can be stated as follows (Algorithm 2):

Algorithm 2: Concept evaluation method.

For i = 1 to n    For j = 1 to Cardinal (C)        For k = 1 to Cardinal (AT)      If (cj ∈ UMHRDi) ^ HAS_ATi(cj, ak) then CONCEPT_ATi(cj, ak) = 1      else CONCEPT_ATi(cj, ak) = 0

To calculate the support for a concept, the following is carried out. For every input ontology, the support for a certain concept of that ontology is obtained by summing the values in the cells of the corresponding matrix for that concept and the counterparts of the rest of the input ontologies when these take the value 1. Formally, the conceptual support for the concept c of the i-th input ontology, written CSⁱ (c), is defined using the following equation:

$$\begin{gathered} {CS}^i(c)={\sum }_{j=1}^N{CONCEPT\_AT}_i(c,a_j)+ \\ {\sum }_{\begin{array}{c}l=1\\ l\ne i\end{array}}^n{\sum }_{j=1}^N{CONCEPT\_AT}_i\left(c,a_j\right){CONCEPT\_AT}_l\left(c,a_j\right) \end{gathered}$$

(4)

where N = Cardinal (AT); i = 1, 2 ,..., n; n = number of input ontologies.

To end, the total conceptual support for the i-th ontology, written TCSⁱ, is defined using the following equation:

$${TCS}^i={\sum }_{j=1}^m{CS}^i\left(c_j\right)\mathrm{with}{\mathrm{MHRD}}_{\mathrm{i}}=\{{\mathrm{c}}_1,{\mathrm{c}}_2,\dots {\mathrm{c}}_{\mathrm{m}}\}$$

(5)

5. Overall Knowledge Evaluation Method

The method for the whole evaluation process of all the views (i.e., ontologies) provided on some topic is built from the view having a richer conceptualization; that is, the MHRD is composed of the greatest number of concepts. Then, in an incremental fashion, every concept of such a selected view is enriched with knowledge (i.e., attributes) of the rest of the views so that an integrated view (i.e., ontology) is generated. Also, the concept(s) of the rest of the MHRDs that are not included in the selected MHRD are included in such a (integrated) view. This generation process can be expressed formally in an algorithmic way as follows (Algorithm 3):

Algorithm 3: Method for assessing general knowledge.

MRHDint = MHRDmax ⋃ (C\MHRDmax), where MHRDmax = MHRD s.t. Cardinal (MHRD) = max i {Card(MHRDi), i = 1, … n}, n = number of input ontologies; For every cj ∈ MRHDint do    For i = 1 to n       For every ck ∈ MRHDi do        If (cj = ck) OR (AT(cj) ⊂ AT(ck)) then AT(cj) = AT(cj) ∪ AT(ck)

At this point, we can proceed in a similar way than before with every single input ontology with respect to the integrated ontology resulting from the application of the above algorithm. Thus, by taking into account the previous sections, it is easy to calculate both knowledge evaluation parameters, namely, the relational support and the conceptual support. In this case, the set of ontologies involved in the evaluation process would be composed of just two ontologies, namely, the selected input ontology and the integrated ontology. Formally, the following can be derived for this case:

The taxonomic support for the i-th ontology with respect to the integrated ontology, written TSintⁱ, is defined using the following equation:

$${\mathrm{T}\mathrm{S}\mathrm{i}\mathrm{n}\mathrm{t}}^{\mathrm{i}}={\sum }_{\mathrm{j},\mathrm{k}}^{\mathrm{N}}{\mathrm{T}\mathrm{M}}_{\mathrm{i}}({\mathrm{c}}_{\mathrm{j}},{\mathrm{c}}_{\mathrm{k}})+{\sum }_{\mathrm{j},\mathrm{k}}^{\mathrm{N}}{{\mathrm{T}\mathrm{M}}_{\mathrm{i}}\left({\mathrm{c}}_{\mathrm{j}},{\mathrm{c}}_{\mathrm{k}}\right)\mathrm{T}\mathrm{M}}_{\mathrm{i}\mathrm{n}\mathrm{t}}\left({\mathrm{c}}_{\mathrm{j}},{\mathrm{c}}_{\mathrm{k}}\right)$$

(6)

where N = Cardinal (C); i = 1, 2,..., n; n = number of input ontologies; TM_int(c_j, c_k) be the value of the cell formed by the concepts c_j and c_k in the evaluation matrix of the taxonomic relationships for the integrated ontology.

The partonomic support for the ith ontology with respect to the integrated ontology, written PSintⁱ, may be defined using the following equation:

$${PSint}^i={\sum }_{j,k}^N{PM}_i(c_j,c_k)+{\sum }_{j,k}^N{{PM}_i\left(c_j,c_k\right)PM}_{int}\left(c_j,c_k\right)$$

(7)

where N = Cardinal (C); i = 1, 2,..., n; n = number of input ontologies; PM_int(c_j, c_k) is the value of the cell formed by the concepts c_j and c_k in the evaluation matrix of the partonomic relationships for the integrated ontology.

The relational support for the i-th ontology with respect to the integrated ontology, written RSintⁱ, is defined using Equation (3) below:

RSintⁱ = TSintⁱ + PSintⁱ

(8)

The conceptual support for the concept c of the i-th input ontology with respect to the integrated ontology, written CSintⁱ (c), is defined using the following equation:

$$\begin{gathered} {CSint}^i\left(c\right)={\sum }_{j=1}^N{CONCEPT\_AT}_i(c,a_j)+ \\ {\sum }_{j=1}^N{CONCEPT\_AT}_i\left(c,a_j\right){CONCEPT\_AT}_{int}\left(c,a_j\right) \end{gathered}$$

(9)

where N = Cardinal (AT); i = 1, 2,..., n; n = number of input ontologies; CONCEPT_AT_int(c, a_j) is the value of the cell formed by the concept c and the attribute a_j in the evaluation matrix of the concept for the integrated ontology.

Finally, the total conceptual support for the i-th ontology with respect to the integrated ontology, written TCSintⁱ, is defined using the following equation:

$${TCSint}^i={\sum }_{j=1}^m{CSint}^i\left(c_j\right)\mathrm{with}{\mathrm{MHRD}}_{\mathrm{i}}=\{{\mathrm{c}}_1,{\mathrm{c}}_2,\dots {\mathrm{c}}_{\mathrm{m}}\}$$

(10)

6. Case Study

6.1. Domain Knowledge Elicitation

An experiment using structured interviews has been conducted to apply the results of this research study. The experiment occurred in the domain of sustainable practices in agriculture and farming. Three experts, namely, E1, E2, and E3, who have each ecological crop production in the Mediterranean biogeographic region of Murcia (Spain), were interviewed separately. They were asked to provide some of the most relevant concepts that should be considered in crops to be managed in a sustainable fashion. The interviews took place during May and October 2023.

For E1, there are two key concepts to be considered in the management of sustainable practices in crops. These concepts are the specific type of plant in question and the irrigation system being used. When this expert was requested to supply more information on which factors related to the irrigation system he considers in crop management, he mentioned that often, the way through which the water comes to their crop is very important. Also, E1 indicated that the features of the source of irrigation were the most important factor for crops, in that the production success is dependent on the source of water (e.g., from pool, river, desalination plant, from the rain, etc.). When this expert was asked to specify the features of these concepts, he provided the following information, grouped by the concepts underlying the information on concepts as described above:

• Crops: size.
• Plants: variety, age.
• Irrigation system: cost, technology.
• Irrigation pipeline: length.
• Surface irrigation: covered area.

List 1. Concepts provided by expert E1

By proceeding similarly with Experts E2 and E3, List 2 and List 3 below present the characteristics of each of the concepts according to experts E2 and E3, respectively. In this respect, in the third interview, E3 pointed out that in irrigation systems, a very important issue that should be considered is the reliability of such systems. He also stated that among irrigation systems, there are two main categories, namely, surface irrigation and drip irrigation.

• Crops: size.
• Plants: variety, annual production.
• Land: pH level
• Water supply: cost, technology.
• Irrigation pipeline: flow.
• Sprinkler irrigation: capacity.

List 2. Concepts provided by expert E2

• Crops: size
• Plants: variety.
• Land: pH level.
• Irrigation system: cost, technology, reliability.
• Irrigation pipeline: flow.
• Drip irrigation: automation, wet bulb.

List 3. Concepts provided by expert E3

6.2. Views Modelling

Following the information provided by the three experts (i.e., E1, E2, and E3), it can be said that they have their corresponding views on some of the relevant concepts for managing a crop in a sustainable fashion. Such views can be represented through three input ontologies, written O₁, O₂, and O_3, respectively, as illustrated in Figure 1, Figure 2 and Figure 3, respectively. In these figures, these ontologies have been represented through graphs, where the nodes contain information on concepts (written in capital letters), and their corresponding attributes (written in small letters) have been allocated into boxes. The arrows in such graphs are of two types, depending on whether they represent taxonomic or partonomic relationships.

6.3. Views Evaluation

As can be seen in these figures, some concepts (i.e., ‘IRRIGATION SYSTEM’ and ‘WATER SUPPLY SYSTEM’) are semantically equivalent (since they have the same conceptual context and share the same attributes), and therefore they need to be integrated carefully. Also, it can be noticed that the ‘PLANT’ concept is present in both input ontologies. Therefore, this concept has more support than the concepts of ‘IRRIGATION SYSTEM’ or ‘WATER SUPPLY SYSTEM,’ which are only present in one of the input ontologies.

For the example introduced above, following the terminology introduced in previous sections, the set of concepts C of all the input ontologies is {CROP, PLANT, LAND, IRRIGATION SYSTEM/WATER SUPPLY SYSTEM, IRRIGATION PIPELINE, SURFACE IRRIGATION, SPRINKLER IRRIGATION, DRIP IRRIGATION} = {c₁, c₂,…, c₈} can be defined. In addition, the set of attributes AT of all input ontologies is {crop.size, plant.variety, plant.age, irrigation system.cost, irrigation system.technology, irrigation pipeline.length, irrigation system.reliability, surface irrigation.covered area, land.ph, plant.annual production, irrigation pipeline.flow, sprinkler irrigation.capacity, drip irrigation.automation, drip irrigation.wet bulb} = {a₁, a₂,…, a₁₄}, where the notation a_i = ‘x.y’ stands for ‘the attribute y linked to the concept x in MHRD₁, MHRD₂ or MHRD_3′.

6.4. Relational Support for Each View

The following semantic relationships evaluation matrices can be obtained upon applying the above method for evaluating semantic relationships to the two input ontologies, namely, O₁, O₂, and O₃, as shown in

Table 1. Taxonomic relationships evaluation matrix for the O₁ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	0	0	0	0	0	0	0	0
c3	0	0	0	0	0	0	0	0
c4	0	0	0	0	0	0	0	0
c5	0	0	0	0	0	0	0	0
c6	0	0	0	1	0	0	0	0
c7	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0

,

Table 2. Taxonomic relationships evaluation matrix for the O₂ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	0	0	0	0	0	0	0	0
c3	0	0	0	0	0	0	0	0
c4	0	0	0	0	0	0	0	0
c5	0	0	0	0	0	0	0	0
c6	0	0	0	0	0	0	0	0
c7	0	0	0	1	0	0	0	0
c8	0	0	0	0	0	0	0	0

,

Table 3. Taxonomic relationships evaluation matrix for the O₃ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	0	0	0	0	0	0	0	0
c3	0	0	0	0	0	0	0	0
c4	0	0	0	0	0	0	0	0
c5	0	0	0	0	0	0	0	0
c6	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0
c8	0	0	0	1	0	0	0	0

,

Table 4. Partonomic relationships evaluation matrix for the O₁ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	1	0	0	0	0	0	0	0
c3	0	0	0	0	0	0	0	0
c4	1	0	0	0	0	0	0	0
c5	0	0	0	1	0	0	0	0
c6	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0

,

Table 5. Partonomic relationships evaluation matrix for the O₂ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	1	0	0	0	0	0	0	0
c3	1	0	0	0	0	0	0	0
c4	1	0	0	0	0	0	0	0
c5	0	0	0	1	0	0	0	0
c6	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0

and

Table 6. Partonomic relationships evaluation matrix for the O₃ input ontology.

	c1	c2	c3	c4	c5	c6	c7	c8
c1	0	0	0	0	0	0	0	0
c2	1	0	0	0	0	0	0	0
c3	1	0	0	0	0	0	0	0
c4	1	0	0	0	0	0	0	0
c5	0	0	0	1	0	0	0	0
c6	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0

:

By applying Equations (1) and (2), the values of the taxonomic and the partonomic support for the ontologies O₁, O₂, and O₃ are TS¹ = 1; TS² = 1; TS³ = 1; PS¹ = 6; PS² = 7; PS³ = 7. Hence, following Equation (3), the values of the relational support for these views (ontologies) are RS¹ = 7, RS² = 8, and RS³ = 8, respectively.

Conceptual Support for Each View

The following concept evaluation matrices can be obtained by applying the above method for evaluating concepts to the three input ontologies, namely, O₁, O₂, and O_3, as shown in

Table 7. Concept evaluation matrix for the O₁ input ontology.

	a1	a2	a3	a4	a5	a6	a7	a8	a9	a10	a11	a12	a13	a14
c1	1	0	0	0	0	0	0	0	0	0	0	0	0	0
c2	0	1	1	0	0	0	0	0	0	0	0	0	0	0
c3	0	0	0	0	0	0	0	0	0	0	0	0	0	0
c4	0	0	0	1	1	0	0	0	0	0	0	0	0	0
c5	0	0	0	0	0	1	0	0	0	0	0	0	0	0
c6	0	0	0	1	0	0	1	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0	0	0	0	0	0	0

,

Table 8. Concept evaluation matrix for the O₂ input ontology.

	a1	a2	a3	a4	a5	a6	a7	a8	a9	a10	a11	a12	a13	a14
c1	1	0	0	0	0	0	0	0	0	0	0	0	0	0
c2	0	0	1	0	0	0	0	0	1	0	0	0	0	0
c3	0	0	0	0	0	0	0	1	0	0	0	0	0	0
c4	0	0	0	1	1	0	0	0	0	0	0	0	0	0
c5	0	0	0	0	0	0	0	0	0	1	0	0	0	0
c6	0	0	0	1	0	0	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0	0	0	1	0	0	0
c8	0	0	0	0	0	0	0	0	0	0	0	0	0	0

and

Table 9. Concept evaluation matrix for the O₃ input ontology.

	a1	a2	a3	a4	a5	a6	a7	a8	a9	a10	a11	a12	a13	a14
c1	1	0	0	0	0	0	0	0	0	0	0	0	0	0
c2	0	0	1	0	0	0	0	0	1	0	0	0	0	0
c3	0	0	0	0	0	0	0	1	0	0	0	0	0	0
c4	0	0	0	1	1	0	0	0	0	0	0	1	0	0
c5	0	0	0	0	0	0	0	0	0	1	0	0	0	0
c6	0	0	0	1	0	0	0	0	0	0	0	0	0	0
c7	0	0	0	0	0	0	0	0	0	0	0	0	0	0
c8	0	0	0	0	0	0	0	0	0	0	0	0	1	1

:

By applying Equations (4) and (5), the values of the total conceptual support for the views are TCS¹ = 11, TCS² = 12, and TCS³ = 14, respectively.

6.5. Overall Knowledge Evaluation Process

Applying the method proposed in Section 4 for the whole evaluation process of all the views, an (integrated) ontology is obtained, as represented in Figure 4.

This schema has been implemented into OWL language using Protégé—see Figure 5 below.

By applying Equations (6)–(11) and proceeding similarly to before in this section for calculating RSint y TCSint for each view,

Table 10. Knowledge evaluation of each view.

Evaluation Parameter	View 1	% View 1	View 2	% View 2	View 3	% View 3
Relational evaluation
RS	7	47%	8	53%	8	53%
RSint	8	44%	10	56%	10	56%
Concept evaluation
TCS	11	48%	12	52%	14	60%
TCSint	13	46%	15	54%	17	62%

summarises the total and relative evaluation of the knowledge contributed by each view.

6.6. Results Analysis

From the contents of Table 10 above, it follows that the integrated view has the highest support in terms of both relational and concept evaluation parameters, as was expected from its very definition. At the same time, that table allows us to assess in relative terms the contribution of each view amongst themselves as well as in relation to the integrated ontology.

7. Discussion

The approach presented here is based on the assumption that the systematization of knowledge is beneficial for knowledge management in organizations. Thus, such systematization contributes to the deployment of competencies related to critical thinking. In turn, such competencies are essential to provide an adaptive response to organizational information technology-driven change, which is more and more necessary in environments with a low digitalization level as the agriculture domain. In addition, although a high number of ontologies are currently publicly available, these are hard to use/exploit for most users as these usually require a high literacy level in ontological engineering or information technologies. So, there is a practical need to standardize practices for the use of ontologies to provide methods that assist individuals and groups, who are not often experts in ontological engineering, in the exploitation of ontologies as sources of sectorial or organization innovation.

In this study, an approach for the exploration and integration of domain knowledge schemas that impact the systematization of ontological knowledge has been proposed. This can be useful in the initial stages of devising IT-driven organizational solutions.

In relation to other related approaches, the ontology learning approach is the one that can have more links with our approach. However, that approach often needs specialized lexical resources, which may be lacking in some application domains. Also, such an approach typically involves large sets of linguistic rules, lexical-syntactic patterns, etc. All these sets have to be pre-defined since well-written texts are required as inputs to find patterns with a certain guarantee of reliability. The approach we propose in this study focuses on the fusion of manually built ontologies [29]. This approach relies only on the expert’s capacity for expressing her/his knowledge in an intuitive manner, that is, by selecting concepts, expressing attributes of each of them, and defining the (taxonomic and partonomic) relations between such concepts. Furthermore, while the knowledge integration approach described in this study ensures the internal coherence of each ontology from each expert, it could be applied to integrate ontologies obtained through ontology learning methods once such ontologies are validated.

8. Conclusions

AI development in recent years has led to the research of knowledge management tools for multi-user environments, among many other AI applications. In the knowledge management field, the construction of ontologies as knowledge repositories using various sources requires a means of evaluation of all: the input ontologies and the integration process on the output ontology. The results obtained from the evaluations serve as guides to measure the quality of the repository.

The research presented in this article describes an approach to evaluate the knowledge provided from several knowledge sources automatically. To carry out such evaluation, an ontology integration process takes place. This approach takes advantage of the structure used to represent the ontologies and measures the knowledge contained in them by means of comparisons between all the ontologies. The evaluation obtained provides a relative rating. However, it can be used for different purposes, including improving the integration process so that it takes previous ratings into account when they must be resolved.

There are three very important aspects that will be addressed in future research studies. Firstly, to apply this approach, it is necessary to have very reliable sources. The rationale for this is that a basic principle underlying this approach is that all input ontologies represent valid knowledge. This leads to the reliability of the source issue. Secondly, the results obtained provide relative measures of knowledge evaluation in the same application domain, so it is not useful for integrations in other domains, where there could be no, or very few, shared concepts.

Finally, the approach presented in this study is valid for knowledge models involving (solely) two popular conceptual relationships, namely, taxonomic and partonomic relationships. We plan to extend our approach to other conceptual relationships, such as chronological and topological ones.