*Article* **Scarce Data in Intelligent Technical Systems: Causes, Characteristics, and Implications**

**Christoph-Alexander Holst \* and Volker Lohweg**

inIT—Institute Industrial IT, Technische Hochschule Ostwestfalen-Lippe, Campusallee 6, 32657 Lemgo, Germany **\*** Correspondence: christoph-alexander.holst@th-owl.de

**Abstract:** Technical systems generate an increasing amount of data as integrated sensors become more available. Even so, data are still often scarce because of technical limitations of sensors, an expensive labelling process, or rare concepts, such as machine faults, which are hard to capture. Data scarcity leads to incomplete information about a concept of interest. This contribution details causes and effects of scarce data in technical systems. To this end, a typology is introduced which defines different types of incompleteness. Based on this, machine learning and information fusion methods are presented and discussed that are specifically designed to deal with scarce data. The paper closes with a motivation and a call for further research efforts into a combination of machine learning and information fusion.

**Keywords:** scarce data; machine learning; information fusion

### **1. Introduction**

In modern industrial applications, data are generated in increasing amounts due to better availability, accessibility, and cost-effectiveness of technical sensors. In fact, modern methods for data analysis often assume the availability of big data. Many machine learning methods not only assume big data but also require it. This is also the case in many industrial use-cases [1], such as predictive maintenance [2] or machine fault diagnosis [3].

However, the reality—also in industrial applications—is that data is not always available in sufficient quantities. It may also be that data is recorded in large quantities, but the data are repetitive containing the same information repeatedly. The presence of only a few data sources or data points is summarised by the term scarce data or data scarcity [4]. The goal in dealing with scarce data must nevertheless be to obtain as much information and as much knowledge as possible from the little data that is available. Causes of scarce data are, for example, measured variables that are difficult to collect, costly measurement methods, or a low number of measurement objects that need to be collected. However, an explicit definition and detailed specification of different types of data scarcity is rare in the current literature. For example, Wang et al. [5] define two types: scarce data due to a limited number of samples and sparse data (e.g., sparse time series or matrices).

The problem of scarce data is recognised in the state of the art of machine learning [6,7]. Approaches to addressing data scarcity include inherently data-efficient algorithms and methods for enabling data-hungry algorithms to be used on scarce data—as identified recently by Adadi [8] in their survey on data-efficient algorithms. Regarding the former, it is generally considered that low-complexity models, such as decision trees or linear regression, require less data than high-complexity models, such as deep neural networks. Regarding the second, various methods have been devised and proposed for highly complex models that are intended to be applicable to scarce data, such as data augmentation [9] or transfer learning [10].

In current machine learning approaches, data scarcity is often only implicitly taken into account by extending and adapting existing algorithms [11–13]. Another research area

**Citation:** Holst, C.-A.; Lohweg, V. Scarce Data in Intelligent Technical Systems: Causes, Characteristics, and Implications. *Sci* **2022**, *4*, 49. https:// doi.org/10.3390/sci4040049

Academic Editor: Johannes Winter

Received: 31 October 2022 Accepted: 30 November 2022 Published: 12 December 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

which focuses on data scarcity is information fusion. Information fusion has developed independently from machine learning. Fusion methods specifically expect data to be uncertain due to scarcity (as well as other data imperfections) [14]. In case of multiple uncertain information sources, e.g., sensors, experts, or machine learning models, fusion aims to create a single output with increased certainty. To achieve this, uncertainties based on data scarcity are explicitly modelled, quantified, and considered.

This article addresses scarce data due to its frequency of occurrence in industrial applications and the implications for data processing methods. The aim of this article is (i) to more specifically detail scarce data in its causes and subtypes, and (ii) to provide an overview of both machine learning and information fusion methods that address scarce data. Towards this end, the following contributions are presented in this article:


The further structure of this paper follows these contributions.

#### **2. A Typology of Scarce Data**

Scarce or incomplete data is a form of data imperfection that affects the ability of algorithms, machine-learned models, or human engineers to extract information and induce knowledge. Incomplete data represents uncertainty in the data, but also leads to uncertainty in the process of induction. In this sense, it is closely related to uncertainties especially to the notion of epistemic uncertainty. It follows an introduction of epistemic uncertainty together with its counterpart aleatoric uncertainty.

**Definition 1** (Aleatoric Uncertainty)**.** *Aleatoric uncertainty refers to the inherent variation of an object, concept, process, or phenomenon. It is random and non-deterministic in nature [15]. Even if data is complete and the underlying process is completely understood, the outcome of this process cannot be predicted with absolute certainty [16,17]. Consequently, gathering more data—or adding new data or information sources—does not reduce aleatoric uncertainty. Take, for example, a classification problem. In such a problem, aleatoric uncertainty is the intra-class distance or variance.*

**Definition 2** (Epistemic Uncertainty)**.** *In contrast, epistemic uncertainty results from a lack of knowledge about a phenomenon. This lack is caused by incomplete—not available—or inconsistent information. Epistemic uncertainty is, in principle, reducible by gathering additional information. In practice, reducing epistemic uncertainty is often not possible, feasible, or valuable [15–17]. In technical or industrial systems, this is due to one or more of the following reasons.*


Scarce Data is, therefore, itself a form of epistemic uncertainty. Handling epistemic uncertainty is one of the major challenges in data analysis. This is also recognised in machine learning research very recently [19–21]. To overcome this challenge, it is crucial to understand the various types of scarcity, their causes, and their interactions.

Several taxonomies and typologies have been proposed in the literature to categorise and relate data or information imperfections, uncertainties, and quality [15,22–26]. An overview of taxonomies and typologies is given by Jousselme et al. [27], which includes some of the works just mentioned. An overview of data quality in databases provided by de Almeida et al. [28] is also of interest. The authors identify data completeness as a major data quality issue. However, work limited to databases will not be discussed further here. Instead, we summarise taxonomies and typologies which focus on or at least address incompleteness, missing data, or missing information in Table 1. Most of the works referenced in Table 1 rely on the term incompleteness which is used interchangeably with scarcity in the table.

**Table 1.** Taxonomies of uncertainty, imperfection, ignorance, and quality which address the topic of data or information incompleteness (in the sense of missing data or information, i.e., scarcity). Incompleteness is recognised as the main concept of imperfection throughout the referenced works. However, a categorisation of the various kinds of missing data or information is not carried out.


This survey shows that incompleteness is recognised broadly as a type of data imperfection, a kind of uncertainty, and a source of ignorance. In nearly all referenced taxonomies, incompleteness is not further subcategorised. A detailed look into various forms of incompleteness and missing data is not provided.

In the following, we present a more detailed typology of incompleteness as a form of imperfection (see Figure 1) based on Smets' [23] taxonomy. This typology perceives incompleteness as a form of data imperfection along with imprecision and inconsistency.

The proposed typology subdivides incompleteness into six categories.

**Undersampled:** Data points always represent only a sample of a distribution or the characteristics of a phenomenon. Sensors only provide a window into the real world. Their observations are a fragmented representation. A phenomenon is undersampled if there is insufficient data available to make sound and significant findings about its characteristics. Due to undersampled data, information remains partially hidden. The aleatoric uncertainty of a phenomenon can only be described inadequately. Figure 2 illustrates two cases of undersampling using a scatter plot in a one-dimensional and a two-dimensional feature space.

As a consequence, training with machine learning methods does not lead to satisfactory results. The generalisation ability of the trained models is questionable at best. Probabilistic methods rely on the availability of statistically sound data or knowledge about prior distributions [30]. Kalman filters, for example, assume zero-mean Gaussian distributed data [31]. In the case of undersampled data, this knowledge cannot be derived from the data itself. Few data points also increase the risk of finding spurious correlations in the

data [32]—especially when many features or data sources are involved. Another threat of undersampled data is that machine-learned models tend to easily overfit [33].

**Figure 1.** A typology of data and information imperfection with a detailed subcategorisation of incompleteness. The typology is based on the work of Smets' [23]. It recognises incompleteness as one of three major sources of imperfection – besides inconsistency and imprecision. Imprecision captures deficiencies that prevent unambiguous statements from being made based on individual data points. Inconsistency refers to situations in which a piece of information is contradictory to existing knowledge or with other information sources. Incompleteness is lacking, absent, or non-complete data and information.

**Figure 2.** Two examples showcasing undersampled data: (**a**) an ill-represented one-dimensional distribution and (**b**) an ill-represented two-dimensional distribution. A two-dimensional scatter plot showcasing undersampled data. The plots show the distributions of phenomena in feature space (red). The distributions are unknown and represent the aleatoric uncertainty of the phenomena. In both examples, the sampled data points (blue) are insufficient to draw conclusions about the distributions. The missing data points are a form of epistemic uncertainty.

**Non-representative:** Data or information is non-representative when only certain parts or subconcepts of a phenomenon are observable or represented in the data. Other subconcepts may be very well represented. Take, for example, a bi-modal distribution of a phenomenon's characteristics. One of the modes may be very well sampled, whereas the other is absent in the data. In extreme cases, complete concepts are missing. In less extreme cases, subconcepts may merely be undersampled. Data in which subconcepts are undersampled are often also referred to as biased. The observation of industrial machines (condition monitoring) often produces non-representative data. Machines are specifically built to run as smoothly and faultlessly as possible. Consequently, data obtained during normal operation is often available in abundance. In contrast, data on fault states or unusual operating conditions are often rare. Reducing this kind of epistemic uncertainty is difficult in practice since running a machine in fault states is either costly or infeasible. Figure 3 shows the multi-modal and condition monitoring examples as a form of nonrepresentative data.

**Figure 3.** Two cases of non-representative data. In (**a**) a bi-modal distribution is shown (red, unknown). One mode is very-well sampled; the second is missing in the data. Plot (**b**) shows a multi-class classification problem, in which certain classes are missing in the data. Such missing data can, for example, be due to unseen fault states of a machine.

**Low-dimensional:** Real-world processes can only be observed by a finite number of sensors. Data may be incomplete due to missing data sources – in this case, the data space is too low-dimensional. A low-dimensional space may be insufficient to handle the aleatoric uncertainty of the phenomenon at hand. Figure 4 illustrates a case where data is scarce with respect to the number of available sources.

**Figure 4.** A classification example in which the addition of a new data source allows us to distinguish two classes perfectly (**b**). In the two-dimensional space shown in (**a**), the aleatoric uncertainty prevents a clear separation of classes. Low-dimensional data is still a form of epistemic uncertainty as it is unknown how the class distributions evolve with new sources.

This epistemic uncertainty is reducible by adding new sources although it is crucial to carefully select new sources that are meaningful.

**Sparse:** Sparse data is caused by sensors or data sources which do not provide data continuously. For example, data is missing over certain time periods or data from different sources cannot be synchronised with each other. Missing data can be caused by defective sensors. This leads to data gaps. Take, for instance, data which is organised in a twodimensional table. Its rows represent data instances and its columns are data sources. Sparse data is then characterised by missing entries throughout this table (think of a sparse matrix).

**Without Context:** Context is needed to extract information and knowledge from data. Roughly speaking, context is itself information that surrounds the phenomenon of interest

and its data-generating process [34]. Context aids in understanding the phenomenon. It can be provided by domain knowledge. Examples of context are labels in classification applications or maps in applications of autonomous driving. Context, and specifically labels, are often costly to produce or provide. If in large datasets only a fraction of data instances are labelled, then the problem relates to undersampled data.

**Drifting/Shifting:** The effectiveness of machine learning algorithms relies heavily on the assumption that training and test data are taken from the same or at least similar distributions [35]. In reality, concepts and phenomena often drift in their distribution over time, e.g., data clusters move through feature space. As a consequence, models which have learned from training data are outdated as soon as significant drift occurs. Adaptation or retraining is usually necessary. Because the drifting data distribution over time is not known, drift is categorised as a form of incomplete information.

These six types of incomplete data have different causes, characteristics, and effects on machine learners or other data-processing algorithms. To overcome the associated challenges, algorithms have to specifically consider each type. This has to be kept in mind in designing data analyses.

#### **3. An Overview of Methods for Working with Scarce Data**

The challenges associated with scarce data have been known and intensively discussed in the research community for some time. Various methods and approaches exist that can deal with scarce data. In the following, we discuss methods of transfer learning, data augmentation, and information fusion that act in very different ways on scarce data. This survey is closely related to the work of Adadi [8], who studied machine learning methods for scarce data. We extend this survey with an insight into information fusion methods. We mainly focus on the problem of undersampled data and non-representative data. In the ensuing discussion, we motivate further research efforts on the combination of machine learning and information fusion methods.

#### *3.1. Transfer Learning*

Transfer learning is a machine learning method in which a model that has been trained in one domain is reused in a related domain. The model is not completely retrained but only adapted by post-training [36,37]. The purpose of transfer learning is to be able to use machine learners even with scarce data. Transfer learning requires a model which has learned as many basic concepts of a domain as possible. For example, these may be geometric shapes in image data, basic patterns such as a Mexican hat in time series, or basic pronunciations or sounds in human speech. Once basic concepts are known to a model, few training examples are required to adapt to a new domain—even zero-shot learning is possible under specific circumstances and depending on the application [38]. Most commonly neural networks and convolutional neural networks are used to transfer learning, but other machine learning methods have been adapted for transfer learning, such as Markov logic networks [39] and Bayesian networks [40]. Transfer learning has been applied to many domains. A survey on machine diagnostics in industrial applications is provided by Yao et al. [41].

Transfer learning comes with several drawbacks and pitfalls. Because a source model is required to know as many concepts as possible, large datasets and resources are necessary to train the source model in the first place. Such a model needs to be trained on a general dataset, which is at best not domain-specific. Secondly, the target domain is still characterised by scarce data. Therefore, some risks remain even if the transfer is learned. Models are still at risk to overfit or detect spurious correlations [37,42]. Finally, performance is affected negatively if the source and target domain do not cover the same concepts or focus on different concepts. This is referred to as negative transfer [43,44]. For example, recent studies have shown that models trained on the ImageNet (https://www.image-net.org/, accessed on 9 November 2022) dataset favour texture over shape [45]. Transferring these models into domains in which textural information is less important and objects are mostly

defined by shape—such as object recognition of machinery parts, screws, or nuts [46–48] will not result in optimal performance.

#### *3.2. Data Augmentation*

Data augmentation refers to methods that artificially increase the amount of available data. The aim is to facilitate machine learners to train on even small amounts of training data. Augmentation creates slightly modified copies of existing data or completely new synthetic data [49]. Data augmentation techniques have been successfully applied to image [49,50], text and natural language [51–53], and time series data [54]. Augmentation has a regularising effect on machine learning models, helps to reduce overfitting, and can improve the generalisability of models [50]. Industrial applications of data augmentation are, for example, given by Dekhtiar et al. [46], Židek et al. [47,48], Parente et al. [55], or Shi et al. [56].

Additional data instances are usually created by applying various transformations to data. In image datasets, these are, e.g., rotations, scaling, cropping, colour transformations, distortions, or erasing random parts of an image [50]. In natural language, parts of a text are randomly swapped, inserted, deleted, or replaced synonymously [52]. Time series transformations take place either in the time or frequency domain. These include cropping, slicing, jittering, or warping among others [54]. These transformations aim to teach a machine learner which information is important for defining a concept. For example, additional rotated images teach that rotation is not important to a concept or class. It is still the same class. By replacing the background in images, models learn to focus on objects in the foreground. Thus, augmenting data by selected transformations allows us to integrate expert knowledge into the machine learning process. However, it is crucial to apply the right transformation for a particular application in order for the data augmentation to be useful. Often data augmentation seems to be carried out in an "ad-hoc manner with little understanding of the underlying theoretical principles"—as stated by Dao et al. [57].

Another approach to data augmentation is to create additional data automatically by generative models such as generative adversarial networks [58]. The expectation is that expert's knowledge will no longer be necessary or will be at least less crucial. A major drawback of generative augmentation is that it is susceptible to perpetrate bias in data [59].

With all these methods, there is a risk of losing important information in the augmentation process. Information may be discarded, e.g., by cropping an image, or may be overwritten by erasing parts of a text randomly [50,52]. It follows that patterns or classes are not correctly preserved. The data instance and its label may then no longer match (The label is not preserved). This problem is aggravated if small details in a data instance are crucial for a concept. Slight changes to the original data may then already be enough to distort or destroy concepts.

#### *3.3. Information Fusion*

Scarce data and epistemic uncertainty are intensively addressed in the research field of information fusion. Information fusion has been researched since the midst of the 20th century as a distinct field in parallel to machine learning [60,61]. While information fusion has similar goals and applications as machine learning—such as classification, regression, detection, or recognition—its focus differs. The aim of information fusion methods is to extract and condense high-quality information from a set of low-quality data sources [62]. Information fusion explicitly assumes that sources provide incomplete or imprecise information. The task of information fusion is to make the best of what imperfect data is available [14]. Fusion methods include a strong focus on modelling uncertain, error-prone, imprecise, and vague information [63]. For instance, fuzzy information is modelled via fuzzy set theory. Missing information or ignorance are modelled via evidence theories, such as the Dempster-Shafer theory. Fusion methods address scarce data with possibility theory. In direct comparison to probability theory, possibility theory is characterised by the fact that incomplete information is represented qualitatively [64]. The possibility theory requires

a smaller amount of data but is less expressive in the final analysis [63,64]. Established methods of machine learning, on the other hand, rarely model missing information or epistemic uncertainty explicitly. Instead, they rely on a quantitative evaluation of data. In the following, we provide an overview of the mathematical tools fusion relies on, that is, the Dempster-Shafer theory, the fuzzy set theory, and the possibility theory.

#### 3.3.1. Dempster-Shafer Theory

The Dempster-Shafer theory of evidence (DST) has been proposed by Shafer [65] on the foundation of Dempster's works on a framework for expressing upper and lower probabilities [66]. In the DST, available evidence forms the basis to express a degree of belief in a proposition that quantifies incomplete knowledge [67]. In this basic sense, it is comparable to Bayesian probability theory. It is motivated by the fact that probability theory is not able to distinguish between ignorance (epistemic uncertainty) and well-informed uncertainty (aleatoric uncertainty) natively [65].

Probability theory (ProbT) operates on a frame of discernment Ω which includes all given propositions or hypotheses *X* as singletons, i.e., Ω = {*X*1, *X*2, . . . , *xn*}. Each proposition is given a probability <sup>0</sup> ≤ *<sup>p</sup>*(*X*) ≤ <sup>1</sup> to be true with the restriction of <sup>∑</sup>*X*∈<sup>Ω</sup> *<sup>p</sup>*(*X*) = 1. In the case of total ignorance, one tends to distribute probabilities uniformly over Ω but this is arbitrary. A uniform distribution is not distinguishable from a situation in which it is known that propositions are actually equally likely. DST allows us to assign evidence to sets of combined propositions. It operates on the power set of the frame of discernment, i.e., P(Ω) = {∅, *X*1, *X*2, . . . , {*X*1, *X*2} . . . , Ω}. By assigning evidence *m* to combined propositions (e.g., {*X*1, *X*2}), a state of incomplete knowledge is expressed. In case of {*X*1, *X*2}, it is unclear whether evidence favours *X*<sup>1</sup> or *X*2. Belief in a proposition is then obtained by Bel(*X*) = <sup>∑</sup>*A*⊆*<sup>X</sup> <sup>m</sup>*(*A*). The usage of the power set allows DST to handle incomplete knowledge due to scarce data better and more properly than probability theory. An example of the difference between ProbT and DST is given in Figure 5.

**Figure 5.** Probability theory versus Dempster-Shafer's theory in a condition monitoring example. The basic propositions are *h*: the monitored object is healthy and *f*<sup>1</sup> , *f*2: the object is in one of two fault states. The distribution modelled with ProbT (**a**) is ambiguous since it cannot distinguish between ignorance (epistemic uncertainty) and well-informed uncertainty (aleatoric uncertainty). Using DST (**b**), it turns out that the expert or model is indeed partly ignorant. This is expressed by *m*({ *f*<sup>1</sup> , *f*2}) = 0.4 (a fault occurred but it is unknown which one) and by *m*(Ω) = 0.2 (nothing is known).

DST is designed with a fusion of independent multiple sources in mind. Having multiple partially ignorant and uncertain sources, the aim is to get to a single estimation with reduced ignorance and increased certainty. To achieve this, most fusion rules involve a reinforcement effect. If, for example, *m*1(*X*) = *m*2(*X*), then the fused mass *m*12(*X*) > *m*1(*X*). Several fusion rules have been proposed over the years, for example, Dempster's rule of combination [66,68], Yager's rule [69], Campos' rule [70], or the Balanced Two-Layer Conflict Solving rule [61], to name just a few.

DST fusion achieves that—if a group of sensors, experts, or machine learning models is uncertain in their assessments because of scarce data—to increase certainty. A popular approach in machine learning is to apply ensemble learners [71]. In ensemble learning, multiple weak learners are trained simultaneously. Their outputs are fused into a single

one. An example of an ensemble is random forests. Although this seems to be an exemplary area of application for DST fusion, most ensemble learners rely on majority votings or averaging functions [72–74]. This motivates further research efforts in combining DST and machine learning methods as a way to handle the effects of scarce data.

#### 3.3.2. Fuzzy Set Theory

Fuzzy set theory (FST) was proposed by Zadeh [75] motivated by the intrinsic vague nature of language. The fuzzy set theory facilitates the modelling of imprecise and vague information (cf. Figure 1). Although FST is not focused on incomplete information, it brings benefits when it comes to scarce data. Zadeh introduces sets with vague boundaries in contrast to crisp sets known from probability theory or Dempster-Shafer theory. In a crisp set, an element either belongs to this set or not. Its membership function *µ* is a mapping of all elements belonging to the frame of discernment Ω to a boolean membership *µ* : Ω → {0, 1}. Fuzzy sets allow degrees of memberships, that is, *µ* : Ω → [0, 1].

The inherent vagueness of fuzzy membership functions can be exploited to learn class distributions from only a few data instances [76]. If class borders are only needed to be modelled imprecisely and vaguely, then less effort has to be put into a training process than learning precise class borders. The fuzzy membership of a data instance is then interpreted as the uncertainty of the classification model. This blurring of class borders results in weaker models with the upside of less data demand.

An approach for this kind of classification is fuzzy pattern classifiers (FPC). Fuzzy pattern classifiers have been introduced and advanced by Bocklisch [77,78]. An FPC learns a unimodal potential function for each data source. This function serves as a membership function. Each membership function is a weak classifier in itself. Seen as a group, the membership functions are similar to an ensemble. They output each a gradual estimate for the predicted membership. This allows to apply fuzzy aggregation rules to fuse the outputs into a singular class membership (see for example previous works by Holst and Lohweg [79–82]).

Unimodal potential functions were proposed by Aizerman et al. [83] as a pattern recognition tool. It was only later that they were applied as membership functions for fuzzy sets. Unimodal potential functions are used to model the distribution of compact and convex classes. Lohweg et al. [84] described a resource-efficient variant optimised for limited hardware:

$$\mu(\mathbf{x}) = \begin{cases} 2^{-d(\mathbf{x}, \mathbf{p}\_{\parallel})} \text{ if } \mathbf{x} \le \overline{\mathbf{x}}, \\ 2^{-d(\mathbf{x}, \mathbf{p}\_{\parallel})} \text{ if } \mathbf{x} > \overline{\mathbf{x}}, \end{cases}$$

$$\text{with } d(\mathbf{x}, \mathbf{p}\_{\parallel}) = \left(\frac{|\mathbf{x} - \overline{\mathbf{x}}|}{C\_{\text{l}}}\right)^{D\_{\text{l}}},$$

$$d(\mathbf{x}, \mathbf{p}\_{\mathbf{r}}) = \left(\frac{|\mathbf{x} - \overline{\mathbf{x}}|}{C\_{\text{r}}}\right)^{D\_{\text{r}}}, \text{and}$$

*x* a data instance (measurement value).

The unimodal potential function has several advantages for the use of scarce data. The function is parameterizable with few parameters. The number of parameters scales with data sources linearly. The parameters are relatively easy to train in data. Training methods can be found in [76,81,84]. The parameters are intuitive to interpret. Therefore, expert knowledge can be integrated easily. On the other hand, FPCs require unimodal and convex data distributions. In this regard, Hempel [85] proposed a multi-modal FPC, although his approach requires more training data in general.

#### 3.3.3. Possibility Theory

The possibility theory (PosT) was introduced by Zadeh in 1978 as an extension of fuzzy set theory [86]. It is designed as a counterpart to probability theory because of its limited ability to represent epistemic uncertainty.

Possibility theory is based on possibility distributions *π*—similar to probability distributions *p*. The possibility 0 ≤ *π*(*x*) ≤ 1 conveys how plausible the event *x* is. A value *π*(*x*) = 1 means completely plausible; *π*(*x*) = 0 completely implausible. At least one *x* is required to be fully plausible (normality requirement). But more than one *x* can be fully plausible. This leads to <sup>∑</sup>*x*∈<sup>Ω</sup> *<sup>π</sup>*(*x*) <sup>≥</sup> 1 or <sup>R</sup> *x*∈Ω *π*(*x*) ≥ 1.

Possibility distributions are similarly defined as fuzzy membership functions, that is, *π*(*x*) = *µ*(*x*) [16]. This has the advantage that mathematical operations defined on fuzzy sets can be directly applied to possibility distributions [87]. Though it has to be verified first if this is sensible. Fuzzy membership functions and possibility distributions differ in interpretation. Let *x* be an alternative for an unknown value *v* and *A* be a fuzzy set. The *π*(*x*) expresses the possibility of *x* = *v* knowing that *x* ∈ *A*. In contrast, *µ*(*x*) expresses the degree of membership of *x* to *A* knowing that *x* = *v*.

Possibility distributions are also a less expressive and weaker model than probability distributions. Roughly speaking, it is easier to conclude that a proposition is possible rather than probable. Moreover, for a proposition to be probable it must preliminarily be possible. This leads to the probability/possibility consistency principle stating that *π*(*x*) ≥ *p*(*x*). In return, possibility distributions] require less effort – meaning training data or expert's knowledge – to construct [88]. They do not require statistically sound data because they model incomplete information qualitatively; whereas probability distributions model random phenomena quantitatively. This distinction is highlighted in Figure 6.

**Figure 6.** A continuous probability (**a**) and a continuous possibility distribution (**b**). The probability distribution models a random phenomenon quantitatively; the possibility of distribution of incomplete information qualitatively. The following applies: R *x*∈Ω *p*(*x*) = 1, R *x*∈Ω *π*(*x*) ≥ 1, and *π*(*x*) ≥ *p*(*x*).

This leads to the conclusion that possibility theory is well-suited to be used in the case of epistemic uncertainty and scarce data.

#### *3.4. Discussion*

Scarce data and epistemic uncertainty remain major challenges to machine learning and data analysis approaches. Missing information in data obstructs inherent aleatoric uncertainty.

In the area of machine learning, several techniques for coping with few training data have been thoroughly studied. Some of the most important are data augmentation, transfer learning, and interpretable models. While data augmentation and transfer learning focus on undersampled data mainly, interpretable models address also non-representative data. But only recently has epistemic uncertainty come into focus. Researchers have begun to explicitly define and quantify epistemic uncertainty of machine learning models [17,20,21,89].

In contrast, the research field of information fusion focuses on scarce data and epistemic uncertainty since its emergence in the mid-twentieth century. Fusion methods apply evidence theories such as DST, fuzzy set theory, and possibility theory to either quantify epistemic uncertainty or reduce its impact on performance.

However, combining fusion and machine learning methods is rare in the state of the art, although research need has been recognised recently [90–92]. Several works have been published that attempt to fill this open research topic. Among these are approaches which apply fusion techniques as a preprocessing step before machine learning [93,94]. These works focus on providing a machine learner with a more robust and condensed data

basis through prior fusion. They do not focus on incomplete information though. Further works devise classifiers based on the Dempster-Shafer theory [95–97]. Finally, machine learning in a possibilistic setting exists but is very rare. A small survey is conducted by Dubois et al. [98]. This leads to the conclusion that further research is needed to more successfully and formally address scarce data in machine data analysis.

#### **4. Conclusions**

Despite the increasing number of sensors and measuring devices, data is often scarce in industrial applications. The scarcity of data stems from limited sensor availability and functionality, limited observation periods, hidden concepts, and the inevitable blind ignorance of engineers. This leads to challenges in data analysis. In this paper, we have typologized missing data and information in more detail based on the works of Smets [23]. According to this new typology, incomplete data is categorised into (1) undersampled, (2) non-representative, (3) low-dimensional, (4) sparse, (5) without context, and (6) drifting data. Existing typologies did not or only insufficiently detail the category of incompleteness [15,22–26,29]. In this respect, we have filled an open gap in existing works.

This paper also explored machine learning and information fusion methods that deal with scarce data and incomplete information. As such, this paper complements Adadi's survey [8], which is limited to machine learning methods. Regarding machine learning, we focused on methods enabling data-hungry algorithms to be used on scarce data. Such methods are data augmentation [9] and transfer learning [10], among other methods. The idea behind transfer learning is to reuse and adapt models which have been trained on large, preferably general, datasets. However, efforts for training a source model are substantial and the risk of negative transfer has to be considered. Data augmentation creates new data points artificially by modifying existing ones. Data augmentation can reduce overfitting at the risk of destroying information.

Information fusion, on the other hand, relies on evidence theories, fuzzy set theory, and possibility theory to model, quantify, and cope with epistemic uncertainty [14]. This paper motivates and calls for further research efforts in combining fusion and machine learning approaches.

**Author Contributions:** C.-A.H. conceptualised the paper, conducted the research, and wrote the article. V.L. supervised the research activity and revised the article. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was partly funded by the Ministry of Economic Affairs, Innovation, Digitalisation and Energy of the State of North Rhine-Westphalia (MWIDE) within the project AI4ScaDa, grant number 005-2111-0016.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **Abbreviations**

The following abbreviations are used in this manuscript:


## **References**


**Siegfried Hackel 1,\*, Shanna Schönhals <sup>1</sup> , Lutz Doering <sup>1</sup> , Thomas Engel <sup>2</sup> and Reinhard Baumfalk <sup>3</sup>**


**\*** Correspondence: siegfried.hackel@ptb.de; Tel.: +49-531-592-1017

**Abstract:** This article depicts the role of the Digital Calibration Certificate (DCC) for an end-to-end digital quality infrastructure and as the basis for developments that are designated by the keyword "Industry 4.0". Furthermore, it describes the impact the DCC has on increasing productivity in the manufacturing of products and in global trade. The DCC project is international in its scope. Calibration certificates document the measurement capability of a measurement system. They do this independently and by providing traceability to measurement standards. Therefore, they do not only play an important role in the world of metrology, but they also make it possible for manufacturing and commercial enterprises to exchange measurement values reliably and correctly at the national and at the international level. Thus, a DCC concept is urgently needed for the end-to-end digitalization of industry for the era of Industry 4.0 and for Medicine 4.0. A DCC brings about important advantages for issuers and for users. The DCC leads to the stringent, end-to-end, traceable and process-oriented organization of manufacturing and trading. Digitalization is thus a key factor in the field of calibration as it enables significant improvements in product and process quality. The reason for this is that the transmission of errors will be prevented, and consequently, costs will be saved as the time needed for distributing and disseminating the DCCs and the respective calibration objects will be reduced. Furthermore, it will no longer be necessary for the test equipment administration staff to update the data manually, which is a time-consuming, tedious and error-prone process.

**Keywords:** D-SI; DCC; digital signature; calibration; Industry 4.0

#### **1. Introduction**

The analogue calibration certificate is currently still used and issued in paper form or as a closed PDF document. These closed documents hardly contribute to improving the production process of a company. The reason for this is that using analogue calibration data from a calibration in subsequent processes is very time-consuming and prone to errors as these data have to be converted once again into the digital formats of the specific manufacturing plant. The DCC [1] overcomes this disadvantage of its analogue counterpart. The objective was to create an internationally recognized DCC format which acts as an interface (exchange format) in the whole field of metrology, and especially in the field of machine-to-machine communication. On the basis of the DCC, we will be able to develop further exchange formats in the future. These might be in the field of legal metrology, for digital type examination certificates, for the **D**igital **T**win (DT) or for developments in many other fields—for example, the **D**igital **C**alibration **R**equest (DCR) format or, in the field of automated accident notification, in the format of the "**I**nternational **S**tandard **A**ccident **N**umber (ISAN): Linking data in accidents and emergencies." The DCC serves for the electronic storage, the authenticable and—if necessary—the encrypted and signed dissemination and the uniform interpretation of the calibration results. Due to the DCC schema to be applied, the DCC is thus both machine-readable and machine-interpretable

**Citation:** Hackel, S.; Schönhals, S.; Doering, L.; Engel, T.; Baumfalk, R. The Digital Calibration Certificate (DCC) for an End-to-End Digital Quality Infrastructure for Industry 4.0. *Sci* **2023**, *5*, 11. https://doi.org/ 10.3390/sci5010011

Academic Editors: Johannes Winter and Claus Jacob

Received: 28 November 2022 Revised: 19 January 2023 Accepted: 9 February 2023 Published: 6 March 2023

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

when using Good Practice (GP) DCCs. Further information on machine interpretability can be found in [2]. The DCC has been developed and agreed upon in a broad community and is constantly being developed further in order to achieve worldwide acceptance. The target groups are all those bodies, authorities and companies worldwide which require proof of the metrological traceability of their measurement results and use these results in modern manufacturing processes in the field of IIoT/Industry 4.0. These bodies, authorities and companies include the metrology institutes, designated institutes, national calibration centres, calibration laboratories and the many companies in industry that require traceable measurement results for their quality management systems.

According to the International Vocabulary of Metrology (VIM), the term "calibration" is defined as follows [3]:

"Operation that, under specified conditions, in a first step, establishes a relation between the quantity values with measurement uncertainties provided by measurement standards and corresponding indications with associated measurement uncertainties and, in a second step, uses this information to establish a relation for obtaining a measurement result from an indication" [4], p. 3.

The ISO IEC 17025 standard [5] describes the general requirements that are placed on qualified calibrations. Section 7.8 of this standard deals with "reporting on results." Although digitalization is increasing worldwide, calibration certificates are still mainly issued in analogue form, meaning that they are generated in paper form. This does not only lead to numerous problems, but it also prevents, in many fields, the complete digitalization of the value chain, not least due to the fact that analogue calibration certificates are often designed differently, even if the calibrations are the same. A uniform design for certificates for calibrations which are basically equal in nature is shown in a Good Practice (GP) DCC. Work on the GP-DCC is currently in process in the most diverse fields (temperature, humidity, ambient pressure, mass, weighing instruments, force, torque...). By means of the GP-DCC, the results are becoming machine-interpretable (as explained above) as, thanks to the GP-DCC, it is exactly known what has been stored, where it has been stored and in which way it has been stored.

Calibrations—and thus also calibration certificates—play an important role in many branches of industry (e.g., in the automotive industry and the pharmaceutical industry). A pharmaceutical company may require several hundreds of thousands of calibrations certificates every year. It is easy to imagine which efforts currently have to be made to archive analogue calibration certificates (either exclusively in paper form or also in other forms). In this publication, the DCC concept is presented. It will solve the abovementioned problems and will bring about many additional advantages for companies and other users. Furthermore, it will allow machines to communicate with each other (M2M). All of these aspects are described in more detail below.

#### **2. The Role of Calibration and Calibration Certificates**

#### *2.1. Quality Infrastructure*

The quality infrastructure of a country is of fundamental importance for the services that are provided to the public as well as for consumer protection. This was published for the first time in 2007 in a report of the World Bank [6]. Meanwhile, various contributions have been issued which were developed on the basis of this report (see also [7–10]). Figure 1 shows the general set-up of an arbitrary national infrastructure, and it also shows digitalization by means of DCC. From this figure, it becomes obvious what an important role calibration certificates play.

Calibration certificates appear several times in Figure 1. It should be stated that besides the "normal" digitalization effect which normally occurs, for example, in the field of eGovernment, an exorbitant increase in quality can be observed, which is due to the fact that the calibration information is more precise and can be used in an automated way. The data are transferred in M2M communication without any transmission errors. The step in which data are transferred manually according to the four-eyes principle and

which so far has been necessary, for example, in the pharmaceutical industry, can then be omitted. Thereby, the time expenditure can be substantially reduced, and errors during data transmission will thus be ruled out. *Sci* **2023**, *5*, x FOR PEER REVIEW 3 of 14

**Figure 1.** Presentation of the national quality infrastructure (based on [6]). **Figure 1.** Presentation of the national quality infrastructure (based on [6]).

Calibration certificates appear several times in Figure 1. It should be stated that besides the "normal" digitalization effect which normally occurs, for example, in the field of eGovernment, an exorbitant increase in quality can be observed, which is due to the fact that the calibration information is more precise and can be used in an automated way. The data are transferred in M2M communication without any transmission errors. The step in which data are transferred manually according to the four-eyes principle and which so far has been necessary, for example, in the pharmaceutical industry, can then be omitted. Thereby, the time expenditure can be substantially reduced, and errors during data transmission will thus be ruled out. After the data have been transferred, the reject rate in manufacturing can After the data have been transferred, the reject rate in manufacturing can immediately be reduced and products can be manufactured in a more resource-saving and more sustainable way. Manufacturing is increasingly helping to save many raw materials, as well as time and energy, and thus often reduces CO<sup>2</sup> emissions to a high extent. This, in turn, will contribute essentially to environmental protection and to achieving the global UN sustainable development goals not only in the industrial countries, but also in developing countries and in countries in transition. The latter can often even skip an innovation cycle (which would normally still be based on a paper-based calibration system) and thus participate more quickly in an international economic system. In addition, they could contribute effectively to the conservation of nature especially in their own countries.

immediately be reduced and products can be manufactured in a more resource-saving and more sustainable way. Manufacturing is increasingly helping to save many raw materials, as well as time and energy, and thus often reduces CO2 emissions to a high extent. This, in turn, will contribute essentially to environmental protection and to achieving the global UN sustainable development goals not only in the industrial countries, but also in developing countries and in countries in transition. The latter can often even skip an innovation cycle (which would normally still be based on a paper-The topic of "data as economic goods" has become particularly important in the value chain of industrial companies and will become more and more important in the future due to the advancing digital transformation. In this regard, the quality of the data which have been gained, among others, via sensors and actuators and the verifiability of the statements and conclusions derived from this data are of central importance especially for industry. It will only be possible to use data successfully as economic goods if the quality of the data is reliably verified and safeguarded.

based calibration system) and thus participate more quickly in an international economic system. In addition, they could contribute effectively to the conservation of nature especially in their own countries. The topic of "data as economic goods" has become particularly important in the value chain of industrial companies and will become more and more important in the future due to the advancing digital transformation. In this regard, the quality of the data which have been gained, among others, via sensors and actuators and the verifiability of For more than 125 years, maintaining the high level of quality of German products has been one of the fundamental pillars of the quality label "Made in Germany" which is recognized worldwide today. One of the fundamental challenges of the national quality infrastructure (QI) is to transfer this quality label to the digitalized world and to establish it there. This includes, in particular, the development of safe and robust calibrated measurement systems to be able to ensure data quality and the trust people have in the data. Metrology plays a decisive role in gaining the trust people have in measurements and in

quality of the data is reliably verified and safeguarded.

the statements and conclusions derived from this data are of central importance especially for industry. It will only be possible to use data successfully as economic goods if the ensuring the quality of the measurement data and measurement results. Figure 1 shows how the different elements of QI interact with each other.

The quality infrastructure of a country consists of seven elements. The **N**ational **M**etrology **I**nstitute (NMI) disseminates the SI units to the national standardization institute. The standardization institute ensures that the level of the enterprises (see the level "enterprises" in the diagram above) is provided with norms and standards. At the same time, it determines the norms and standards for the national accreditation body. The accreditation body has the task of accrediting calibration and testing laboratories, inspection bodies and certification bodies and of supervising the accreditation. In this way, the accreditation bodies are able to supply the level of the enterprises with verified expertise.

The second task of the national metrology institutes is to provide the accredited calibration laboratories with traceable systems. In this way, it is ensured that the measurement standards of the accredited calibration laboratories are linked up with the national measurement standards. The accredited calibration laboratories will then ensure the traceability of the measurement standards of the test laboratories, of the inspection bodies and of the other calibration laboratories. Calibration thus has a special position in the national quality infrastructure (see the red arrows in Figure 1).

So far, the certification bodies have not been particularly active in the field of digitalization. However, it has turned out that especially in the field of software development, considerable damage has occurred. This is due to the fact that different systems of units are used. The software certification bodies will therefore play a particularly important role in this field.

DCCs have not yet been the subject of considerations. However, using DCCs, and thanks to the fact that the error-free transmission of data becomes possible using DCCs, a high increase in quality will be achieved as the transmission problems will be eliminated.

In addition, there is the advantage that in a DCC, larger amounts of data can be transmitted, and that the calibration information can be integrated in a **D**igital **T**win (DT) in an automated and secure way.

#### *2.2. Task of the Calibration*

The calibration of measurement instruments is the main pillar of measurements which are comparable with each other and whose contents are correct. It is also one of the main pillars of the distributed manufacturing processes that are common in industry today. Without calibration, it is impossible to make a sound statement on the quality of a manufactured product. The calibration is based on the measurement standards provided by the NMIs and the **D**esignated **I**nstitutes (DIs). These measurement standards are embedded in the SI system of units [11]. Figure 2 illustrates this relationship. In many calibration fields, the factors indicated in the figure are clearly higher. It can be assumed that in Germany alone, several millions of calibrations are carried out every year. Via large-scale comparison measurements (called "interlaboratory comparisons"), as well as via audits and other procedures, the NMIs and the DIs ensure the provided measurement standards agree within the scope of the conventional and/or stated measurement uncertainties. In addition to this, the NMIs and the DIs of different countries stay in close contact with each other in order to ensure there are harmonized measurement standards worldwide. The calibration pyramid shows that the measurement uncertainty increases from the top to the bottom.

**Figure 2.** Representation of the calibration pyramid of the individual national metrology institutes/designated institutes and their integration in international interlaboratory comparisons with other NMIs/DIs. **Figure 2.** Representation of the calibration pyramid of the individual national metrology institutes/designated institutes and their integration in international interlaboratory comparisons with other NMIs/DIs.

#### **3. Research for Practical Applications 3. Research for Practical Applications**

Both industry and the economy are waiting for—and urgently require—a successful and lawful digital transformation so that data, information and certificates can securely and robustly be transmitted throughout the QI processes. Such a digital transformation is the precondition for both industry and the economy to be able to keep their promise of quality also efficiently and effectively in a digitalized world. Since August 2020, a project consortium has been established in Germany which is known as GEMIMEG-II [12] and is funded by the BMWK (Federal Ministry for Economic Affairs and Climate Action). It consists of 12 partners as well as of further institutions (the latter in a consultative capacity). The partners are contributing research achievements in the field of DCC [13,14] and are setting up a digital calibration infrastructure. The overriding objective and the central challenge of the GEMIMEG-II project is to bring the framework requirements of QI and the practical demands of industry together in a joint, holistic approach which is applicable in practice. Both industry and the economy are waiting for—and urgently require—a successful and lawful digital transformation so that data, information and certificates can securely and robustly be transmitted throughout the QI processes. Such a digital transformation is the precondition for both industry and the economy to be able to keep their promise of quality also efficiently and effectively in a digitalized world. Since August 2020, a project consortium has been established in Germany which is known as GEMIMEG-II [12] and is funded by the BMWK (Federal Ministry for Economic Affairs and Climate Action). It consists of 12 partners as well as of further institutions (the latter in a consultative capacity). The partners are contributing research achievements in the field of DCC [13,14] and are setting up a digital calibration infrastructure. The overriding objective and the central challenge of the GEMIMEG-II project is to bring the framework requirements of QI and the practical demands of industry together in a joint, holistic approach which is applicable in practice.

The focus of this project is on metrology (calibration, referencing, measurement, i.e., on trustable information on the quality of measuring instruments, sensor networks, digital twin and data analysis methods). The overall objective is to make information for the realization of reliable, connected measuring systems available in a secure, lawful and legally compliant end-to-end way. The results of the research fields are implemented in four so-called "RealBeds" (i.e., demonstrators which are geared to different fields of application), and these demonstrators thus take on the status of a prototype. The four RealBeds are connected calibration facilities, Industry 4.0 applications, the pharmaceutical/process industry and autonomous driving. The legal questions of the research fields and of the RealBeds will be investigated in addition in a legal simulation The focus of this project is on metrology (calibration, referencing, measurement, i.e., on trustable information on the quality of measuring instruments, sensor networks, digital twin and data analysis methods). The overall objective is to make information for the realization of reliable, connected measuring systems available in a secure, lawful and legally compliant end-to-end way. The results of the research fields are implemented in four so-called "RealBeds" (i.e., demonstrators which are geared to different fields of application), and these demonstrators thus take on the status of a prototype. The four RealBeds are connected calibration facilities, Industry 4.0 applications, the pharmaceutical/process industry and autonomous driving. The legal questions of the research fields and of the RealBeds will be investigated in addition in a legal simulation study.

study. In these investigations, two aspects are important: (1) In how far is reliable and trustworthy information on measuring instruments and measurement data digitally available, and (2) how can a secure and robust orchestration of the measuring systems be achieved? The investigations also include the development of a digital metrology system in the sense of an end-to-end, digitalized, traceable and legally secure measuring and calibration chain for complex sensor networks. Thereby, the sensors—which, so far, have been considered individually—will increasingly be connected with each other in a In these investigations, two aspects are important: (1) In how far is reliable and trustworthy information on measuring instruments and measurement data digitally available, and (2) how can a secure and robust orchestration of the measuring systems be achieved? The investigations also include the development of a digital metrology system in the sense of an end-to-end, digitalized, traceable and legally secure measuring and calibration chain for complex sensor networks. Thereby, the sensors—which, so far, have been considered individually—will increasingly be connected with each other in a network by the IIoT (**I**ndustrial **I**nternet **o**f **T**hings) in Industry 4.0. A basic set-up of a generic, massive, connected sensor system is shown in Figure 3. The processing of the sensor data follows the

[15,16].

*Sci* **2023**, *5*, x FOR PEER REVIEW 6 of 14

*Sci* **2023**, *5*, x FOR PEER REVIEW 6 of 14

fog/edge computing approach. This allows computing functions and memory assignments to take place on suitable nodes of a network, but completely decentralized solutions for use in mobile systems are also possible by means of identical stack and software technologies. and memory assignments to take place on suitable nodes of a network, but completely decentralized solutions for use in mobile systems are also possible by means of identical stack and software technologies. stack and software technologies.

network by the IIoT (**I**ndustrial **I**nternet **o**f **T**hings) in Industry 4.0. A basic set-up of a generic, massive, connected sensor system is shown in Figure 3. The processing of the sensor data follows the fog/edge computing approach. This allows computing functions

network by the IIoT (**I**ndustrial **I**nternet **o**f **T**hings) in Industry 4.0. A basic set-up of a generic, massive, connected sensor system is shown in Figure 3. The processing of the sensor data follows the fog/edge computing approach. This allows computing functions and memory assignments to take place on suitable nodes of a network, but completely decentralized solutions for use in mobile systems are also possible by means of identical

**Figure 3.** Architecture for a massive sensor network based on the OpenFog reference architecture **Figure 3.** Architecture for a massive sensor network based on the OpenFog reference architecture [15,16]. This technical and automated interconnection in a network significantly increases the

This technical and automated interconnection in a network significantly increases the requirement to determine the quality of data and their availability if, on the one hand, the integrity of the citizens and of the environment and, on the other hand, the economic success of the companies is to be ensured. It can thus be expected that the need for developing and setting up sensor networks as holistically calibrated, digital measuring This technical and automated interconnection in a network significantly increases the requirement to determine the quality of data and their availability if, on the one hand, the integrity of the citizens and of the environment and, on the other hand, the economic success of the companies is to be ensured. It can thus be expected that the need for developing and setting up sensor networks as holistically calibrated, digital measuring systems will continuously increase. requirement to determine the quality of data and their availability if, on the one hand, the integrity of the citizens and of the environment and, on the other hand, the economic success of the companies is to be ensured. It can thus be expected that the need for developing and setting up sensor networks as holistically calibrated, digital measuring systems will continuously increase. At this point, special requirements will not only arise for the quality of data, but also

systems will continuously increase. At this point, special requirements will not only arise for the quality of data, but also for the communication infrastructure and for the management of data sources and data sinks. Figure 4 shows a schematic overview of the GEMIMEG-II concept. At this point, special requirements will not only arise for the quality of data, but also for the communication infrastructure and for the management of data sources and data sinks. Figure 4 shows a schematic overview of the GEMIMEG-II concept. for the communication infrastructure and for the management of data sources and data sinks. Figure 4 shows a schematic overview of the GEMIMEG-II concept.

**Figure 4.** GEMIMEG concept in a schematic overview.

This concept extends from the sensor measurement values, including calibration and the characteristic numbers QoS (QoS: **Q**uality **o**f **S**ensing), via sensor fusion and data aggregation with the characteristic numbers QoD (**Q**uality **o**f **D**ata) to data application (e.g., in the digital twin) with the characteristic numbers QoI (**Q**uality **o**f **I**nformation). The quality characteristic numbers can also be described as QoX, where the X stands for **S**ensing, **D**ata, or **I**nformation.

That step of the measurement process which contains the sensor measurement values and the calibration of the sensors is supposed to abstract the measurement values (by exploiting the domain knowledge during the measurement process) and to decouple them from the specific characteristics of the actually used sensor. The qualified measurement values and the quality characteristic numbers which have been obtained in this way in the subsequent step (sensor and data aggregation) can be processed without the concrete measurement system being known any further. The measurement values are thus decoupled from the corresponding measurement modalities in the best possible way and associated QoS trust measures have been determined. After these input quantities have been processed during the sensor and data aggregation step, new data are generated as output quantities—after data fusion, or via software-based sensors, or via data analysis using neural networks, or via artificial intelligence—which, in turn, can then have the quality characteristic numbers QoD. This step can be available several times. It can be arranged in parallel (for different measurement values) and/or in a cascaded way (multi-step processing). The data with QoD can be used in different digital twins which, in turn, can access the data with QoD from the multi-step sensor and data aggregation.

If pieces of information are then derived from these data (in an application such as a digital twin), this information can also be supplemented by means of quality characteristic numbers for the QoI information—where the QoI describes, for example, a level of trust or the reliability of this information.

The DCCs, which have already been addressed several times in this article, bring about several conceptual advantages for the project. Avoiding changes between different media—and thus ruling out transmission errors—is an obvious advantage. The calibration results are immediately available for further processing. It is thus, for example, possible to extract different measuring uncertainties automatically in the areas that are important for the manufacturing process.

#### **4. The DCC Concept**

In its digital form, the DCC is defined via an XSD schema [13]. For the information obtained in a calibration, the schema contains clearly defined fields for entering mandatory specifications, as well as open text fields in which the calibration laboratory can enter further information. In this way, the entire information content obtained during a calibration will be included in the DCC and transmitted.

The Digital System of Units (D-SI [17,18]) is used to express the units for the measurement values. It supports both the seven base units and the measurement units derived from these—including decimal multiples or dividers, among others—as well as the disseminated imperial units. Even unusual systems of units are possible as long as they have been clearly defined. Actually, this is necessary today in order to ensure, on the one hand, international acceptance, and to be able, on the other hand, to cope without difficulties with all the applications that exist today.

Figure 5 shows an initial rough generic view of the way in which the DCC is embedded in the existing environment of norms, standards, terms and technical regulations.

In addition to the DCC, Figure 5 also shows the **D**igital **C**alibration **R**equest (DCR). This is a standardized digital document by means of which a desired calibration can be requested and specified. Furthermore, Figure 5 shows a document named the **D**igital **C**alibration **A**nswer (DCA). In the DCA, the calibration laboratory can enter further information on the calibration which—for formal reasons (e.g., according to the ISO IEC 17025 standard [5])—is not supposed to appear in an official calibration document. The aim is that all documents should be based on a joint structured document schema DX which, in turn, must take the norms, standards, nomenclatures and technical regulations into account which are shown in Figure 5 below the level "Digital Document Schema DX." The purpose

of this is to ensure that the calibration is documented digitally in a way that is legally valid and in compliance with the norms and standards.

*Sci* **2023**, *5*, x FOR PEER REVIEW 8 of 14

**Figure 5.** Generic view of the digital calibration document "ecosystem".

#### **Figure 5.** Generic view of the digital calibration document "ecosystem". *4.1. Structure of the DCC Files*

In addition to the DCC, Figure 5 also shows the **D**igital **C**alibration **R**equest (DCR). This is a standardized digital document by means of which a desired calibration can be requested and specified. Furthermore, Figure 5 shows a document named the **D**igital **C**alibration **A**nswer (DCA). In the DCA, the calibration laboratory can enter further information on the calibration which—for formal reasons (e.g., according to the ISO IEC 17025 standard [5])—is not supposed to appear in an official calibration document. The aim is Technically, the DCC is a text document which has an XML structure. XML documents are stored as plain text files, based on the Unicode character set (UTF-8 format [19]), and are thus suitable for long-term storage. The files can be opened and processed with an arbitrary text editor. Version 1.0, which is used for DCC, is widespread and has been standardized for more than 20 years now, which is why XML is ideally suited for the use of such important documents as calibration reports. The calibration reports can be ported from XML to other data formats (e.g., JSON).

that all documents should be based on a joint structured document schema DX which, in turn, must take the norms, standards, nomenclatures and technical regulations into ac-More information on the structure of a DCC can be found on the DCC homepage [20].

#### count which are shown in Figure 5 below the level "Digital Document Schema DX." The *4.2. Prologue including a Processing Instruction*

purpose of this is to ensure that the calibration is documented digitally in a way that is legally valid and in compliance with the norms and standards. *4.1. Structure of the DCC Files*  Technically, the DCC is a text document which has an XML structure. XML documents are stored as plain text files, based on the Unicode character set (UTF-8 format [19]), and are thus suitable for long-term storage. The files can be opened and processed with an arbitrary text editor. Version 1.0, which is used for DCC, is widespread and has been standardized for more than 20 years now, which is why XML is ideally suited for the use Every XML document must start with a single-spaced prologue. The prologue is an instruction which has the form (<?xml...?>) and is named XML **P**rocessing **I**nstruction (PI). The prologue gives the reading program instructions on what must be observed during reading. Apart from the version (which can be indicated via the "version" attribute), it is also the agreed character set which can be stored in the document file. The characters in documents are encoded according to various ISO standards. The ISO 8859 standard [21] specifies the characters of different languages in ten subdivisions. Further characters are defined in ISO standard IEC 10646 [22]. DCC uses the character set UTF-8. This character set can map all official languages in the world.

of such important documents as calibration reports. The calibration reports can be ported from XML to other data formats (e.g., JSON). More information on the structure of a DCC can be found on the DCC homepage One of the main advantages of XML is that it can be read and interpreted by machines. In addition, it is also readable by humans. A decisive factor is furthermore that XML is a data format that is suitable for long-term storage. This is of special importance as it must still be possible to read the files in several decades. Further information on data formats that are suitable for long-term storage can be found, for example, in [23,24].

#### Every XML document must start with a single-spaced prologue. The prologue is an *4.3. Cryptographic Signatures in the DCC*

*4.2. Prologue including a Processing Instruction* 

[20].

instruction which has the form (<?xml...?>) and is named XML **P**rocessing **I**nstruction (PI). The prologue gives the reading program instructions on what must be observed during reading. Apart from the version (which can be indicated via the "version" attribute), it is also the agreed character set which can be stored in the document file. The characters in documents are encoded according to various ISO standards. The ISO 8859 standard [21] In order to ensure the authenticity and the originality of each calibration certificate, the XML document can be provided by the issuing body (the calibration body) with a corresponding hash value and can be signed digitally. According to ISO standard 17025, adding a signature is **not mandatory**. As a measure to create trust between the calibration institute und the customer, however, adding a signature is recommended. As the signature

specifies the characters of different languages in ten subdivisions. Further characters are

can be verified at any time, the recipients of a DCC are in a position to verify the authenticity of the content of a DCC as well as the issuer of a DCC themselves. For this purpose, public software tools are available free of cost [25]. The cryptographic procedures can be applied to XML data structures in a robust and probative way [23,26]. Further information on XML is to be found in the literature, e.g., in [27]. The VDI/VDE has already developed a very broad approach for data exchange [28] and—although this is a national standard for Germany—it is already used in various other countries. In this approach, the workflow in industry, in which the DCC can be integrated without any problems, is taken into account. On the one hand, the integrity and authenticity of a DCC must be ensured. On the other hand, however, electronically stored data can easily be changed and/or copied as often as desired. The use of cryptographic security procedures for the DCC is therefore highly recommended. Good summaries on the topic of cryptography can be found in [29,30].

Unfortunately, this report cannot describe this concept in more detail.

#### **5. The DCC within the Scope of the Manufacturing Process**

In this section, we, first of all, show the processes which take place within the scope of discrete manufacturing. In Figure 6, the logic relationship between the "parts supplier," the "machine" and the "factory" is shown in the form of models. All models include, in particular, commissioning and production processes. *Sci* **2023**, *5*, x FOR PEER REVIEW 10 of 14

**Figure 6.** Hierarchy of the different process models in the context of a factory, similar to [31] and adapted. **Figure 6.** Hierarchy of the different process models in the context of a factory, similar to [31] and adapted.

For these processes, calibrated measurement components are required. However, different pieces of information on the calibration are currently only available as calibration certificates in paper form (Figure 7). For these processes, calibrated measurement components are required. However, different pieces of information on the calibration are currently only available as calibration certificates in paper form (Figure 7).

When discrete or continuous manufacturing processes are automated, this offers considerable potential for using a DCC. Whereas in conventional automation with static hierarchies, it is sufficient to manage the calibration data of process sensors analogously and/or decentrally (Figure 8). Due to the fact that there are only a few clearly defined input points for this information, the situation in an Industry 4.0 architecture is different.

**Figure 7.** Comparison of how far the calibration information is available in an analogue calibration certificate (in the diagram in Figure 7, this is each time shown above the respective model) and in a

When discrete or continuous manufacturing processes are automated, this offers considerable potential for using a DCC. Whereas in conventional automation with static

DCC.

adapted.

DCC.

certificates in paper form (Figure 7).

**Figure 6.** Hierarchy of the different process models in the context of a factory, similar to [31] and

For these processes, calibrated measurement components are required. However, different pieces of information on the calibration are currently only available as calibration

**Figure 7.** Comparison of how far the calibration information is available in an analogue calibration certificate (in the diagram in Figure 7, this is each time shown above the respective model) and in a **Figure 7.** Comparison of how far the calibration information is available in an analogue calibration certificate (in the diagram in Figure 7, this is each time shown above the respective model) and in a DCC. hierarchies, it is sufficient to manage the calibration data of process sensors analogously and/or decentrally (Figure 8). Due to the fact that there are only a few clearly defined input points for this information, the situation in an Industry 4.0 architecture is different.

**Figure 8.** Conventional automation architecture as compared to Industry 4.0 or IIoT structures based on a Cyber Physical System (CPS) based on [32]. **Figure 8.** Conventional automation architecture as compared to Industry 4.0 or IIoT structures based on a Cyber Physical System (CPS) based on [32].

CPS-based architectures [33] react adaptively and continuously if it becomes necessary to change manufacturing processes, and this is exactly what is stipulated for Industry 4.0 or IIoT networks. The consequence of this is as follows: The process sensors, which are also used in automation, must not be able any longer to only make the generated measurement values but also—as an independent element—the sensor information (such as calibration certificates) available to other process components independently and in changing architectures. The massive sensor networks, which have already been presented above, might be an option for this and—together with the DCC—fulfil the requirements which have been discussed here. CPS-based architectures [33] react adaptively and continuously if it becomes necessary to change manufacturing processes, and this is exactly what is stipulated for Industry 4.0 or IIoT networks. The consequence of this is as follows: The process sensors, which are also used in automation, must not be able any longer to only make the generated measurement values but also—as an independent element—the sensor information (such as calibration certificates) available to other process components independently and in changing architectures. The massive sensor networks, which have already been presented above, might be an option for this and—together with the DCC—fulfil the requirements which have been discussed here.

Figures 7 and 8 clearly show that the calibration information that is contained in a DCC is not only available in the immediate manufacturing processes in which the calibrated measuring components are used. As the DCCs can be seamlessly integrated into the digital infrastructure of a factory, the calibration information can also be used in other Figures 7 and 8 clearly show that the calibration information that is contained in a DCC is not only available in the immediate manufacturing processes in which the calibrated measuring components are used. As the DCCs can be seamlessly integrated into the digital infrastructure of a factory, the calibration information can also be used in other processes

processes such as in enterprise planning, commissioning or quality management. The VDI/VDE has already developed a very broad approach for data exchange [28] and—alt-

tries. In this approach, the workflow in industry, in which the DCC can be integrated

Within the scope of the GEMIMEG-II project, we are currently working on a software library in the programming language Python (PyDCC). The aim of PyDCC is to facilitate access to the contents of the DCC and their processing so that the advantages of the DCC can be prepared for digital manufacturing in a simple and easy way. As soon as the GEM-IMEG-II project has been concluded, this software package, which will be an open-source software, will become generally available. Figure 9 shows the current development and planning status of this software project as well as the focal areas of the future development

In this way, a basic functionality is also available for using the DCC on edge and smart field devices. The idea is to grant all users easy access to the information contained in the DCC by making this software functionality (which is non-competitive) available as an open-source version that includes the standardized DCC, which is provided with an unambiguous version identification and with the respective XML schema. Publishing the approach as an open-source version, which will reach many users, is supposed to minimize the effort that is needed to implement the basic functionalities in such a way that the

without any problems, is taken into account.

tasks.

such as in enterprise planning, commissioning or quality management. The VDI/VDE has already developed a very broad approach for data exchange [28] and—although this is a national standard for Germany—it is already used in various other countries. In this approach, the workflow in industry, in which the DCC can be integrated without any problems, is taken into account.

Within the scope of the GEMIMEG-II project, we are currently working on a software library in the programming language Python (PyDCC). The aim of PyDCC is to facilitate access to the contents of the DCC and their processing so that the advantages of the DCC can be prepared for digital manufacturing in a simple and easy way. As soon as the GEMIMEG-II project has been concluded, this software package, which will be an open-source software, will become generally available. Figure 9 shows the current development and planning status of this software project as well as the focal areas of the future development tasks. *Sci* **2023**, *5*, x FOR PEER REVIEW 12 of 14 DCC can be used by a specific company and/or can be connected with the Enterprise Resource Planning/PLM (Product Lifecycle Management).

system.

**Figure 9.** PyDCC Software Management (currently unreleased, see Tobola, Andreas "Introducing PyDCC—a Python module for the DCC" [34], p. 101). **Figure 9.** PyDCC Software Management (currently unreleased, see Tobola, Andreas "Introducing PyDCC—a Python module for the DCC" [34], p. 101).

**6. Conclusions & Outlook**  In this article, we showed that all aspects of calibration are of great economic interest. The end user often does not realize how many components have contributed to the quality assurance for a product. Establishing the use of DCCs is a task for both the industrial and trade partners and should be achieved all over the world. When the approach of the digital calibration certificate is recognized and used internationally, the DCC will play a central role. For example, language barriers which still exist today can be eliminated by means of the digital document as the contents will be standardized and can automatically be transferred to the respective language. In this way, the DCC will make an important In this way, a basic functionality is also available for using the DCC on edge and smart field devices. The idea is to grant all users easy access to the information contained in the DCC by making this software functionality (which is non-competitive) available as an open-source version that includes the standardized DCC, which is provided with an unambiguous version identification and with the respective XML schema. Publishing the approach as an open-source version, which will reach many users, is supposed to minimize the effort that is needed to implement the basic functionalities in such a way that the DCC can be used by a specific company and/or can be connected with the Enterprise Resource Planning/PLM (Product Lifecycle Management).

contribution to the internationalization of measurement technology and the calibration

undergone maintenance work. For this purpose, only the type or level of the issuing authority must be mentioned in the DCC document. Here, the end-to-end use of a DCC

In principle, the DCC system suggested here can also be used for other applications in measurement technology. These range from legal metrology to qualification measure-

Currently, establishing the DCC as a component in an asset administration shell

would bring about great advantages for all process partners.

(AAS [35]) in a sub-model is envisaged.

#### **6. Conclusions & Outlook**

In this article, we showed that all aspects of calibration are of great economic interest. The end user often does not realize how many components have contributed to the quality assurance for a product. Establishing the use of DCCs is a task for both the industrial and trade partners and should be achieved all over the world. When the approach of the digital calibration certificate is recognized and used internationally, the DCC will play a central role. For example, language barriers which still exist today can be eliminated by means of the digital document as the contents will be standardized and can automatically be transferred to the respective language. In this way, the DCC will make an important contribution to the internationalization of measurement technology and the calibration system.

In principle, the DCC system suggested here can also be used for other applications in measurement technology. These range from legal metrology to qualification measurements that are carried out on a system in order to prove its functionality or after it has undergone maintenance work. For this purpose, only the type or level of the issuing authority must be mentioned in the DCC document. Here, the end-to-end use of a DCC would bring about great advantages for all process partners.

Currently, establishing the DCC as a component in an asset administration shell (AAS [35]) in a sub-model is envisaged.

**Author Contributions:** Conceptualization, methodology, draft preparation and visualization, S.H., S.S., L.D., T.E. and R.B.; supervision, validation, writing—reviewing and editing, L.D. and T.E. All authors have read and agreed to the published version of the manuscript.

**Funding:** The GEMIMEG II project is funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK), grant reference GEMIMEG 01 MT20001A.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** See 'DCC—GoodPractice · GitLab', GitLab. Available online: https:// gitlab.com/ptb/dcc/dcc-goodpractice (accessed on 16 January 2023).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
