A Hardware-Aware Application Execution Model in Mixed-Criticality Internet of Things

Abstract

The Real-Time Internet of Things is an emerging technology intended to enable real-time information communication and processing over a global network of devices at the edge level. Given the lessons learned from general real-time systems, where the mixed-criticality scheduling concept has proven to be an effective approach for complex applications, this paper formalizes the paradigm of the Mixed-Criticality Internet of Things. In this context, the evolution of real-time scheduling models is presented, reviewing all the key points in their development, together with some connections between different models. Starting from the classical mixed-criticality model, a mathematical formalization of the Mixed-Criticality Internet of Things concept, together with a specifically tailored methodology for scheduling mixed-criticality applications on IoT nodes at the edge level, is presented. Therefore, a novel real-time hardware-aware task model for distributed mixed-criticality systems is proposed. This study also offers a model for setting task parameters based on an IoT node-related affinity score, evaluates the proposed mapping algorithm for task scheduling, and presents some use cases.

1. Introduction

In the era of telecommunication and interconnectivity, the Internet of Things (IoT) is a promising new technology for connecting intelligent objects that surround us into a large network, often distributed over large geographic regions [1,2]. Real-time and non-real-time, critical, and non-critical systems coexist with increasingly complex and sometimes conflicting demands from the network. Thus, new concepts for the provisioning, management, and monitoring of these systems and their components must be developed. The development of such concepts has led to an entirely new field, the Real-Time Internet of Things (RT-IoT) [3], which promises better connectivity and efficient use of next-generation embedded devices. As the field of real-time systems has developed in recent years, it is natural to integrate new concepts from RT systems into RT-IoT. This concept, which is currently gaining attention in classical RT systems, is the concept of Mixed-Criticality Systems (MCSs), which define systems running real-time tasks of different criticality levels sharing the same platform. As many critical real-time applications have already been implemented using distributed heterogeneous architectures [4], the MCS concept can also be applied to RT-IoT, leading to a new scheduling paradigm called MC-IoT (for a full list of abbreviations, please see Appendix B).

The need for comprehensive solutions that integrate systems with strict timing constraints/requirements into IoT architectures is not new and has been extensively discussed in [5]. This need is also reflected in the design and implementation of real-time/time-triggered Ethernet protocols with mixed-criticality support [4]. However, only a few examples of real-time systems in their classical form or mixed-criticality systems have been integrated into IoT architectures. In [1], the placement of services and data in fog devices is discussed with regard to the optimization of the system. The challenges of both soft and hard real-time systems are addressed in [3] by introducing the term RT-IoT. Papers such as [6] and [7] present examples of autonomous IoT devices and applications, while [8] presents an example of a collaborative homogeneous IoT system, and [9], a system development platform for reducing the complexity of developing IoT-enabled applications with different criticalities. In [10], the deep learning approach is used to schedule real-time edge services, while Jin et al. [11] proposed a mechanism for data aggregation in mobile privacy-aware crowdsourcing systems. Furthermore, a real-time system that proposes the approximate image sharing and energy-aware adaptation to obtain higher bandwidth and energy efficiency, called BEES, was introduced in [12].

From our perspective, MC-IoT can effectively address the problem of resource management at the edge level of IoT platforms running a variety of applications with different criticalities and different time requirements, such as sensor fusion for automated driving applications [13], smart buildings [14], and healthcare mixed-criticality applications [15]. The scheduling problem in real-time distributed heterogeneous systems is not trivial and has been insufficiently addressed, despite its potential [16]. A few papers addressing this problem have been published [17,18,19,20].

The heterogeneous nature of the IoT architecture has a direct influence on applications that run on these different hardware components.

• The application code execution time is strongly influenced by the target system running the application.
• In the same manner, the power consumption owing to the execution of the same code may be different for target j than for target k.

Therefore, new application models are required for the MC-IoT applications. These models need to be considered in addition to the temporal and sometimes criticality aspects as well as hardware particularities.

In RT systems in general and in RT-IoT in particular, temporal behavior is a significant aspect, as important as the correctness and accuracy of the provided result [21]. Thus, the application models for MC-IoT, as a particular case of RT-IoT, should consider the influence of the hardware particularities.

In this study, we formalized the basic concepts of MC scheduling in the IoT domain and defined an MC–IoT paradigm. We also provide a methodology for scheduling MC applications in MC-IoT.

The main contributions of this research are:

• Definition and mathematical formalization of the Mixed-Criticality Internet of Things scheduling concept;
• The scheduling problem in MC-IoT and RT-distributed systems was mathematically formulated;
• A novel hardware-aware real-time task model for distributed mixed-criticality systems such as MC-IoT was proposed;
• A methodology for mapping tasks on heterogeneous targets, considering both timing and hardware particularities, was proposed and analyzed.

The evolution of real-time scheduling models is presented in Section 2, reviewing all the key points in their development. We created a mathematical formalization of the scheduling problem in Section 3. We adapted the classical MC task model to the MC-IoT paradigm in Section 4. The methodology for scheduling tasks in MC-IoT is proposed together with example algorithms for setting task parameters and distributing tasks on different IoT nodes in Section 5. The proposed function for task distribution is evaluated and compared with the classical mapping function in Section 6. The article ends with conclusions and future perspectives in Section 7.

2. Related Work

Tasks are basic execution units of an application. Depending on their activation patterns, they can be periodic or sporadic. Periodic tasks are activated at a constant rate [22], whereas sporadic tasks have a minimum inter-arrival time [23,24]. In the mixed-criticality paradigm, each task has an assigned criticality level and a set of properties [25,26] (for a full list of symbols, please see Appendix B).

$${\mathbf{\tau }}_i=\{ {\mathit{T}}_{\mathit{i}},{\mathit{D}}_{\mathit{i}},{\mathit{L}}_{\mathit{i}},\{{\mathit{C}}_{\mathit{i},\mathit{j}}|\mathit{j}\in 1\dots \mathit{l}\}\}$$

(1)

where l represents the number of criticality levels, T_i—the period for periodic tasks or the (minimum) arrival interval between two consecutive jobs of the same task i, D_i is the time by which any job execution needs to be completed relative to its release time, L_i is the criticality level (1 being the lowest level), and C_i,j is the computation time (vector of values—one per criticality level j, for levels lower or equal to the criticality level L_i (j ≤ L_i) expressing the worst-case execution time for each criticality level).

A task consists of a series of jobs, with each job inheriting the set of parameters of the task (T_i, D_i, L_i) to which it adds its parameters [27]. Thus, the k-th job of task τ_i is characterized as:

$${\mathit{J}}_{\mathit{i},\mathit{k}}=\left\{{\mathit{a}}_{\mathit{i},\mathit{k}},{\mathit{d}}_{\mathit{i},\mathit{k}},{\mathit{c}}_{\mathit{i},\mathit{k}},{\mathit{T}}_{\mathit{i}},{\mathit{D}}_{\mathit{i}},{\mathit{L}}_{\mathit{i}}\right\})$$

(2)

where a_i,k represent the arrival time a_i,k+1 − a_i,k >= T_i; d_i,k represents the absolute deadline (d_i,k+1 = a_i,k + D_i); c_i,k represents the execution time, which is dependent on the criticality mode of the system (for L_j, c_i,k = C_i,j); and T_i, D_i, L_j have the same meaning as in the task model.

The classical model has been the source of other simplified models, such as those introduced by Burns [28]. In [29], it was proven that Vestal’s model is a generalization of Burns’ model. Another classical simplification of the model considers only two levels of criticality (LO and HI) [30].

In time, other parameters were transformed from a simple scalar value to a vector of values depending on the criticality level. Such is the case of the period in [31]. In the same manner, the worst-case memory access was considered and added to the general model as a vector of values in [32], and a vector of values describing the QoS was added in [33]. In [34], a new model called Elastic Mixed-Criticality (E-MC) was introduced, which introduces the idea of variable periods for low-criticality tasks.

In addition to general mixed-criticality models derived from hard real-time task models, another task model for cyber-physical mixed-criticality systems was proposed in [35]. This model can be considered as a translation of the model presented in [36] for real-time tasks into an MCS task model by expressing the number of tolerable deadline misses as a function of the criticality level instead of a constant value, as in the classical real-time case.

On the one hand, we have the task-level models presented in the previous paragraph, which are mostly used for sets of independent tasks and are insufficient for sets of tasks with precedence constraints. However, we have models derived from graph-based real-time task models [37], such as those proposed in [38]. The mode-switching Diagraph Real-Time (MS-DRT) task model is such an example, which was proposed by Ekberg and Yi in [39]. These types of models are more complex and describe, in addition to temporal behavior, functional dependencies between tasks. Another graph-based model that specifies the allowed interference between tasks, called interference constraint graph (ICG), was proposed in [40]. The task parameters are the same as those in the Vestal model, and the main difference lies in the fact that the graph models the relationship between tasks, namely, task interference.

However, we have graph-based models for which we could not identify a central influential model but encountered several variants, from which we mention Ekberg and Yi (2016) and Huang et al. (2013).

Figure 1 summarizes the above paragraphs, presenting the main task models for RTS (real-time systems) and MCS (mixed-criticality systems), their main differences, and their evolution. For uniformity reasons, the parameter representing the criticality level has been marked with L even if in some of the original articles other symbols were used. Vestal’s model (2007) is the most influential model, which was developed from the sporadic task model proposed by Mok in 1983 and was further customized into several simplified models, among which we consider worth mentioning the Burns and Baruah model (2013). In addition to general task models, there are models specifically for firm/soft real-time tasks with criticality levels (Lee and Shin, 2017).

Although there are a relatively large number of task models, there are only a few core models that give birth to different variations. Most of the variations are compatible with one another, but they focus on different aspects, generalizing or simplifying certain behaviors/parameters, thus differing in the assumptions regarding the task parameters (see Appendix A).

3. Application Execution Model and Scheduling Problem Formulation

With the development of 6G networks, the IoT applications running at the edge level are integrating more and more real-time and mixed-criticality functionalities [33].

Mixed-Criticality Systems represent a special class of real-time systems in which application functions with different criticalities share the same resources in terms of computation and/or communication [12]. The MCS concept offers important advantages when integrated into different types of systems that directly interact with the environment [12]. IoT is also a type of system that interacts directly with the user and/or environment. As IoT systems are rapidly evolving towards real-time IoT systems, where tasks with different time constraints and criticality levels coexist, it is only natural that the application scheduling in such systems becomes increasingly complex, and the need to integrate the concept of mixed-criticality becomes stringent.

Adapting the MCS concept to IoT architectures gives birth to a new scheduling paradigm called MC-IoT, for which we propose the following definition.

Definition 1. 

Mixed Criticality-Internet of Things (MC-IoT) is a type of system that run real-time tasks of different criticalities at the edge level of IoT architectures.

In MCSs, as in any real-time system, applications are split into basic units of execution called tasks. The scheduling issue of these tasks on multiprocessors and distributed systems have become prominent in RT-IoT systems such as 6G networks [33].

Tasks can be independent of one another or have a precedence dependence on different tasks. Each task or set of dependent tasks is allocated to a specific node in a process called global level scheduling, and each set of independent or dependent tasks is scheduled at the device level in a process called local scheduling.

Thus, the scheduling problem at the edge level can be divided into two sub-problems:

(1)

Task mapping (at an intermediate/Fog level): Each task must be allocated or mapped to a processing element.

(2)

Local task scheduling (at the device/Edge node level): all tasks allocated to a certain processing element must be schedulable.

Until now, the task mapping problem was not approached by taking into consideration the hardware heterogeneity. Thus, we further propose a hardware-aware solution to this problem.

In this paper, we assume the following hypotheses:

The system is represented by an IoT layer architecture with heterogeneous edge nodes. The architecture contains three main computation levels (Edge, Fog, Cloud) [41].

These applications were implemented as systems for periodic/sporadic independent mixed-criticality tasks.

These tasks have stringent timing requirements (i.e., hard real-time tasks).

The worst-case execution time (WCET) was determined via static analysis for high criticality levels and estimated through measurements for low criticality levels.

Each task was statically assigned to a processing element (PE) and could not migrate during runtime.

Each processing element starts in the lowest criticality mode, L₁, and remains in that mode as long as all jobs are executed within their criticality level computation times C_i,1. If any job is executed for more than C_i,1 without signaling completion, the system immediately changes to L₂ and so on [25].

For simplicity, we consider that, for each criticality mode L_j, tasks with a criticality lower than L_j are dropped [25]. Still, the models and partitioned algorithms proposed in this article are valid, even if we consider more complicated local task scheduling algorithms, which do not drop lower criticality tasks but schedule them using processor slack time.

The system is heterogeneous, which means that the task execution time varies depending on the PEs.

4. The Proposed MC-IoT Task Model

As was presented in Section 2, the current MC task models do not take into consideration the hardware heterogeneity of the system.

We begin with the classical MCS model proposed by Vestal in 2007 [25] and consider the parameters from Equation (1). In addition to the temporal behavior already included in this model, we propose an extension consisting of a new parameter, affinity score A_i, which is defined as a vector of values, one per host, representing the affinity score of the task for each host. The affinity score is an integer between zero and p, where p is the number of processing elements. A higher value indicates higher affinity, and zero indicates no affinity.

Moreover, we propose to express the computation time not only as a function of the criticality level (L_i) but also as a function of the processing hardware element running the task (PE_q). Thus, task model (1) becomes:

$${\mathbf{\tau }}_i=\left\{{\mathit{T}}_{\mathit{i}},{\mathit{D}}_{\mathit{i}},{\mathit{L}}_{\mathit{i}},\left\{{\mathit{C}}_{\mathit{i},\mathit{j},\mathit{q}}\right|\mathit{j}\in 1\dots \mathit{l},\mathit{q}\in 1\dots \mathit{p}\},\{{\mathit{A}}_{\mathit{i},\mathit{q}}|\mathit{q}\in 1\dots \mathit{p}\}\right\}$$

(3)

where C_i is a matrix of size p × l (p represents the number of PEs, and l is the number of criticality levels).

The task affinity score can be set statically by the task creator or computed using an algorithm based on the resources required by the task and the task computation time for different processing elements. If a task cannot be scheduled on a particular PE, the WCET value can be set to infinity, which translates to an affinity score of zero.

In the following analysis, we consider a dual criticality system. However, these algorithms can also be extended to platforms with multiple criticality levels. In criticality-aware heuristics, it makes more sense to use utilization in the high criticality (Hi) mode on a certain processing element q during the assignment of Hi-criticality tasks and utilization in the low criticality (Lo) mode during the assignment of Lo-criticality tasks. Following this principle, the affinity score is set for Hi-criticality tasks using their Hi-criticality WCET and for Lo-criticality tasks using their Lo-criticality WCET.

5. A Methodology for Mapping Tasks on Different Processing Elements

Having the previously defined MC-IoT task model and scheduling problem formalized in Section 4, we propose a hardware-aware methodology for mapping tasks into our proposed MC-IoT architecture. The methodology comprises different methods for setting the newly introduced task parameter, namely the affinity score, and defining a suitable mapping function to respect both application and resource constraints.

5.1. Setting Affinity Score

While the task parameters inherited from the classical MCS model are assigned considering only the temporal behavior [25], the affinity score is computed by considering the particularities of the processing elements. This can be achieved using the following methodology.

5.1.1. Set Affinity Score Based on Computation Time

The affinity score can be set based on the computation time using the algorithm presented in Algorithm 1.

Algorithm 1: SetAffinity_WCET

Input: $C_{i, 2,q}$

Output: $A_{i, q}$

1  for $i\in \left\{1, 2, \dots , n\right\}$do

2   for $q\in \left\{1, 2, \dots , p\right\}$ do

3     $X_{i, 2,q}\leftarrow C_{i,2,q}$

4   end for

5  end for

6  for $i\in \left\{1, 2, \dots , n\right\}$do

7   $a\leftarrow 1$

8   for $q\in \left\{1, 2, \dots , p\right\}$ do

9     $index\leftarrow \max \left(X_{i, 2,q}\right)$

10    $A_{i,index}\leftarrow a$

11     $a\leftarrow a+1$

12     $X_{i,2,index}\leftarrow 0$

13   end for

14  end for

Step  1:

Extract the task computation time for the highest criticality level (2 in the case of a dual criticality system) on each PE (the second column in the generated computation time matrix C_i,j,q) by copying it into an array of structures X_i,2,q. Each structure has a computation time value X_i,2,q, and a PE index (q), where i represents the task number, which is fixed, and 2 represents the highest criticality level. Index q varies from one to p.

Step  2:

Extract the matrix line index of the maximum value for X_i,2,q from the array when q varies from one to p.

Step  3:

In the affinity array, set the affinity score A_i,q to 1 for the PE corresponding to the highest X_i,2,q value, and then 2 for the PE corresponding to the highest X_i,2,q from the array after setting the element with the highest value from the first iteration X_i,2,q to 0, 3 for the remaining highest value, and so on, while q ≤ p.

5.1.2. Set Affinity Score Based on Criticality Level

To distribute tasks on different processing elements considering the criticality level of the task, we can employ carefully selected algorithms, such as those described in [42].

An alternative method for mapping the tasks that consider the affinity score is proposed as Algorithm 2.

Algorithm 2: SetAffinity_criticality (p > l)

Input: $L_i$, $C_{i, 2,q}$

Output: $A_{i, q}$

1  for $i\in \left\{1, 2, \dots , n\right\}$ do

2   for $q\in \left\{1, 2, \dots , p\right\}$  do

3      $X_{i,2,q}\leftarrow C_{i, 2,q}$

4   end for

5  end for

6  for $q\in \left\{1, 2, \dots , p\right\}$ do

7   $P{E}_{q, j}\leftarrow mod\left(q, l\right)$

8   if $mod\left(q, l\right)=0$ then

9      $P{E}_{q, j}\leftarrow l$

10   end if

11  end for

12  for $i\in \left\{1, 2, \dots , n\right\}$ do

13   $a\leftarrow 0$

14   for $q\in \left\{1, 2, \dots , p\right\}$ do

15   $\left\{maxVal,index\right\}\leftarrow \max (X_{i, 2,q}|P{E}_{q, j}~=L_i)$

16      if $maxVal~=0$ then

17       $a\leftarrow a+1$

18       $A_{i, index}\leftarrow a$

19       $X_{i,2,index}\leftarrow 0$

20      end if

21   end for

22  end for

23  for $i\in \left\{1, 2, \dots , n\right\}$  do

24   $a2\leftarrow p-a$

25   for$q\in \left\{1, 2, \dots , p\right\}$ do

26   $\left\{maxVal,index\right\}\leftarrow \max (X_{i, 2,q}|P{E}_{q,j}=L_i)$

27      if $maxVal~=0$ then

28       $a2\leftarrow a2+1$

29     $A_{i, index}\leftarrow a2$

30     $X_{i, 2,index}\leftarrow 0$

31    end if

32   end for

33  end for

Step  1:

Build an array of structures PE_q,j. Each structure has a criticality level L_j that represents the expected criticality level of the tasks to be partitioned on PE_q and PE index (q). Index q varies from 1 to p, whereas L_j is given by computing L_j = PE_q mod l, where l is the number of criticality levels and mod represents the modulo operation. If L_j is zero, we consider L_j = l.

Step  2:

Assign each task τ_i according to criticality level L_i. We have two subsets: PEs with expected criticality PE_q,j equal to L_i, and PEs with expected criticality PE_q,j not equal to L_i. The affinity scores A_i,q had the highest values for the first subset of PEs. For each subset of PEs, an affinity score was assigned according to the computation time, as shown in Algorithm 1.

If the number of criticality levels exceeded the number of PEs, the affinity score was set according to Algorithm 3.

Algorithm 3: SetAffinity_criticality (l>p)

Input: $L_i$, $C_{i,2,q}$

Output: $A_{i, q}$

1  for $i\in \left\{1, 2, \dots , n\right\}$ do

2   for $q\in \left\{1, 2, \dots , p\right\}$ do

3      $X_{i, 2,q}\leftarrow C_{i,2,q}$

4   end for

5  end for

6  for $i\in \left\{1, 2, \dots , n\right\}$ do

7    $q\leftarrow mod\left(L_i, p\right)$

8   if $q=0$ then

9      $q\leftarrow p$

10   end if

11   $A_{i,q}\leftarrow p$

12   $X_{i,l,q}\leftarrow 0$

13  end for

14  for $i\in \left\{1, 2, \dots , n\right\}$ do

15   $a\leftarrow 0$

16   for $q\in \left\{1, 2, \dots , p-1\right\}$ do

17      $index\leftarrow \max \left(X_{i, l,q}\right)$

18      $a\leftarrow a+1$

19      $A_{i, index}\leftarrow a$

20      $X_{i, l,index}\leftarrow 0$

21   end for

22  end for

Step  1:

For each taskτ_i, obtain the PE on which it is expected to run by computing PE_q = L_j mod p, where p is the number of PEs, and mod represents the modulo operation. If PE_q is 0, we consider PE_q = p. Next, we set the affinity score A_i,q for PE_q as p.

Step  2:

For each task τ_i, set the remaining affinity scores A_i,q according to the computation time (Algorithm 1), where q ranges from 1 to p-1.

5.2. Task Mapping

As highlighted in the literature [33], the partitioned scheduling approach can achieve better schedulability results than the global scheduling approach. Thus, we also propose a partitioned approach.

Given a set of independent MC tasks and processing elements, the problem involves determining the following function:

$$M:\tau \to P$$

(4)

where τ is the set of tasks in the system, P is the set of processing elements, and M is a morphism.

Considering the working hypotheses presented above, tasks cannot migrate during runtime; therefore, each task was assigned to a single PE. M (τ_i) represents the PE at which τ_i runs. Figure 2 illustrates the assignment of tasks to PEs, where p is the total number of PEs, and n is the number of tasks in the system.

After each task is assigned to a processing element, the tasks allocated to PE_q form a task subset defined as Ψ_q [27]. If task set τ is successfully partitioned, then

$$\mathit{\tau }={{\displaystyle \mathit{\Psi }}}_1\cup {\mathit{\Psi }}_2\cup {\mathit{\Psi }}_3\cup \dots \cup {\mathit{\Psi }}_p$$

(5)

where Ψ_q can also be an empty subset {Ø}.

The goal is to find a suitable M function to comply with the next request given the following priority order:

(1)

Suitable subsets of tasks Ψ_q are created, such that each subset is schedulable by the local scheduling algorithm running on its assigned PE_q.

(2)

Respect task affinity score.

(3)

Optimize resource usage.

Owing to its increased complexity, the mapping function should be implemented at the intermediate (Fog) level.

5.2.1. Local Task Scheduling

According to Vestal [25], a task set is schedulable if the deadline of each task τ_i is greater than or equal to its worst-case response time.

$${\mathit{R}}_{\mathit{i}}\le {\mathit{D}}_{\mathit{i}}$$

(6)

where WCET is the maximum duration between completion and activation of each job of task τ_i [43].

$${\mathit{R}}_{\mathit{i}}={\mathit{C}}_{\mathit{i}}+{\mathit{I}}_{\mathit{i}}$$

(7)

where I_i is the inference from higher priority tasks.

The value of R_i is dependent on the scheduling algorithm and can be iteratively computed as the least fixed point of (8), where hp(i) is a subset of tasks with a priority higher than or equal to that of task τ_i but not containing τ_i, C_i is the worst-case execution time, and T_i is the task period [25]:

$${\mathit{R}}_{\mathit{i}}={\displaystyle \sum }_{{\mathbf{\tau }}_{\mathit{j}}\mathbf{ϵ}\mathit{h}\mathit{p}\left(\mathit{i}\right)}\lceil \frac{{\mathit{R}}_{\mathit{i}}}{{\mathit{T}}_{\mathit{j}}}\rceil {\mathit{C}}_{\mathit{j}}$$

(8)

Scheduling in mixed-critic systems is feasible if the following two conditions are satisfied [44].

Condition 1.

If all jobs run no more than their C_i,j values for the current L_j criticality mode, then all jobs with a criticality level L_i higher than or equal to L_j must be completed before their deadline.

Condition 2.

If at least one job exceeds its C_i,j execution time value for the current L_j criticality mode, then the current criticality mode changes to the next criticality value (L_j+1), for which Condition 1 must also be true.

Thus, Condition 1 must be satisfied for all the criticality modes available in the system.

For a k criticality mode (L_k) on the PE_q processing element, Equation (8) becomes:

$${\mathit{R}}_{\mathit{i},\mathit{k},\mathit{q}}={\displaystyle \sum }_{{\mathbf{\tau }}_{\mathit{j}}\mathbf{ϵ}\mathit{h}\mathit{p}\left(\mathit{i}\right)}\lceil \frac{{\mathit{R}}_{\mathit{i}.\mathit{k}.\mathit{q}}}{{\mathit{T}}_{\mathit{j}}}\rceil {\mathit{C}}_{\mathit{j},\mathit{k},\mathit{q}}$$

(9)

Checking Condition 1, for the L_k criticality mode, reduces to the verification of Condition (6), particularly for the L_k criticality mode on the PE_q processing element:

$${\mathit{R}}_{\mathit{i},\mathit{k},\mathit{q}}\le {\mathit{D}}_{\mathit{i}}$$

(10)

is true for all the tasks in the task set.

Moreover, for all deadlines to be satisfied, the processing units must not be overloaded with the tasks. Thus, a necessary but sufficient condition bound on the load metric for any processing unit is given by [44]:

For each q = 1…p:

$${\mathit{U}}_{{\mathit{L}}_{\mathit{j}}{\mathit{\Psi }}_{\mathit{q}}}\le 1, \mathit{j}=1\dots \mathit{l}$$

(11)

For periodical/sporadic tasks, the total processor utilization in execution mode L_j is:

$${\mathit{U}}_{{\mathit{L}}_{\mathit{j}}{\mathit{\Psi }}_{\mathit{q}}}={\displaystyle \sum }_{{\mathbf{\tau }}_{\mathit{i}}\mathbf{ϵ}\mathit{h}\mathit{c}\left({\mathit{L}}_{\mathit{j}}\right), \mathit{k}=\mathit{j}}^{\mathit{k}\le \mathit{l}}\frac{{\mathit{C}}_{\mathit{i},\mathit{k},\mathit{q}}}{{\mathit{T}}_{\mathit{i}}}$$

(12)

where hc(L_j) is a subset of tasks from Ψ_q with criticality higher than or equal to L_j and l is the number of criticality levels. L_{k = j…l} represents the criticality levels higher than or equal to L_j, and C_i,k,q represents the WCET of task i, which runs in criticality mode k on the processing element PE_q.

Moreover, many local task-scheduling algorithms have sufficient conditions. For example, a sufficient condition for a local task scheduling algorithm called EDF-VD when the system has two criticality levels has been proven to be [27].

$$\mathit{m}\mathit{a}\mathit{x}\left({\mathit{U}}_{{\mathit{L}}_1{\mathit{\Psi }}_{\mathit{q}}},{\mathit{U}}_{{\mathit{L}}_2{\mathit{\Psi }}_{\mathit{q}}}\right) \le \frac{3}{4}$$

(13)

5.2.2. Affinity Score Deviation Function

The task affinity score can be set statically by the task creator or computed using an algorithm similar to that proposed in Section 5.1, based on the resources needed by the task and the task computation time for different processing elements. The mapping function should also consider the affinity score for each task, provided that the local feasibility conditions are not violated. To evaluate this, we propose a function representing the total affinity-score deviation A_dτ for a particular mapping as follows:

$${\mathit{A}}_{\mathit{d}\mathit{\tau }}={\displaystyle \sum }_{\mathit{i}=1}^{\mathit{n}}\left(\mathit{p}-{\mathit{A}}_{\mathit{i},q}\right)$$

(14)

where q is the index of subset Ψ containing task i and p is the number of PEs (p is also equal to the highest affinity score value).

Our goal is to minimize A_dt with respect to the feasibility conditions imposed by the local scheduler (Condition 1).

5.2.3. Proposed Mapping Function—Best Affinity First

Based on the methods used to set the affinity score introduced in the first paragraphs of this section, we developed a mapping algorithm (see Algorithm 4) that considers the affinity value when partitioning tasks to PEs, namely Best Affinity Fit (BAF). This algorithm is evaluated later in the experimental section.

Algorithm 4: Best Affinity Fit (BAF)

Input: ${\tau }_i$

Output: ${\Psi }_q$, $U_{L_j{\Psi }_q}$

1  for $i\in \left\{1, 2, \dots , n\right\}$ do

2     $assign\leftarrow 1$

3    $\left\{maxVal,index\right\}\leftarrow \max \left(A_{i,q}\right)$

4    while $U_{L_j{\Psi }_{index}}+C_{i,j,index}/T_i>1$ do

5     $A_{i,index}\leftarrow 0$

6    $\left\{maxVal,index\right\}\leftarrow \max \left(A_{i, q}\right)$

7     if $maxVal=0$ then

8      $assign\leftarrow 0$

9     end if

10    end while

11    if $assign\leftarrow 1$ then

12     $add\left({\Psi }_{index}, {\tau }_i\right)$

13     $U_{L_j{\Psi }_{index}}\leftarrow U_{L_j{\Psi }_{index}}+C_{i,j,q}/T_i$

14    end if

15  end for

For each task ${\tau }_i$:

Step  1:

First, we assume that the task can be assigned to PE. Therefore, we set the variable assigned to 1. We find the PE with the highest affinity score for task i, which has enough space to accept task τ_i (PE utilization must respect Equation (14)).

Step  2:

If a certain PE_index does not have sufficient space for task i, the affinity score A_i,index to 0. If there are no PEs that can host task i, then the set is assigned to 0.

Step  3:

If we have found a PE_indexwhich can accept task τ_i, then add task i to subset Ψ_index and update the PE utilization.

6. Examples and Comparative Evaluation

We conducted a series of simulation experiments to evaluate the effectiveness of our mapping technique: the Best Affinity Fit. It has been compared against two other relevant mapping methods, Best Fit Decreasing Utilization (BFDU) and Best Fit Decreasing Criticality (BFDC), which are some of the most frequently used algorithms [45]. BAF dispatches tasks to processors according to the Affinity value, while BFDU and BFDC tasks are first ordered by decreasing utilization for BFDU or by decreasing criticality for BFDC and then assigned to each processor. The processors are also ordered by their decreasing utilization. Two affinity assignment strategies were also evaluated: one allocates values according to the WCET, whereas the other depends on the criticality level.

6.1. Random Task Set Generation

All tasks were randomly generated in Matlab (R2018b) using the task set generation algorithm introduced in [46], which is a slight modification of the workload generation algorithm proposed by Guan et al. [47]. We consider a dual-criticality system {Lo, Hi}. Each new task, τ_i, was generated as follows:

Task criticality level: L_i = Hi with probability P_Hi; otherwise, L_i = Lo.

Task period: T_i is a randomly generated value drawn from a uniform distribution [10, 100].

Task deadline: D_i = T_i, because of the implicit deadline constraint.

The utilization of each task U_i,j,q is a matrix p×l, where p is the number of PEs in the system and l represents the number of criticality levels. To generate the utilization values, we considered five input parameters [46].

$U_{bound}$:

$$\max \left(U_{Lo}\left(\tau \right), U_{Hi}\left(\tau \right)\right)=U_{bound}$$

(15)

$$U_{Lo}\left(\tau \right)={\displaystyle \sum }_{{\tau }_i\in \pi ,q=0}^{q<p}{U}_{i, \mathit{Lo},q}$$

(16)

$$U_{Hi}\left(\tau \right)={\displaystyle \sum }_{{\tau }_i\in Hi\left(\pi \right),q=0}^{q<p}{U}_{i, \mathit{Hi},q}$$

(17)

where π is the task set, and Hi(π) is a subset of π that contains only Hi criticality tasks.

• [U_l, U_U] Utilizations are generated uniformly from this range, with:
  $0\le U_L\le U_U\le 1$
  [Z_l, Z_U]: Ratio between the Hi-criticality utilization of a task and its Lo-criticality utilization with
  $0\le Z_L\le Z_U$.
• WCET for criticality level Lo:
  $C_{i,\mathit{Lo},q}=U_{i,\mathit{Lo},q}·T_i$.
• WCET for criticality level Hi:
  $C_{i,\mathit{Hi},q}=U_{i,\mathit{Hi},q}·T_i if L_i=Hi$. $\mathrm{Otherwise}, C_{i, \mathit{Hi},q}=C_{i, \mathit{Lo},q}$.
• The affinity values were assigned using one of the two methods described in the previous section.

6.2. Best Affinity First Performance Evaluation Results

The parameters used to generate the task sets are provided in graph captions. For each plot, one parameter was varied, whereas the others were fixed. The utilization of each processor must satisfy the necessary condition bound on the load metric for any m-processing unit system (9). Each data point is determined by randomly generating 100 task sets.

6.2.1. Proposed MC Task Model vs. Classical MC Task Model

The main differences between our proposed model and the classical MC task model are as follows: a new parameter called affinity is introduced, and the computation time, in the form of worst-case execution time, becomes a bi-dimensional array (with one dimension being the criticality level and one the processing element).

Table 1. Parameter Comparison Between the Proposed MC Task Model and the Classical MC Task Model.

Parameters	Proposed MC Task Model	Classical MC Task Model
Period	$T_i$	$T_i$
Deadline	$D_i$	$D_i$
Criticality level	$L_i$	$L_i$
WCET	$\left\{C_{i,\mathrm{j},q}\right| j\in 1\dots l, q\in 1\dots p\}$	$\left\{C_{i,\mathrm{j}}\right| j\in 1\dots l\}$
Affinity	$\left\{A_{i,q}\right|q\in 1\dots p\}$	-

briefly illustrates the differences.

Next, we consider an example of an application model using both the proposed and classical models. We also consider a sensor node modeled as a dual-criticality system with three processing elements. The sensor node runs the aforementioned IoT sensing application and consists of four tasks. The affinity score for each task is computed in

Table 2. Task Model Example 1.

Task	Functionality	Proposed MC Task Model						Classical MC Task Model
		${\mathit{T}}_{\mathit{i}}$	${\mathit{D}}_{\mathit{i}}$	${\mathit{L}}_{\mathit{i}}$	${\mathit{C}}_{\mathit{i},\mathit{j},\mathit{q}}$	Ai,q		Ti	Di	Li	Ci,j,q	Ti
						Algorithm 1	Algorithm 2
$M_1$	sensing	8	8	Lo	$\begin{array}{cc}2& 2\\ 1& 1\\ 4& 4\end{array}$	$\begin{array}{ccc}2& 3& 1\end{array}$	$\begin{array}{ccc}3& 1& 2\end{array}$	8	8	Lo	4	4
$M_2$	sensing	17	17	Hi	$\begin{array}{cc}4& 7\\ 6& 8\\ 3& 5\end{array}$	$\begin{array}{ccc}2& 1& 3\end{array}$	$\begin{array}{ccc}1& 3& 2\end{array}$	17	17	Hi	6	8
$M_3$	log	24	24	Lo	$\begin{array}{cc}5& 5\\ 8& 8\\ 7& 7\end{array}$	$\begin{array}{ccc}3& 1& 2\end{array}$	$\begin{array}{ccc}3& 1& 2\end{array}$	24	24	Lo	8	8
$M_4$	communication	42	42	Hi	$\begin{array}{cc}5& 10\\ 8& 12\\ 13& 17\end{array}$	$\begin{array}{ccc}3& 2& 1\end{array}$	$\begin{array}{ccc}2& 3& 1\end{array}$	42	42	Hi	13	17

according to the two algorithms presented previously: Algorithms 1 and 2 (because the number of PEs exceeds the number of criticality levels).

The tasks are mapped on processing elements using the following methods: on the one hand, BAF (by computing the affinity score according to Algorithm 1), and BFDU and BFDC using the classical MC task model, considering the worst-case execution time vector (the line with the highest values from the C_i,j,q matrix). The task-mapping results are presented in

Table 3. Task Mapping Example 1.

BAF (using Algorithm 1)	$\left\{M_3, M_4\right\}\in {\Psi }_1$ $\left\{M_1\right\}\in {\Psi }_2$ $\left\{M_2\right\}\in {\Psi }_3$	BFDU	$\left\{M_1, {\mathrm{M}}_2\right\}\in {\Psi }_1$ $\left\{M_3,M_4\right\}\in {\Psi }_2$
BAF (using Algorithm 2)	$\left\{M_1, M_3\right\}\in {\Psi }_1$ $\left\{M_2, M_4\right\}\in {\Psi }_2$	BFDC	$\left\{M_2, {\mathrm{M}}_4\right\}\in {\Psi }_1$ $\left\{M_1,M_3\right\}\in {\Psi }_2$

.

For a system with four criticality levels, {Lo, M1, M2, Hi}, and three processing elements, the affinity score is computed in

Table 4. Task Model Example 2.

Task	Functionality	Proposed MC Task Model						Classical MC Task Model
		${\mathit{T}}_{\mathit{i}}$	${\mathit{D}}_{\mathit{i}}$	${\mathit{L}}_{\mathit{i}}$	${\mathit{C}}_{\mathit{i},\mathit{j},\mathit{q}}$	Ai,q		Ti	Di	Li	Ci,j,q	Ti	Di	Li
						Algorithm 1	Algorithm 3
$M_1$	sensing	15	15	Lo	$\begin{array}{cccc}2& 2& 2& 2\\ 1& 1& 1& 1\\ 5& 5& 5& 5\end{array}$	$\begin{array}{ccc}2& 3& 1\end{array}$	$\begin{array}{ccc}3& 2& 1\end{array}$	15	15	Lo	5	5	5	5
$M_2$	sensing	20	20	M1	$\begin{array}{cccc}4& 5& 5& 5\\ 3& 6& 6& 6\\ 6& 9& 9& 9\end{array}$	$\begin{array}{ccc}3& 2& 1\end{array}$	$\begin{array}{ccc}2& 3& 1\end{array}$	20	20	M1	6	9	9	9
$M_3$	sensing	30	30	Hi	$\begin{array}{cccc}5& 7& 9& 12\\ 8& 10& 13& 15\\ 4& 8& 11& 14\end{array}$	$\begin{array}{ccc}3& 1& 2\end{array}$	$\begin{array}{ccc}2& 1& 3\end{array}$	30	30	Hi	8	10	13	15
$M_4$	sensing	40	40	M2	$\begin{array}{cccc}8& 14& 17& 17\\ 9& 10& 13& 13\\ 11& 14& 16& 16\end{array}$	$\begin{array}{ccc}1& 3& 2\end{array}$	$\begin{array}{ccc}3& 2& 1\end{array}$	40	40	M2	11	14	16	16

according to Algorithms 1 and 3 (because the number of PEs is lower than the number of criticality levels).

The tasks were mapped using the same partitioning heuristics as those listed in Table 3. The mapping results are presented in

Table 5. Task Mapping Example 2.

BAF (using Algorithm 1)	$\left\{M_2, M_3\right\}\in {\Psi }_1$ $\left\{M_1,M_4 \right\}\in {\Psi }_2$	BFDU	$\left\{M_1, M_2, M_3\right\}\in {\Psi }_1$ $\left\{M_4 \right\}\in {\Psi }_2$
BAF (using Algorithm 3)	$\left\{M_1, M_4\right\}\in {\Psi }_1$ $\left\{M_2\right\}\in {\Psi }_2$ $\left\{M_3\right\}\in {\Psi }_3$	BFDC	$\left\{ M_2, M_3, M_4\right\}\in {\Psi }_1$ $\left\{M_1\right\}\in {\Psi }_2$

.

In conclusion, by considering a matrix of computation times instead of a vector, we can model the computation time not only as a function of the criticality level but also by considering the influence of the hardware particularities of the processing elements on the task computation time. Moreover, the introduction of a new parameter called affinity offers a certain degree of versatility to the task model because this parameter can be computed according to any optimization function. Our proposed model offers better resource management by achieving lower total utilization, as discussed in the following paragraphs.

6.2.2. BAF vs. BFDU

In the following paragraphs, we compare the results of different mapping functions in terms of total utilization. To reduce the gap between the two MC task models for all mapping functions, the computation time was considered as a matrix, as in our proposed model. The only difference between the task models used for BAF and BFDU is that BAF also uses the affinity parameter, whereas BFDU and BFDC do not. For the BAF, affinity values were assigned according to the WCET in the Hi mode for Hi-criticality tasks and the WCET in the Lo mode for Lo-criticality tasks.

In Figure 3a, the task set utilization bound (x-axis) ranges from 0.4 to 1.0 times the number of processors in steps of 0.1; in step 3.b, the number of processors (x-axis) ranges from 2 to 12 in steps of 2. In Figure 3c, the number of tasks (x-axis) ranges from 10 to 24 in steps 2 and 3.d the percentage of Hi-criticality tasks (x-axis) was varied, ranging from 0.2 to 0.6 in steps of 0.1. The average total utilization is depicted on the y-axis.

From the simulation results presented in Figure 3, we obtain the following results: by varying the number of processors, BAF shows better results with an average value of 1.64, meaning almost 60%, and by varying the number of tasks, there is a reduction in the total utilization when using BAF compared to BFDU with 1.29, meaning about 40%.

6.2.3. BAF vs. BFDC

In this case, affinity values were assigned according to the criticality level of each task. Four experiments were conducted (Figure 4) similar to the first set. Each data point is determined by randomly generating 100 task sets. The average affinity deviations for the task sets used for Figure 4a are depicted in Figure 5 based on Equation (14).

In Figure 4, by varying the number of processors, BAF has better results with an average value of 1.2, meaning about 43%, and by varying the number of tasks, there is a reduction in the total utilization when using BAF compared to BFDC with 0.51, meaning about 15%.

In conclusion, the average total utilization functions for BFDU and BFDC have a similar growth tendency compared to BAF, which is reaching better results with almost a constant factor (between 0.5-1 for BFDU and between 0.2-0.7 for BFDC). The case where the BAF function growth is significantly better is when the number of processors/ processing units is increased.

7. Conclusions and Future Perspectives

With the continuous improvement in IoT technology and increasingly complex applications, some of them with strict timing constraints are envisioned to be developed. Under these circumstances, the implementation of new and effective RT-IoT architectures may offer much-required technological support. From our perspective, combining the underlying mechanisms of mixed-criticality systems with the Internet of Things offers a huge opportunity for the research and development of complex distributed applications.

In this study, we mathematically formalized the MC-IoT concept. We also propose an effective methodology for scheduling mixed-criticality applications on IoT nodes. A novel hardware-aware extension of the classical MCS task model was proposed to support this methodology, along with a new task mapping function: Best Affinity First.

The performance evaluations prove that BAF has significantly better results than the existing BFDU and BFDC task-mapping techniques in terms of total task utilization. The difference in performance increases with the number of processing elements. These results are underlying the suitability of the proposed partitioning algorithm to the IoT scenario where more numerous processing elements are used compared to the classic MCSs.

Moreover, a hardware-aware scheduling approach increases the potential for the development of the distributed heterogeneous systems running real-time mixed-criticality applications such as 6G networks and their applications.

The proposed model can be further extended in order to include other aspects besides the hardware affinity. Depending on the other resources in the system that need to be more efficiently used, other functions for setting the affinity score can also be developed.