Fuzzy Logic Testing Approach for Measuring Software Completeness

Abstract

Due to advancements in science and technology, software is constantly evolving. To adapt to newly demanded requirements in a piece of software, software components are modified or developed. Measuring software completeness has been a challenging task for software companies. The uncertain and imprecise intrinsic relationships within software components have been unaddressed by researchers during the validation process. In this study, we introduced a new fuzzy logic testing approach for measuring the completeness of software. We measured the fuzzy membership value for each software component by a fuzzy logic testing approach called the fuzzy test. For each software component, the system response was tested by identifying which software components in the system required changes. Based on the measured fuzzy membership values for each software component, software completeness was calculated. The introduced approach scales the software completeness between zero and one. A software component with a complete membership value indicates that the software component does not require any modification. A non-membership value specifies that the existing software component is no longer required in the system or that a new software component is required to replace it. The partial membership value specifies that the software component requires few new functionalities according to the new software requirements. Software with a partial membership value requires partial restructuring and design recovery of its components. Symmetric design of software components reduces the complexity in the restructuring of software during modification. In the study, we showed that by using the introduced approach, high-quality software that is faultless, reliable, easily maintained, efficient, and cost-effective can be developed.

1. Introduction

Soft computing techniques such as fuzzy logic have been extensively used in software engineering to address uncertainty and imprecision [1,2,3,4]. In this study, we measured software completeness by introducing the fuzzy logic testing approach. In this study, we classified the software requirements into different types. The fuzzy membership value scaled the software completeness between zero and one. The software completeness value exactly extracted the software components that required changes according to the latest requirements. By this approach, complete membership, partial membership, and non-membership values for the software components were determined by validating the software components against its requirements.

A software component, as defined by Pang [5], is “a software unit of functionality that manages a single abstraction.” In the literature, completeness has been considered as an essential quality attribute. Davis et al., in [6], measured the quality of software requirements by quality attribute (e.g., completeness) and, in [7], forward engineering completeness for software was measured, whereas incompleteness can cause unsuccessful effects [8,9]. Software completeness is defined as the state of software having all the required software components with appropriate functionality and features and it certifies that no functionality or feature is left out.

The evolution of software occurs due to technology drivers and business needs. During the software evolution process, its functionalities and features continuously change, and as software size increases, ultimately, its complexity also increases. Software products are changed based on global changes in business. The change process comprises modifications, additions, and deletions of software components or modules in the software system. Software generally grows in terms of features, so it must be designed to accommodate the demanded changes and satisfy a set of new requirements [7,10,11,12].

A feature is defined as [13] “an optional or incremental unit of functionality.” S_F = {f₁, f₂, f₃ … f_r−1, f_r}, where S_F represents a set of features within the system. In this study, each feature is designed based on a requirement or set of requirements R and is verified with the expected system response $S{R}_k{C}_i$ to a specific status, $f_q$ = (R, $S{R}_k{C}_i$), where (1 ≤ k ≤ n) and (1 ≤ q ≤ m). $S{R}_k{C}_i$ denotes the system response tested for the i^th software component $C_i$, that contains n number of system responses. System response is defined as the set of all possible expected output results for each input data, to determine whether a feature is correctly working or not [14,15,16,17,18].

Consider an example where a client asks the software engineer about an object F: what is F made of → component, what can F do → feature. Examples of components include fee checkers, registered students, and publications. Examples of features include the following: can detect the total amount of fees paid by the students, can determine the total number of students registered in a specific course during a given semester at a university, can generate reports about faculty publications.

According to GenVoca theory, now called feature-oriented software development, features are implemented as applied functions, whereas the compositions of features are implemented as composite functions, which are signified by •. In most cases, the order of composition does not affect the process, but in other cases, the composition order matters [19].

$$\begin{array}{r}\mathrm{Books}\\ \mathrm{On-hold} • \mathrm{Books}\\ \mathrm{Purchase} • \mathrm{Books}\\ \mathrm{On-hold},\mathrm{Purchase} • \mathrm{Books}\end{array}$$

Software requirements depict functionalities and features of the system to be developed in descriptive form, as well as constraints on the services of the system. The software requirement known as completeness is defined as a set of requirements or a single requirement that satisfies the demanded quality, such that no information regarding the software to be developed is left out [9,20]. Fuzzy logic is based on membership values, where non-membership is represented by zero, partial membership values vary between zero and one, and complete membership is represented by one [21,22,23].

The novel fuzzy logic testing approach identified the missing functionalities, features, and new software requirements. The software was developed by component-based development. Therefore, by applying the fuzzy test, errors were extracted from the exact functionality. The developed software components were reliable, upgradable, cost effective, reusable, and time effective and were developed with improved quality. In this study, the authors classified the software requirements into functional, non-functional, inverse, non-inverse, content, and non-content requirements. The classification of software requirements helped to represent the software requirements in the precise form and in identification of imprecise software requirements.

The rest of the sections in this paper are organized as follows: Section 2 defines the preliminary concepts, Section 3 describes related work, Section 4 describes fuzzy membership value of the software components, Section 5 explains the measurement approach for software completeness, Section 6 describes materials and methods, Section 7 shows results, and finally, Section 8 concludes the paper.

2. Preliminaries

In this section, we define the necessary concepts related to software requirements, fuzzy functionality similarity between software modules, completeness of software components, linguistic variables, proof of software security level, and programming language faults.

2.1. Software Requirements

Extracting rules or requirements from natural language has been a challenging task. In this study, we used the modality concept to define the requirement types. Modality, as defined by Sleimi et al., is [24] “a verb indicating the modality of the action, e.g., must, shall.”

(i). Functional Requirements and Non-Functional Requirements: Functional requirements (FR) describe how the system should act according to statements of services defined in the software specification. One example of a FR is the following: the system shall allow users to search all sets of books related to course work or select subsets from it. In this modality, shall → functional requirement.

Non-Functional Requirements (NFR) are limitations on system functions or services such as timing, security, usability, availability, performance, etc. NFR are applied to the entire system, not to specific features or services of the system. One example of an NFR is the following: the system shall be available 24/7. Modality: the requirement in which “shall” is used to represent the limitations on system functions → non-functional requirement. Data privacy is a non-functional requirement that assures the personal data are secure from unauthorized access. In data privacy, the following types of information are considered sensitive, by legal mandates and also by the general public [25]: personally identifiable information (PII), personal health information (PHI), personally identifiable financial information (PIFI), and student records. Consider an example of an NFR related to data privacy: the system shall protect personal data from unauthorized users. Timing issues: This is an outset with the inclusion of timing definition in the software requirements. In testing timing issues, the task of the software tester is to find the temporal error in the input situations, if the execution takes the shortest or longest time to complete the tasks [26].

(ii). Inverse Requirements and Non-Inverse requirements: As defined in [27,28], these can be functional requirements or non-functional requirements in which customers specify “what the system shall not do” or the system shall display a text box in red if the user does not enter the data according to the specified condition of the system. Inverse requirements that describe the functional aspects of the system are represented by IR, whereas those that describes non-functional aspects of the system are known as non-inverse requirements, represented by NIR. An example of an IR is the following: the system shall not allow users to enroll in more than 12 credit hours in a semester. An example of an NIR is the following: the system shall not be available from 12:00 a.m. to 3:00 a.m. The modality “shall” in negative form (shall not) → inverse requirements, whereas the requirement in which it is used to represent the limitations on system functions → non-inverse requirements.

(iii). Content Requirements and Non-Content Requirements: As defined by the authors, these can be functional requirements or non-functional requirements in which customers specify “what the system must do.” Content requirements are used for customized software. Content requirements are represented by CR, whereas non-content requirements are represented by NCR. The purpose of content requirements is to elaborate on the differences between the software requirements demanded by different organizations. A CR would be as follows: the system must ask the student to complete the teacher evaluation form before redirecting him/her to the main screen. An NCR would be as follows: the system must respond to user requests within 3 s. The statements of requirement with the modal verb “must” are specified as content requirements, whereas the requirement in which the modality “must” is used to represent the limitation on the system, e.g., performance (within 3 s) → non-content requirement.

In this study, each software component was validated against its software requirements. The software components were developed by using the process of component-based development. To determine whether the software components were required in the software, we used a requirements validation framework [7]. As symmetry creates balance in design, therefore, each software component was developed to be symmetric in design, whereas a few software components were developed to be asymmetric in order to grab attention. Software completeness denotes the membership value that covers the requirements in new software. For this, we used a fuzzy test. Software requirements were gathered from the users of the software in natural language. The requirements were classified using the concept of modality. The inverse and content requirements were classified based on different demanded features and functionalities or constraints within the same software component or overall software.

2.2. Fuzzy Functionality Similarity between Software Modules

Consider software S, which consists of n software components $C_i$. $R_j^i$ represents the i^th software component, whereas j represents the software requirement that corresponds to a particular functionality and feature of a software component. $F{C}_k^i$ represents the functionality of the component, i represents the software component and k represents the functionality of the i^th software component. In this study, we tested two software modules, $M_f$ and $M_p$. We used the fuzzy if–then rule to determine the functionality similarity between software modules.

$$\mathrm{If} R_j^i \left(M_f\right) \mathrm{is} C_m \mathrm{then} R_j^i \left(M_p\right) \mathrm{is} C_m$$

Figure 1 represents fuzzy functionality similarity, and an increase in color shade denotes an increase in the functionalities within software components, whereas white represents software components with single functionality.

2.3. Completeness of Software Components

$$\begin{gathered} \left(R_1^i , R_2^i, R_3^i, \dots R_n^i\right)\in \mathrm{S} \\ {R}_j^i \stackrel{\wedge}{=} F{C}_k^i \end{gathered}$$

As in the experimental requirements, $R_1^i,$ $R_2^i, R_3^i$, $R_4^i$, $R_5^i$, $R_6^i,$ $R_7^i$, $R_8^i,$ $R_9^i, \dots R_n^i$ represent $F{R}_1$ & $NF{R}_1$, $F{R}_2$ & $NF{R}_2$, $C{R}_3$ & $NC{R}_3$, $I{R}_4$ & $NI{R}_4$, $I{R}_5$ & $NI{R}_5$, $F{R}_6$ & $NF{R}_6$, $F{R}_7$ & $NF{R}_7$, $F{R}_8$ & $NF{R}_8$, $C{R}_9$ & $NC{R}_9$, $\dots$ $F{R}_{10}$ & $NF{R}_{10}$, respectively.

$$\begin{array}{c}F{R}_1 \& NF{R}_1 \stackrel{\wedge}{=} F{C}_1^i\\ F{R}_2 \& NF{R}_2 \stackrel{\wedge}{=} F{C}_2^i \\ C{R}_3 \& NC{R}_3 \stackrel{\wedge}{=} F{C}_3^i \\ I{R}_4 \& NI{R}_4 \stackrel{\wedge}{=} F{C}_4^i\\ :\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}:\\ :\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}:\\ F{R}_n \& NF{R}_n \stackrel{\wedge}{=} F{C}_n^i \end{array}$$

Each software component consists of one or more features and functionalities within a single component.

$${\displaystyle \sum }_{i=1}^{n}F{C}_k^i= F{C}_1^i + F{C}_2^i + F{C}_3^i + \dots +F{C}_n^i$$

2.3.1. Complete Membership

Considering the software component $C_{\mathrm{v}}$ with three functionalities $F{C}_1^{\mathrm{v}}$, $F{C}_2^{\mathrm{v}}$ and $F{C}_3^{\mathrm{v}}$, the expected system responses are represented by $S{R}_1{C}_{\mathrm{v}}$, $S{R}_2{C}_{\mathrm{v}}$ and $S{R}_3{C}_{\mathrm{v}}$, respectively. All functionalities of the software component are represented in blue if the software component possesses a complete membership value. Figure 2 represents a software component that possesses complete membership.

2.3.2. Partial Membership

Considering the software component $C_{\mathrm{y}}$ with three functionalities $F{C}_1^{\mathrm{y}}$, $F{C}_2^{\mathrm{y}}$ and $F{C}_3^{\mathrm{y}}$, the expected system responses are represented by $S{R}_1{C}_{\mathrm{y}}$, $S{R}_2{C}_{\mathrm{y}}$ and $S{R}_3{C}_{\mathrm{y}}$, respectively. This software component possesses a partial membership value; the functionalities with complete membership values are represented in blue, whereas the functionality with a partial membership value is represented in brown. Figure 3 represents a software component that possesses partial membership.

2.3.3. Non-Membership

Considering the software component $C_{\mathrm{z}}$ with three functionalities $F{C}_1^{\mathrm{z}}$, $F{C}_2^{\mathrm{z}}$ and $F{C}_3^{\mathrm{z}}$, the expected system responses are represented by $S{R}_1{C}_{\mathrm{z}}$, $S{R}_2{C}_{\mathrm{z}}$ and $S{R}_3{C}_{\mathrm{z}}$, respectively. This software component possesses a non-membership value, and functionalities with non-membership values are represented in green. Figure 4 represents a software component that possesses non-membership.

2.4. Linguistic Variable

Linguistic variable [29] values are in sentences or words of natural language. For example, “change” is linguistic variable, whereas “minor changes”, “major changes” and “complex changes” are its values. It represents the semantic concepts in mathematical representation [30], which consist of more than one term, called a fuzzy set. Linguistic variable uses the values of fuzzy variable. Consider the linguistic variable: (c, T(c), U, M), the variable name is represented by c, whereas its term set is denoted by T(c), and each value defined in U is a fuzzy variable. M: each variable consists of associated semantic rules (membership), c: “change” is a linguistic variable.

2.5. Proof of Software Security Level

Software security levels are tested in different perspectives during software development in order to build secure software that is invulnerable before it deployment. As described by Murray [31], a software program is insecure and imperfect if it “deviates from its intended behavior (e.g., arbitrary code execution), or reveal sensitive data (data leakage), or modify data that should remain protected (integrity violation).” Researchers have conducted research on proving secure software, such as identifying security flaws [32] in protocols and programs and also on transport layer security (TLS) implementations [33], operating system kernels [34], cryptographic algorithms [35], etc.

In order to make the software secure, we tested the software components with the fuzzy test for all possible system responses ($S{R}_k{C}_i$) by entering different input data, e.g., actions ($A_1$–$A_n$). The input data are entered implicitly or explicitly in order to check all the possible positive and negative system responses.

2.6. Programming Language Faults

Software program $P_S$ consists of n number of software components $C_i$, (1 ≤ i ≤ n), in which software component $C_{34}$ in $P_S$ is faulty, if $P_S$ cannot be correctly verified with respect to existing implementation of $C_i$, but it can be verified by the implementation specification for $C_i$. Figure 5 represents types of programming language faults.

Types of programming language faults are as follows [17].

2.6.1. Algorithm Fault

This fault occurs in the program when the logic of the algorithm does not deliver the correct output (result) for a certain input due to incorrect processing, e.g., the system should display a list of journals recognized by Web of Science if the user selects the Web of Science indexing option but the system displays a list of journals recognized by EI (Engineering Index) due to a fault.

2.6.2. Computational Fault

This occurs when the implementation of program is wrong or the program is not capable of computing the desired output. The system displays unexpected output due to combining the floating point and integer variables.

2.6.3. Syntax Fault

This fault occurs in the program due to incorrect syntax,

e.g., >> y = y ∗ ((3 + 5 ∗ y); fault: % extra parenthesis

>> x = ‘Chongqing fault: % missing quote

2.6.4. Documentation Fault

The program documentation describes the software in the form of illustrations and written text. It also describes how to operate it or use it. A documentation fault occurs when this does not match with the program.

2.6.5. Overload Fault

During the program implementation, a stack, array or queue is used for memory purposes. If more elements are added than its limited size, then an overload fault occurs in the program.

2.6.6. Timing Fault

If the system does not respond to a user due to the occurrence of a failure in the program, then the system reports a timing fault.

2.6.7. Hardware Fault

Software companies are trendsetters for the hardware industry. Hardware is updated in order to support new features and functionalities of the software. This fault occurs if the hardware does not work properly with the software. This problem can be avoided if hardware specifications are completely specified in the technical feasibility.

2.6.8. Software Fault

Software faults occur if the software does not function properly, or is not supported by the operating system, such Microsoft Windows, Linux, Apple macOS, Apple iOS or Android.

2.6.9. Omission Fault

This fault occurs “when we fail to enter correct information” [17], e.g., in a program in which a variable is not initialized (fault: int y =; correct: int y = 10;).

2.6.10. Commission Fault

This fault occurs “when we enter something into a representation that is incorrect” [17], e.g., a program in which char is initialized to int variable incorrectly (int n = a;).

3. Related Work

In [36], MacDonell and Gray applied a fuzzy logic modeling tool set to estimate the size of a product and to determine the risks of the project. Kacprzvk and Yager [30] used fuzzy logic on data sets for the extraction of linguistic summaries, and the researchers also focused on which methods could be applied to assess the goodness of the linguistic summary. According to Barry Boehm [37], software contains fixed features such as the database size, product complexity and required reliability that are not controllable by management.

Most size estimations are calculated based on imperfect, incomplete knowledge and assumptions about software features [38]. In software engineering practices, it has been observed that more user engagement in the software development process increases the software success rate [39]. Researchers [40,41,42] concluded that user feedback plays a vital role in software quality and in detecting new design concepts for software evolution.

In the literature, researchers concluded that fuzzy logic can be used in different phases of the system development life cycle (SDLC), such as for system design [43], software testing [44,45], estimation of product size, estimation of product risk related to which diverse input parameters are used [46], etc. In [47], researchers applied fuzzy logic for the prioritization of test cases which minimizes the cost of regression testing. Figure 6 represents the extraction of different solutions/activities when fuzzy logic is applied in different phases of the system development life cycle (SDLC) [48,49,50,51,52,53,54,55,56].

In the literature, different techniques for gathering requirements have been introduced by researchers [57,58,59,60], such as one-on-one interviews, group interviews, facilitated sessions, joint application design (JAD), a group support system (GSS), prototyping, questionnaires, use cases, following people around, requests for proposals (RFPs) and brainstorming. In a survey conducted with software engineers, these techniques were also found to possess limitations. Therefore, in this study, we used a requirements validation framework [7] for the validation of software components. In [61], researchers introduced tsDetect which is an open source tool used to detect test smells. tsDetect can detect following types of test smells: assertion roulette, conditional test logic, constructor initialization, default test, duplicate assert, eager test, empty test, exception handling, general fixture, ignored test, lazy test, magic number test, mystery guest, redundant print, redundant assertion, resource optimism, sensitive equality, sleepy test and unknown test. In this study, we did not find any similar approaches in which the completeness of software was measured by our introduced approach.

4. Fuzzy Membership Value of the Software Components

To measure the fuzzy membership value for each software component, we used the introduced fuzzy logic testing approach called the fuzzy test.

Definition 1.

Let$m_{k,i}$represent the membership value of the system response for software component$C_i$, which is the ratio of successful system responseskin$C_i$to the total number of expected system responses during the occurrence of$C_i$. The fuzzy membership value$m_i$is the sum of$m_{k,i}$for the software component$C_i$that containsksystem responses.

$$m_i = {\displaystyle \sum }_{k=1}^n{m}_{k,i}= {\displaystyle \sum }_{k=1}^n \frac{S{R}_k{C}_i }{SR{C}_i}$$

(1)

Rules for Fuzzy Membership Value

  if ($m_{k,i}$= $SR{C}_i$) then

    $m_i$= 1

  else if ($m_{k,i}$ ≠ $SR{C}_i$) then

     0 < $m_i$ < 1

  else

    $m_i$= 0

  end if

5. Software Completeness

In this study, we measured the software completeness with our introduced quantitative approach. Each software component was validated against its corresponding requirements. By this approach, missing functionalities or features and new software requirements were identified. $S_c$, $S_I$, $S_E$, $C_C$, $C_P$, $C_N$ and $C_T$ represent the software completeness, software components completeness index, engineering of software (reverse or forward engineering), complete membership of the software components, partial membership of the software components, non-membership of the software components and the total number of software components, respectively.

$$S_I = \left[1 - \right[\left(C_C/C_T\right) + \left(C_P/C_T\right) + \left(C_N/C_T\right)]]$$

(2)

$$S_c = S_E + S_I$$

(3)

By substituting Equation (2) into Equation (3),

$$S_c = S_E + \left[1 - \right[\left(C_C/C_T\right) + \left(C_P/C_T\right) + \left(C_N/C_T\right)]]$$

5.1. Proof

In this section, the authors prove the software completeness equation. According to Lotif Zadeh [23], fuzzy logic is defined as:

membership value → [0, 0.01, 0.02, …, 0.98, 0.99, 1]

In this, zero represents non-membership, one represents complete membership and the value between zero and one represents partial membership. Fuzzy logic evolved in the context of the theory of fuzzy sets. Fuzzy sets are extension of Boolean and crisp sets.

Fuzzy membership values for each software component are calculated by a fuzzy logic testing approach (fuzzy test). If the fuzzy membership value is equal to one, then it is assigned to $C_C$, else if the fuzzy membership value is equal to partial (varies between zero and one), then it is assigned to $C_P$, else it is assigned to $C_N$.

Therefore,

0 ≤ m ≤ 1

Therefore, as software membership value is determined by the $S_c$ equation, by substituting the values of $C_C$, $C_P$, $C_N$ and $S_E$.

$$0 \le S_E + \left[1 - \right[\left(C_C/C_T\right) + \left(C_P/C_T\right) + \left(C_N/C_T\right)]] \le 1.$$

5.1.1. Complete Membership Value

If the software does not require any changes due to evolution of the business or the operating environment, then the complete membership value of the software components $C_C$ is equal to the membership value of the total software components $C_T$. The partial membership value of a software component $C_P$ becomes equal to the non-membership value of the software components because the software does not require any minor or major modifications during the validation process. If the software possesses complete membership, then $C_C$ = $C_T$, $C_P$ = 0 and $C_N$ = 0, whereas $C_P$ = $C_N$ = 0. $C_C$ = $C_T$ because the software is not missing any features nor does it require any newly demanded features. In this section, we substitute the values of $C_C$, $C_P$ and $C_N$ in Equation (2), to prove that the software membership value is equal to the non-membership value.

$$\begin{gathered} S_I = [1 - [(C_C/C_T) + (C_P/C_T) + (C_N/C_T\left)\right]] = [1 - \left[\right(C_T/C_T) + (0/C_T) + (0/C_T\left)\right]] \\ =[1-[1+0+0\left]\right]=1-1=0 \end{gathered}$$

Hence, this proves that the membership value is zero when the software exhibits completeness equal to one. In this case, $S_E$ = 0 because the software does not require restructuring.

5.1.2. Non-Membership Value

If all the software components do not support the functionalities or features demanded in the new requirements due to low business values and low component quality, then $C_N$ = 0 for all software components. If the software possesses a non-membership value, then $C_N$ = 0, C_P = 0 and C_C = 0. The new software is completely developed by using a forward engineering approach. In this case, the components with non-membership value $C_N$ are completely satisfied/validated by the user of the software. By substituting the values of $C_C$, $C_P$ and $C_N$ in Equation (2),

$$\begin{gathered} S_I = [1 - [(C_C/C_T) + (C_P/C_T) + (C_N/C_T\left)\right]] = [1 - \left[\right(0/C_T) + (0/C_T) + (0/C_T\left)\right]] \\ =[1-[0+0+0\left]\right]=1-0=1 \end{gathered}$$

Hence, this proves that the membership value is one when the software exhibits a membership value equal to zero. $S_E$ = 1, when the existing software does not support the new user requirements, the software is outdated or the software does not support changes, and the system is a candidate for scrapping because keeping it in operation would be expensive [6,27,53].

5.1.3. Partial Membership Value

If the software requires minor or major changes in its components according to new user requirements, then the membership value of the software is equal to partial membership. The partial membership value is determined by validating each software component against its corresponding software requirements. Therefore, software membership is a combination of complete membership, partial membership and non-membership values, whereas $S_c$ determines the partial membership value of software. In this case, the software is updated by a reverse engineering approach. Figure 7 represents the software components in the conceptual model.

According to this figure, the $C_C$, $C_P$, $C_N$ and $C_T$ values are calculated as

$$C_i = C_1 + C_2 + C_3 + \dots + C_{24}$$

$C_1$ = $F{C}_1^1$ = 1 → complete membership; $C_2$ = $(F{C}_1^2$ + $F{C}_2^2)$= ((0.3 + 1)/2) = 0.65 → partial membership; $C_3$ = $F{C}_1^3$ = 0.9 → partial membership; $C_4$ = $(F{C}_1^4$ + $F{C}_2^4)$= ((1 + 1)/2) = 1 → complete membership; $C_5$ = $(F{C}_1^5$ + $F{C}_2^5+F{C}_3^5$) = ((1 + 1 + 0.8)/3) = 0.93 → partial membership; $C_6$ = $(F{C}_1^6$ + $F{C}_2^6)$= ((0.1 + 0.6)/2) = 0.35 → partial membership; $C_7$ = $(F{C}_1^7$ + $F{C}_2^7+F{C}_3^7$) = ((0 + 0 + 0)/3) = 0 → no membership; $C_8$ = $F{C}_1^8$ = 0 → no membership; $C_9$ = $(F{C}_1^9$ + $F{C}_2^9+F{C}_3^9$ + $F{C}_4^9$) = ((1 + 1 + 0.4 + 1)/4) = 0.85 → partial membership; $C_{10}$ = $(F{C}_1^{10}$ + $F{C}_2^{10})$= ((1 + 1)/2) = 1 → complete membership; $C_{11}$ = $(F{C}_1^{11}$ + $F{C}_2^{11})$= ((0.7 + 0.6)/2) = 0.65 → partial membership; $C_{12}$ = $(F{C}_1^{12}$ + $F{C}_2^{12}+F{C}_3^{12}$ + $F{C}_4^{12}$) = ((0 + 0 + 0 + 0)/4) = 0 → no membership; $C_{13}$ = $(F{C}_1^{13}$ + $F{C}_2^{13})$= ((1 + 1)/2) = 1 → complete membership; $C_{14}$ = $(F{C}_1^{14}$ + $F{C}_2^{14})$= ((0.9 + 0.9)/2) = 0.9 → partial membership; $C_{15}$ = $(F{C}_1^{15}$ + $F{C}_2^{15})$= ((0.5 + 0.7)/2) = 0.6 → partial membership; $C_{16}$ = $(F{C}_1^{16}$ + $F{C}_2^{16})$= ((0.7 + 0.9)/2) = 0.8 → partial membership; $C_{17}$ = $F{C}_1^{17}$ = 1 → complete membership; $C_{18}$ = $(F{C}_1^{18}$ + $F{C}_2^{18})$= ((1 + 0)/2) = 0.5 → partial membership; $C_{19}$ = $(F{C}_1^{19}$ + $F{C}_2^{19})$= ((0.6 + 0.3)/2) = 0.45 → partial membership; $C_{20}$ = $(F{C}_1^{20}$ + $F{C}_2^{20})$= ((0.1 + 0.6)/2) = 0.35 → partial membership, $C_{21}$ = $(F{C}_1^{21}$ + $F{C}_2^{21})$= ((1 + 0.7)/2) = 0.85 → partial membership; $C_{22}$ = $(F{C}_1^{22}$ + $F{C}_2^{22})$= ((1 + 1)/2) = 1 → complete membership; $C_{23}$ = $(F{C}_1^{23}$ + $F{C}_2^{23})$= ((0.8 + 0.8)/2) = 0.8 → partial membership; $C_{24}$ = $(F{C}_1^{24}$ + $F{C}_2^{24}+F{C}_3^{24}$) = ((1 + 1 + 0)/3) = 0.33 → partial membership;

$$\begin{array}{l}{C}_C = C_1 + C_4 + C_{10} + C_{13} + C_{17} + C_{22} = 1 + 1 + 1 + 1 + 1 + 1 = 6 \\ C_P = C_2 + C_3 + C_5 + C_6 + C_9 + C_{11} + C_{14} + C_{15} + C_{16} + C_{18} + C_{19} + C_{20} + C_{21} + C_{23} + C_{24} \\ = 0.65 + 0.9 + 0.93 + 0.35 + 0.85 + 0.65 + 0.9 + 0.6 + 0.8 + 0.5 + 0.45 + 0.35 + 0.85 + 0.8 + 0.33 \\ = 9.91\\ C_N = C_7 + C_8 + C_{12} = 0 + 0 + 0 = 0\end{array}$$

By substituting the values in Equation (2),

$$\begin{array}{l}{S}_I = \left[1 - \right[\left(C_C/C_T\right) + \left(C_P/C_T\right) + \left(C_N/C_T\right)]] \\ S_I = \left[1 - \right[\left(6/24\right) + \left(9.91/24\right) + \left(0/24\right)]]\\ S_I = \left[1 - \right[0.25 + 0.41 + 0)\left]\left] = \right[1 - \right[0.25 + 0.41 + 0)]] \\ S_I = 1 - 0.66 = 0.34 \\ S_c = S_E + S_I \\ S_c = 0 + 0.34 = 0.34 \end{array}$$

In this case, $S_E$ = 0 because software is updated by the reverse engineering approach. As the partial membership value varies between complete membership and non-membership, 0 < partial membership value < 1. Therefore, 0 < 0.34 < 1, and we obtain partial membership.

Section 6 describes in detail how to calculate the membership value for each software component.

6. Materials and Methods

Measuring software completeness has been a challenging task for software companies. Due to changes in the operating environment, software is changing day by day. In this study, we introduced a fuzzy logic testing approach for measuring software completeness. Software requirements were gathered in natural language and were classified into different types. To measure the fuzzy membership value for each software component, each functionality was tested with the newly demanded system responses.

The dataset in this study consisted of 560 requirements, as shown in

Table 1. Dataset for software requirements.

Requirement Type	Requirement Class	Number of Requirements	Percentage
FR	Functional	365	65.18%
NFR	Security	14	2.50%
	Usability	21	3.75%
	Portability	10	1.79%
	Availability	7	1.25%
	Capacity	11	1.96%
	Efficiency	12	2.14%
	Reliability	15	2.68%
	Performance	13	2.32%
	Maintainability	15	2.68%
	Accuracy	8	1.43%
	Error-handling	10	1.79%
	Operational	19	3.39%
	Scalability	11	1.96%
IR	Inverse	9	1.61%
NIR	Security	2	0.36%
	Availability	1	0.18%
	Performance	1	0.18%
CR	Content	11	1.96%
NCR	Security	1	0.18%
	Usability	1	0.18%
	Performance	1	0.18%
	Operational	2	0.36%
Total		560	100%

. There were 365 functional requirements, 166 non-functional requirements, 9 inverse requirements, 4 non-inverse requirements, 11 content requirements and 5 non-content requirements. There were thirteen requirement classes of non-functional requirements, named security, usability, portability, availability, capacity, efficiency, reliability, performance, maintainability, accuracy, error-handling, operational and scalability.

Based on the difference between the previously expected system responses and newly expected system responses, the partial membership value was determined. Other software components were tested based on the system responses. If any new functionality was required within the software component or it was omitted during the development due to any reason, it was considered zero membership $C_N$ = 0 during the validation process. In this study, we tested the software components, and Figure 8 represents a fuzzy test for measuring the fuzzy membership value of the $C_5$ software component.

The tested login software component consisted of six expected system responses that were successfully tested with system responses $S{R}_1{C}_5$, $S{R}_2{C}_5$, $S{R}_3{C}_5$, $S{R}_4{C}_5$, $S{R}_5{C}_5$ and $S{R}_6{C}_5$. The fuzzy membership value of this software component was $m_5$ = 1.

Figure 9 represents a fuzzy test for measuring the fuzzy membership value of the $C_{19}$ software component.

This tested passport software component consisted of four expected system responses and two new expected system responses were recommended during the requirements validation process that were successfully tested with system responses $S{R}_1{C}_{19}$, $S{R}_2{C}_{19}$, $S{R}_3{C}_{19}$, $S{R}_4{C}_{19}$, $S{R}_5{C}_{19}$ and $S{R}_6{C}_{19}$. The ✓ symbol indicates that system response was successfully tested, whereas the ✗ symbol indicates that it is new recommendation. If $S{R}_k{C}_i$ was successfully performed, then $S{R}_k{C}_i$ was assigned one membership value, else a zero membership value was assigned. The fuzzy membership value of this software component was $m_{19}$ = 0.68. The fuzzy membership value was calculated by equation 1 given below.

$$m_i = {\displaystyle \sum }_{k=1}^n{m}_{k,i}= {\displaystyle \sum }_{k=1}^n \frac{S{R}_k{C}_i }{SR{C}_i}$$

During the validation process, the following changes were required in the software, and we classified them into content requirements and inverse requirements, as shown in

Table 2. Content requirements.

Description	Software Components
$R_1^7$: The system must display a drop-down list for the selection of the course category (professional compulsory course, professional elective course, public basic course, public compulsory course). $C_P$ = 0.81
$R_2^8$: The system must display the selected course category and ask the user to select a course code offered by the university during the current semester. $C_P$ = 0.88
$R_3^{12}$: The system must calculate the CGPA according to the university format and ask the user to enter obtain marks and total marks for each subject. $C_P$ = 0.87
$R_4^{52}$: The system must display the recommended books for the entered course code. $C_P$ = 0.64
$R_5^{189}$: The system must display a text box to enter a student ID. (The system generates a student card) $C_P$ = 0.91
$R_6^{190}$: The system must display a text box to enter a faculty ID. (The system generates a faculty card) $C_P$ = 0.88
$R_7^{193}$: The system must display a text box to enter a faculty ID and research interest. (The system generates a profile) $C_P$ = 0.83
$R_8^{280}$: The system must display a text box to enter the ID of a degree course. (The system generates a list of courses offered during the current semester) $C_P$ = 0.94
$R_9^{311}$: The system must display a text box to enter a student class ID. (The system generates a timetable for the student) $C_P$ = 0.92
$R_{10}^{312}$: The system must display a text box to enter a faculty class ID. (The system generates a timetable for the faculty member) $C_P$ = 0.88
$R_{11}^{317}$: The system must display a text box to enter a student ID and a degree ID. (The system generates a transcript) $C_P$ = 0.95

and

Table 3. Inverse requirements.

Description	Software Components
${\mathrm{R}}_1^{13}$: The system shall not print a semester invoice after the due date. (This option is for offline payment and bank name will be printed on the semester invoice) $C_P$ = 0.91
$R_2^{16}$: The system shall display the text box in red if a user enters more than or less than eight digits for their national identity card number. $C_P$ = 0.83
$R_3^{25}$: The system shall display the text box for age in red if the age of a candidate is less than 18 years or more than 35 years. A text box in lighter blue indicates that the age will be automatically calculated. (Red text box indicates the ineligibility of a candidate) $C_P$ = 0.80
$R_4^{32}$: The system shall not accept a native language other than Chinese/English. $C_P$ = 0.89
$R_5^{33}$: The system shall not accept a nationality of a candidate other than that of their country of residence. $C_P$ = 0.91
$R_6^{240}$: The system shall display a red text box if a user enters a percentage of completeness of a master degree that is less than 70%. $C_P$ = 0.88
$R_7^{296}$: The system shall not upload a photograph for a user profile if its size is less than (300 × 400) pixels. $C_P$ = 0.73
$R_8^{301}$: The system shall not accept journal publications other than those from the SCI/SCIE. $C_P$ = 0.60
$R_9^{302}$: The system shall not accept conference publications that are not recognized by the CCF/EC. $C_P$ = 0.70

. In this study, we measured the completeness of software by using our introduced approach. The software components were designed with single functionality or with sub-functionalities which are cohesive. However, in order to integrate different software components together, software components were loosely coupled.

The following are the complexity factors affecting developed methodology: size, goals and objectives, changes, ambiguity and data.

Size: The size of the software component is considered as an indication of complexity, and for this software components were designed to be symmetric with a minimum size. Goals and objectives: If goals and objectives are not defined properly, then the tested software component neglects to test the important features. Changes: Changes in the functionality of software components are affected by internal and external influences. They imply adaption to changing requirements. Ambiguity: In order to avoid ambiguity, each software component was validated by a requirements validation framework. Data: Incomplete data fail to test the expected system response. This fuzzy test was designed by considering software requirements and the dataset.

The software consisted of 360 software components, 323 existing software components and 37 new software components. In this study, 21 software components required partial changes, as shown in Table 2 and Table 3, and Figure 9 also represents a software component that required partial changes. All software components were tested by a fuzzy test, to find the $C_C$, $C_P$ and $C_N$ values.

The details regarding the functionalities are as follows: 342 software components consisted of a single functionality, 11 consisted of 2 functionalities and 7 consisted of 3 functionalities. Table 1 illustrates the detail of the dataset. Figure 8 and Figure 9 represent a fuzzy test of the software components $C_5$ and $C_{19}$, respectively.

Rules for Assigning the Fuzzy Membership Value

  if the fuzzy membership value of $m_i$ is equal to one, then

    $C_C$ = $m_i$

  else if the fuzzy membership value of $m_i$ varies between zero and one, then

    $C_P$ = $m_i$

  else

    $C_N$ = $m_i$

  end if

$C_C$ = 302 and $C_P$ = 0.81 + 0.88 + 0.87 + 0.64 + 0.91 + 0.88 + 0.83 + 0.94 + 0.92 + 0.88 + 0.95 + 0.91 + 0.83 + 0.80 + 0.89 + 0.91 + 0.88 + 0.73 + 0.60 + 0.70 + 0.68 = 17.44. In this study, 37 new software components required development. Therefore, $C_N$ = 0 for the 37 software components. The total software components became $C_T$ = 323 + 37 = 360. By substituting the values in Equation (2),

$$\begin{array}{l}{S}_I = \left[1 - \right[\left(C_C/C_T\right) + \left(C_P/C_T\right) + \left(C_N/C_T\right)]] \\ S_I = \left[1 - \right[\left(302/360\right) + \left(17.44/360\right) + \left(0/360\right)]] \\ S_I = \left[1 - \right[0.84 + 0.04 + 0]] \\ S_I = 1 - 0.88 = 0.12 \\ S_c = S_E + S_I = 0 + 0.12 = 0.12\end{array}$$

The software obtained a completeness of 0.88, and to obtain the complete membership value, software required a reverse engineering membership value of 0.12. $S_E$= 0 because the existing software was maintained.

7. Results

In the experiment, we observed that if software completeness had a membership value less than a 0.30, then it would be developed by forward engineering. For reverse engineering, existing software components required minor changes, whereas few new software components were developed. As the software size increased, its complexity also increased and resembled an Egyptian pyramid. Figure 10 represents software completeness. Membership is represented on the Y-axis and changes on the X-axis.

In the experiment, the software consisted of three hundred twenty-three existing software components, out of which twenty-one software components were partially changed. To update the software to a new version or change it according to the latest software requirements, twenty-one software components were restructured for new functionalities and features, and thirty-seven software components were newly developed. Therefore, the total number of software components after the reverse engineering process was three hundred sixty. In the figure, 0.88 represents the fuzzy membership value of software completeness. To achieve the complete membership value (one) for the software, new software components were developed, and a few were partially changed. The change in the membership value was 0.12.

To determine whether the software required minor, major or complex changes, we used the linguistic variable: (c, T(c), U, M), as c: “change” is a linguistic variable. T(c) = {minor changes, major changes, complex changes}, U: U = {0, 360}, M (minor changes) = fuzzy set for change in less than 90 software components with membership of μ_{minor changes.} In the experiment, there were 37 new software components and 21 software components required partial changes. In this study, the software was developed by component-based development. The details regarding the line of code (LOC) for the software components are as follows: 342 software components were developed with a single functionality which consisted of approximately 32 LOCs per software component, 11 software components were developed with two functionalities which consisted of approximately 60 LOCs per software component and seven software components were developed with three functionalities which consisted of approximately 105 LOCs per software component. The total LOC of the complete software was 12,439.

We validated the software components by the requirements validation framework described in Section 2.1. To validate the introduced approach, we applied a machine learning confusion matrix, which is used by a prediction model to store incorrect and correct decisions [62]. Software modification means changes required in the software components according to new software requirements or due to errors or defects in the software components. If a software component is classified as correct when it is truly correct, then the classification is a true positive (TP). If the software component is classified as correct when it is actually modified, then the classification is a false positive (FP). If the software component is classified as a modification when it is actually correct, then the classification is a false negative (FN). If the software component is classified as a modification when it is actually modified, then the classification is a true negative (TN).

In this study, the total number of predictions regarding the software component changes (partial membership) was 360 and there were 352 correct predictions; true positive (TP) = 322, false positive (FP) = 6, false negative (FN) = 2, true negative (TN) = 30.

• Accuracy = $\left(TP+TN\right)/Total$ = $\left(322+30\right)/360$ = 0.97
• Sensitivity = Recall = $TP/\left(FN+TP\right)$ = $322/\left(2+322\right)$ = 0.99
• Specificity = $TN/\left(TN+FP\right)$ = $30/\left(30+6\right)$ = 0.83
• Precision = $TP/\left(TP+FP\right)$ = $322/\left(322+6\right)$ = 0.98

In this research, we used a machine learning technique for measuring the accuracy of the introduced approach, which is 0.97.

8. Conclusions

In this study, software completeness was measured with a quantitative approach. Software completeness scaled the membership value between zero and one. Values closer to one indicated that minor modifications or design recoveries were required by reverse engineering. If the value was closer to zero, then the software was developed by forward engineering. Each software component was tested by a fuzzy test. The fuzzy test extracted the errors and missing features within software components. The software components with a partial membership value were easily maintained due to the identification of an error from the exact line of code. The mathematical formula of $S_c$ was proved with three conditions: complete membership, partial membership and non-membership. The classification of software requirements helped to developed software components with improved quality, and these software components were also reliable, upgradable and reusable. The introduced approach developed high-quality software, which reduces the development cost and as well as the testing cost. It has been observed that after two to three years, in only a few circumstances, software fulfills the demanded requirements and possesses a software completeness value equal to one. However, in most circumstances, software exhibits partial membership.

In future work, we are interested to measure fuzzy maintenance cost and fuzzy maintenance line of code based on the number of errors or defects identified by the fuzzy test.