Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing
Abstract
:1. Introduction
- We have proposed six new MRs for dilation and erosion operations and ascertain the effectiveness of these MRs while also assessing improvements in them using mutation testing.
- We have compared our six proposed MRs with the eight existing MRs for dilation and erosion operations.
- In existing literature, only two mutation operators are used for the evaluation of edge detection and morphological image operations. We have used nine mutation operators to improve the effectiveness of edge detection and morphological image operation (dilation and erosion) MRs. We have also compared the result of our proposed framework with the existing techniques.
- We have also checked the effectiveness of mutation operators to determine which operator is more effective in generating and killing a maximum number of mutants.
2. Related Work
- In literature, test cases are selected and generated randomly. Random selection leads to an unfair distribution of parametric values, which ultimately affects the testing process.
- In existing techniques, MR evaluation is conducted through mutation testing. This evaluation is not comprehensive, as only a few mutation operators are used to check the fault detection rate of MRs. The total number of mutants generated through these mutation operators is quite low which makes the testing weak.
- In existing literature, no work has been conducted to check the effectiveness of mutation operators. It is not highlighted that which operator is more valuable to generate and kill maximum number of mutants.
3. Metamorphic Relations
3.1. MRs for Edge Detection
3.2. MRs for Dilation and Erosion
3.2.1. Existing MRs for Dilation and Erosion
3.2.2. Proposed MRs for Dilation and Erosion
4. Evaluation of Metamorphic Relations
4.1. Source Test Case Generation
4.2. Metamorphic Testing
4.3. MR Evaluation Using Mutation Testing
4.4. Advantages of Proposed Framework
- In the proposed fraework, we have used nine applicable mutation operators to ascertain the effectiveness of MRs. The higher the number of mutation operators used, the higher the number of faults detected. The use of these operators contribute to the improvement of software quality, the effectiveness of test suites, and improvement in test coverage.
- We have proposed new MRs in the field of IP. Overall, proposing new metamorphic relations in image processing contributes to the advancement of testing methodologies, algorithm validation techniques, and research practices, ultimately leading to more reliable, robust, and efficient image processing systems and applications.
5. Experiment Design
5.1. Proposed Evaluation
- Sobel edge detection program
- Dilation and erosion programs
5.2. Source Code
5.2.1. Pseudocode of Dilation/Erosion Operation
- FUNCTION dilation(image, kernel_size, kernel):
- n, m = shape_of(image)
- transpose_kernel = transpose(kernel)
- create empty array edges_img with shape (n, m)
- x = y = 0
- WHILE y < m + 1 − kernel_size DO
- IF x < n + 1 − kernel_size THEN
- local_pixels = get_subarray(image, x, y, kernel_size)
- val = minimum(local_pixels + kernel)
- IF val > 255 THEN
- set edges_img[x + (kernel_size//2), y + (kernel_size//2)] = 255
- ELSE IF val < 0 THEN
- set edges_img[x + (kernel_size//2), y + (kernel_size//2)] = 0
- ELSE
- set edges_img[x + (kernel_size//2), y + (kernel_size//2)] = maximum/minimum(local_pixels)
- END IF
- increment x by 1
- ELSE
- increment y by 1
- set x to 0
- END IF
- END WHILE
- Convert edges_img to uint8 data type
- RETURN edges_img
- END FUNCTION
- FUNCTION display(images):
- titles = [‘original’, ‘After Dilation’]
- FOR i FROM 1 TO length_of(images) DO
- CREATE a new figure
- SET title of the figure to titles[i]
- DISPLAY the i-th image with gray colormap
- END FOR
- SHOW all figures
- END FUNCTION
- SET path to image file path
- LOAD image from path
- images = [image]
- APPEND dilation(image, 5, ones(5,5)) to images
- DISPLAY images
5.2.2. Pseudocode of Edge Detection Operation
- FUNCTION sad():
- vertical_filter = [[−1,−2,−1], [0,0,0], [1,2,1]]
- horizontal_filter = [[−1,0,1], [−2,0,2], [−1,0,1]]
- path = “path_to_image.jpg”
- img = read_image(path)
- n, m, = dimensions_of_image(img)
- edges_img = copy_of(img)
- x = y = 0
- WHILE y < m + 1 − 3 DO:
- IF x < n + 1 − 3 THEN:
- local_pixels = get_local_pixels(img, x, y)
- vertical_transformed_pixels = vertical_filter * local_pixels
- vertical_score = sum_of_elements(vertical_transformed_pixels)/4
- horizontal_transformed_pixels = horizontal_filter * local_pixels
- horizontal_score = sum_of_elements(horizontal_transformed_pixels)/4
- edge_score = square_root_of((vertical_score^2) + (horizontal_score^2))
- edges_img[x + 1, y + 1] = [edge_score] * 3
- x = x + 1
- ELSE:
- y = y + 1
- x = 0
- END IF
- END WHILE
- edges_img = edges_img/edges_img.max()
- display_image(edges_img)
- save_image(‘output_path.jpg’, edges_img)
- IF _name_ == “_main_”:
- sad()
5.3. Dataset
5.4. Source Test Cases
5.5. Source Test Cases
5.6. Mutation Operators
- AOD—arithmetic operator deletion
- AOR—arithmetic operator replacement
- ASR—assignment operator replacement
- BCR—break continue replacement
- COD—conditional operator deletion
- COI—conditional operator insertion
- CRP—constant replacement
- DDL—decorator deletion
- EHD—exception handler deletion
- EXS—exception swallowing
- IHD—hiding variable deletion
- IOD—overriding method deletion
- IOP—overridden method calling position change
- LCR—logical connector replacement
- LOD—logical operator deletion
- LOR—logical operator replacement
- ROR—relational operator replacement
- SCD—super calling deletion
- SCI—super calling insert
- SIR—slice index remove
- CDI—class method decorator insertion
- OIL—one iteration loop
- RIL—reverse iteration loop
- SDI—static method decorator insertion
- SDL—statement deletion
- SVD—self variable deletion
- ZIL—zero iteration loop
- AOD → Unary arithmetic operator deletion
- AOR → Arithmetic operation replacement
- LOR → Logical operator replacement
- ROR → Relational operator Replacement
- OIL → One iteration loop
- RIL → Reverse iteration loop
- SIR → Slice index remove
- SDL → Statement deletion
- ZIL → Zero iteration loop
6. Results and Discussions
6.1. Effectiveness of Mutation Operators
6.1.1. Effectiveness of Mutation Operators Used in Edge Detection
6.1.2. Effectiveness of Mutation Operators Used in Dilation and Erosion
6.2. Effectiveness of Metamorphic Relations
6.2.1. Effectiveness of Edge Detection MRs
6.2.2. Effectiveness of Dilation and Erosion MRs
6.2.3. Effectiveness of Proposed MRs
6.3. Comparison of Proposed Framework with Existing Techniques
6.3.1. Comparison Results of Edge Detection
6.3.2. Comparison Results of Dilation and Erosion
6.4. Comparison Results of Proposed MRs with Existing MRs of Dilation and Erosion
7. Conclusions
- In this research, we have used only one dataset for the experiments. We can use more datasets for convincing testing.
- One of the limitations of this research is the use of only one algorithm for edge detection. There are several other edge detection algorithms that can be used to make the experiments more substantial.
- We have used only one programming language, i.e., Python. We could use more languages, such as Java, C++, etc., so that more mutation operators can be used.
- In this research, statement coverage and branch coverage are used for code coverage. In the future, the proposed research for the evaluation of MRs can be strengthened by using more coverage criteria for improved coverage. The future work involving coverage criterion may include multiple condition coverage (MCC), where every combination of conditions’ outcomes is tested at least once in a decision; modified condition/decision coverage (MCDC) where a decision’s potential outcomes are determined by each condition contained within the decision, all def-use (definition usage) coverage where all-def coverage is attained when all defs of any variable are covered and all-uses coverage is attained when a path from each def to each use of that def has been exercised, etc.
- The research can also be strengthened by using additional mutation operators that will cover the fault types not used in the proposed research, such as logical connector replacement (LCR), break continues replacement (BCR), constant replacement (CRP), class method decorator insertion (CDI), etc.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Bolchini, C.; Cassano, L.; Mazzeo, A.; Miele, A. Usability-based Cross-Layer Reliability Evaluation of Image Processing Applications. In Proceedings of the IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT) 2021, Athens, Greece, 6–8 October 2021. [Google Scholar]
- Raid, A.M.; Khedr, W.M.; El-dosuky, M.A.; Aoud, M. Image Restoration Based on Morphological Operations. Int. J. Comput. Sci. Eng. Inf. Technol. (IJCSEIT) 2014, 4, 3. [Google Scholar] [CrossRef]
- Ur Rehman, F.; Izurieta, C. An Approach For Verifying And Validating Clustering Based Anomaly Detection Systems Using Metamorphic Testing. In Proceedings of the IEEE International Conference On Artificial Intelligence Testing (AITest) 2022, Newark, CA, USA, 15–18 August 2022. [Google Scholar]
- Memon, A.; Banerjee, I.; Nagarajan, A. What Test Oracle Should I Use for effective GUI Testing. In Proceedings of the 18th IEEE International Conference on Automated Software Engineering, Montreal, QC, Canada, 6–10 October 2003. [Google Scholar]
- Saha, P.; Kanewala, U. Fault Detection Effectiveness of Source Test Case Generation Strategies for Metamorphic Testing. In Proceedings of the MET ’18: Proceedings of the 3rd International Workshop on Metamorphic Testing, Gothenburg, Sweden, 27 May 2018. [Google Scholar]
- Chen, T.Y.; Cheung, S.C.; Yiu, S.M. Metamorphic testing: A new approach for generating next test cases. In Proceedings of the AST ‘15: Proceedings of the 10th International Workshop on Automation of Software Test, Florence, Italy, 16–24 May 2015. [Google Scholar]
- Segura, S.; Troya, J.; Duran, A.; Cortes, A.R. Performance Metamorphic Testing: Motivation and Challenges. In Proceedings of the IEEE/ACM 39th International Conference on Software Engineering: New Ideas and Emerging Technologies Results Track (ICSE-NIER), Buenos Aires, Argentina, 20–28 May 2017. [Google Scholar]
- Luu, Q.H.; Liu, H.; Chen, T.Y.; Vu, L.H. Testing Ocean Software with Metamorphic Testing. In Proceedings of the 7th International Workshop on Metamorphic Testing 2022, Pittsburgh, PA, USA, 9 May 2022. [Google Scholar]
- Hong, T.P.; Chiu, C.C.; Su, J.H.; Chen, C.H. Applicable Metamorphic Testing for Erasable-Itemset Mining. IEEE Access 2022, 10, 38545–38554. [Google Scholar] [CrossRef]
- Niu, X.; Sun, Y.; Wu, H.; Li, G.; Nie, C.; Yu, L.; Wang, X. Enhance Combinatorial Testing with Metamorphic Relations. J. Latex Cl. Files 2015, 14, 8. [Google Scholar] [CrossRef]
- Arrieta, A. Multi-Objective Metamorphic Follow-up Test Case Selection for Deep Learning Systems. In Proceedings of the Genetic and Evolutionary Computation Conference 2022, Boston, MA, USA, 9–13 July 2022. [Google Scholar]
- Sim, K.Y.; Wong, D.M.L.; Hii, T.Y. Evaluating the Effectiveness of Metamorphic Testing on Edge Detection Programs. Int. J. Innov. Manag. Technol. 2013, 4, 6–10. [Google Scholar]
- Mayer, J.; Guderlei, R. On Random Testing of Image Processing Applications. In Proceedings of the Sixth International Conference on Quality Software (QSIC’06), Beijing, China, 27–28 October 2006. [Google Scholar]
- Jameel, T.; Chao, L. Test Oracles Based on Metamorphic Relations for Image Processing Applications. In Proceedings of the IEEE 16th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Takamatsu, Japan, 1–3 June 2015. [Google Scholar]
- Jafari, F.; Nadeem, A.; Zaman, Q. Evaluation of Metamorphic Testing for Edge Detection in MRI Brain Diagnostics. Appl. Sci. 2022, 12, 8684. [Google Scholar] [CrossRef]
- Ding, j.; Hu, X.H. Application of Metamorphic Testing Monitored by Test Adequacy in a Monte Carlo Simulation Program. Softw. Qual. J. 2017, 25, 3. [Google Scholar] [CrossRef]
- Ding, J.; Zhang, D.; Hu, X. An Application of Metamorphic Testing for Testing Scientific Software. In Proceedings of the 1st International Workshop on Metamorphic Testing, Austin, TX, USA, 14–22 May 2016. [Google Scholar]
- Jiang, M.; Chen, Y.T.; Kuo, F.C.; Towey, D.; Ding, Z. A metamorphic testing approach for supporting program repair without the need for a test oracle. J. Syst. Softw. 2017, 126, 127–140. [Google Scholar] [CrossRef]
- Jiang, C.; Huang, S.; Hui, Z. Metamorphic Testing of Image Region Growth Programs in Image Processing Applications. In Proceedings of the IEEE International Conference on Software Quality, Reliability and Security Companion, Lisbon, Portugal, 16–20 July 2018. [Google Scholar]
- Jameel, T.; Mengxiang, L.; Chao, L. Automatic Test Oracle for Image Processing Applications Using Support Vector Machines. In Proceedings of the 6th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 23–25 September 2015; pp. 1110–1113. [Google Scholar]
- Chan, W.K.; Ho, J.C.F.; Tse, T.H. Piping Classification to Metamorphic Testing: An Empirical Study towards Better Effectiveness for the Identification of Failures in Mesh Simplification Programs. In Proceedings of the 31st Annual International Computer Software and Applications Conference (COMPSAC), Beijing, China, 24–27 July 2007. [Google Scholar]
- Chen, T.Y.; Kuo, F.C.; Liu, H.; Poon, P.L.; Towey, D.; Tse, T.H.; Zhou, Z.Q. Metamorphic Testing: A Review of Challenges and Opportunities. ACM Comput. Surv. 2017, 51, 1–27. [Google Scholar] [CrossRef]
- Ding, J.; Wu, T.; Lu, J.Q.; Hu, X.H. Self-Checked Metamorphic Testing of an Image Processing Program. In Proceedings of the Fourth IEEE International Conference on Secure Software Integration and Reliability Improvement, Singapore, 9–11 June 2010; pp. 190–197. [Google Scholar]
- Sun, C.; Liu, B.; Fu, A.; Liu, Y.; Liu, H. Path-directed source test case generation and prioritization in metamorphic testing. J. Syst. Softw. 2022, 183, 111091. [Google Scholar] [CrossRef]
- Segura, S.; Towey, D.; Zhou, Z.Q.; Chen, T.Y. Metamorphic Testing: Testing the Untestable. IEEE Softw. 2019, 37, 46–53. [Google Scholar] [CrossRef]
- Swathika, R.; Sharmila, T.S.; Bhattacharya, D. Edge Detection Using Simple Image Arithmetic. In Proceedings of the International Conference on Computer, Communication, and Signal Processing (ICCCSP), Chennai, India, 22–23 February 2018. [Google Scholar]
Ref Papers | SUT | Image Generation Method | Testing Method | Mutation Operators |
---|---|---|---|---|
Sim et al. [12] | Sobel Edge Detection Program | Randomly selected from published image libraries | Mutation Testing | LOR, ROR |
Jameel et al. [14] | Dilation and Erosion Programs | Randomly Selected | Mutation Testing | ROR |
Jiang et al. [19] | Image Region Growth program | Segmental symbolic evaluation method | Mutation Testing | Milu operators |
Jameel et al. [20] | Dilation Program | Ground truth images are chosen randomly | Mutation Testing | ROR |
Chan et al. [21] | Mesh Simplification Program of Image Rendering | Randomly Selected | Mutation Testing | MuJava operators |
Ding et al. [17] | Discrete Dipole Approximation Program | Random Generation | Structural Testing and Mutation Testing | ABS, ROR |
Ding et al. [16] | Monte Carlo Program and 3D Structure Reconstruc-tion Program | Random or Category Based Selection | Structural Testing and Mutation Testing | Modify coefficient, add constant, AOR, COR, CRP, SDL |
MR | Mathematical Property |
---|---|
MR1: Counter clock wise rotation at 90 degree | C(E(Im)) = E(C(Im)) |
MR2: Transposition | T(E(Im)) = E(T(Im)) |
MR3: Reflection at the ordinate | Mx(E(Im)) = E(Mx(Im)) |
MR4: Reflection at abscissa | My(E(Im)) = E(My(Im)) |
MR | Mathematical Property |
---|---|
R1: Reflection at the ordinate | |
R2: Reflection at abscissa | |
R3: Duality | c) c) where c is the complement of an image I. |
R4: Non-inverses | |
R5: Size of image object changes | |
R6: No. of objects in image changes | |
R7: Commutative | |
R8: Translation invariance |
Proposed MRs | Explanation |
---|---|
Counter clock wise rotation at 90 degree | Where, Im is the input Image, C(.) is the counter clockwise rotation at 90 degree, is the dilation and is the erosion operation. The image output of counter-clock wise rotation at 90 degree followed by morphological operations should be similar to image output of morphological operations followed by counter-clock wise rotation at 90 degree. |
Transposition | Where, T(.) is the transpose of an image. The image output of transposition followed by dilation and erosion should be similar to image output of dilation and erosion followed by transposition. |
Enhanced Associative Property | Where, A is the input image. B and C are the structuring elements. Image dilated with structuring element B and then dilated with structuring element C should give same results when we first dilate the image with structuring element C and then dilated with structuring element B. This property is specific to dilation as erosion does not fulfill this property. |
Image Translation | Where, Trans(.) is the image translation. The output of image translation followed by erosion should be similar to the output of erosion followed by image translation. This property is not applicable on dilation operation. |
Mutation Operators in Existing Techniques | Mutation Operators in Proposed Framework |
---|---|
ROR—Relational operator replacement LOR—Logical operator replacement | AOD—Unary arithmetic operator deletion AOR—Arithmetic operation replacement LOR—Logical operator replacement ROR—Relational operator Replacement OIL—One iteration loop RIL—Reverse iteration loop SIR—Slice index remove SDL—Statement deletion ZIL—Zero iteration loop |
Operation | Source |
---|---|
Sobel Edge Detection | https://towardsdatascience.com/edge-detection-in-python-a3c263a13e03 “(access on 23 April 2021)” |
Dilation | https://python.plainenglish.io/image-dilation-explained-easily-e085c47fbac2 “(access on 17 May 2022)” |
Erosion | https://medium.com/analytics-vidhya/2d-convolution-using-python-numpy-43442ff5f381 “(access on 17 May 2022)” |
Edge Detection Program | Dilation Program | Erosion Program | |
---|---|---|---|
No. of Test Cases | 95 | 95 | 95 |
No. of Statements | 41 | 46 | 46 |
No. of Covered Statements | 41 | 46 | 46 |
Statement Coverage (%) | 100% | 100% | 100% |
No. of Branches | 10 | 12 | 12 |
No. of Covered Branches | 10 | 12 | 12 |
Branch Coverage (%) | 100% | 100% | 100% |
Mutation Operators | Number of Mutants Generated | % of Generated Mutants | Mutants Killed by Each MR | FDR of Mutation Operators |
---|---|---|---|---|
AOD | 6 | 3.70% | MR1: 6 | MR1: 100% |
MR2: 6 | MR2: 100% | |||
MR3: 6 | MR3: 100% | |||
MR4: 6 | MR4: 100% | |||
AOR | 98 | 60.49% | MR1: 73 | MR1: 74.48% |
MR2: 74 | MR2: 75.51% | |||
MR3: 62 | MR3: 63.26% | |||
MR4: 62 | MR4: 63.26% | |||
COI | 2 | 1.23% | MR1: 2 | MR1: 100% |
MR2: 2 | MR2: 100% | |||
MR3: 2 | MR3: 100% | |||
MR4: 2 | MR4: 100% | |||
ROR | 30 | 18.51% | MR1: 22 | MR1: 73.33% |
MR2: 22 | MR2: 73.33% | |||
MR3: 22 | MR3: 73.33% | |||
MR4: 22 | MR4: 73.33% | |||
OIL | 2 | 1.23% | MR1: 1 | MR1: 50% |
MR2: 1 | MR2: 50% | |||
MR3: 1 | MR3: 50% | |||
MR4: 1 | MR4:50% | |||
RIL | 2 | 1.23% | MR1: 1 | MR1: 50% |
MR2: 2 | MR2: 100% | |||
MR3: 0 | MR3: 0% | |||
MR4: 0 | MR4: 0% | |||
SIR | 4 | 2.46% | MR1: 4 | MR1: 100% |
MR2: 4 | MR2: 100% | |||
MR3: 4 | MR3: 100% | |||
MR4: 4 | MR4: 100% | |||
SDL | 16 | 9.87% | MR1: 13 | MR1: 81.25% |
MR2: 13 | MR2: 81.25% | |||
MR3: 13 | MR3: 81.25% | |||
MR4: 13 | MR4: 81.25% | |||
ZIL | 2 | 1.23% | MR1: 2 | MR1: 100% |
MR2: 2 | MR2: 100% | |||
MR3: 2 | MR3: 100% | |||
MR4: 2 | MR4: 100% |
Mutation Operators | Number of Mutants Generated | % of Generated Mutants | Mutants Killed by Each MR | FDR of Mutation Operators |
---|---|---|---|---|
AOR | 95 | 73.07% | MR1: 51 | MR1: 53.68% |
MR2: 50 | MR2: 52.63% | |||
MR3: 51 | MR3: 53.68% | |||
MR4: 53 | MR4: 55.78% | |||
MR5: 56 | MR5: 58.94% | |||
MR6: 56 | MR6: 58.94% | |||
COI | 4 | 3.07% | MR1: 1 | MR1: 25% |
MR2: 1 | MR2: 25% | |||
MR3: 1 | MR3: 25% | |||
MR4: 2 | MR4: 50% | |||
MR5: 1 | MR5: 25% | |||
MR6: 2 | MR6: 50% | |||
ROR | 20 | 15.38% | MR1: 3 | MR1: 15% |
MR2: 3 | MR2: 15% | |||
MR3: 3 | MR3: 15% | |||
MR4: 6 | MR4: 30% | |||
MR5: 3 | MR5: 15% | |||
MR6: 5 | MR6: 25% | |||
OIL | 1 | 0.76% | MR1: 1 | MR1: 100% |
MR2: 0 | MR2: 0% | |||
MR3: 1 | MR3: 100% | |||
MR4: 1 | MR4: 100% | |||
MR5: 0 | MR5: 0% | |||
MR6: 0 | MR6: 0% | |||
RIL | 1 | 0.76% | MR1: 0 | MR13: 0% |
MR2: 0 | MR14: 0% | |||
MR3: 0 | MR15: 0% | |||
MR4: 0 | MR16: 0% | |||
MR5: 0 | MR17: 0% | |||
MR6: 1 | MR18: 100% | |||
SIR | 2 | 1.53% | MR1: 1 | MR1: 50% |
MR2: 1 | MR2: 50% | |||
MR3: 1 | MR3: 50% | |||
MR4: 1 | MR4: 50% | |||
MR5: 1 | MR5: 50% | |||
MR6: 2 | MR6: 100% | |||
SDL | 6 | 4.61% | MR1: 1 | MR1: 16.6% |
MR2: 1 | MR2: 16.6% | |||
MR3: 1 | MR3: 16.6% | |||
MR4: 1 | MR4: 16.6% | |||
MR5: 1 | MR5: 16.6% | |||
MR6: 1 | MR6: 16.6% | |||
ZIL | 1 | 0.76% | MR1: 1 | MR1: 100% |
MR2: 1 | MR2: 100% | |||
MR3: 1 | MR3: 100% | |||
MR4: 1 | MR4: 100% | |||
MR5: 1 | MR5: 100% | |||
MR6: 1 | MR6: 100% |
MR | Total No. of Mutants | No. of Killed Mutants | FDR (%) |
---|---|---|---|
MR1 | 162 | 124 | 76.54% |
MR2 | 162 | 126 | 77.77% |
MR3 | 162 | 112 | 69.13% |
MR4 | 162 | 112 | 69.13% |
MR | Total No. of Mutants | No. of Killed Mutants | FDR (%) |
---|---|---|---|
R1 | 130 | 70 | 53.84% |
R2 | 130 | 68 | 52.30% |
R3 | 130 | 60 | 46.15% |
R4 | 130 | 67 | 51.53% |
R5 | 130 | 52 | 40.00% |
R6 | 130 | 45 | 34.61% |
R7 | 130 | 68 | 52.30% |
R8 | 130 | 65 | 50.00% |
MR | Total No. of Mutants | No. of Killed Mutants | FDR (%) |
---|---|---|---|
MR1 | 130 | 58 | 44.61% |
MR2 | 130 | 56 | 43.07% |
MR3 | 130 | 58 | 44.61% |
MR4 | 130 | 64 | 49.23% |
MR5 | 130 | 62 | 47.69% |
MR6 | 130 | 67 | 51.53% |
Ref Papers | SUT | No. of Test Cases | Mutation Operator | No. of Mutants Generated | Language |
---|---|---|---|---|---|
Sim et al. [12] | Edge detection | 30 | LOR, ROR | 31 | C |
Jameel et al. [14] | Dilation and Erosion | Random selection | ROR | 33 | Mex C |
Proposed framework | Dilation, Erosion, and Edge detection | 95 | AOD, AOR, OIL, SIR, ZIL, ROR, SDL, COI | 162: edge detection 130: dilation and erosion | Python |
MR | FDR by Sim et al. Technique [12] | FDR by Proposed Framework |
---|---|---|
MR1 | 45% | 76.54% |
MR2 | 77% | 77.77% |
MR3 | 68% | 69.13% |
MR4 | 45% | 69.13% |
MR | FDR by Jameel et al. Technique [14] | FDR by Proposed Framework |
---|---|---|
R1 | 15% | 53.84% |
R2 | 58% | 54.30% |
R3 | 97% | 46.15% |
R4 | 51% | 51.53% |
R5 | 58% | 40.00% |
R6 | 18% | 34.61% |
R7 | 73% | 52.30% |
R8 | 21% | 50.00% |
Mutation Operators | No. of Mutants Generated |
---|---|
AOR | 95 |
COI | 4 |
ROR | 20 |
OIL | 1 |
RIL | 1 |
SIR | 2 |
SDL | 6 |
ZIL | 1 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jafari, F.; Nadeem, A. Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing. J. Imaging 2024, 10, 87. https://doi.org/10.3390/jimaging10040087
Jafari F, Nadeem A. Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing. Journal of Imaging. 2024; 10(4):87. https://doi.org/10.3390/jimaging10040087
Chicago/Turabian StyleJafari, Fakeeha, and Aamer Nadeem. 2024. "Measuring Effectiveness of Metamorphic Relations for Image Processing Using Mutation Testing" Journal of Imaging 10, no. 4: 87. https://doi.org/10.3390/jimaging10040087