Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images

Tanfoni, Marco; Ceroni, Elia Giuseppe; Marziali, Sara; Pancino, Niccolò; Maggini, Marco; Bianchini, Monica

doi:10.3390/electronics13163161

Open AccessArticle

Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images

by

Marco Tanfoni

^*,†

,

Elia Giuseppe Ceroni

^*,†

,

Sara Marziali

,

Niccolò Pancino

,

Marco Maggini

and

Monica Bianchini

Department of Information Engineering and Mathematics, University of Siena, 53100 Siena, Italy

^*

Authors to whom correspondence should be addressed.

^†

These authors contributed equally to this work.

Electronics 2024, 13(16), 3161; https://doi.org/10.3390/electronics13163161

Submission received: 16 July 2024 / Revised: 27 July 2024 / Accepted: 2 August 2024 / Published: 10 August 2024

(This article belongs to the Special Issue Deep Learning Approach for Secure and Trustworthy Biometric System)

Download

Browse Figures

Versions Notes

Abstract

Facial biometrics are widely used to reliably and conveniently recognize people in photos, in videos, or from real-time webcam streams. It is therefore of fundamental importance to detect synthetic faces in images in order to reduce the vulnerability of biometrics-based security systems. Furthermore, manipulated images of faces can be intentionally shared on social media to spread fake news related to the targeted individual. This paper shows how fake face recognition models may mainly rely on the information contained in the background when dealing with generated faces, thus reducing their effectiveness. Specifically, a classifier is trained to separate fake images from real ones, using their representation in a latent space. Subsequently, the faces are segmented and the background removed, and the detection procedure is performed again, observing a significant drop in classification accuracy. Finally, an explainability tool (SHAP) is used to highlight the salient areas of the image, showing that the background and face contours crucially influence the classifier decision.

Keywords: fake detection; image forgery; explainability; interpretability; segmentation; SHAP; DeepLabV3+; MobileNetV3 Large

Share and Cite

MDPI and ACS Style

Tanfoni, M.; Ceroni, E.G.; Marziali, S.; Pancino, N.; Maggini, M.; Bianchini, M. Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images. Electronics 2024, 13, 3161. https://doi.org/10.3390/electronics13163161

AMA Style

Tanfoni M, Ceroni EG, Marziali S, Pancino N, Maggini M, Bianchini M. Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images. Electronics. 2024; 13(16):3161. https://doi.org/10.3390/electronics13163161

Chicago/Turabian Style

Tanfoni, Marco, Elia Giuseppe Ceroni, Sara Marziali, Niccolò Pancino, Marco Maggini, and Monica Bianchini. 2024. "Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images" Electronics 13, no. 16: 3161. https://doi.org/10.3390/electronics13163161

APA Style

Tanfoni, M., Ceroni, E. G., Marziali, S., Pancino, N., Maggini, M., & Bianchini, M. (2024). Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images. Electronics, 13(16), 3161. https://doi.org/10.3390/electronics13163161

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Generated or Not Generated (GNG): The Importance of Background in the Detection of Fake Images

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI