Conditional face image manipulation detection : combining algorithm and human examiner decisions
Ibsen, Mathias and Nichols, Robert and Rathgeb, Christian and Robertson, David J. and Davis, Josh P. and Løvåsdal, Frøy and Raja, Kiran and Jenkins, Ryan E. and Busch, Christoph; (2024) Conditional face image manipulation detection : combining algorithm and human examiner decisions. In: Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security. Association for Computing Machinery, ESP, pp. 41-46. ISBN 9798400706370 (https://doi.org/10.1145/3658664.3659649)
Preview |
Text.
Filename: Ibsen-etal-IH_MMSec-2024-Conditional-face-image-manipulation-detection.pdf
Final Published Version License: Download (10MB)| Preview |
Abstract
It has been shown that digitally manipulated face images can pose a security threat to automated authentication systems (e.g., when such systems are used for border control). In such scenarios, a malicious actor can, in many countries, apply for an identity document using a manipulated face image, which can then be used to gain fraudulent access to a system. Research has shown that humans and algorithms struggle to detect digitally manipulated face images, especially when the type of manipulation is unknown or when evaluated across multiple types of manipulations. In this work, we consider the detection performance of algorithms and humans on datasets consisting of retouched, face swapped and morphed images. Specifically, we investigate the joint performance of algorithms and humans in a differential detection scenario where both a trusted and suspected image are presented simultaneously. To this end, we propose a conditional face image manipulation detection approach where the human decision is only considered when the algorithm is unsure about the decision outcome. The results show that the automated algorithm performs better than the human detectors and that combining the decisions of algorithms and humans, in general, leads to an increased detection performance. To our knowledge, this is the first study to explore the joint detection performance of algorithms and humans in a differential face manipulation detection scenario and when using a variety of face image manipulations.
ORCID iDs
Ibsen, Mathias, Nichols, Robert, Rathgeb, Christian, Robertson, David J. ORCID: https://orcid.org/0000-0002-8393-951X, Davis, Josh P., Løvåsdal, Frøy, Raja, Kiran, Jenkins, Ryan E. and Busch, Christoph;-
-
Item type: Book Section ID code: 89778 Dates: DateEvent24 June 2024Published12 April 2024AcceptedSubjects: Science > Mathematics > Electronic computers. Computer science
Philosophy. Psychology. Religion > PsychologyDepartment: Faculty of Humanities and Social Sciences (HaSS) > Psychological Sciences and Health > Psychology Depositing user: Pure Administrator Date deposited: 28 Jun 2024 15:13 Last modified: 04 Oct 2024 00:13 URI: https://strathprints.strath.ac.uk/id/eprint/89778