{rfName}
Ge

Indexed in

Citations

62

Altmetrics

Analysis of institutional authors

Liz López, HelenaCorresponding AuthorHuertas-Tato JAuthorCamacho DAuthor

Share

November 13, 2023
Publications
>
Article
No

Generation and detection of manipulated multimodal audiovisual content: Advances, trends and open challenges

Publicated to: Information Fusion. 103 102103- - 2024-03-01 103(), DOI: 10.1016/j.inffus.2023.102103

Authors:

Liz-López, H; Keita, M; Taleb-Ahmed, A; Hadid, A; Huertas-Tato, J; Camacho, D
[+]

Affiliations

Institut d'Electronique de Microélectronique et de Nanotechnologie (IEMN) - Author
Sorbonne Univ Abu Dhabi, Sorbonne Ctr Artificial Intelligence, Abu Dhabi, U Arab Emirates - Author
Sorbonne University Abu Dhabi - Author
Univ Politecn Madrid, Comp Syst Dept, Calle Alan Turing S-N, Madrid 28031, Spain - Author
Univ Polytech Hauts De France, Inst Elect Microelect & Nanotechnol IEMN, F-59313 Valenciennes, France - Author
Universidad Politécnica de Madrid - Author
See more

Abstract

Generative deep learning techniques have invaded the public discourse recently. Despite the advantages, the applications to disinformation are concerning as the counter-measures advance slowly. As the manipulation of multimedia content becomes easier, faster, and more credible, developing effective forensics becomes invaluable. Other works have identified this need but neglect that disinformation is inherently multimodal. Overall in this survey, we exhaustively describe modern manipulation and forensic techniques from the lens of video, audio and their multimodal fusion. For manipulation techniques, we give a classification of the most commonly applied manipulations. Generative techniques can be exploited to generate datasets; we provide a list of current datasets useful for forensics. We have reviewed forensic techniques from 2018 to 2023, examined the usage of datasets, and given a comparative analysis of each modality. Finally, we give another comparison of end-to-end forensics tools for end-users. From our analysis clear trends are found with diffusion models, dataset granularity, explainability techniques, synchronisation improvements, and learning task diversity. We find a roadmap of deep challenges ahead, including multilinguality, multimodality, improving data quality (and variety), all in an adversarial ever-changing environment.
[+]

Keywords

audiodeep learningdeepfakesfacesfakemultimedia data forensicsmultimodalspeaker verificationvideoAudioDeep learningDeep neural-networksMultimedia data forensicsMultimedia data manipulation generationMultimodalVideo

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal Information Fusion due to its progression and the good impact it has achieved in recent years, according to the agency WoS (JCR), it has become a reference in its field. In the year of publication of the work, 2024 there are still no calculated indicators, but in 2023, it was in position 4/204, thus managing to position itself as a Q1 (Primer Cuartil), in the category Computer Science, Artificial Intelligence. Notably, the journal is positioned above the 90th percentile.

Independientemente del impacto esperado determinado por el canal de difusión, es importante destacar el impacto real observado de la propia aportación.

Según las diferentes agencias de indexación, el número de citas acumuladas por esta publicación hasta la fecha 2025-12-21:

  • Google Scholar: 26
  • WoS: 9
  • Scopus: 25
[+]

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-12-21:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 63.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 67 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 4.
  • The number of mentions in news outlets: 1 (Altmetric).
[+]

Leadership analysis of institutional authors

This work has been carried out with international collaboration, specifically with researchers from: France; United Arab Emirates.

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (Liz-López H) and Last Author (CAMACHO FERNANDEZ, DAVID).

the authors responsible for correspondence tasks have been LIZ LÓPEZ, HELENA and Liz-López H.

[+]