ON THE CHOICE OF THE OPTIMAL TEMPORAL SUPPORT FOR AUDIO CLASSIFICATION WITH PRE-TRAINED EMBEDDINGS - Equipe Signal, Statistique et Apprentissage
Conference Papers Year : 2024

ON THE CHOICE OF THE OPTIMAL TEMPORAL SUPPORT FOR AUDIO CLASSIFICATION WITH PRE-TRAINED EMBEDDINGS

Abstract

Current state-of-the-art audio analysis systems rely on pretrained embedding models, often used off-the-shelf as (frozen) feature extractors. Choosing the best one for a set of tasks is the subject of many recent publications. However, one aspect often overlooked in these works is the influence of the duration of audio input considered to extract an embedding, which we refer to as Temporal Support (TS). In this work, we study the influence of the TS for well-established or emerging pre-trained embeddings, chosen to represent different types of architectures and learning paradigms. We conduct this evaluation using both musical instrument and environmental sound datasets, namely OpenMIC, TAU Urban Acoustic Scenes 2020 Mobile, and ESC-50. We especially highlight that Audio Spectrogram Transformer-based systems (PaSST and BEATs) remain effective with smaller TS, which therefore allows for a drastic reduction in memory and computational cost. Moreover, we show that by choosing the optimal TS we reach competitive results across all tasks. In particular, we improve the state-of-the-art results on OpenMIC, using BEATs and PaSST without any fine-tuning.
Fichier principal
Vignette du fichier
Pre_print_ICASSP_Paper.pdf (444.79 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04360221 , version 1 (21-12-2023)

Identifiers

Cite

Aurian Quelennec, Michel Olvera, Geoffroy Peeters, Slim Essid. ON THE CHOICE OF THE OPTIMAL TEMPORAL SUPPORT FOR AUDIO CLASSIFICATION WITH PRE-TRAINED EMBEDDINGS. ICASSP, IEEE, Apr 2024, Séoul, South Korea. ⟨hal-04360221⟩
380 View
87 Download

Altmetric

Share

More