SALMONN 7B

31.8648% · SALMONN 7B · Dynamic-SUPERB · LLM-C classification macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.4088% · SALMONN 7B · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.1335% · SALMONN 7B · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.825% · SALMONN 7B · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.9461% · SALMONN 7B · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.3624% · SALMONN 7B · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.0483% · SALMONN 7B · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.9265% · SALMONN 7B · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.944% · SALMONN 7B · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.7661% · SALMONN 7B · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.6924% · SALMONN 7B · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36 TER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.11% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

226.21 % TER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.67% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.16% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.82% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.17% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.97% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.72% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.33% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.33% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.0777 MSE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.1105 KTAU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.1354 LCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.1325 SRCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.82% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

118.4047 Miss Time · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

165.0617 ACC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.79% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.06% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31652.7877 MSE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

91.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.17% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

104.02 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.8126 Sacre Bleu · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

356.36 % MER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.25 % IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.43% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.32% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.25% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.72% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.11% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.17% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.449 MEDAE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.58% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.08% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.29% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.53% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.09% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.84% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5285 Abs Diff · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.88% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.81% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.4857 Abs Diff · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.36 % PER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.79 % PER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.88% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.06% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.76% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.14% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.77% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.29% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0633 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0292 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.67% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.27% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0427 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.26% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.56% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.75% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.28% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.75% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.26% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.99 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

293.5 % MER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.14% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.46 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.37 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.45 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.52 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.83 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.88 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.88 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.44 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

216.51 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98.96 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

609.28 % CER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

273.9 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.07% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.41% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

95.65% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9325 Slot Type F1 · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4059 Slot Value CER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.43% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.9675 POS · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7583 POS · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.38% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.7884 Sacre Bleu · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.32 TER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.13% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.33% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.381 MSE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.233 KTAU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.024 LCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.221 SRCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98.664 Miss Time · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.38% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

283.705 ACC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.1% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98768.729 MSE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.67% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

104 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.813 Sacre Bleu · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5636 MER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.19 IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.72% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

64% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

81% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.684 MEDAE · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.75% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.9% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.69% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.176 ACC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

367.562 Abs Diff · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.22% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.191 ACC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.19 Abs Diff · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.36 % PER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.79 % PER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.2% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.27% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.067 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.62% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.091 IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.029 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.7% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.27% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.043 PCC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.48% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.3% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.64% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

293.5 % MER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.14% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.46 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.37 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.45 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.52 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.83 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.88 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

69.88 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.44 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

216.51 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

98.96 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.11 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

609.28 % CER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

273.9 % WER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.93% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.967 ACC · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.4% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.8% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

67.39% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.212 IoU · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.6% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.923 Slot Type F1 · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.237 Slot Value CER · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.9675 POS · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.7583 POS · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.7884 Sacre Bleu · SALMONN 7B · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗