WavLLM

32.3205% · WavLLM · Dynamic-SUPERB · LLM-C classification macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.8671% · WavLLM · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.2035% · WavLLM · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.4533% · WavLLM · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.6% · WavLLM · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.661% · WavLLM · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.069% · WavLLM · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35.5725% · WavLLM · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.935% · WavLLM · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.3267% · WavLLM · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.9283% · WavLLM · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.76% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.25 TER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.78% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

136.11 % TER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.33% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.58% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.82% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.15% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.89% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.42% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.89% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.96% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.421 MSE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0103 KTAU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.0216 LCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0118 SRCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.27% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

111.7759 Miss Time · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96.88% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100 ACC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.84% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.21% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3581.0352 MSE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

155.48 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.8969 Sacre Bleu · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

481.13 % MER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.9 % IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.43% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.83% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.33% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.72% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.47% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.35% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9011 MEDAE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93.75% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8681 Angle Diff · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · Angle Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.08% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.08% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.22% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.29% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.55% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.9109 Abs Diff · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.04% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.11% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45754.9952 Abs Diff · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.99 % PER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.99 % PER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.88% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.84% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.46% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.88% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.67% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.53% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0439 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.12% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1304 IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.78% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.86% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0222 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.98% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0461 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.83% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.51% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.16% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.77% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.85% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.51% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90.53 % DER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · DER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.91 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

172.03 % MER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.92% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.56 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.23 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.9 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.35 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.14 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.56 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90.11 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.26 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.93 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.87 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.17 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

149.15 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.38 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

136.24 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

445.36 % CER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

266.26 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.67% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93.42% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

60% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.27% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.96% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.17% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93.75% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8681 F1 · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

91.85% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0222 IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.907 Slot Type F1 · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4752 Slot Value CER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.15% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.1126 POS · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.0463 POS · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.7423 Sacre Bleu · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.86% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.48 % TER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.71 TER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.58% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.42% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.13% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.276 MSE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.03 KTAU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.005 LCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.038 SRCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.64% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

81.349 Miss Time · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

81.25% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-49.772 ACC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3243.74 MSE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.33% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

155.5 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.897 Sacre Bleu · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.8113 MER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.171 IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.83% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.745 MEDAE · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.81% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.138 ACC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.867 Abs Diff · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.81% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.161 ACC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51811.886 Abs Diff · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.99 % PER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.99 % PER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

79.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.89% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.093 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.12% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.174 IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.18% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.054 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.86% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.094 PCC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

83.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.7% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.8% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

35% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.11% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.9 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

172.03 % MER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.4% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.92% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.56 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.23 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.9 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.35 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.14 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.56 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90.11 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.26 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

62.93 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.87 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.17 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

149.15 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.38 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

136.24 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

445.36 % CER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

266.26 % WER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.53% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.934 ACC · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.9% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.2% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.1% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.48% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.693 IoU · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.6% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.896 Slot Type F1 · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.551 Slot Value CER · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63.5% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.1126 POS · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.0463 POS · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.3% · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.7423 Sacre Bleu · WavLLM · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗