Whisper-LLaMA

34.7868% · Whisper-LLaMA · Dynamic-SUPERB · LLM-C classification macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.2335% · Whisper-LLaMA · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.0841% · Whisper-LLaMA · Dynamic-SUPERB · Environmental audio · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.2217% · Whisper-LLaMA · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.5194% · Whisper-LLaMA · Dynamic-SUPERB · Music & harmony · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.3955% · Whisper-LLaMA · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.169% · Whisper-LLaMA · Dynamic-SUPERB · Speech content & structure · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.399% · Whisper-LLaMA · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.399% · Whisper-LLaMA · Dynamic-SUPERB · Speech recognition & speakers · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.2765% · Whisper-LLaMA · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

34.7743% · Whisper-LLaMA · Dynamic-SUPERB · Voice & speech behavior · LLM-C macro average · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92.43% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.0065 TER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.56% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

146.28 % TER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.58% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.71% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.23% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.22% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.17 MSE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.3333 KTAU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.6747 LCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

-0.5 SRCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54.55% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

135.6077 Miss Time · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

84.38% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

204.3094 ACC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.56% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.3657 MSE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

72.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.68 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.1093 Sacre Bleu · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

101.93 % MER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

80% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.17 % IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.73% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

91.39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 Event-based F1 · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.02% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.17% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.6651 MEDAE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

81.25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4006 Angle Diff · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · Angle Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.08% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.93% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.86% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.86% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.56% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.5378 Abs Diff · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

19.19% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.236 Abs Diff · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.12 % PER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.17 % PER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.77% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.42% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.73% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.71% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.44% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.38% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.72% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.44% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0185 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.38% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.2179 IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14.67% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0055 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

26.39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39.43% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.0435 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.81% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.44% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.33% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.37% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

94.87% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92.18% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

95.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.53% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

93% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.81 % DER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · DER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

31.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.73 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

424.58 % MER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.8 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.03 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

72.49 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

71.09 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

85.38 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.96 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76.79 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.63 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.75 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.96 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.14 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.04 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.93 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.71 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.67 % CER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

143.02 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

88.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

86.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

80.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

83.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

83.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.46% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

77.78% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

11.08% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

81.25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.4006 F1 · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

90.76% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0 IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.8637 Slot Type F1 · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.3178 Slot Value CER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.43% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.5722 POS · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9081 POS · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.78% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.372 Sacre Bleu · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · GPT-4o evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Chord Classification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Key Detection (GiantSteps - Key) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

95.14% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.45 % TER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Transcription (MAESTRO) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Tonic Classification (Mridangam Tonic) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Pitch Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.86 TER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Pitch Extraction By Lyrics (CSD) · TER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Classificaiton In Songs (EMOTIFY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Emotion Detection (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.07% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Genre Classification (ISMIR04) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Genre Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MagnaTagATune) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Music Tagging (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Genre Classification (FMA) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.98% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Classification (Beijing Opera Percussion) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Percussion Instruments Stroke Classification (Mridangam Stroke) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Combination Recognition (OpenMIC-2018) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Instrument Source Classification (Nsynth) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Instrument Classification (MTG) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

99.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

6.567 MSE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.333 KTAU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · KTAU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.615 LCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · LCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.5 SRCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Automatic MOS Prediction (SingMOS) · SRCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.64% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

59.763 Miss Time · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Music Beat Tracking (ASAP) · Miss Time · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

153.683 ACC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Editing Identification (People’s Speech) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Scene Fake Detection (ASPIRE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Deep Fake Detection (LJSpeech, WaveFake, MUSDB18HQ) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Singing Voice Deepfake Detection (CtrSVDD, ACEKiSing, M4Singer) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.366 MSE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Duration Prediction (NTUML2021) · MSE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Music Speech Classification (MAESTRO, Librispeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Effect Detection (RemFx) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

58.7 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Children Song Transcript Verification (CSD) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.109 Sacre Bleu · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Lyric Translation (SingSet) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.0193 MER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Song Lyric Recognition (SingSet) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

8.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · MARBLE Vocal Technique Detection (VocalSet) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

86% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.37 IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Segment Retrieval (Clotho) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Sound Event Detection (DCASE2016 Task2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-channel Sound Event Understanding (STARSS23) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

9.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Animal Classification (WaveSource-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Bird Sound Detection (Warblrb10k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cat Emotion Classification (Cat Sound Classification Dataset V2) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Cornell Birdcall Identification · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

14% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Animals) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

92.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Beehive States Classification (Beehive States) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

97.22% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound Event Detection (DESED-PublicEval) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.48 Event-based F1 · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Domestic Environment Sound EventDetection (DESED-PublicEval) · Event-based F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emergency Traffic Detection (Large-Scale-Audio-dataset) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Exterior And Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Interior And Domestic Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Natural Soundscapes And Water Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (UrbanSound8K-Urban Noises) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environment Recognition (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.7% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Environmental Sound Classification (ESC50) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.33% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Vocal Imitation Classification (Vocal Imitations) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.711 MEDAE · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Audio Spatial Distance Prediction (Spatial LibriSpeech) · MEDAE · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · How Far Are You (3DSpeaker) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sound Position Prediction · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

54% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Emotion Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

1.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emoji Grounded Speech Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Change Detection (Ravdess) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Emotion Recognition (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Emotion Recognition (CREMAD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.92% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Emotion Recognition (RAVDESS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Covid19 Cough Audio Classification (CoughVid) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Environmental Sound Classification (ESC50-Human And Non Speech Sounds) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

3.6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Non Speech Sound Recognition (Nonspeech7k, CommonVoice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Human Screaming Detection (Environmentdb) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Vocal Sound Recognition (VocalSound) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.151 ACC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.409 Abs Diff · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Segment Counting (Librispeech-words) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.151 ACC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.215 Abs Diff · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phone Segment Counting (VoxAngeles) · Abs Diff · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.12 % PER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Phoneme Recognition (LibriSpeech-TestClean) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100.17 % PER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phoneme Recognition (LibriSpeech-TestOther) · PER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Consonant Place Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Manner Of Articulation) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

5.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Phone) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Frontness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Height) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

41.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Phonological Feature Classification (VoxAngeles-Vowel Roundedness) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Heteronym Differentiation (HeteronymEn) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

22.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15.44% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.026 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Accuracy (speechocean762-scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-lingual Pronunciation Similarity (VoxAngeles) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Accent Classification (Accentdb Extended) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

15% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stress Detection (MIRSD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

25% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.042 IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Third Tone Sandhi Recognition (NCCU Corpus of Spoken Taiwan Mandarin) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

13.59% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.021 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Fluency (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.6% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Ranking) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

27.86% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.09 PCC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · L2 English Prosodic (speechocean762-Scoring) · PCC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

57.1% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Lexical) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

61.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Prosody Naturalness (ProsAudit-Protosyntax) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2015) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoof Detection (ASVspoof2017) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

37.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Deep Fake Voice Recognition (DEEP-VOICE) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Enhancement Detection (LibriTTS-TestClean, WHAM) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (CallHome) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Promo) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

94.9% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Fraud Robocall Recognition (Robocall) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

88.68% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Language Identification (VoxLingua107) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

96% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Language Identification (VoxForge) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

12.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Age Classification (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Gender Recognition by Voice (Common Voice) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

32.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

20.39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · HEAR Speaker Count Identification (LibriCount) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

18% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Counting (LibriTTS-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

56.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

55% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speaker Verification (VCTK) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Diarization (Libri2Mix-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

52% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speaker Verification (SUPERB Hidden Set) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (LJSpeech, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

47.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

53% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise Detection (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

23.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Gaussian) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Music) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

30% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Noise) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Noise SNR Level Prediction (VCTK, MUSAN-Speech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

46% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (LJSpeech, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

44.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-LargeRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-MediumRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Reverberation Detection (VCTK, RirsNoises-SmallRoom) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · N-Best Correction (Librispeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.7 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · AAVE Speech Recognition (CORAAL) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

424.58 % MER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-switch Speech Recognition (NTUML2021) · MER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.2% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

28.3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Speech Recognition (ASCEND) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.8 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-de) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.03 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-en) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

72.49 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-es) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

71.09 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-fr) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

85.38 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-it) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

74.96 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-nl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

76.79 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pl) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

73.63 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Lingual Speech Recognition (MLS-pt) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

45.75 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PTBR Speech Recognition (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33.96 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB ASR (LibriSpeech-TestClean) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

42.14 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · ASR (LibriSpeech-TestOther) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.04 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Ar) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

10.93 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Es) (Common Voice) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

65.71 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Spontaneous) (CHIME6-Test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29.67 % CER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB OOD ASR (Zh) (Common Voice) · CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

143.02 % WER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Target Speaker ASR (AMI-test) · WER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Multi-Speaker Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

4.93% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.889 ACC · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Command Recognition (AudioMNIST) · ACC · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

87% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

82.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Text Matching (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

77% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestClean) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

77% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LibriSpeech-TestOther) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

83.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Term Detection (LJSpeech) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

75.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Keyword Spotting (Speech Commands V1-Test) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

48.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Query by Example (Quesst14-Eval) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

49.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Stuttering Detection (SEP28k) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

16.3% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Voice Disorder Classification (VOICED) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

77.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Conversation Matching · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

21.8% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (SLUE-HVB) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

29% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Classification (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Dialogue Act Pairing (Daily Talk) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

33% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

24.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Action) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

36.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Intent Classification (SLURP-Intent) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

100% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Localization (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.48% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.019 IoU · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Named Entity Recognition (SLUE-VoxPopuli) · IoU · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

38.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sarcasm Detection (Mustard) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

50.4% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Semantic Textual Similarity (SpokenSTS) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

39% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Speech Sentiment Analysis (MELD) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

43.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Spoken Digit Arithmetic (AudioMNIST) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

40.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · NAR · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.857 Slot Type F1 · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Type F1 · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.354 Slot Value CER · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Slot Filling (AudioSnips-Test) · Slot Value CER · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

51.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Code-Switching Semantic Grammar Acceptability Comparison (CSZS-zh-en) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

63% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Nonce Word Detection (sWUGGY) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

2.5722 POS · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

0.9081 POS · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · PoS Estimation with transcription (LibriTTS) · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

68.5% · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · Sentence Grammar Acceptability (sBLIMP) · LLM-C · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗

17.372 Sacre Bleu · Whisper-LLaMA · BigAudioBench / Dynamic-SUPERB · Dynamic-SUPERB Phase 2 · SUPERB Speech Translation (CoVoST2-Test) · Sacre Bleu · Llama-3.1-8B-Instruct evaluator · source task-level evaluation · Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ↗