| Language | Region | Tier | Unit | Sets | Conditions | Utterances | Hours | Hub | Models | Dataset |
|---|
Click a language to see its sets. Arabic and English are maintained in arabic-arena-ar and arabic-arena-en and join the board unchanged.
Each row is a language, each column a speaking condition. The cell shows how much test audio the language has in that condition. Hover a cell for its sets; the Languages tab holds the same numbers as a table.
| Lang | Set | Condition | Domain | Utterances | Hours | Clusters | Reference | Licence | Hosting |
|---|
What a set is
Each set is a deterministic subset of one public test split. Clips of 1 to 30 seconds are ranked by
sha256("<source>|<id>") and taken in that order until the set holds 1,000 utterances (FLEURS
and small test sets are used whole). Audio is stored as 16 kHz mono FLAC; references are stored raw, and
scoring normalizes both sides with a per-language normalizer.
- Independent samples. Every set has a cluster key (speaker, recording, talk or sentence). No cluster may hold more than max(20, 2n/clusters) rows, so a few speakers cannot dominate a set; sets with fewer than 300 rows or 100 clusters carry a low-power flag.
- Clean references. A reference is dropped when fewer than half of its letters are in the language's scripts, when it has more than 40 letters per second of audio (it cannot belong to the clip), or when the clip is silent.
- No hidden duplicates. Each clip carries the sha256 of its 16 kHz PCM. A clip may appear in only one set of a language; copies of the same corpus under another name are removed.
- Reproducible. Every set has a manifest with the pinned source revision, every row id, and the sha256 of its audio and text.
Conditions
References
Each set records how its references were made. Sets with machine references are shown as diagnostics and are never ranked.
Tiers
Tiers describe how deep a language's data is, not how important the language is. Tier 1 (full): at least four independent real-condition sets beyond FLEURS and Common Voice. Tier 2 (standard): two or three. Tier 3 (minimal): FLEURS and/or Common Voice only; shown, not ranked in the global board.
Hosting
Sets whose licence allows it are re-hosted in one dataset per language, kadirnar/stt-arena-<lang>,
one config per set, each keeping its source licence. Sets from gated sources (the authors' access terms), Common
Voice (the Mozilla Data Collective terms forbid mirrors) and sources without a redistribution licence ship as
manifests and are rebuilt from the source; every rebuilt row must match the manifest's hashes.
from datasets import load_dataset
ds = load_dataset("kadirnar/stt-arena-tr", "fleurs_tr", split="test")
# columns: id, audio, text (raw reference), references, duration, lang, variety,
# cluster, speaker, stratum, audio_sha256, metaScoring (next release)
- Headline unit per language: WER (space-insensitive) for space-delimited scripts, CER without spaces for Chinese, Japanese, Korean, Thai, Lao, Khmer and Burmese. CER is always shown next to WER.
- Normalizer: a universal layer, a script module and a per-language profile (Turkish casing, Hebrew niqqud, Persian yeh and ZWNJ, Indic nukta, Thai sara am, ...), with golden tests per language.
- Each model gets its own language code for every set; an auto-detect run is reported as a diagnostic.
- Per-language score: weighted geometric mean over sets, weights split by condition (FLEURS low); cluster bootstrap intervals; a zero-shot view that hides sets whose corpus a model was trained on.