登顶 HF 热榜、下载达20万：有道「子曰4 -R2T2」模型技术报告发布

Summary: This is an open-source model for real-time speech transcription, focused on stable output and adjustable latency.

1. NetEase Youdao's Confucius4-R2T2 topped Hugging Face's automatic speech recognition trending leaderboard four days after release, with about 200,000 global downloads—roughly 62,000 for the main model and 140,000 combined for quantized versions.

2. It builds on Qwen3-ASR-1.7B and is post-trained on about 1,500 hours of Chinese and English audio. Its core design is to emit only text that is already stable and never revise committed text, reducing subtitle jitter and translation rewrites.

3. The quality-first configuration achieves a 5.76% aggregate macro-average error rate across 13 main test sets, below X-ASR's 8.97%. With 160 ms audio chunks, the low-latency configuration has average backtracking latency of about 0.53 seconds for Chinese and 0.42 seconds for English; 80 ms chunks can go below 0.5 seconds but require a quality trade-off.

4. It can be used for real-time subtitles, dictation, simultaneous interpretation, and voice assistants. After open-sourcing, quantized versions and runtime environment adaptations allow developers to tune quality and speed for their scenarios.

Summary: Youdao uses this open-source model to strengthen its real-time speech technology brand and showcase R&D capabilities through a model matrix.

1. Brand visibility: R2T2 topped Hugging Face's ASR trending leaderboard just four days after release, with about 200,000 global downloads. The technical report went live on arXiv, creating a combined rollout across open-source release, leaderboard performance, technical report, and community adaptation.

2. Product R&D: The model builds on Qwen3-ASR-1.7B and is post-trained on about 1,500 hours of Chinese and English audio. Through longest stable prefix, explicit waiting, and streaming commitment, it incorporates when to output into training; the two stages include supervised fine-tuning and GRPO reinforcement learning.

3. User value: Aimed at real-time subtitles, simultaneous interpretation, and voice assistants, it does not revise committed text, reducing subtitle jitter, translation rewrites, and redundant inference. Quality-first and low-latency configurations are adjustable to cover different experience requirements.

4. Brand matrix: Youdao also has Confucius4 multimodal, T3PO streaming translation, and TTS speech synthesis, covering listening, understanding, translation, and expression, while using open-source community feedback to expand validation in real-world scenarios.

Summary: The open-sourcing of real-time speech recognition creates application opportunities, but quality, latency, and commitment stability must be balanced.

1. Market opportunity: R2T2 topped the HF ASR trending leaderboard four days after release and reached about 200,000 downloads, indicating growing interest in real-time transcription, subtitles, simultaneous interpretation, and voice agents. Quantized versions and runtime adaptations lower the barrier for developers to experiment and can drive related applications and hardware scenarios.

2. Shifting demand: Real-time applications require not only accurate recognition but also timeliness and stability. If committed text is revised, it can cause subtitle jitter, translation rewrites, or redundant inference, making stable incremental text the foundation for downstream processing.

3. Positive and negative effects: The quality-first configuration has a 5.76% aggregate macro-average error rate, while the low-latency configuration has average backtracking latency of about 0.53 seconds for Chinese and 0.42 seconds for English. However, earlier output may introduce recognition errors, so audio chunk size and provisional tail must be tuned by scenario.

4. Lessons: Advance partnerships through open-source models plus community quantization, technical reports, and a model matrix. T3PO can be chained with R2T2 to support simultaneous speech translation, forming real-time listen, translate, and speak capabilities.

Summary: R2T2 provides integrable open-source recognition capabilities for real-time voice product design, with emphasis on localization, adjustable latency, and stable output.

1. Product design needs: Real-time subtitles, dictation, simultaneous interpretation, voice assistants, and agents all need continuously arriving speech converted into stable incremental text. Committed content is not revised, reducing terminal UI jitter and repeated processing.

2. Production and deployment implications: The model builds on Qwen3-ASR-1.7B. After open-sourcing, quantized versions and adaptations for different runtime environments allow the community to build local real-time dictation. On the inference side, cache reuse, bounded context, and provisional-tail validation reduce redundant computation in streaming updates.

3. Business opportunities: With 160 ms audio chunks, the quality-first configuration has a 5.76% aggregate macro-average error rate, and the low-latency configuration has average backtracking latency of about 0.53 seconds for Chinese and 0.42 seconds for English. With 80 ms chunks, latency can fall below 0.5 seconds but requires a quality trade-off, making selection dependent on device compute and interaction requirements.

4. Digital direction: Speech recognition can be chained with T3PO streaming translation and TTS speech synthesis to cover listening, understanding, translation, and expression, providing an incremental text entry point for meetings, customer service, hardware terminals, and other scenarios.

Summary: The pain point in real-time voice services is stable incremental output; R2T2 offers a solution with adjustable quality and latency.

1. Customer pain points: Real-time subtitles, simultaneous interpretation, and voice agents require speed, accuracy, and stability. If text already handed to downstream systems is revised, it causes subtitle jitter, translation rewrites, or redundant inference. Output too early is error-prone, while waiting too long slows interaction.

2. Technical approach: It uses longest stable prefix (LSP) to construct partial-audio supervision and adds explicit WAIT, so the model commits when evidence is sufficient and waits when information is insufficient. The two-stage process uses supervised fine-tuning and GRPO reinforcement learning, with rewards covering accuracy, redundancy, repetition, language consistency, tail continuation, and WAIT placement.

3. Inference service: The commitment policy keeps published text unchanged and combines cache reuse, bounded context, and provisional-tail validation. With 160 ms audio chunks, the quality-first configuration has a 5.76% aggregate macro-average error rate, and the low-latency configuration has average backtracking latency of about 0.53 seconds for Chinese and 0.42 seconds for English.

4. Industry trends: Although post-trained only on Chinese and English, the model achieves lower error than the base model on 18 test sets in 10 other languages, indicating cross-lingual transfer. Youdao's model matrix covers multimodal, ASR, streaming translation, and TTS, enabling combined services.

Summary: Open-source platforms and community adaptation amplify R2T2's reach; platform operators need to focus on distribution, developer experience, and stability risks.

1. Platform needs: Real-time speech models must balance recognition quality, output latency, and stable commitment. Developers need quantized versions, adaptations for different runtime environments, and adjustable parameters, while platforms should facilitate model downloads, evaluation, and scenario integration.

2. Latest practices: R2T2 was open-sourced on Hugging Face, topped the ASR trending leaderboard in four days, and reached about 200,000 downloads—roughly 62,000 for the main model and 140,000 combined for quantized versions. The technical report went live on arXiv, forming a closed loop of open-source project, paper, and community feedback.

3. Operational management: On the inference side, quality-first and low-latency configurations are provided. With 160 ms audio chunks, the aggregate macro-average error rate is 5.76%, and the low-latency configuration has average backtracking latency of about 0.53 seconds for Chinese and 0.42 seconds for English. With 80 ms chunks, latency can fall below 0.5 seconds but requires a quality trade-off, so platforms can set default parameters by scenario.

4. Risk avoidance: Not revising committed text is key to real-time applications. Platforms need to avoid premature commitment, duplicate output, excessive completion, and mistaking transcription for translation; provisional-tail validation, cache reuse, and bounded context can reduce experience risks.

Summary: R2T2 incorporates when to commit into streaming ASR training and inference, providing a verifiable technical path and experimental evidence.

1. New problem: Real-time transcription must not only recognize content but also determine which words can be committed when an utterance is unfinished. Output too early is error-prone, waiting too long slows downstream processing, and committed text cannot be revised.

2. Method innovation: Based on Qwen3-ASR-1.7B, the audio encoder and projection layer are frozen, and only the language decoder is updated. It is post-trained on about 1,500 hours of Chinese and English audio. It constructs the longest stable prefix (LSP), records model outputs at 80 ms intervals, and adds explicit WAIT. The two stages are supervised fine-tuning and GRPO reinforcement learning.

3. Theoretical properties: LSP remains prefix-monotonic as audio advances, matches the reference output on complete audio, and is maximal among candidate labels satisfying the constraints. During training, 90% of samples are truncated at an audio prefix, while 10% retain complete audio and transcription.

4. Experimental implications: On eight English and five Chinese test sets, the quality-first configuration achieves a 5.76% aggregate macro-average error rate, below X-ASR's 8.97%; with 160 ms low-latency configuration, average backtracking latency is about 0.53 seconds for Chinese and 0.42 seconds for English. Chinese-English post-training transfers to 18 test sets in 10 other languages.