Introducing Muse Voice Transcribe

We’re excited to introduce Muse Voice Transcribe, the first real-time audio perception model developed by Meta Superintelligence Labs.

Muse Voice Transcribe marks a first milestone in bringing our real-time voice models to you. It delivers real-time streaming ASR, diarization with 20+ speakers, and endpointing. It is multilingual with seamless code-switching and improves accuracy with language, keyword, and context biasing.

We rank first on Artificial Analysis on streaming speech-to-text and on public diarization benchmarks. Model inclusion and rankings as of September 1, 2026.

Streaming ASR as the Foundation

Muse Voice Transcribe is an autoregressive multimodal model from the Muse Spark family.

Audio is processed in 80ms chunks (12.5 Hz), each of which is transformed into a single soft token. At each audio chunk, the model decides to either continue listening to the next audio chunk or emit a text token. When the model decides to continue listening, it predicts a <|next_audio|> special token and replaces <|next_audio|> with the actual audio chunk for the next input.

When the audio stream stops, we insert a special <|empty_audio|> token to inform the model there are no more audio chunks. The model emits all remaining text tokens without producing any <|next_audio|> tokens after seeing <|empty_audio|>.

Loading demo

Since the model has full control over when to listen, it decides on the amount of audio context before transcribing a word (we call this “delay”). There is a tradeoff between accuracy and delay. The longer the model waits to predict, the more accurate the transcript, but the higher the latency. Muse Voice Transcribe has “adaptive delay,” dynamically changing delay for each word based on difficulty. This is enabled with reinforcement learning (RL), where word error rate (WER) reward and a delay reward are combined multiplicatively.

With adaptive delay, Muse Voice Transcribe achieves the Pareto front on speed-accuracy trade-off measured by time to final transcription.

Building Diarization and Endpointing on Top of ASR

With streaming ASR as the foundation, we can easily support other audio perception tasks by introducing additional special tokens.

For diarization, we introduce a <|start_of_turn|> token to mark potential speaker switch and <|speaker_{A-Z}|> tag to differentiate the speaker. <|start_of_turn|> is predicted as soon as speaker switches and speaker tag prediction is delayed to the end of the chunk. The audio from the same speaker may also be segmented by <|start_of_turn|> but the speaker tag will be the same for all these segments.

Here is an example token sequence for diarization (we omit <|next_audio|> for simplicity):

<|start_of_turn|>Hello, how are you doing?<|speaker_A|><|start_of_turn|> Did anything fun over the weekend?<|speaker_A|><|start_of_turn|> Hey I'm good!<|speaker_B|>

For endpointing, we introduce a <|speech_onset|> token to mark beginning of the speech and <|speech_endpoint|> token to mark when the user finishes speaking.

Here is an example token sequence for endpointing (we omit <|next_audio|> for simplicity):

<|speech_onset|>Hey Meta, what's the weather in Menlo Park?<|speech_endpoint|> [silence] <|speech_onset|>What should I wear today?<|speech_endpoint|>

We train both tasks together with streaming ASR and add extra rewards on top of the ASR reward for diarization and endpointing respectively.

Capabilities

Language Coverage

Muse Voice Transcribe is trained with 70+ languages, of which 25 are extensively verified. In the initial release, we recommend trying these 25 validated languages, with support for additional languages also available.

ChineseFrenchHindiJapaneseSpanishVietnamese

我这个月去了十二次健身房，体重从 78.5 公斤直接降到 75 公斤，体脂率也掉了 2.5%。我今天早上跑完五公里，花了 28 分钟，算一算平均心率是 145，感觉体能比三个月前好多了。

En ce moment j'apprends les 50 états des États-Unis. Il faut aller en ordre alphabétique. Il y a quatre États qui commencent par la lettre A, trois par la lettre C et ainsi de suite. J'espère bientôt tous les mémoriser.

Start demo: Muse Voice Transcribe Chinese transcription demoClick to play demoTap to play demo

我这个月去了十二次健身房，体重从 78.5 公斤直接降到 75 公斤，体脂率也掉了 2.5%。我今天早上跑完五公里，花了 28 分钟，算一算平均心率是 145，感觉体能比三个月前好多了。

Play Muse Voice Transcribe Chinese transcription demo

Muse Voice Transcribe Chinese transcription demo playback position0:00 of 0:19

Code-Switching

Code-switching is very common for bilingual speakers. Muse Voice Transcribe natively supports arbitrary code-switching, either within sentence or between sentence. In this audio clip, it also uses context biasing to further improve recognition accuracy.

Long Context

Our model natively supports long audio input exceeding one hour and up to 20+ speakers, with no required post-processing.

Applications

With one click, voice dictation across Meta AI and Muse Code is now powered by Muse Voice Transcribe. It can be used with any application and work through tasks with Meta AI and any window on your screen — just hold 'Fn' to try.

Muse Voice Transcribe is available today via Meta Model API, Meta AI for Mac, and Muse Code.