Inside Veena: Reverse-Engineering Maya Research's Open-Weight Hindi TTS Model

This is a deep-dive follow-up to the main Voice AI guide. Part 2 mentioned that Indic text-to-speech is the weakest layer of most frontier voice stacks and that open Hindi models are closing the gap. This post pulls apart the best open Hindi TTS model of 2025 — Maya Research's Veena — and shows exactly how it is built, using only its public files on Hugging Face.

The companion posts in this series apply the same findings: Part 5 is a practical recipe for training a Hindi TTS model that beats Veena. Part 6 shows how we run Veena in production behind OpenAI Realtime and Twilio with sub-second turn latency.

What Maya Research publishes

Maya Research is an India-based lab. On Hugging Face they publish two open-weight voice models under Apache 2.0:

• `maya-research/maya1` — English-first expressive TTS with emotion and voice-design tokens.
• `maya-research/Veena` — Hindi and English TTS, their most-downloaded model and the basis of this write-up.

Both ship with `config.json`, `tokenizer_config.json`, `chat_template.jinja`, a `prompt.txt` that documents the voice-design DSL, an `emotions.txt` list and a `vllm_streaming_inference.py` reference implementation. Everything below is pulled from those files plus the `safetensors` header, read with HTTP range requests so there is no need to download the 6 GB of weights.

There is no magic — it is a published recipe applied well

Both models are the same shape:

Meta Llama-3.2-3B fine-tuned to autoregressively generate SNAC audio codec tokens instead of (or in addition to) text, with the vocabulary extended by exactly 7 × 4096 = 28,672 codec tokens plus a few control and emotion tokens.

This is the same recipe as Canopy Labs' Orpheus-TTS, and before it SmolLM-TTS and Spark-TTS. The "good voice" comes from four engineering choices, not from a secret model.

Hard evidence from the architecture

The safetensors header and `config.json` give identical numbers to stock `meta-llama/Llama-3.2-3B`, except for the vocabulary size.

Tensor names are vanilla Llama: `model.embed_tokens`, `model.layers.N.self_attn.{q,k,v,o}_proj`, `mlp.{gate,up,down}_proj`, the two layernorms per block and `model.norm`. There is no `lm_head` tensor — tied embeddings. There are no audio-specific modules in the model. All audio modelling happens through the vocabulary extension.

What the extra tokens are

The 156,960 − 128,256 = 28,704 new vocabulary slots decompose as:

• 28,672 SNAC codec tokens at ids 128,266 to 156,937 — exactly 7 frames × 4,096 codebook entries.
• A reused block of Llama's generic `<custom_token_0..9>` as control tokens, with 128,257 as start-of-speech and 128,258 as end-of-speech. Both are confirmed in `vllm_streaming_inference.py`.
• One `<|audio|>` marker at 156,939.
• 20 named emotion and paralinguistic tokens at 156,940 to 156,959: `<angry>`, `<appalled>`, `<chuckle>`, `<cry>`, `<curious>`, `<disappointed>`, `<excited>`, `<exhale>`, `<gasp>`, `<giggle>`, `<gulp>`, `<laugh>`, `<laugh_harder>`, `<mischievous>`, `<sarcastic>`, `<scream>`, `<sigh>`, `<sing>`, `<snort>`, `<whisper>`.

Veena has 9 fewer tokens because it drops the emotion set — it is pure TTS.

How the audio pipeline actually works

All of this is lifted directly from Maya's own `vllm_streaming_inference.py`:

• Codec: `hubertsiuzdak/snac_24khz`, a hierarchical audio codec at 24 kHz mono, open weights, MIT. Maya did not train the codec.
• Frame layout: 7 flattened tokens per hierarchical SNAC frame, packed as `[L1, L2_even, L3_0, L3_1, L2_odd, L3_2, L3_3]`. Each slot is `offset(128266) + level_code`, so all three SNAC levels share one flat `7 × 4096` vocabulary slice.
• De-tokenisation: `code = (token − 128266) mod 4096`, then regroup to SNAC's 3 hierarchical levels and call `snac_model.quantizer.from_codes` followed by `snac_model.decoder`.
• Streaming trick: slide a 4-frame (28-token) window, decode every 7 new tokens, and keep only the middle 2,048 samples of each decode. This cross-fade removes the "pop" artifacts that plague naïve chunked codec decoding.
• Sampling defaults: `temperature=0.4`, `top_p=0.9`, `repetition_penalty=1.1`, `min_tokens=28` so output is never a one-frame glitch.
• Logits processor `OnlyAudioAfterSOS` masks every non-SNAC token to −∞ after start-of-speech. The model physically cannot hallucinate text mid-speech.

The prompt format — their actual voice-design DSL

The chat template is the unmodified Llama-3.1 Instruct template. The whole voice-control contract lives inside the content of one user turn:

<description="Realistic male voice in the 30s age with american accent.
Normal pitch, warm timbre, conversational pacing, neutral tone
delivery at med intensity."> Hello! <excited> This is amazing!

`prompt.txt` is the data-generation spec Maya used to label training captions. It enumerates the closed vocabulary the model actually learned to condition on:

• Age: 20s, 30s, 40s.
• Gender: male, female.
• Accent: american, indian, middle_eastern, asian_american, british.
• Pitch: low, normal, high.
• Timbre: deep, warm, gravelly, smooth, raspy, nasally, throaty, harsh — plus robotic and ethereal for the creative split.
• Pacing: very_slow, slow, conversational, brisk, fast, very_fast.
• Emotion: neutral, energetic, excited, sad, sarcastic, dry.
• Intensity: low, med, high.
• Plus a "realistic vs creative" split with 8 realistic domains and roughly 30 creative character types such as pirate, mafia, villain or cyborg.

"Describe a voice in natural language" is the marketing framing. In reality the training captions were templated from this fixed enum. Descriptions written in that style work well. Freeform descriptions degrade.

Why the voice is actually good

1. SNAC at 24 kHz is a better-sounding codec at low token rate

At about 7 tokens per frame, SNAC's hierarchy preserves prosody and timbre with far less "codec warble" than EnCodec or DAC. Maya inherited this for free by picking SNAC.

2. 3 B parameters is big enough for long-range prosody

Earlier open TTS language models were 0.5 to 1 B parameters. Jumping to 3 B measurably improves naturalness and cross-sentence consistency. The 8k context with Llama-3 RoPE scaling (factor 32, original 8192) covers minutes of audio.

3. Inference engineering, not training, kills artifacts

The `OnlyAudioAfterSOS` logits mask plus the sliding-window SNAC decode is where most open TTS implementations sound worse than Maya even with the same weights. This is the easiest part to copy — and the easiest to ship badly.

4. Caption-constrained training gives a dense conditioning signal

Because the 60-plus voice attributes come from a closed enum in `prompt.txt`, the model gets a consistent conditioning signal instead of noisy free text. Each attribute reliably moves the voice. The "free-form voice design" UX is a thin natural-language wrapper over that enum.

How to replicate Veena end-to-end

A minimum viable Veena clone takes six steps:

• Base LM: `meta-llama/Llama-3.2-3B` or its Instruct variant.
• Extend the tokenizer by 7 × 4,096 SNAC codec tokens plus start-of-speech, end-of-speech, `<|audio|>` and any emotion specials. Resize `embed_tokens` and keep `tie_word_embeddings=true`.
• Build training pairs of (caption, audio). Encode audio with `hubertsiuzdak/snac_24khz` and flatten the three hierarchical levels into Maya's 7-slot per-frame layout.
• Caption everything with the closed enum from `prompt.txt` plus inline paralinguistics such as `<laugh>`. Use automated age, gender and accent classifiers, a pacing detector and an LLM polish pass.
• Supervised fine-tuning on `prompt_tokens + <SOS> + snac_token_stream + <EOS>` with the standard causal LM loss. SNAC stays frozen.
• At inference, apply the `OnlyAudioAfterSOS` logits mask and the 28-token / 2,048-sample sliding-window decode.

Honest caveats

• Training data is marked `dataset: proprietary` on the Veena card. Hours, sources and speaker count are not disclosed.
• The weights alone do not say whether Maya did a continued-pretraining phase on a large audio corpus before supervised fine-tuning, or SFT only. The published Orpheus and Spark recipes are two-stage. Given the quality, Maya likely did the same.
• Veena uses `eos_token_id=128001` (`<|end_of_text|>`) rather than maya1's audio `<custom_token_2>=128258` as the audio stop. That suggests Veena was trained with less strict text / audio phase separation. maya1 is the newer and more refined model.

What this changes for Indian voice AI

Veena matters because it moves the Hindi quality bar inside reach of anyone with a single L4 GPU and a few thousand hours of audio. Combined with a strong Indic STT like Sarvam Saarika and a frontier LLM, it closes most of the gap with English-only voice stacks for Indian callers. The next two posts show how to push past Veena (Part 5) and how to run it in production (Part 6).