Skip to Content
ModelsElevenLabs

ElevenLabs family

ElevenLabs’ speech models, two in total: Eleven v4 for speech synthesis and Scribe v2 for transcription. Both are called through the OpenAI-compatible audio endpoints.

Model IDs (copy them exactly)
Eleven v4 (speech synthesis)elevenlabs/eleven_v4POST /v1/audio/speech
Scribe v2 (transcription)elevenlabs/scribe_v2POST /v1/audio/transcriptions

For the common endpoint parameters, see Audio Speech API and Audio Transcriptions API. For live prices, see the model page : speech synthesis is billed per character, and transcription is billed by audio duration.

Eleven v4 speech synthesis

Key pointvoice takes a voice ID, not a voice name; OpenAI voices such as alloy are not supported.
eleven_tts.py
import os
from openai import OpenAI
 
client = OpenAI(base_url="https://api.ofox.io/v1", api_key=os.environ["OFOX_API_KEY"])
 
with client.audio.speech.with_streaming_response.create(
    model="elevenlabs/eleven_v4",
    voice="21m00Tcm4TlvDq8ikWAM",
    input="Hello, and welcome to OfoxAI speech synthesis.",
    response_format="mp3",
) as resp:
    resp.stream_to_file("speech.mp3")
ParameterDescription
voiceRequired. An ElevenLabs voice ID (20 characters), e.g. 21m00Tcm4TlvDq8ikWAM. Find voice IDs in the ElevenLabs voice library . Passing a voice name (e.g. Rachel) returns 404
response_formatmp3 (default, 44.1 kHz), opus (48 kHz, Ogg container), wav (16 kHz), pcm (16 kHz, 16-bit mono). aac and flac are not supported
language_codeLanguage code, e.g. zh, en; detected automatically if omitted
provider_optionsPasses ElevenLabs-specific parameters through, e.g. {"elevenlabs": {"voice_settings": {"stability": 0.3}}}. For parameter definitions, see the official ElevenLabs documentation 

In our tests, a Chinese sentence of about 20 characters returned in about 2 seconds, producing about 4 seconds of audio.

Scribe v2 transcription

In addition to the transcribed text, Scribe v2 returns start and end times for each word and can identify different speakers.

scribe.py
import json
import os
from openai import OpenAI
 
client = OpenAI(base_url="https://api.ofox.io/v1", api_key=os.environ["OFOX_API_KEY"])
 
with open("meeting.mp3", "rb") as f:
    result = client.audio.transcriptions.create(
        model="elevenlabs/scribe_v2",
        file=f,
        extra_body={"provider_options": json.dumps({"elevenlabs": {"diarize": True}})},
    )
 
print(result.text)
for w in (result.model_extra or {}).get("words", [])[:10]:
    print(w.get("speaker"), w["start"], w["end"], w["word"])

Response structure:

FieldDescription
textFull transcribed text
words[]Word-level results with word, start, end (seconds); with speaker diarization enabled, also speaker (e.g. speaker_0, speaker_1)
languageDetected language, e.g. zho, eng
durationAudio duration (seconds)
usage{"type": "duration", "seconds": ...}, billed by duration
  • Speaker diarization must be enabled through provider_options: {"elevenlabs": {"diarize": true}}, passed as a JSON string in a form field.
  • response_format supports json and verbose_json, which return the same content. If you need a subtitle file, generate it yourself from the timestamps in words.

Common errors

HTTPError messageCauseFix
404A voice with voice_id 'Rachel' was not found.voice was given a voice name or a nonexistent IDUse a voice ID
400unsupported ElevenLabs response_format: aacUnsupported formatUse mp3, opus, wav, or pcm
400Unsupported response_format 'srt'. Only 'json' and 'verbose_json' are supported on this gatewayTranscription used text, srt, or vttUse json

For more errors, see Audio Speech API and Audio Transcriptions API.

Official docs

FAQ

What do I put in voice for ElevenLabs?

A voice ID (20 characters), such as 21m00Tcm4TlvDq8ikWAM, which you can find in the ElevenLabs voice library. A voice name (such as Rachel) or an OpenAI voice such as alloy returns 404 voice_id not found.

Which audio formats does Eleven v4 support?

mp3 (default), opus, wav, and pcm. aac and flac are not supported.

Can Scribe v2 distinguish speakers?

Yes. Pass {"elevenlabs": {"diarize": true}} through provider_options, and each word in the response words will carry a speaker field, such as speaker_0 or speaker_1.

Can Scribe v2 output srt subtitles directly?

No. response_format supports only json and verbose_json. You can generate subtitles yourself from the start and end timestamps in the response words.

How is ElevenLabs speech billed?

Eleven v4 is billed by the number of input characters and Scribe v2 by audio duration; see the model page for rates.

Last updated on
Hailuo (MiniMax)Gemini TTS