Open Source Contribution to BerriAI/litellm
BerriAI/litellm is an AI gateway and Python SDK that lets developers call 100+ LLM providers through one OpenAI-compatible interface, including the /v1/audio/transcriptions endpoint. When a request does not specify a response_format, LiteLLM picks a provider-specific transcription config that decides which format to send upstream.
For OpenAI models, that config was chosen by a name check: only models containing gpt-4o got the GPT transcription config, which sends no forced format. Every other model fell through to the Whisper config, which forces verbose_json. The newer gpt-transcribe model has no "4o" in its name, so it was routed to the Whisper config. That caused three visible problems:
audio.transcriptions.create(model="gpt-transcribe", file=...) with no response_format was rejected because verbose_json is not compatible with gpt-transcribe. Open WebUI's mic button and file upload surfaced this as an error toast.GET /health?model=gpt-transcribe answered 503 with the same error, so the model looked unhealthy even though it was fine.response_format: json workaround, uploads whose length the gateway cannot measure (an .m4a file, for example) logged $0 spend, so cost tracking and budgets silently missed them.Instead of special-casing response_format for one model inside the Whisper config, the fix routes gpt-*transcribe models to the GPT transcription config:
gpt-transcribe now behaves exactly like gpt-4o-transcribe, so the two cannot drift apart.whisper-1 still receives verbose_json by default.response_format set on the request or on the model already worked and still does.Fixing the 400 exposed the third problem: gpt-transcribe is priced per second, but when the gateway cannot measure an upload's length, the cost logged as $0. For these models, OpenAI returns usage.seconds in the JSON reply, so the final PR uses it as a fallback:
Logging spend that used to be $0 can affect budgets, so the PR description leads with a breaking-change note. Deployments on the response_format: json workaround see the new charges too, so admins with tight transcription budgets are told to check them. Unmeasurable text replies carry no usage data and still log $0; that gap is tracked separately.
The OpenAI branch of the audio transcription config lookup. The model check was widened from a single substring to the whole GPT transcription family, so gpt-transcribe gets the GPT config instead of the Whisper one:
elif litellm.LlmProviders.OPENAI == provider:
- if "gpt-4o" in model:
+ if "gpt-4o" in model or (model.startswith("gpt-") and "transcribe" in model):
return litellm.OpenAIGPTAudioTranscriptionConfig()
else:
return litellm.OpenAIWhisperAudioTranscriptionConfig()Adds the seconds fallback for billing. Two pieces work together:
def _bills_provider_transcription_seconds(model: str | None, custom_llm_provider: str | None) -> bool:
if model is None or custom_llm_provider != LlmProviders.OPENAI.value:
return False
transcription_config: Final = ProviderConfigManager.get_provider_audio_transcription_config(
model=model.removeprefix(f"{LlmProviders.OPENAI.value}/"), provider=LlmProviders.OPENAI
)
return isinstance(transcription_config, OpenAIGPTAudioTranscriptionConfig)completion_cost, the measured duration (hidden params, then the response's own duration) is evaluated first. Only if that comes back empty does the code use usage.seconds from the reply, and only for OpenAI GPT transcribe models:audio_transcription_file_duration = _hidden.get(
"audio_transcription_duration",
getattr(completion_response, "duration", 0.0),
) or (
completion_response.usage.seconds
if isinstance(completion_response, TranscriptionResponse)
and isinstance(completion_response.usage, TranscriptionUsageDurationObject)
and _bills_provider_transcription_seconds(requested_model, custom_llm_provider)
else 0.0
)Tests the default response_format by checking the request each model actually produces, rather than checking which config class was selected:
[
("whisper-1", {}, "verbose_json"),
("my-transcribe-whisper", {}, "verbose_json"),
("whisper-1", {"response_format": "srt"}, "srt"),
("gpt-4o-mini-transcribe", {}, None),
("gpt-transcribe", {}, None),
("gpt-transcribe", {"response_format": "text"}, "text"),
]my-transcribe-whisper contains "transcribe" but does not start with gpt-, so it correctly stays on the Whisper path.srt or text format is always respected.A parametrized test builds a real transcription response (reported usage.seconds of 2) and checks the computed cost for each case:
A second test confirms a custom-priced deployment (input_cost_per_second set on the deployment) bills at its own price rather than the cost map's.
End-to-end tests run through the real gateway against a scripted OpenAI server that rejects verbose_json for gpt-transcribe with a 400, exactly as the vendor does. They cover:
verbose_json Upstream: a default call returns 200, and the upstream request contains only the model and the uploaded file, byte for byte..m4a bills the 2 seconds OpenAI reports, at both the cost-map price and a deployment price, and the response cost header, the spend log row, and the key's total spend all agree.Registers openai/gpt-transcribe as an audio_transcription model and checks that the health report shows one healthy and zero unhealthy endpoints, and that the probe sent to OpenAI contains only the model with no verbose_json.
I opened the PR without an OpenAI API key, so I could not test against the live API. Instead I ran a local HTTP server that stands in for OpenAI and records exactly what litellm.transcription sends. That showed the routing fix on the real code path, with only the upstream response mocked:
# Before (main)
gpt-transcribe: response_format sent = verbose_json
gpt-4o-transcribe: response_format sent = (not sent)
whisper-1: response_format sent = verbose_json
# After (fix branch)
gpt-transcribe: response_format sent = (not sent)
gpt-4o-transcribe: response_format sent = (not sent)
whisper-1: response_format sent = verbose_jsonI listed what I could not verify as caveats in the PR, including that gpt-transcribe is priced per second but json replies carry no duration, so cost might log as $0. That caveat became the billing fix in the final version.
Because of my missing API key, a contributor from the BerriAI team took over and helped carry the PR forward. They widened the scope to cover health checks and billing, and re-ran the proof against the real OpenAI API on 2026-10-16. Each side ran on its own 2-worker proxy with Postgres, built from the merge base and from the PR tip with the same config. Clients were the OpenAI Python SDK 3.31.0 and the Open WebUI 0.12.2 interface from the original issue, driven in Chromium. The test clip was a 1.35 s recording, also encoded as AAC, which the gateway cannot measure.
| Case | Before | After |
|---|---|---|
| SDK, no format, m4a | 400 | 200, $0.00015 |
| SDK, no format, wav | 400 | 200, $0.000101 |
| Open WebUI mic and upload | Error toast | Works |
| curl, no format | 400 | 200 |
| /health | 503 | 200 |
| json workaround, m4a | No cost logged | $0.00015 |
| Custom-priced deployment, m4a | 400 | 200, $0.0006 |
| Open WebUI API | 500 | 200 |
| whisper-1, json, m4a (control) | No cost logged | No cost logged |
The PR also went through two structured checks against the merge base:
gpt-transcribe and gpt-live-transcribe no longer get verbose_json added. Whisper, gpt-4o-mini-transcribe, chat, streaming chat and the readiness probe matched exactly. It classified the new spend logging and key budget counting as backward-incompatible by design, and named one regression risk: an OpenAI-compatible host serving a gpt-*transcribe name that accepts only verbose_json would lose that default.verbose_json for gpt-transcribe exactly as the vendor does), with nothing inside the proxy mocked.| Contract | Base | Head |
|---|---|---|
| SDK call with no format | 400 | 200 |
| Unmeasured m4a, cost-map price | $0 | $0.00015 |
| Unmeasured m4a, deployment price | $0 | $0.0006 |
| Measured wav beats reported seconds | $0.0003 | $0.0003 |
| /health probe of gpt-transcribe | Unhealthy | Healthy |
A notable aspect of the review lifecycle was BerriAI's orchestration of multiple automated AI agents alongside human maintainers to validate correctness, prevent regressions, and assess risk before merging:
@mateo-berri actively steered the review by rebasing the branch, running sanity audits, and invoking specialized AI agents via repository comment commands (e.g., @litereview code-review, bugbot run, and @greptileai).litereview): Triggered via @litereview code-review, this agent performed deep semantic validation of the diff. It verified that gpt-*transcribe models cleanly bypassed verbose_json, confirmed zero correctness regressions, and caught a documentation inaccuracy regarding Azure billing scope that Mateo promptly addressed.b7bd696), Bugbot performed static and symbolic checks on the diff to ensure no unexpected exceptions or null-dereference regressions were introduced.usage.seconds fallback was finalized and tested, Greptile upgraded the review to a 5/5 Confidence Score, clearing the PR for squash-and-merge.text replies carry no usage data, so unmeasurable text uploads still log $0, tracked separately.whisper-1 json uploads the gateway cannot measure still log $0, as before.response_format: text, gpt-transcribe returns a JSON body, unchanged by this PR.gpt-*transcribe name, and a live Azure gpt-transcribe deployment were not verified live.The PR was merged into BerriAI:main, fixing issues #43867 and #44543. Default gpt-transcribe calls now return transcripts, the health check passes, and unmeasurable uploads are billed instead of logging $0.
Hi! I'm Irvallen's AI assistant. Ask me anything about his projects or experience.