Skip to main content

v1.63.11-stable

Krrish Dholakia
Ishaan Jaffer

These are the changes since v1.63.2-stable.

This release is primarily focused on:

  • [Beta] Responses API Support
  • Snowflake Cortex Support
  • UI - Credential Management, re-use credentials when adding new models
  • UI - Test Connection to LLM Provider before adding a model
info

This release will be live on 03/16/2025

Demo Instance​

Here's a Demo Instance to test changes:

New / Updated Models in Model Cost Map​

  • Add support for Amazon Nova Canvas model PR
  • Add pricing for Jamba new models PR
  • Add pricing for Amazon EU models PR
  • Add Bedrock Deepseek R1 model pricing PR
  • Update Gemini pricing: Gemma 3, Flash 2 thinking update, LearnLM PR
  • Mark Cohere Embedding 3 models as Multimodal PR
  • Add Azure Data Zone pricing PR
    • LiteLLM Tracks cost for azure/eu and azure/us models

LLM Translation​

  1. New Endpoints
  1. New LLM Providers
  1. New models
  • Support OpenRouter reasoning_content on streaming Get Started
  • Support Bedrock converse cache token tracking Get Started
  1. Bug Fixes
  • Return code, param and type on OpenAI bad request error More information on litellm exceptions
  • Fix Bedrock converse chunk parsing to only return empty dict on tool use PR
  • Fix Azure Function Calling Bug & Update Default API Version to 2025-02-01-preview PR
  • Fix Perplexity incorrect streaming response PR
  • Fix Triton streaming completions bug PR
  • Fix: String data: stripped from entire content in streamed Gemini responses PR
  • Fix: Support bytes.IO when handling audio files for transcription PR
  • Fix: "system" role has become unacceptable in Ollama PR
  • Handle HTTP 201 status code in Vertex AI response PR

Spend Tracking Improvements​

  1. Cost Tracking for Responses API Get Started
  2. Fix Azure Whisper cost tracking PR

UI​

Re-Use Credentials on UI​

You can now onboard LLM provider credentials on LiteLLM UI. Once these credentials are added you can re-use them when adding new models

Test Connections before adding models​

Before adding a model you can test the connection to the LLM provider to verify you have setup your API Base + API Key correctly

General UI Improvements​

  1. Add Models Page
    • Allow adding Cerebras, Sambanova, Perplexity, Fireworks, Openrouter, TogetherAI Models, Text-Completion OpenAI on Admin UI
    • Allow adding EU OpenAI models
    • Fix: Instantly show edit + deletes to models
  2. Keys Page
    • Fix: Instantly show newly created keys on Admin UI (don't require refresh)
    • Fix: Allow clicking into Top Keys when showing users Top API Key
    • Fix: Allow Filter Keys by Team Alias, Key Alias and Org
    • UI Improvements: Show 100 Keys Per Page, Use full height, increase width of key alias
  3. Users Page
    • Fix: Show correct count of internal user keys on Users Page
    • Fix: Metadata not updating in Team UI
  4. Logs Page
    • UI Improvements: Keep expanded log in focus on LiteLLM UI
    • UI Improvements: Minor improvements to logs page
    • Fix: Allow internal user to query their own logs

Security​

  1. Support for Rotating Master Keys Getting Started
  2. Fix: Internal User Viewer Permissions, don't allow internal_user_viewer role to see Test Key Page or Create Key Button Role based access controls
  3. Emit audit logs on All user + model Create/Update/Delete endpoints Get Started
  4. JWT
    • Support multiple JWT OIDC providers Get Started
    • Fix JWT access with Groups not working when team is assigned All Proxy Models access
  5. Using K/V pairs in 1 AWS Secret Get Started

Logging Integrations​

  1. Prometheus: Track Azure LLM API latency metric Get Started here
  2. Allow switching off storing Error Logs in DB Get Started here
  3. Added tags, user_feedback and model_options to additional_keys which can be sent to Athina Get Started here

Performance / Reliability improvements​

  1. Fix Redis cluster mode for routers PR
  2. Delegate router Azure client init logic to Azure provider PR
  3. Fix Azure AI services URL PR
  4. Support extra_headers on Bedrock PR

General Improvements​

  1. UI API Playground for testing LiteLLM translation PR
  2. Fix: Correctly use PROXY_LOGOUT_URL when set PR
  3. Bing Search Pass Through endpoint PR
  4. OpenWebUI Integration - display thinking tokens
  • Guide on getting started with LiteLLM x OpenWebUI. Get Started
  • Display thinking tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) Get Started

Complete Git Diff​

Here's the complete git diff