liteLLM

Backed by Y CombinatorGive DevelopersAzure AccessGemini AccessBedrock AccessOpenAI AccessAnthropic AccessAzure AccessGemini AccessBedrock AccessOpenAI AccessAnthropic AccessAzure AccessGemini AccessBedrock AccessOpenAI AccessAnthropic AccessAzure AccessGemini AccessBedrock AccessOpenAI AccessAnthropic AccessAI Gateway to provide model access, fallbacks and spend tracking across 100+ LLMs. All in the OpenAI format.Deploy LiteLLM On-Prem🚅 LiteLLMCost TrackingBatches APIGuardrailsModel AccessBudgetsLLM ObservabilityRate LimitingPrompt Managements3 LoggingPass-Through EndpointsUser240M+docker pulls1B+requests served80%uptime1,005+contributorsWhat is LiteLLM?LiteLLM simplifies model access, spend tracking and fallbacks across 100+ LLMs. Watch this demo, to learn more.FeaturesLiteLLM makes it easy for Platform teams to give developers LLM accessSpend TrackingBudgets & Rate LimitsOpenAI-CompatibleLLM FallbacksAccurately charge teams for their usage.Attribute cost to key/user/team/org -> Automatic spend tracking across OpenAI/Azure/Bedrock/GCP/etc. ->Tag-based spend tracking ->Log spend to s3/gcs/etc. ->Prompt formatting support for HF models ->The Best Pricing PlansOpen Source$0Free100+ LLM Provider Integrations Langfuse, Arize Phoenix, Langsmith, OTEL LoggingVirtual Keys, Budgets, Teams Load Balancing, RPM/TPM limitsLLM GuardrailsGet StartedEnterpriseGet In TouchCloud or Self-HostedFor giving LLM access to a large number of developers and projects.Everything in OSSEnterprise Support + Custom SLAsJWT Auth, SSO, Audit LogsAll Enterprise Features - DocsGet QuoteLearn about Enterprise ->Netflix uses LiteLLM to give developers Day 0 LLM accessDavid LeenStaff Software Engineer - NetflixLiteLLM has let my team provide the latest LLM models to our users usually within a day of them being released. Without LiteLLM this would be hours of work each time a new model is announced. It means we don't have to transform inputs and outputs across providers and has saved us months of work.Mark KoltnukPrincipal Architect (GenAI Platform) - LemonadeOur experience with LiteLLM and Langfuse at Lemonade has been outstanding. LiteLLM streamlines the complexities of managing multiple LLM models✨Connect to ContentAdd layers or components to infinitely loop on your page. --- Contact SalesRequest QuoteBook a demoRequest a price quote from our Sales Team. --- Backed by Y CombinatorLiteLLM EnterpriseIdeal for organizations giving LLM access to a large number of developers and projects. Available in both SaaS and self-managed deployment options, LiteLLM Enterprise adds:SSO, SCIM24/7 SupportOIDC/JWT-AuthLive upgrade assistanceTeam/Org AdminsKey/Team-based GuardrailsSecret ManagersKey RotationsContact SalesSee Evaluation ChecklistGet 30 day trial keyTry LiteLLM Enterprise, if you need features like SSO, SCIM, and OIDC/JWT Auth.Try LiteLLM Enterprise, if you need features like SSO, SCIM, and OIDC/JWT Auth.FeaturesAll available on LiteLLM Open-Source.All available on LiteLLM Open-Source.Delegate accessLiteLLM Enterprise allows you to delegate access of teams and organizations to admins. Reducing the time you spend managing the proxy.Generate tokens on the flyWith OIDC/JWT Auth and 'group-based access', developers can automatically generate temporary tokens to call models via LiteLLM. Reducing time spent managing keys and model access.Monitor Proxy in ProductionLiteLLM Enterprise provides Prometheus Metrics, Support with SLA's, and additional alerting features, to help you guarantee reliability in production.Trusted by --- Join LiteLLM Community Get help on Slack / DiscordFill out this form, to join us on Slack + Discord.First NameEmailLinkedIn URLCompanySubmitAre you using LiteLLM Proxy or SDK today?