MLflow

Deliver High-Quality AI, FastBuilding AI products is all about iteration.MLflow lets you move 10x faster by simplifying how you debug, evaluate, and monitor your LLM applications, Agents, and Models.Get StartedView Docs30M+ Downloads/moLLMs & AgentsModel TrainingObservabilityCapture complete traces of your LLM applications and agents to get deep insights into their behavior. Built on OpenTelemetry and supports any LLM provider and agent framework. Monitor production quality, costs, and safety.Quickstart→CodeEvaluationRun systematic evaluations, track quality metrics over time, and catch regressions before they reach production. Choose from 50+ built-in metrics and LLM judges, or define your own with highly flexible APIs.Quickstart→CodePrompts & OptimizationVersion, test, and deploy prompts with full lineage tracking. Automatically optimize prompts with state-of-the-art algorithms to improve performance.Quickstart→CodeAI GatewayUnified API gateway for all LLM providers. Route requests, manage rate limits, handle fallbacks, and control costs through a unified OpenAI-compatible interface.Quickstart→CodeAgent ServerDeploy agents to production with a single command. The MLflow Agent Server provides a FastAPI-based hosting solution with automatic request validation, streaming support, and built-in tracing — so you can go from prototype to production endpoint in minutes.Quickstart→from mlflow.agent_server import AgentServer, invoke, streamfrom mlflow.types.agent import ResponsesAgentRequest, ResponsesAgentResponse @invoke()async def run_agent(request: ResponsesAgentRequest) -> ResponsesAgentResponse: msgs = [i.model_dump() for i in request.input] result = await Runner.run(agent, msgs) return ResponsesAgentResponse( output=[item.to_input_item() for item in result.new_items] ) # Start the serveragent_server = AgentServer("MyAgent")agent_server.run(app_import_string="server:app")CodeMost Adopted Open-Source AIOps PlatformBacked by Linux Foundation, MLflow has been fully committed to open-source for 5+ years. Now trusted by thousands of organizations and research teams worldwide to power their LLMOps and MLOps workflows.mlflow/mlflow30 Million+Package Downloads / MonthWorks With Any FrameworkFrom LLM agent frameworks to traditional ML libraries - MLflow integrates seamlessly with 100+ tools across the AI ecosystem. Supports Python, TypeScript/JavaScript, Java, R, and natively integrates with OpenTelemetry.Why Teams Choose MLflowFocus on building great AI, not managing infrastructure. MLflow handles the complexity so you can ship faster.Open Source100% open source under Apache 2.0 license. Forever free, no strings attached.No Vendor Lock-inWorks with any cloud, framework, or tool you use. Switch vendors anytime.Production ReadyBattle-tested at scale by Fortune 500 companies and thousands of teams.Full VisibilityComplete tracking and observability for all your AI applications and agents.Community20K+ GitHub stars, 900+ contributors. Join the fastest-growing AIOps community.IntegrationsWorks out of the box with LangChain, OpenAI, PyTorch, and 100+ AI frameworks.Get Started in 3 Simple StepsFrom zero to full-stack LLMOps in minutes. No complex setup or major code changes required.Get Started →1Start MLflow ServerOne command to get started. Docker setup is also available.bashuvx mlflow server~30 seconds2Enable LoggingAdd minimal code to start capturing traces, metrics, and parameterspythonimport mlflow mlflow.set_tracking_uri( "http://localhost:5000")mlflow.openai.autolog()~30 seconds3Run your codeRun your code as usual. Explore traces and metrics in the MLflow UI.pythonfrom openai import OpenAI client = OpenAI()client.responses.create( model="gpt-5-mini", input="Hello!",)~1 minuteFrequently Asked QuestionsVisit our FAQ page for everything you need to know about MLflow.What is MLflow?MLflow is the largest open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data. With over 30 million monthly downloads, thousands of organizations rely on MLflow each day to ship AI to production with confidence.MLflow's comprehensive feature set for agents and LLM applications includes production-grade observability, evaluation, prompt management, prompt optimization, an AI Gateway for managing costs and model access, and more. Learn more at MLflow for LLMs and Agents.For machine learning (ML) model development, MLflow provides experiment tracking, model evaluation capabilities, a production model registry, and model deployment tools.Why do I need an AI engineering platform like MLflow?Is MLflow free?How does MLflow compare to other LLMOps/MLOps tools?Can I use MLflow with my existing AI infrastructure?Do I need to use Python to use MLflow?Can I use MLflow in my enterprise organization?BlogLatest newsView allMar 23, 2026Testing and Refining Claude Code Skills with MLflowMar 18, 2026Your Agents Need an AI PlatformMar 4, 2026Agent Trace Evaluation with TruLens Scorers in MLflowView allGET INVOLVEDConnect with the open source communityJoin millions of MLflow usersDocumentationRead DocsGitHub20k starsLinkedIn69k followersYouTubeView tutorialsXFollow us on XSlackJoin our Slack --- Open Source AI Engineering PlatformConfidently ship agents and LLM applications to production with built-in observability, evaluation, prompt management, monitoring, cost controls, and much more.Get StartedView Docs30M+ Downloads/moObservabilityCapture complete traces of your LLM applications and agents to get deep insights into their behavior. Built on OpenTelemetry and supports any LLM provider and agent framework. Monitor production quality, costs, and safety.Quickstart→CodeEvaluationRun systematic evaluations, track quality metrics over time, and catch regressions before they reach production. Choose from 50+ built-in metrics and LLM judges, or define your own with highly flexible APIs.Quickstart→CodePrompts & OptimizationVersion, test, and deploy prompts with full lineage tracking. Automatically optimize prompts with state-of-the-art algorithms to improve performance.Quickstart→CodeAI GatewayUnified API gateway for all LLM providers. Route requests, manage rate limits, handle fallbacks, and control costs through a unified OpenAI-compatible interface.Quickstart→CodeAgent ServerDeploy agents to production with a single command. The MLflow Agent Server provides a FastAPI-based hosting solution with automatic request validation, streaming support, and built-in tracing — so you can go from prototype to production endpoint in minutes.Quickstart→from mlflow.agent_server import AgentServer, invoke, streamfrom mlflow.types.agent import ResponsesAgentRequest, ResponsesAgentResponse @invoke()async def run_agent(request: ResponsesAgentRequest) -> ResponsesAgentResponse: msgs = [i.model_dump() for i in request.input] result = await Runner.run(agent, msgs) return ResponsesAgentResponse( output=[item.to_input_item() for item in result.new_items] ) # Start the serveragent_server = AgentServer("MyAgent")agent_server.run(app_import_string="server:app")CodeMost Adopted Open Source AI PlatformBacked by Linux Foundation, MLflow has been fully committed to open source for 5+ years. Trusted by thousands of organizations and research teams worldwide to power their LLMOps workflows.mlflow/mlflow30 Million+Package Downloads / MonthWorks with Any LLM and Agent FrameworkFrom LLM providers to agent frameworks — MLflow integrates seamlessly with 100+ tools across the AI ecosystem. Supports any programming language and natively integrates with OpenTelemetry and MCP.Why Teams Choose MLflowFocus on building great AI, not managing infrastructure. MLflow handles the complexity so you can ship faster.Open Source100% open source under Apache 2.0 license. Forever free, no strings attached.No Vendor Lock-inWorks with any cloud, framework, or tool you use. Switch vendors anytime.Production ReadyBattle-tested at scale by Fortune 500 companies and thousands of teams.Full VisibilityComplete tracking and observability for all your AI applications and agents.Community20K+ GitHub stars, 900+ contributors. Join the fastest-growing LLMOps community.IntegrationsWorks out of the box with LangChain, OpenAI, PyTorch, and 100+ AI frameworks.Get Started in 3 Simple StepsFrom zero to production-ready agents in minutes. No complex setup or major code changes required.Get Started →1Start MLflow ServerOne command to get started. Docker setup is also available.bashuvx mlflow server~30 seconds2Enable LoggingAdd minimal code to start capturing traces, metrics, and parameterspythonimport mlflow mlflow.set_tracking_uri( "http://localhost:5000")mlflow.openai.autolog()~30 seconds3Run your codeRun your code as usual. Explore traces and metrics in the MLflow UI.pythonfrom openai import OpenAI client = OpenAI()client.responses.create( model="gpt-5-mini", input="Hello!",)~1 minuteBlogLatest newsView allMar 23, 2026Testing and Refining Claude Code Skills with MLflowMar 18, 2026Your Agents Need an AI PlatformMar 4, 2026Agent Trace Evaluation with TruLens Scorers in MLflowView allGET INVOLVEDConnect with the open source communityJoin millions of MLflow usersDocumentationRead DocsGitHub20k starsLinkedIn69k followersYouTubeView tutorialsXFollow us on XSlackJoin our Slack --- ObservabilityLLM and Agent ObservabilityGain visibility into your agent or LLM application's logic to debug issues, improve quality and understand user behavior.Get StartedView DocsRich, detailed traces for every requestEnd to end observabilityCapture your agent or LLM application's inputs, outputs, and step-by-step execution: prompts, retrievals, tool calls, and more.Quickstart→CodeVisualize execution flowDeep dive into your agent or LLM application's logic and latency with a comprehensive and intuitive UI for effective debugging.CodeQuality monitoringTrack and analyze the quality of your agent or LLM application over time, and take action to fix issues before impact spreads.CodeSpot trends and patterns at scaleZoom out with a simplified summary UI to quickly review many traces at once to understand how your agent or LLM application is performing overall.CodeCodeAutomatic tracing for your entire stackAuto-trace 50+ LLM providers and agent frameworks with a single line of code. LLM tracing captures every execution step, and MLflow is OpenTelemetry compatible, supporting any programming language, agent, or LLM.Get Started in 3 Simple StepsFrom zero to full observability in minutes. No complex setup or major code changes required.Get Started →1Start MLflow ServerOne command to get started. Docker setup is also available.bashuvx mlflow server~30 seconds2Enable LoggingAdd minimal code to start capturing traces, metrics, and parameterspythonimport mlflow mlflow.set_tracking_uri( "http://localhost:5000")mlflow.openai.autolog()~30 seconds3Run your codeRun your code as usual. Explore traces and metrics in the MLflow UI.pythonfrom openai import OpenAI client = OpenAI()client.responses.create( model="gpt-5-mini", input="Hello!",)~1 minute --- EvaluationsAgent EvaluationConfidently evaluate quality in development and production to identify issues and iteratively test improvements.Get startedFind quality issues using LLM judges and human feedbackPre-built LLM judgesQuickly start with built-in LLM judges for safety, hallucination, retrieval quality, and relevance. Our research-backed judges provide accurate, reliable quality evaluation aligned with human expertise.CodeCustomized LLM judgesAdapt our base model to create custom LLM judges tailored to your business needs, aligning with your human expert's judgment.CodeCollect human feedbackGather feedback from end users and domain experts directly within your application. Use human annotations to validate LLM judge accuracy, identify blind spots, and continuously improve evaluation quality.CodeCodeIteratively improve qualityTest new agent versionsMLflow's GenAI evaluation API lets you test new agent versions (prompts, models, code) against evaluation and regression datasets. Each version is linked to its evaluation results, enabling tracking of improvements over time.CodeCustomize with code-based metricsCustomize evaluation to measure any aspect of your app's quality or performance using our custom metrics API. Convert any Python function—from regex to custom logic—into a metric.from mlflow.genai.scorers import scorer @scorerdef response_length(request, response): """Check response is within length limits.""" length = len(response.text.split()) return length <= 500 results = mlflow.genai.evaluate( data=eval_data, scorers=[response_length],)Identify root causes with evaluation review UIsUse MLflow's Evaluation UI to visualize a summary of your evals and view results record-by-record to quickly identify root causes and further improvement opportunities.CodeCompare versions side-by-sideCompare evaluations across agent versions to understand if your changes improved or regressed quality. Review individual questions side-by-side in the Trace Comparison UI to find differences, debug regressions, and inform your next version.CodeCodeGet Started in 4 Simple StepsFrom zero to evaluating your agent in minutes. No complex setup required.Get Started →1Start MLflow ServerOne command to get started. Docker setup is also available.bashuvx mlflow server~30 seconds2Enable TracingAdd minimal code to start capturing traces from your agent or LLM app.pythonimport mlflow mlflow.set_tracking_uri( "http://localhost:5000")mlflow.openai.autolog()~30 seconds3Run your codeRun your code as usual. Explore traces and metrics in the MLflow UI.pythonfrom openai import OpenAI client = OpenAI()client.responses.create( model="gpt-5-mini", input="Hello!",)~1 minute4Evaluate with LLM JudgesRun built-in LLM judges to automatically score your app's quality.pythonfrom mlflow.genai.scorers import ( Safety, Correctness,) traces = mlflow.search_traces()mlflow.genai.evaluate( data=traces, scorers=[ Safety(), Correctness(), ],)~1 minuteGET INVOLVEDConnect with the open source communityJoin millions of MLflow usersDocumentationRead DocsGitHub20k starsLinkedIn69k followersYouTubeView tutorialsXFollow us on XSlackJoin our Slack