Flyte v1.3.0

Preview Flyte 2 for production  Hosted on Union.ai6,908JoinIntroducing Flyte 2The most intuitive, developer-loved way to orchestrate AI workflows in open source. Now available for local execution.AI orchestration & runtimeDynamically orchestrate complex, long-running, and agentic workflows with autoscaling and infrastructure awareness.Author in pure PythonWrite workflows in actual Python, no need to learn a DSL. Write, test, and version workflows locally, then run them at scale.Durable by defaultBuild fault-tolerant, resilient workflows that retry automatically, pick up where they leave off, and make failures inconsequential.Choose your engineFlyte 2 OSSBuild durable AI/ML pipelines and agents with OSS.Open-sourceBuild and scale dynamic AI/ML workflows using Flyte’s open-source platform and community.Infra-aware orchestrationAuthor in pure Python to provision and scale resources for workflows.Dynamic workflow executionWorkflows can make on-the-fly decisions at runtime with real-time logic, conditions, and retries.Self-healing workflowsWorkflows can autonomously recover from failures and continue where they left off.Run locallyTest and debug tasks in your local environment using the same Python SDK that runs in production on Kubernetes.Try nowUnion.aiThe enterprise Flyte platform. Build scalable AI and agents in your cloud.Everything in Flyte 2 OSS, plus:Massive scale at 50k+ actions/runMassive scale and ultra-low latency to accelerate AI from experiment to productionOrchestrate, train, and serveOrchestrate, deploy, and optimize AI/ML systems one unified platform.Real-time inferenceServe performant agents and models with sub-second latency.Live remote debuggerDebug remote tasks, line-by-line, on the actual infrastructure where your tasks run.Reusable, warm-start containersAchieve task startup time of <100ms by eliminating cold starts.ObservabilityGet visibility into resource usage, data lineage, and versioning.White-glove supportGet dedicated help from a team of expert AI engineers.Try for freeTrusted by thousands of AI buildersFeaturesMake your AI, ML, and agentic workflows fly.Build dynamic, self-healing workflows in open source. Our infra-aware platform orchestrates data, models, & compute.Build in pure PythonAuthor dynamic, production workflows in pure Python. No DSL required.Learn moreIterate intelligentlyDevelop and debug locally before deploying to production.Learn moreRecover & reproduceBuilt-in caching and versioning ensure fast, repeatable runs.Learn moreVisualizeRender plots and visualize data with reports.Learn moreDeployPromote workflows to cloud or on-prem without infra complexities.Learn moreAdapt at runtimeBuild truly agentic workflows with stateful execution with automatic failure recovery.Learn moreScaleAutoscale compute dynamically to match workload demand.Learn moreIntegrationsExpand your workflows with powerful integrations.Apache SparkRun Spark jobs on ephemeral clusters.BigQueryQuery a BigQuery table.PyTorch Elastic v1Pytorch-native multi-node distributed training.RayConnect to Ray cluster to perform distributed model training and hyperparameter tuning.SnowflakeQuery a Snowflake service.Weights & Biases v1Best in class ML/AI experiment- and inference-time tracking.Explore IntegrationsCommunityAsk questions, share ideas, and
get advice.Explore CommunityEventsJoin us for events, workshops, and trainings.ResourcesSee the latest docs and articles.Case StudiesBlogDocsSpotify cuts quarterly forecast time in half with FlyteRead case study ↗ Wayve accelerates autonomous driving R&D with Flyte’s scalable orchestrationRead case study ↗ Cradle accelerates ML development for protein design with FlyteRead case study ↗ Warner Bros. Discovery accelerates ML workflow delivery and reduces costs with FlyteRead case study ↗ Kineo accelerates AI delivery and cuts orchestration costs with FlyteRead case study ↗ MethaneSAT uses Flyte to orchestrate global methane reduction from spaceRead case study ↗ Load moreWe Compared the Data Models of Every Major AI OrchestratorRead article ↗ Flyte 2 OSS: Now Available for Local ExecutionRead article ↗ Union.ai Completes $38.1 Million Series A to Power a New Era of AI Development InfrastructureRead article ↗ Building Crash-Proof AI SystemsRead article ↗ What Changes When Experiment Tracking Is Native to the Orchestrator?Read article ↗ Flyte vs. Ray vs. Flyte + Ray: Choosing the Right Tool for Distributed AI WorkflowsRead article ↗ Load moreIntroducing Union.aiThe enterprise Flyte platformOrchestrate, ship, and scale AI systems from experiment to production. Union.ai’s platform accelerates teams through AI orchestration, training, real-time inference, and observability.Learn moreFlyte is an open-source workflow orchestration platform created and shared by Union.aiProductFeaturesIntegrationsUnion.ai ↗ResourcesEventsCase StudiesBlogVideosCommunityDocsDocsAPI Reference ↗Latch SDK ↗Pterodactyl ↗SocialGitHub ↗Slack ↗X ↗LinkedIn ↗By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.DenyAcceptPreferencesPrivacy Preference CenterWhen you visit websites, they may store or retrieve data in your browser. This storage is often necessary for the basic functionality of the website. The storage may be used for marketing, analytics, and personalization of the site, such as storing your preferences. Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website.Reject all cookiesAllow all cookiesManage Consent Preferences by CategoryEssentialAlways ActiveThese items are required to enable basic website functionality.MarketingEssentialThese items are used to deliver advertising that is more relevant to you and your interests. They may also be used to limit the number of times you see an advertisement and measure the effectiveness of advertising campaigns. Advertising networks usually place them with the website operator’s permission.PersonalizationEssentialThese items allow the website to remember choices you make (such as your user name, language, or the region you are in) and provide enhanced, more personal features. For example, a website may provide you with local weather reports or traffic news by storing data about your current location.AnalyticsEssentialThese items help the website operator understand how its website performs, how visitors interact with the site, and whether there may be technical issues. This storage type usually doesn’t collect information that identifies a visitor.Confirm my preferences and close --- Preview Flyte 2 for production  Hosted on Union.ai6,908JoinFlyteWhere meticulous design meets production-grade scale. AI-native orchestration that makes your workflows fly.BuildIterateAnalyzeVisualizeDeployMonitorScaleMenuBuildFlyte enables you to build your data and ML workflows with robustness and scalability from the ground up. It hand-holds you throughout the creation process with best-in-class features.Strongly typed interfacesValidate your data at every step of the workflow by defining data guardrails using Flyte types. Integrate Flyte with Pandera and Great Expectations natively to perform in-depth validation and ensure that your data meets the necessary standards. Copied to clipboard! from typing import List from flytekit import task, workflow @task def mean(x: List[float]) -> float: return sum(x) / len(x) @task def variance(x: List[int], mu: float) -> float: return sum([(val - mu) ** 2 for val in x]) / len(x) @workflow def variance_wf(x: List[float] = [1.3, 2.4, 3.9]) -> float: return variance(x=x, mu=mean(x=x)) float and int types don’t match, hence compilation results in a type mismatch error. flytekit.core.type_engine.TypeTransformerFailedError: Cannot convert literal scalar { primitive { float_value: 1.3 } } to Explore Flyte types ->Any languageYou should be able to adopt the programming language of your choice to orchestrate data and ML workflows. Flyte lets you write code in any language using raw containers, or choose Python, Java, Scala or JavaScript SDKs to develop your Flyte workflows. You can use the languages you are most comfortable with, making it easier to design and manage your workflows.PythonJavaScala Copied to clipboard! import pandas as pd from flytekit import Resources, task @task(limits=Resources(cpu="2", mem="150Mi")) def total_pay(hourly_pay: float, hours_worked: int, df: pd.DataFrame) -> pd.DataFrame: return df.assign(total_pay=hourly_pay * hours_worked) Copied to clipboard! import com.google.auto.service.AutoService; import com.google.auto.value.AutoValue; import org.flyte.flytekit.SdkRunnableTask; import org.flyte.flytekit.jackson.JacksonSdkType; @AutoService(SdkRunnableTask.class) public class TotalPayTask extends SdkRunnableTask { public TotalPayTask() { super(JacksonSdkType.of(Input.class), JacksonSdkType.of(Output.class)); } @AutoValue public abstract static class Input { public abstract double hourlyPay(); public abstract long hoursWorked(); } @AutoValue public abstract static class Output { public abstract double output(); public static Output create(double output) { return new AutoValue_TotalPayTask_Output(output); } } @Override public Output run(Input input) { return Output.create(input.hourlyPay() * input.hoursWorked()); } } Copied to clipboard! import org.flyte.flytekit.SdkRunnableTask import org.flyte.flytekitscala.SdkScalaType case class TotalPayInput(hourlyPay: Double, hoursWorked: Long) case class TotalPayOutput(output: Double) class TotalPayTask extends SdkRunnableTask(SdkScalaType[TotalPayInput], SdkScalaType[TotalPayOutput]) { def run(input: TotalPayInput): TotalPayOutput = TotalPayOutput(output = input.hourlyPay + input.hoursWorked) } Language shouldn’t be a barrier for your orchestration needs →“We got over 66% reduction in orchestration code when we moved to Flyte — a huge win!”Seth Miller-Zhang, Senior Software Engineer at ZipRecruiterMap tasksParallel processing is useful for scenarios from hyperparameter optimization and complex data processing. Flyte’s map tasks enable parallel processing by dividing larger computations into independent pieces. Copied to clipboard! @task def mappable_task(string_item: str) -> dict[str, int]: return dict(Counter(string_item.split())) @workflow def wf(string_list: list[str] = ["Deer Bear", "Car Deer"]) -> dict[str, int]: mapped_out = map_task(mappable_task)(string_item=string_list).with_overrides( retries=1 ) return coalesce(string_list=mapped_out) Run a task in parallel by enclosing it in map_task construct.Map a task with multiple inputs →Dynamic workflowsDynamism in DAGs is crucial to building dynamic workflows, particularly in ML. Build dynamic DAGs in Flyte effortlessly using the @dynamic decorator. This allows you to create flexible and adaptable workflows that can change and evolve as needed, making it easier to respond to changing requirements. Copied to clipboard! @dynamic def evaluate_model( models: List[JoblibSerializedFile], X: List[List[float]], y: List[float], ) -> List[List[str]]: final_result = [] for model_ser in models: model = joblib.load(model_ser) cv = RepeatedStratifiedKFold(n_splits=7, n_repeats=2, random_state=1) scores = cross_val_score(model, X, y, scoring="accuracy", cv=cv, n_jobs=-1) final_result.append( evaluate_flyte_task(name=str(model_ser), scores=scores.tolist()) ) return final_result A dynamic workflow to compute cross-validation scores across different models.Learn how dynamic workflows are perceived by Flyte →BranchingThe conditional statement in Flyte allows you to selectively execute branches of your workflow based on static or dynamic data produced by other tasks or input data. That’s useful when you need to make decisions based on the results of previous tasks, or on external data. Copied to clipboard! @workflow def conditional_wf(): conditional("stop_if_fails").if_(validated.is_false()).then( summary_drift_report( new_data=target_df, reference_data=get_reference_data(), ) ).else_().then(make_predictions(input_data=target_df, output_path="./data")) Based on the outcome of the data validation, the conditional statement selects the appropriate task to execute.Explore more conditional examples →FlyteFile & FlyteDirectoryFlyte simplifies the process of transferring files and directories between local and cloud storage. It establishes a seamless connection and streamlines the movement of data from one to the other. Copied to clipboard! @task def normalize_columns( csv_url: FlyteFile, column_names: List[str], ) -> FlyteFile: with open(csv_url, newline="\n") as input_file: ... return FlyteFile(path=out_path, remote_path=output_location) @task def download_files(csv_urls: List[str]) -> FlyteDirectory: ... return FlyteDirectory(path=local_dir) Eliminate the need for boilerplate code when downloading and uploading files and directories with Flyte.“FlyteFile is a really nice abstraction on a distributed platform. [I can say,] ‘I need this file,’ and Flyte takes care of downloading it, uploading it and only accessing it when we need to. We generate large binary files in netcdf format, so not having to worry about transferring and copying those files has been really nice.”Nicholas LoFaso, Senior Platform Software Engineer at MethaneSATStructured datasetEffortlessly convert dataframes between types and enforce column-level type checking using the abstract 2D representation provided by Structured Dataset. Extract subsets of columns and specify the storage backend, all without the need to write boilerplate code. Copied to clipboard! @task def analyze_df( df: vaex.dataframe.DataFrameLocal, ) -> Annotated[StructuredDataset, kwtypes(age=float)]: result_df = df.groupby(by="animal").agg({"age": "mean"}) return StructuredDataset(result_df) @task def read_df(df: StructuredDataset) -> float: pandas_df = df.open(pd.DataFrame).all() return pandas_df["age"].min() @workflow def sd_wf( vaex_df: vaex.dataframe.DataFrameLocal = vaex.from_arrays( animal=["dog", "cat", "guinea pig"], age=[2, 5, 1], ) ) -> float: return read_df(df=analyze_df(df=vaex_df)) A Vaex dataframe can easily be converted to a Pandas dataframe.Empower your dataframes with the power of Structured Datasets →Wait for external inputsYou might need your execution to halt, wait for external inputs, or await manual approval. By using sleep(), wait_for_input(), and approve(), you can pause execution for a specified duration, wait for external input signals, and necessitate explicit approval before resuming execution, respectively. Copied to clipboard! @workflow def gate_nodes_wf(): # sleep sleeping = sleep(timedelta(seconds=10)) # wait for an external input title_input = wait_for_input("title", timeout=timedelta(hours=1), expected_type=str) # wait for explicit approval approve(final_report, "approve-final-report", timeout=timedelta(hours=2)) More on configuring gate nodes ->ImageSpecData scientists and ML practitioners often find Dockerfiles challenging. With Flyte’s ImageSpec, you can create images without needing Dockerfiles. You can define Python packages, apt packages and environment variables directly in the ImageSpec. Copied to clipboard! pandas_image_spec = ImageSpec( base_image="ghcr.io/flyteorg/flytekit:py3.8-1.6.2", packages=["pandas", "numpy"], python_version="3.8", apt_packages=["git"], env={"Debug": "True"}, registry="ghcr.io/flyteorg", ) @task(container_image=pandas_image_spec) def get_pandas_dataframe() -> typing.Tuple[pd.DataFrame, pd.Series]: df = pd.read_csv("https://storage.googleapis.com/download.tensorflow.org/data/heart.csv") return df[["age", "thalach", "trestbps", "chol", "oldpeak"]], df.pop("target") ImageSpec is the simplest approach to build images ->IterateMake workflow orchestration a collaborative experience. Don’t struggle to iterate on your data and ML workflows with your team. Leverage the native features to accelerate your development cycles.Recover from failuresDebugging can be time-consuming and costly — more so if it involves rerunning successful tasks. Flyte makes it easy to recover from failures by allowing you to rerun only the failed tasks in a workflow. Simply click the Recover button on the user interface, and Flyte will automatically recover the successful tasks and rerun only the failed tasks. This can save you time and resources, and make debugging more efficient.One-click recovery.Rerun a single taskDebug issues quickly: Rerun workflows at the most granular level without modifying the previous state of a data/ML workflow. Run Flyte tasks independently any number of times with the click of a button.Versioned workflowsData and ML practitioners should be able to experiment in isolation and try multiple iterations. Versioning lets them reproduce results and roll back to a previous workflow version any time. Flyte versions the entire workflow and allows switching versions just like TV channels!Launching a specific version of the workflow is as simple as selecting the required version.Learn why versioning is tied to reproducibility →“Workflow versioning is quite important: When it comes to productionizing a pipeline, there are only a few platforms that provide this kind of versioning. To us, it's critical to be able to roll back to a certain workflow version in case there is a bug introduced into our production pipeline.”Pradithya Aria Pura, Principal Software Engineer at GojekCache outputsDon’t waste resources while iterating on data/ML workflows. Cache your task outputs by passing the cache=True argument to your @task decorator to accelerate your runs. Copied to clipboard! @task(cache=True, cache_version="1.0") def train_mnist_pipeline(hp: Hyperparameters) -> train_outputs: kwargs = {"num_workers": 1, "pin_memory": True} if torch.cuda.is_available() else {} training_data_loader = mnist_dataloader(hp.batch_size, train=True, **kwargs) ... cache_version is helpful to invalidate the previous cache if task functionality changes.Understand how caching works →Intra-task checkpointingFlyte task boundaries are natural checkpoints. They can be expensive, however, in scenarios, such as training a model: Training can be time-consuming and resource-intensive. Intra-task checkpoints in Flyte help checkpoint progress within a task execution. Copied to clipboard! @task def train_model(): try: checkpoint = flytekit.current_context().checkpoint prev_checkpoint = checkpoint.read() except (NotImplementedError, ValueError): checkpoint, prev_checkpoint = None, False if prev_checkpoint: start_epoch, model, opt = torch.load(BytesIO(prev_checkpoint)) else: start_epoch = 0 model = ... opt = ... for epoch in range(start_epoch, n_epochs): ... if checkpoint: model_io = BytesIO() torch.save((epoch, model, opt), model_io) model_io.seek(0) checkpoint.write(model_io.read()) Checkpointing saves the model state and the number of epochs.More on intra task checkpointing →Multi-tenancyMulti-tenancy supports a centralized infrastructure for your team and organization, so multiple users can share the same platform while maintaining their own distinct data and configurations. Flyte is multi-tenant at its core. That’s important for managing and organizing resources effectively as well as facilitating team collaboration within your organization. Copied to clipboard! @reference_task( project="flytesnacks", domain="development", name="core.flyte_basics.files.normalize_columns", version="2NRd4zAkKIf9hpqBAMLjzw==", ) def normalize_columns( csv_url: FlyteFile, column_names: List[str], columns_to_normalize: List[str], Effortlessly reference a Flyte task (and a workflow) to foster collaboration among teams.“The multi-tenancy that Flyte provides is obviously important in regulated spaces where you need to separate users and resources and things like amongst each other within the same organization.”Jake Neyer, Software Engineer at Striveworks“We get a lot of reusable workflows, and it makes it fairly easy to share complex machine learning and different dependencies between teams without actually having to put all the dependencies into one container.”Bernhard Stadlbauer, Data Engineer at PachamaTimeoutTo ensure that a system is always making progress, tasks must end reliably. Flyte lets you define a timeout period, after which the task is marked as failure. Copied to clipboard! @task(timeout=datetime.timedelta(hours=1)) def train_mnist_pipeline(hp: Hyperparameters) -> train_outputs: kwargs = {"num_workers": 1, "pin_memory": True} if torch.cuda.is_available() else {} training_data_loader = mnist_dataloader(hp.batch_size, train=True, **kwargs) ... Task execution will be terminated if the runtime exceeds 1 hour.“Because we are a spot-based company, a lot of our workflows run into the majority of issues. Thankfully, with Flyte, we can debug and do quick iterations.”Varsha Parthasarathy, Senior Software Engineer at Woven PlanetImmutabilityImmutable executions help ensure reproducibility by preventing any changes to the state of an execution. This lets you completely restructure a data/ML workflow between versions without worrying about potential deficits in production. Flyte maintains immutability in your workflows.“Flyte has this concept of immutable transformation — it turns out the executions cannot be deleted, and so having immutable transformation is a really nice abstraction for our data-engineering stack.”Jeev Balakrishnan, Software Engineer at FreenomeAnalyzeGet visibility into your data at every step of your data/ML workflow, across versions. Avoid the uncertainty of the “data blackbox.” Data lineageData lineage can help find the cause of an error. Track the movement and transformation of data throughout the lifecycle of your data and ML workflows with Flyte.VisualizeVisualization makes data easier to comprehend. Flyte provides first-class support for rendering data plots.Data visualizationUse FlyteDecks to visualize data, monitor models and view training history through plots.Plots generated with FlyteDecks.DeployDeployment shouldn’t be a tough nut to crack. Flyte’s cost-saving, cloud-native features simplify deployment to the cloud or on-prem.Dev to prodPromoting your workflows from development to production is as simple as changing your domain from development or staging to production. Copied to clipboard! pyflyte register --project ml-finance-forecast --domain staging 2022-finance-insights.py Copied to clipboard! pyflyte register --project ml-finance-forecast --domain production 2022-finance-insights.py What is a domain? →“One thing that I really like compared to my previous experience with some of these tools: the local dev experience with pyflyte and the sandbox are super, super nice to reduce friction between production and dev environment.”Krishna Yeramsetty, Principal Data Scientist at InfinomeSpot or preemptible instancesLeveraging spot instances doesn’t have to be hard. Schedule your workflows on spot instances by setting interruptible to True in the @task decorator to cut costs. Copied to clipboard! @task(interruptible=True) def data_processing(google_trend_csv: pyspark.sql.DataFrame): google_trend_all = google_trend_csv.withColumn( "Date", F.regexp_extract(google_trend_csv.week, "(.*?) -", 1) ).withColumn( "State", F.regexp_extract(google_trend_csv.file, "Rossmann_DE_(.*)", 1) ) ... Scheduling on spot instances is as simple as setting the interruptible argument.Avail 90% savings on on-demand prices leveraging spot instances →“You can say, ‘Give me imputation’ and [Flyte will] launch 40 spot instances that are cheaper than your on-demand instance that you're using for your notebook and return the results back in memory.”Calvin Leather, Staff Engineer and Tech Lead at Embark Veterinary“Given the scale at which some of these tasks run, compute can get really expensive. So being able to add an interruptible argument to the task decorator for certain tasks has been really useful to cut costs.”Jeev Balakrishnan, Software Engineer at FreenomeCloud-native deploymentDeploy Flyte on AWS, GCP, Azure and other cloud services, and get infinite scalability.Deploy Flyte on cloud effortlessly →“We're mainly using Flyte because of its cloud native capabilities. We do everything in the cloud, and we also don't want to be limited to a single cloud provider. So having the ability to run everything through Kubernetes is amazing for us.”Maarten de Jong, Python Developer at Blackshark.aiSchedulingUse Flyte to schedule your data and machine learning workflows to run at a specific time. Copied to clipboard! cron_lp = LaunchPlan.get_or_create( name="fetch_data_lp", workflow=fetch_data_wf, schedule=CronSchedule( schedule="0 1 * * *", kickoff_time_input_arg="kickoff_time", ), ) The workflow is set to run at 1 a.m. every day.Activate a schedule →SecretsAccess secrets in your Flyte tasks, whether locally or remotely, by mounting them as files or environment variables. Copied to clipboard! @task( secret_requests=[ Secret( group=SECRET_GROUP, key=SECRET_NAME, mount_requirement=Secret.MountType.ENV_VAR, ) ] ) def secret_file_task() -> str: ... Create secrets with a secret manager ->MonitorKeep tabs on the status of your data/ML workflows with Flyte. Identify potential bottlenecks to debug issues quickly.NotificationsStay informed about changes to your workflow's state by configuring notifications through Slack, PagerDuty or email.Configure notifications on Flyte →Timeline viewUse the timeline view to evaluate the duration of each of your Flyte tasks and identify potential bottlenecks.ScaleFlyte is built for high performance. Effortlessly scale your data and ML workflows leveraging the infrastructure-as-code approach.GPU accelerationGPUs are becoming essential to run data and ML workflows. Enable and control your tasks’ GPU demands by requesting resources in the @task decorator — just like that! Copied to clipboard! @task(requests=Resources(gpu="2"), limits=Resources(gpu="4")) def train_mnist_pipeline(hp: Hyperparameters) -> train_outputs: kwargs = {"num_workers": 1, "pin_memory": True} if torch.cuda.is_available() else {} training_data_loader = mnist_dataloader(hp.batch_size, train=True, **kwargs) ... Request GPUs and control the usage by setting limits.Configure Flyte to access GPUs →Dependency isolation via containersDifferent tasks may have different resource requirements and library dependencies. That can cause conflicts if they aren’t managed carefully. With Flyte, you can maintain separate sets of dependencies for your tasks so no dependency conflicts arise. Copied to clipboard! @task(container_image="ghcr.io/flyteorg/flytecookbook:core-latest") def get_data() -> Tuple[np.ndarray, np.ndarray]: from sklearn import datasets iris = datasets.load_iris() X, y = (iris.data[:, :2], iris.target) return (X, y) @task(container_image="ghcr.io/flyteorg/flytekit:py3.9-1.2.7") def normalize(X: np.ndarray) -> np.ndarray: return (X - X.mean(axis=0)) / X.std(axis=0) @workflow def wf() -> np.ndarray: X, _ = get_data() X = normalize(X=X) return X No more dependency hell!Learn how to configure images dynamically →ParallelismFlyte tasks are inherently parallel to optimize resource consumption and improve performance, so you don't have to do anything special to enable parallelism.“When you write Python scripts, everything runs and takes a certain amount of time, whereas now for free we get parallelism across tasks. Our data scientists think that's really cool.”Dylan Wilder, Engineering Manager at SpotifyAllocate resources dynamicallySometimes, the resources needed for a task may depend on user-provided inputs or real-time resource calculations. Flyte makes it easy to allocate resources using the with_overrides method and dynamic workflows. This allows you to adjust resources on the fly, ensuring that your tasks have the resources they need to run efficiently. Copied to clipboard! @task def mean(x: List[float]) -> float: return sum(x) / len(x) @dynamic def dynamic_wf(x: List[float], mem: str) -> float: return mean(x=x).with_overrides(requests=Resources(mem=mem)) @workflow def mean_wf(x: List[float] = [1.3, 2.4, 3.9], mem: str = "600Mi") -> float: return dynamic_wf(x=x, mem=mem) Override the default resources of a task from within a Flyte workflow in mere seconds.More on customizing task resources →Board Flyte to leverage the best-in-class features to build data, ML & analytics workflowsInstall locally ↗ Try hosted ↗ Introducing Union.aiThe enterprise Flyte platformOrchestrate, ship, and scale AI systems from experiment to production. Union.ai’s platform accelerates teams through AI orchestration, training, real-time inference, and observability.Learn moreFlyte is an open-source workflow orchestration platform created and shared by Union.aiProductFeaturesIntegrationsUnion.ai ↗ResourcesEventsCase StudiesBlogVideosCommunityDocsDocsAPI Reference ↗Latch SDK ↗Pterodactyl ↗SocialGitHub ↗Slack ↗X ↗LinkedIn ↗By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.DenyAcceptPreferencesPrivacy Preference CenterWhen you visit websites, they may store or retrieve data in your browser. This storage is often necessary for the basic functionality of the website. The storage may be used for marketing, analytics, and personalization of the site, such as storing your preferences. Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website.Reject all cookiesAllow all cookiesManage Consent Preferences by CategoryEssentialAlways ActiveThese items are required to enable basic website functionality.MarketingEssentialThese items are used to deliver advertising that is more relevant to you and your interests. They may also be used to limit the number of times you see an advertisement and measure the effectiveness of advertising campaigns. Advertising networks usually place them with the website operator’s permission.PersonalizationEssentialThese items allow the website to remember choices you make (such as your user name, language, or the region you are in) and provide enhanced, more personal features. For example, a website may provide you with local weather reports or traffic news by storing data about your current location.AnalyticsEssentialThese items help the website operator understand how its website performs, how visitors interact with the site, and whether there may be technical issues. This storage type usually doesn’t collect information that identifies a visitor.Confirm my preferences and close --- Preview Flyte 2 for production  Hosted on Union.ai6,908JoinMarch 4, 2026•#Min ReadFlyte 2 Is Here: Open-Source AI Orchestration Is Now Available LocallyUnion TeamTable of ContentsTable of ContentsFlyte 2 is dynamic, self-healing AI orchestration. It is available today in open source for local execution, and for production preview hosted on Union.ai.What is Flyte 2 and why is it the standard for AI orchestration?Flyte 2 is fully dynamic, self-healing AI orchestration. As an agent harness and execution runtime, it uses infrastructure-as-context to fix failures autonomously as workflows run.Flyte 2’s self-healing workflows autonomously fix failures at runtimeWorkflow authors are feeling the pain of maintaining brittle pipelines and juggling fragmented devtools – most of which are designed for data pipelines, not AI. With the growing complexity of agents and AI, workflows need to be extremely durable, scalable, and adaptable at runtime. Since we first announced the Flyte 2 beta last year, our open source community has come alive with excitement and feedback.And now, we’re thrilled to announce that Flyte 2 is available locally in open source. We’re also making Flyte 2 for distributed execution available as part of our production preview, hosted on Union.ai.Early feedback from the Flyte 2 betaLet’s start with a massive thank you to all the contributors and beta users who have helped shape the future of open source AI orchestration. Since launching the beta, we’ve seen…60 new releases508 merged PRs35 unique contributorsThose efforts have helped us get to this major milestone, and we’re incredibly grateful to everyone who contributed. Our most recent feedback shows how powerful Flyte 2 truly is:Ranjit Parva, Mistral AIHow Flyte 2 improves the way you orchestrate AIPure Python authoringFlyte 2 lets you author workflows in actual Python without learning a new DSL. You can write, test, and version your workflows locally in pure Python. Migrating existing scripts is simple, and there's no need to rewrite logic just to fit a framework.

TUI with simplified DevEx (tasks simply call tasks)

Self-healing workflowsMost “durable execution” providers can’t handle infrastructure-caused failures. That means out of memory errors, container pre-emption, etc. can’t be automatically addressed. Flyte 2 is an agent harness, so pipelines that fail can recover and continue where they left off. It uses infrastructure-as-context to handle both logic and infrastructure failures autonomously. The workflows and agents on Flyte 2 are self-healing. With built-in caching, retries, and support for custom error handling, Flyte ensures that failures are isolated and never bring down your entire workflow.Self-healing workflows automatically heal from logic and infra failuresIntuitive DebuggingFor anything beyond automation, Flyte 2 makes debugging fast and intuitive. You can observe execution state, logs, and failures at every step, catch errors as they happen, and even rerun workflows interactively in a live debugger.Dynamically rerun with more memory after OOMDynamic, on-the-fly orchestrationFlyte 2 delivers fully dynamic workflows that adapt in real time. From branching logic and loops to dynamic resource allocation, your AI systems and agents can make decisions on the fly at runtime. *You still have the option of static DAGs, too.Plan and execute agentic workflows at runtimeOptimized, scalable workflowsFlyte 2 handles large task fanout and parallelism with ease, scaling with you as you grow your AI systems. Use native multi-container pipelines. Auto-scaling (including scale-to-zero) means you only pay for the compute infra you need.Seamlessly scale out to thousands of containersRapid iterationFlyte 2 automatically versions your executions, supports multi-tenancy in dev/stage/prod environments, and provisions infrastructure only when needed, keeping costs and complexity low.Flyte provides native multi-tenancy through projects and domainsWhat can Flyte 2 do locally today?The Flyte 2 SDK is available today locally for one machine. Running Flyte 2 locally offers a bunch of cool features that illustrate why Flyte’s architecture is changing the game for AI orchestration.What Flyte 2 can do locallyDurable agent workflows — test and run AI orchestration locally without a clusterTerminal UI — launch, inspect, and manage workflows straight from your laptopAsync I/O — non-blocking, concurrent task execution out of the boxLocal caching — reuse prior results to speed up iteration and rerunsLocal serving — serve apps that call workflows and execute agentic logic locally‍Observability — real-time reports and execution state at every step

Demo Flyte 2 in browser

‍Flyte 2 for productionReleasing Flyte 2 locally is a major stepping stone in building open infrastructure for AI development. Our next milestone is releasing a powerful Flyte 2 backend for distributed execution. This backend will enable scalable, fault-tolerant workflow runs, bringing autoscaling, resource awareness, and resilient state management to production environments.In the meantime, we’re giving teams the chance to access Flyte 2 for production as part of our production preview, hosted on Union.ai, the enterprise Flyte platform.Get started

Try Flyte 2

 Try Flyte 2 for distributed execution as part of our production preview, hosted on Union.ai Available today for local execution.Other helpful linksTry our in-browser demo and see Flyte 2 in actionCheck out our Github pageJoin our Slack communityIntroducing Union.aiThe enterprise Flyte platformOrchestrate, ship, and scale AI systems from experiment to production. Union.ai’s platform accelerates teams through AI orchestration, training, real-time inference, and observability.Learn moreFlyte is an open-source workflow orchestration platform created and shared by Union.aiProductFeaturesIntegrationsUnion.ai ↗ResourcesEventsCase StudiesBlogVideosCommunityDocsDocsAPI Reference ↗Latch SDK ↗Pterodactyl ↗SocialGitHub ↗Slack ↗X ↗LinkedIn ↗By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.DenyAcceptPreferencesPrivacy Preference CenterWhen you visit websites, they may store or retrieve data in your browser. This storage is often necessary for the basic functionality of the website. The storage may be used for marketing, analytics, and personalization of the site, such as storing your preferences. Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website.Reject all cookiesAllow all cookiesManage Consent Preferences by CategoryEssentialAlways ActiveThese items are required to enable basic website functionality.MarketingEssentialThese items are used to deliver advertising that is more relevant to you and your interests. They may also be used to limit the number of times you see an advertisement and measure the effectiveness of advertising campaigns. Advertising networks usually place them with the website operator’s permission.PersonalizationEssentialThese items allow the website to remember choices you make (such as your user name, language, or the region you are in) and provide enhanced, more personal features. For example, a website may provide you with local weather reports or traffic news by storing data about your current location.AnalyticsEssentialThese items help the website operator understand how its website performs, how visitors interact with the site, and whether there may be technical issues. This storage type usually doesn’t collect information that identifies a visitor.Confirm my preferences and close --- Preview Flyte 2 for production  Hosted on Union.ai6,908JoinIntegrationsExpand the capabilities of your Flyte workflows with a range of powerful integrations.DataFramesData ValidationDatabases & Data WarehousesData ProcessingFeature StoresModel TrainingDistributed Model TrainingModel DeploymentMonitoringKubernetes-NativeMiscellaneousMenuDataFramesCrunch data however you want. Automatically transition data from one dataframe type to the other using Structured Dataset.VaexVisualize and explore big tabular datasets.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-vaexPolarsUse Polars dataframe as a native Flyte type.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-polarsModinScale your Pandas workflows.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-modinHuggingFaceUse HuggingFace dataset as a native Flyte type.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-huggingfaceData ValidationValidate data at every step of your Flyte workflow.PanderaValidate dataframe-like objects.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-panderaGreat ExpectationsValidate your data with expectations.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-great-expectationsDatabases & Data WarehousesManage and connect to databases and warehouses seamlessly.SQLAlchemyExecute SQL queries as Flyte tasks.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-sqlalchemySnowflakeQuery a Snowflake service.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-snowflakeDuckDBRun intricate analytical queries with DuckDBDocumentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-duckdbHiveQuery a Hive service.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-hiveDoltHubApply git-like versioning to your SQL databases.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-doltBigQueryQuery a BigQuery table.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-bigqueryData ProcessingProcess and analyze your data with data-crunchers.Apache SparkRun Spark jobs on ephemeral clusters.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-sparkDBTTransform data in your warehouses with DBT.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-dbtDatabricksSchedule, monitor and orchestrate Databricks jobs.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-sparkAWS AthenaQuery an AWS Athena service.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-athenaFeature StoresStore, share and manage features for ML models.FeastManage and serve ML features with Feast.Documentation ↗View on GitHub ↗Backend Setup ↗Model TrainingSimplify the model training process.PyTorch Elastic v1Pytorch-native multi-node distributed training.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-kfpytorchAWS SagemakerTrain ML models on Sagemaker from within Flyte.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-awssagemakerDistributed Model TrainingPerform distributed model training to speed up the model development process.RayConnect to Ray cluster to perform distributed model training and hyperparameter tuning.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-rayKubeflow TensorFlowRun distributed TensorFlow training jobs.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-kftenorflowKubeflow MPIRun distributed training with an MPI operator.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-kfmpiKubeflow PyTorchRun distributed PyTorch training jobs.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-kfpytorchHorovodRun distributed deep learning workflows.Documentation ↗View on GitHub ↗Backend Setup ↗DaskRun Dask jobs natively on a Kubernetes cluster.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-daskModel DeploymentStreamline the model deployment process.ONNX TensorFlowGenerate ONNX models from TensorFlow models.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-onnxtensorflowONNX PyTorchGenerate ONNX models from PyTorch models.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-onnxpytorchONNX Scikit LearnGenerate ONNX models from Scikit Learn models.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-onnxscikitlearnMonitoringMonitor data and models from within Flyte.Weights & Biases v1Best in class ML/AI experiment- and inference-time tracking.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-wandbWhylogsLog any kind of data and generate summaries of datasets.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-whylogsMLFlowTrack your machine learning metrics with MLFlowDocumentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-mlflowKubernetes-NativeExercise greater control over Kubernetes resources.Kubernetes PodsConfigure pods for arbitrary workloads.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-podMiscellaneousPapermillExecute Jupyter notebooks.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-papermillAWS BatchRun batch tasks on AWS.Documentation ↗View on GitHub ↗Backend Setup ↗pip install flytekitplugins-awsbatchBuild your own integrationCreate your own integration and submit it to the Flyte repository.Start building ↗Join our community to discuss and share ideas for integrationsIntroducing Union.aiThe enterprise Flyte platformOrchestrate, ship, and scale AI systems from experiment to production. Union.ai’s platform accelerates teams through AI orchestration, training, real-time inference, and observability.Learn moreFlyte is an open-source workflow orchestration platform created and shared by Union.aiProductFeaturesIntegrationsUnion.ai ↗ResourcesEventsCase StudiesBlogVideosCommunityDocsDocsAPI Reference ↗Latch SDK ↗Pterodactyl ↗SocialGitHub ↗Slack ↗X ↗LinkedIn ↗By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.DenyAcceptPreferencesPrivacy Preference CenterWhen you visit websites, they may store or retrieve data in your browser. This storage is often necessary for the basic functionality of the website. The storage may be used for marketing, analytics, and personalization of the site, such as storing your preferences. Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website.Reject all cookiesAllow all cookiesManage Consent Preferences by CategoryEssentialAlways ActiveThese items are required to enable basic website functionality.MarketingEssentialThese items are used to deliver advertising that is more relevant to you and your interests. They may also be used to limit the number of times you see an advertisement and measure the effectiveness of advertising campaigns. Advertising networks usually place them with the website operator’s permission.PersonalizationEssentialThese items allow the website to remember choices you make (such as your user name, language, or the region you are in) and provide enhanced, more personal features. For example, a website may provide you with local weather reports or traffic news by storing data about your current location.AnalyticsEssentialThese items help the website operator understand how its website performs, how visitors interact with the site, and whether there may be technical issues. This storage type usually doesn’t collect information that identifies a visitor.Confirm my preferences and close