Flyte
Orchestration

Flyte

Released: 
Jan 2020
  •  
Documentation
  •  
License:  
Apache-2.0 License
355
Github open issues
1787
Github stars
15 Nov
Github last commit
12
Stackoverflow questions

What is Flyte?

Flyte is an open-source, container-native, modular, structured programming and distributed processing workflow automation platform for large-scale ML processes. It helps create scalable, maintainable, and highly concurrent workflows for data processing and machine learning.

Flyte was developed at Lyft in collaboration with Spotify, Freenome, and many others.  It is built directly on Kubernetes and offers all containerization benefits like scalability, portability, and reliability. 


How does Flyte help?

Executing large-scale compute jobs is not just critical but also problematic from an operational standpoint. Scaling, monitoring, and managing compute clusters becomes very challenging for teams with complex data dependencies and iterations involved.

Flyte’s prime focus is to accelerate the development velocity for ML and data processing workloads. It enables large-scale compute execution without any operational overhead and helps automate complex, mission-critical ML and data processes. It helps teams to focus on the business goals instead of working on infrastructure concerns. Flyte supports: 

  • Multi-tenant service, versioned code, and reproducible executions
  • Cross-cloud portable pipelines
  • Enhanced team collaboration with simplified complexities of multi-owner and multi-step workflows
  • Ergonomic SDK’s in Java, Python, and Scala
  • FlyKit extensions and Backend plugins to extend tasks


Key Features of Flyte

Simplified workflows

Flyte eliminates the need for YAML-based ML and data workflow configurations reducing users’ infrastructure worries. It offers user-friendly and intuitive SDKs to build and run workflows instantly.

Built for resiliency

Flyte being a Kubernetes-native ML orchestration platform, helps unify data and ML processes. The containerized and microservices-driven architecture is resilient by design.

Automated lineage tracking and caching

Flyte tool facilitates automated lineage tracking and caching with a deep understanding of data lineage and provenance. It allows caching repeated action across multiple runs, tracking the generated data, and visualizing dependencies.

Controlled extensibility

Flyte supports built-in platform extensions like Distributed TensorFlow, AWS Sagemaker, Kubernetes Pods, AWS Batch, along with Python and other user-defined extensions.

Self-service data platform

Flyte empowers teams to self-service their ML and data pipelines with a centralized infrastructure platform and security governance.

Companies using

Flyte

gojek
latchbio
lyft
No items found.

Liked the content? You'll love our emails!

The best MLOps and AI Observability content handpicked and delivered to your email twice a month

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Censius automates model monitoring

so that you can 

boost healthcare

improve models

scale businesses

detect frauds

boost healthcare

improve models

scale businesses

detect frauds

boost healthcare

Start Monitoring