User Guide

If this is your first time using Flyte, check out the Getting Started guide.

The User Guide and Tutorials sections of the Flyte documentation covers all of the key features of Flyte organized by topic. Each of the sections below introduces a topic and discusses how you can use Flyte to address a specific problem. Code for all of the examples in the user guide be found in the flytesnacks repo.

Flytesnacks comes with a highly customized environment to make running, documenting and contributing samples easy. If this is your first time running these examples, follow the setup guide below to get started.

Setting up your environment to run the examples

Prerequisites

• Make sure you have docker and git installed.

• Install flytectl. flytectl is a commandline interface for flyte.

OSX

brew install flyteorg/homebrew-tap/flytectl

To upgrade, run:

brew upgrade flytectl

Most other platforms

curl -s https://raw.githubusercontent.com/lyft/flytectl/master/install.sh | bash

Steps

1. Install the python Flytekit SDK and clone the flytesnacks repo:

   Tip

   It’s recommended to create a new python virtual environment to make sure it doesn’t interfere with your development environment.

   git clone --depth 1 git@github.com:flyteorg/flytesnacks.git flytesnacks
   cd flytesnacks
   pip install -r cookbook/core/requirements.txt
   

2. Run hello_world.py locally

   python cookbook/core/flyte_basics/hello_world.py
   

   Expected Output

   Running my_wf() hello world
   

   🎉 Congratulations 🎉

   You have just run your first workflow. Now, let’s run it on the sandbox cluster deployed earlier.

3. We’ve packaged all the required components to run a sandboxed flyte cluster into a single docker image. You can start one by running:

   flytectl sandbox start --source ${PWD}
   

   Tip

   In case make start throws any error please refer to the troubleshooting guide here Troubleshoot

   Check status:

   flytectl sandbox status
   

   Teardown:

   flytectl sandbox teardown
   

4. Take a minute to explore Flyte Console through the provided URL.

   

   A quick visual tour for launching your first Workflow.

5. Register all examples from cookbook/core into the cluster. This step compiles your python code into the intermediate flyteIdl language and store them on the control plane running inside the cluster.

   REGISTRY=cr.flyte.org/flyteorg make fast_register
   

   Note

   If the images are to be re-built, run make register command.

6. Let’s launch our first execution from the UI. Visit the console, click launch.

7. Give it a minute and once it’s done, check out “Inputs/Outputs” on the top right corner to see your greeting.

   

   A quick visual tour for launching a workflow and checking the outputs when they’re done.

Recap

You have successfully:

1. Run a flyte workflow locally,

2. Run a flyte sandbox cluster,

3. Run a flyte workflow on a cluster.

🎉 Congratulations, now you can interactively explore Flyte’s features outlined in the Table of Contents 🎉

Table of Contents

🔤 Flyte Basics

Learn about tasks, workflows, launch plans, caching, and working with files and directories.

🚰 Control Flow

Implement conditionals, nested and dynamic workflows, map tasks, and even recursion!

⌨️ Type System

Improve pipeline robustness with Flyte’s portable and extensible type system.

⏱ Scheduled Workflows

Learn about scheduled workflows.

⚗️ Testing

Test tasks and workflows with Flyte’s testing utilities.

📦 Containerization

Easily manage the complexity of configuring the containers that run Flyte tasks.

🚢 Configuring Production Features

Ship and configure your machine learning pipelines on a production Flyte installation.

🎮 Remote Access

Register, inspect, and monitor tasks and workflows on a Flyte backend.

🔌 Integrations

Leverage a rich ecosystem of plugins from compute infrastructure to jupyter notebooks.

🏗 Extending Flyte

Define custom plugins that aren’t currently supported in the Flyte ecosystem.