Deploy to the Cloud

Prerequisites

Make sure you have Docker and Docker Daemon is running. Some of our users noted that being connected to a VPN may cause problems downloading the image.

Install Flytectl

  1. Install flytectl. Flytectl is a commandline interface for Flyte.

    brew install flyteorg/homebrew-tap/flytectl
    

    Upgrade existing installation using the following command:

    flytectl upgrade
    
    curl -sL https://ctl.flyte.org/install | sudo bash -s -- -b /usr/local/bin # You can change path from /usr/local/bin to any file system path
    export PATH=$(pwd)/bin:$PATH # Only required if user used different path then /usr/local/bin
    

    Upgrade existing installation using the following command:

    flytectl upgrade
    

    Test if Flytectl is installed correctly (your Flytectl version should be >= 0.1.34.) using the following command:

    flytectl version
    
  2. Flyte can be deployed locally using a single Docker container — we refer to this as the flyte-sandbox environment. You can also run these getting started steps in a hosted or pre-provisioned environment. Refer to Deployment section to learn how to deploy a Flyte cluster.

    Tip

    Want to dive under the hood into flyte-sandbox? Refer to Sandbox Overview.

    Here, the ‘.’ represents the current directory, assuming you have changed into myflyteapp — the git-cloned directory you created.

    flytectl sandbox start --source .
    

    Setup flytectl sandbox config

    flytectl config init
    

    NOTE if having trouble with starting the sandbox refer to Troubleshooting Guide.

Build & Deploy Your Application to the Cluster

  1. Flyte uses Docker containers to package the workflows and tasks and sends them to the remote Flyte cluster. Therefore, there is a Dockerfile already included in the cloned repo. You can build the Docker container and push the built image to a registry.

    Since flyte-sandbox runs locally in a Docker container, you do not need to push the Docker image. You can combine the build and push step by simply building the image inside the Flyte-sandbox container. This can be done using the following command:

    flytectl sandbox exec -- docker build . --tag "myapp:v1"
    

    Tip

    1. Why are we not pushing the Docker image? To understand the details, refer to Sandbox Overview

    2. Recommended: Use the bundled ./docker_build_and_tag.sh. It will automatically build the local Dockerfile, name it and tag it with the current git-SHA. This helps in achieving GitOps style workflows.

  2. Next, package the workflow using the pyflyte cli bundled with Flytekit and upload it to the Flyte backend. Note that the image is the same as the one built in the previous step.

    pyflyte --pkgs flyte.workflows package --image "myapp:v1"
    
  3. Upload this package to the Flyte backend. We refer to this as registration. The version here v1 does not have to match the version used in the commands above, but it is generally recommended to match the versions to make it easier to track.

    Note

    Note that we are simply using an existing project flytesnacks and an existing domain development to register the workflows and tasks. It is possible to create your own project and configure domains. Refer to Control Plane to understand projects and domains.

    flytectl register files --project flytesnacks --domain development --archive flyte-package.tgz --version v1
    
  4. Finally, visualize the registered workflow.

    flytectl get workflows --project flytesnacks --domain development flyte.workflows.example.my_wf --version v1 -o doturl
    

Execute on the Flyte Cluster

Use the FlyteConsole to launch an execution and keep tabs on the window!

A quick visual tour for launching a workflow and checking the outputs when they're done.

Alternatively, you can execute using the command line.

Launch and monitor from the CLI using Flytectl. More details can be found here.

  1. Generate an execution spec file.

    flytectl get launchplan --project flytesnacks --domain development flyte.workflows.example.my_wf --latest --execFile exec_spec.yaml
    
  2. Create an execution using the exec spec file.

    flytectl create execution --project flytesnacks --domain development --execFile exec_spec.yaml
    
  3. Monitor the execution by providing the execution name from the create execution command.

    flytectl get execution --project flytesnacks --domain development <execname>
    

Recap

🎉 You have successfully packaged your workflow and tasks and pushed them to a Flyte cluster. Let’s learn how to iterate.