Getting Started: Iterate

A step-by-step guide to building, deploying, and iterating on Flyte tasks and workflows

Iterate on Your Ideas

Modify Code and Test Locally

  1. Open example.py in your favorite editor.

    myapp/workflows/example.py
    
  2. Add name: str as an argument to both my_wf and say_hello functions. Then update the body of say_hello to consume that argument.

    @task
    def say_hello(name: str) -> str:
        return f"hello world, {name}"
    
    @workflow
    def my_wf(name: str) -> str:
        res = say_hello(name=name)
        return res
    
  3. Update the simple test at the bottom of the file to pass in a name, e.g.

    print(f"Running my_wf(name='adam') {my_wf(name='adam')}")
    
  4. When you run this file locally, it should output hello world, adam.

    python myapp/workflows/example.py
    
  5. Flytekit also provides ways to programmatically tests your tasks and workflows and provides easy mocking hooks to mock out certain functions.

[Bonus] Build & Deploy Your Application “Fast”er!

  1. To deploy this workflow to the Flyte cluster (sandbox), you can repeat the steps previously covered in Prerequisites. Flyte provides a faster way to iterate on your workflows. Since you have not updated any of the dependencies in your requirements file, it is possible to push just the code to Flyte backend without re-building the entire Docker container. To do so, run the following commands.

    pyflyte --pkgs myapp.workflows package --image myapp:v1 --fast --force
    

    Note

    --fast flag will take the code from your local machine and provide it for execution without having to build the container and push it. The --force flag allows overriding your previously created package.

    Caution

    The fast registration method can only be used if you do not modify any requirements (that is, you re-use an existing environment). But, if you add a dependency to your requirements file or env you have to follow the Prerequisites method.

  2. The code can now be deployed using Flytectl, similar to what we’ve done previously. flytectl automatically understands that the package is for fast registration. For this to work, a new storage block has to be added to the Flytectl configuration with appropriate permissions at runtime. The storage block configures Flytectl to write to a specific S3 / GCS bucket. If you’re using the sandbox, this is automatically configured by Flytectl, so you can skip this for now. But do take a note for the future.

    flytectl register files --project flytesnacks --domain development --archive flyte-package.tgz  --version v1-fast1
    
  3. Finally, visit the sandbox console, click launch, and give your name as the input.

    A quick visual tour for launching a workflow and checking the outputs when they're done.

    Alternatively, use flytectl. To pass arguments to the workflow, update the execution spec file (previously generated).

    1. Generate an execution spec file. This will prompt you to overwrite and answer ‘y’ on it.

      flytectl get launchplan --project flytesnacks --domain development myapp.workflows.example.my_wf --latest --execFile exec_spec.yaml
      
    2. Modify the execution spec file and update the input params and save the file. Notice that the version would be changed to your latest one.

      ....
      inputs:
       name: "adam"
      ....
      version: v1-fast1
      
    3. Create an execution using the exec spec file.

      flytectl create execution --project flytesnacks --domain development --execFile exec_spec.yaml
      
    4. Monitor the execution by providing the execution name from the create execution command.

      flytectl get execution --project flytesnacks --domain development <execname>
      

Recap

🎉 Congratulations! You have now:

  1. Run a Flyte workflow locally,

  2. Started a Flyte sanbox cluster,

  3. Run a Flyte workflow on a cluster,

  4. Iterated on a Flyte workflow.

Next Steps: User Guide

To experience the full capabilities of Flyte, take a look at the User Guide.