Iterate and Re-deploy#

In this guide, you’ll learn how to iterate on and re-deploy your tasks and workflows.

Modify Code and Test Locally#

Open example.py in your favorite editor.

Add message: str as an argument to both my_wf and say_hello functions. Next, update the body of say_hello to consume that argument.

@task
def say_hello(message: str) -> str:
   return f"hello world, {message}"
@workflow
def my_wf(message: str) -> str:
   res = say_hello(message=message)
   return res

Update the simple test at the bottom of the file to pass in a message, e.g.

print(f"Running my_wf(message='what a nice day it is!') {my_wf(message='what a nice day it is!')}")

When you run this file locally, it should output hello world, what a nice day it is!.

python flyte/workflows/example.py

Quickly Re-deploy Your Application#

To re-deploy this workflow to the sandbox Flyte cluster, you can repeat the steps covered in previous section. Flyte provides a faster way to iterate on your workflows. Since you have not updated any of the dependencies in your requirements file, it is possible to push just the code to Flyte backend without re-building the entire Docker container. To do so, run the following commands.

pyflyte --pkgs flyte.workflows package --image my_flyte_project:v1 --fast --force

Note

--fast flag will take the code from your local machine and provide it for execution without having to build the container and push it. The --force flag allows overriding your previously created package.

Caution

The fast registration method can only be used if you do not modify any requirements (that is, you re-use an existing environment). But, if you add a dependency to your requirements file or env you have to follow the Deploy to the Cloud method.

The code can now be deployed using flytectl, similar to what we’ve done previously. flytectl automatically recognizes that the package is for fast registration. The admin provides a signed URL link to the configured bucket (S3/GCS bucket) for the client to upload the package.

Tip

If you’re using the sandbox, this is automatically configured by flytectl.

The dropdown below provides more information on the required configuration depending on your cloud infrastructure.

Next, you can simply register your packaged workflows with:

flytectl register files --project flytesnacks --domain development --archive flyte-package.tgz  --version v1-fast1

Execute Your Re-deployed Workflow#

Finally, you can execute the updated workflow programmatically with flytectl.

To pass arguments to the workflow, update the execution spec file that we generated in the previous step.

Generate an execution spec file. This will prompt you to overwrite and answer ‘y’ on it.

flytectl get launchplan --project flytesnacks --domain development flyte.workflows.example.my_wf --latest --execFile exec_spec.yaml

Modify the execution spec file and update the input params and save the file. Notice that the version would be changed to your latest one.

....
inputs:
   message: "what's up doc? 🐰🥕"
....
version: v1-fast1

Create an execution using the exec spec file.

flytectl create execution --project flytesnacks --domain development --execFile exec_spec.yaml

You should see an output like:

execution identifier project:"flytesnacks" domain:"development" name:"<execution_name>"

Monitor the execution by providing the execution name from the create execution command.

flytectl get execution --project flytesnacks --domain development <execution_name>

Tip

Alternatively, visit the Flyte sandbox console at http://localhost:30081/console/projects/flytesnacks/domains/development/workflows/flyte.workflows.example.my_wf, click launch, and provide a message as input.

Recap#

In this guide, we:

  1. Setup a Flyte project with pyflyte init my_flyte_project and ran your workflows locally.

  2. Started a Flyte sandbox cluster and ran a Flyte workflow on a cluster.

  3. Iterated on a Flyte workflow and updated the workflows on the cluster.

What’s Next?#

To experience the full power of Flyte on distributed compute, take a look at the Deployment Guides.

Total running time of the script: ( 0 minutes 0.000 seconds)

Gallery generated by Sphinx-Gallery