Single Cluster Simple Cloud Deployment#
These instructions are suitable for the main cloud providers.
Prerequisites#
In order to install Flyte, you will need access to the following:
At least one IAM role on AWS, GCP, etc. This is the role for the Flyte backend service to assume. You can provision another role for user code to assume as well.
As Flyte documentation cannot keep up with the pace of change of the cloud provider APIs, please refer to their official documentation for each of these prerequisites.
Note
Union.AI plans to open-source a reference implementation of these requirements for the major cloud providers in early 2023.
Installation#
Flyte is installed via a Helm chart. First, add the Flyte chart repo to Helm:
helm repo add flyteorg https://flyteorg.github.io/flyte
Then download and update the values files:
curl -sL https://raw.githubusercontent.com/flyteorg/flyte/master/charts/flyte-binary/eks-starter.yaml
Finally, install the chart:
helm install flyte-backend flyteorg/flyte-binary \
--dry-run --namespace flyte --values eks-starter.yaml
When ready to install, remove the --dry-run
switch.
Verify the Installation#
The values supplied by the eks-starter.yaml
file provides only the simplest
installation of Flyte. The core functionality and scalability of Flyte will be
there, but no plugins are included (e.g. Spark tasks will not work), there is no
DNS or SSL, and there is no authentication.
Port Forward Flyte Service#
To verify the installation therefore you’ll need to port forward the Kubernetes service.
kubectl -n flyte port-forward service/flyte-binary 8088:8088 8089:8089
You should be able to navigate to http://localhost:8088/console.
The Flyte server operates on two different ports, one for HTTP traffic and one for gRPC traffic, which is why we port forward both.
From here, you should be able to run through the Getting Started
examples again. Save a backup copy of your existing configuration if you have one
and generate a new config with flytectl
.
mv ~/.flyte/config.yaml ~/.flyte/bak.config.yaml
flytectl config init --host localhost:8088
This will produce a file like:
~/.flyte/config.yaml
#admin:
# For GRPC endpoints you might want to use dns:///flyte.myexample.com
endpoint: dns:///localhost:8088
authType: Pkce
insecure: true
logger:
show-source: true
level: 0
Test Workflow#
You can test a workflow by cloning the flytesnacks
repo and running the
hello world example:
git clone https://github.com/flyteorg/flytesnacks
cd flytesnacks/cookbook
pyflyte run --remote core/flyte_basics/hello_world.py my_wf
What’s Next?#
Congratulations ⭐️! Now that you have a Flyte cluster up and running on the cloud, you can productionize it by following the Single Cluster Production-grade Cloud Deployment guide.