Athena Plugin#
This guide provides an overview of setting up Athena in your Flyte deployment.
Note
Please note that the Athena plugin requires a Flyte deployment in the AWS cloud; it won’t work with demo/GCP/Azure.
Set up the AWS Flyte cluster#
- Ensure you have a functional Flyte cluster up and running in AWS
- Verify that you possess the correct
kubeconfig
and have selected the appropriate Kubernetes context - Double-check that your
~/.flyte/config.yaml
file contains the correct Flytectl configuration
Specify plugin configuration#
Edit the relevant YAML file to specify the plugin.
tasks:
task-plugins:
enabled-plugins:
- container
- sidecar
- k8s-array
- athena
default-for-task-types:
- container: container
- container_array: k8s-array
- athena: athena
Create a file named values-override.yaml
and include the following configuration:
configmap:
enabled_plugins:
tasks:
task-plugins:
enabled-plugins:
- container
- sidecar
- k8s-array
- athena
default-for-task-types:
container: container
sidecar: sidecar
container_array: k8s-array
athena: athena
Ensure that the propeller has the correct service account for Athena.
Upgrade the Flyte Helm release#
helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
Replace <RELEASE_NAME>
with the name of your release (e.g., flyte-backend
),
<YOUR_NAMESPACE>
with the name of your namespace (e.g., flyte
),
and <YOUR_YAML_FILE>
with the name of your YAML file.
helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
Replace <RELEASE_NAME>
with the name of your release (e.g., flyte
)
and <YOUR_NAMESPACE>
with the name of your namespace (e.g., flyte
).