Athena Plugin Setup#

This guide gives an overview of how to set up Athena in your Flyte deployment. Athena plugin needs Flyte deployment in AWS cloud; sandbox/GCP/Azure wouldn’t work.

Setup the AWS Flyte cluster#

  • Make sure you have up and running flyte cluster in AWS

  • Make sure you have correct kubeconfig and selected the correct kubernetes context

  • make sure you have the correct FlyteCTL config at ~/.flyte/config.yaml

Specify Plugin Configuration#

Create a file named values-override.yaml and add the following config to it. Please make sure that the propeller has the correct service account for Athena.

    # -- Tasks specific configuration [structure](
      # -- Plugins configuration, [structure](
        # -- [Enabled Plugins]( Enable sagemaker*, athena if you install the backend
        # plugins
          - container
          - sidecar
          - k8s-array
          - athena
          container: container
          sidecar: sidecar
          container_array: k8s-array
          athena: athena

Upgrade the Flyte Helm release#

helm upgrade -n flyte -f values-override.yaml flyteorg/flyte-core

Register the Athena plugin example#

flytectl register files --archive -p flytesnacks -d development

Launch an execution#

  • Navigate to Flyte Console’s UI (e.g. sandbox) and find the workflow.

  • Click on Launch to open up the launch form.

  • Submit the form.