Athena Plugin Setup

This guide gives an overview of how to set up Athena in your Flyte deployment. Athena plugin needs Flyte deployment in AWS cloud; sandbox/GCP/Azure wouldn’t work.

  1. Create a file named values-athena.yaml and add the following config to it. Please make sure that the propeller has the correct service account for Athena.

    # -- Tasks specific configuration [structure](
      # -- Plugins configuration, [structure](
        # -- [Enabled Plugins]( Enable sagemaker*, athena if you install the backend
        # plugins
          - container
          - sidecar
          - k8s-array
          - athena
          container: container
          sidecar: sidecar
          container_array: k8s-array
          athena: athena
  1. Upgrade the Flyte Helm release.

    helm upgrade -n flyte -f values-athena.yaml flyteorg/flyte
  2. Register the Athena plugin example.

    flytectl register files --archive -p flytesnacks -d development
  3. Lastly, fetch the launch plan, create and monitor the execution.

    flytectl get launchplan --project flytesnacks --domain development  athena.athena.full_hive_demo_wf  --latest --execFile exec_spec.yaml
    flytectl create execution --project flytesnacks --domain development --execFile exec_spec.yaml
    flytectl get execution --project flytesnacks --domain development <execution_id>