OpenAI Batch Agent#

This guide provides an overview of how to set up the OpenAI Batch agent in your Flyte deployment.

Specify agent configuration#

Edit the relevant YAML file to specify the agent.

kubectl edit configmap flyte-sandbox-config -n flyte

tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - k8s-array
      - agent-service
    default-for-task-types:
      - container: container
      - container_array: k8s-array
      - openai-batch: agent-service

Create a file named values-override.yaml and add the following configuration to it:

configmap:
  enabled_plugins:
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - k8s-array
          - agent-service
        default-for-task-types:
          container: container
          sidecar: sidecar
          container_array: k8s-array
          openai-batch: agent-service

Add the OpenAI API token#

Install flyteagent pod using helm:

helm repo add flyteorg https://flyteorg.github.io/flyte
helm install flyteagent flyteorg/flyteagent --namespace flyte

Set Your OpenAI API Token as a Secret (Base64 Encoded):

SECRET_VALUE=$(echo -n "<OPENAI_API_TOKEN>" | base64) && \
kubectl patch secret flyteagent -n flyte --patch "{\"data\":{\"flyte_openai_api_key\":\"$SECRET_VALUE\"}}"

Restart development:

kubectl rollout restart deployment flyteagent -n flyte

Upgrade the Flyte Helm release#

helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>

Replace <RELEASE_NAME> with the name of your release (e.g., flyte-backend), <YOUR_NAMESPACE> with the name of your namespace (e.g., flyte), and <YOUR_YAML_FILE> with the name of your YAML file.

helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml

Replace <RELEASE_NAME> with the name of your release (e.g., flyte) and <YOUR_NAMESPACE> with the name of your namespace (e.g., flyte).

You can refer to the documentation to run the agent on your Flyte cluster.