OpenAI Batch Agent

This guide provides an overview of how to set up the OpenAI Batch agent in your Flyte deployment.

Specify agent configuration

Edit the relevant YAML file to specify the agent.

kubectl edit configmap flyte-sandbox-config -n flyte
tasks:
  task-plugins:
    enabled-plugins:
      - container
      - sidecar
      - k8s-array
      - agent-service
    default-for-task-types:
      - container: container
      - container_array: k8s-array
      - openai-batch: agent-service
plugins:
  agent-service:
    supportedTaskTypes:
    - openai-batch

Add the OpenAI API token

  1. Install flyteagent pod using helm:

helm repo add flyteorg https://flyteorg.github.io/flyte
helm install flyteagent flyteorg/flyteagent --namespace flyte
  1. Get the base64 value of your OpenAI API token:

echo -n "<OPENAI_API_TOKEN>" | base64
  1. Edit the flyteagent secret:

    kubectl edit secret flyteagent -n flyte
    
    apiVersion: v1
    data:
      FLYTE_OPENAI_API_KEY: <BASE64_ENCODED_OPENAI_API_TOKEN>
    kind: Secret
    metadata:
      annotations:
        meta.helm.sh/release-name: flyteagent
        meta.helm.sh/release-namespace: flyte
      creationTimestamp: "2023-10-04T04:09:03Z"
      labels:
        app.kubernetes.io/managed-by: Helm
      name: flyteagent
      namespace: flyte
      resourceVersion: "753"
      uid: 5ac1e1b6-2a4c-4e26-9001-d4ba72c39e54
    type: Opaque
    

Upgrade the Flyte Helm release

helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>

Replace <RELEASE_NAME> with the name of your release (e.g., flyte-backend), <YOUR_NAMESPACE> with the name of your namespace (e.g., flyte), and <YOUR_YAML_FILE> with the name of your YAML file.

You can refer to the documentation to run the agent on your Flyte cluster.