OpenAI Batch Agent

Tags: Integration, Intermediate, OpenAI

The Batch API agent allows you to submit requests for asynchronous batch processing on OpenAI. You can provide either a JSONL file or a JSON iterator, and the agent handles the upload to OpenAI, creation of the batch, and downloading of the output and error files.

Installation

To use the OpenAI Batch agent, run the following command:

pip install flytekitplugins-openai

Example usage

For a usage example, see OpenAI Batch agent example usage.

Local testing

To test an agent locally, create a class for the agent task that inherits from SyncAgentExecutorMixin or AsyncAgentExecutorMixin. These mixins can handle synchronous and synchronous tasks, respectively, and allow flytekit to mimic FlytePropeller’s behavior in calling the agent. For more information, see “Testing agents locally”.

Flyte deployment configuration

Note

If you are using a managed deployment of Flyte, you will need to contact your deployment administrator to configure agents in your deployment.

To enable the OpenAI Batch agent in your Flyte deployment, refer to the OpenAI Batch agent setup guide.