Flyte agents

Flyte agents are long-running, stateless services that receive execution requests via gRPC and initiate jobs with appropriate external or internal services. They enable two key workflows: asynchronously launching jobs on hosted platforms (e.g. Databricks or Snowflake) and calling external synchronous services, such as access control, data retrieval, and model inferencing.

Each agent service is a Kubernetes deployment that receives gRPC requests from FlytePropeller when users trigger a particular type of task (for example, the BigQuery agent handles BigQuery tasks). The agent service then initiates a job with the appropriate service. Since agents can be spawned in process, they allow for running all services locally as long as the connection secrets are available. Moreover, agents use a protobuf interface, thus can be implemented in any language, enabling flexibility, reuse of existing libraries, and simpler testing.

You can create different agent services that host different agents, e.g., a production and a development agent service:

Agent Service

Table of contents

Testing agents locally

Whether using an existing agent or developing a new one, you can quickly test the agent in local Python environment without needing to configure your Flyte deployment.

Enabling agents in your Flyte deployment

After you have tested an existing agent in a local Python environment, you must configure your Flyte deployment for the agent to use it in production.

Developing agents

If the agent you need doesn’t exist, follow these steps to create a new agent.

Testing agents in a local development cluster

After developing your new agent and testing it in a local Python environment, you can test it in a local development cluster to ensure it works well remotely.

Deploying agents to the Flyte sandbox

Once you have tested your new agent in a local development cluster and want to use it in production, you should test it in the Flyte sandbox.

Implementing the agent metadata service

If you want to develop an agent server in a language other than Python (e.g., Rust or Java), you must implement the agent metadata service in your agent server.