Testing agents in a local development cluster#
The Flyte agent service runs in a separate deployment instead of inside FlytePropeller. To test an agent server in a local development cluster, you must run both the single binary and agent server at the same time, allowing FlytePropeller to send requests to your local agent server.
Backend plugin vs agent service architecture#
To understand why you must run both the single binary and agent server at the same time, it is helpful to compare the backend plugin architecture to the agent service architecture.
Backend plugin architecture#
In this architecture, FlytePropeller sends requests through the SDK:
Agent service architecture#
With the agent service framework:
Flyteplugins send gRPC requests to the agent server.
The agent server sends requests through the SDK and returns the query data.
Configuring the agent service in development mode#
Start the demo cluster in dev mode:
flytectl demo start --dev
Start the agent grpc server:
pyflyte serve agent
(Optional) Update the timeout configuration for each request to the agent deployment in the single binary YAML file:
cd flyte
vim ./flyte-single-binary-local.yaml
plugins:
# Registered Task Types
agent-service:
defaultAgent:
endpoint: "localhost:8000" # your grpc agent server port
insecure: true
timeouts:
# CreateTask, GetTask and DeleteTask are for async agents.
# ExecuteTaskSync is for sync agents.
CreateTask: 5s
GetTask: 5s
DeleteTask: 5s
ExecuteTaskSync: 10s
defaultTimeout: 10s
Start the Flyte server with the single binary config file:
make compile
./flyte start --config ./flyte-single-binary-local.yaml
Set up your secrets: In the development environment, you can set up your secrets on your local machine by adding secrets to
/etc/secrets/SECRET_NAME
.
Since your agent server is running locally rather than within Kubernetes, it can retrieve the secret from your local file system.
Test your agent task:
pyflyte run --remote agent_workflow.py agent_task
Note
You must build an image that includes the plugin for the task and specify its config with the --image
flag when running pyflyte run
or in an ImageSpec definition in your workflow file.