Testing agents in a local development cluster

The Flyte agent service runs in a separate deployment instead of inside FlytePropeller. To test an agent server in a local development cluster, you must run both the single binary and agent server at the same time, allowing FlytePropeller to send requests to your local agent server.

Backend plugin vs agent service architecture

To understand why you must run both the single binary and agent server at the same time, it is helpful to compare the backend plugin architecture to the agent service architecture.

Backend plugin architecture

In this architecture, FlytePropeller sends requests through the SDK:

image.png

Agent service architecture

With the agent service framework:

  1. Flyteplugins send gRPC requests to the agent server.

  2. The agent server sends requests through the SDK and returns the query data.

image.png

Configuring the agent service in development mode

  1. Start the demo cluster in dev mode:

flytectl demo start --dev

  1. Start the agent grpc server:

pyflyte serve agent

  1. Update the config for the task handled by the agent in the single binary yaml file.

cd flyte
vim ./flyte-single-binary-local.yaml

:emphasize-lines: 9
tasks:
  task-plugins:
    enabled-plugins:
      - agent-service
      - container
      - sidecar
      - K8S-ARRAY
    default-for-task-types:
      - sensor: agent-service
      - container: container
      - container_array: K8S-ARRAY

plugins:
  # Registered Task Types
  agent-service:
    defaultAgent:
      endpoint: "localhost:8000" # your grpc agent server port
      insecure: true
      timeouts:
        # CreateTask, GetTask and DeleteTask are for async agents.
        # ExecuteTaskSync is for sync agents.
        CreateTask: 5s
        GetTask: 5s
        DeleteTask: 5s
        ExecuteTaskSync: 10s
      defaultTimeout: 10s

  1. Start the Flyte server with the single binary config file:

make compile
./flyte start --config ./flyte-single-binary-local.yaml

  1. Set up your secrets: In the development environment, you can set up your secrets on your local machine by adding secrets to /etc/secrets/SECRET_NAME.

Since your agent server is running locally rather than within Kubernetes, it can retrieve the secret from your local file system.

  1. Test your agent task:

pyflyte run --remote agent_workflow.py agent_task

Note

You must build an image that includes the plugin for the task and specify its config with the --image flag when running pyflyte run or in an ImageSpec definition in your workflow file.