Implementing the agent metadata service#
About the agent metadata service#
Before FlytePropeller sends a request to the agent server, it needs to know four things:
The name of the agent
Which task category the agent supports
The version of the task category
Whether the agent executes tasks synchronously or asynchronously
After FlytePropeller obtains this metadata, it can send a request to the agent deployment using the correct gRPC method.
Note
An agent can support multiple task categories.
We will use the combination of [task category][version] to identify the specific agent’s deployment and know whether the task is synchronous or asynchronous in FlytePropeller.
The task category is
task_type
in flytekit.
Using the BigQuery Agent as an example:
The agent’s name is
BigQuery Agent
.The agent supports
bigquery_query_job_task
.The agent’s version is
0
.By default, the agent executes tasks asynchronously.
Implement the agent metadata service#
To implement the agent metadata service, you must do two things:
Implement the agent metadata service.
Add the agent metadata service to the agent server.
You can refer to base_agent.py, agent_service.py, and serve.py to see how the agent metadata service is implemented in flytekit’s agent server.
Those gRPC methods are generated by flyteidl and you can import them from here.
Note
You can search the keyword metadata
to find implementations in those files.