Protocol Documentation

flyteidl/service/admin.proto

AdminService

The following defines an RPC service that is also served over HTTP via grpc-gateway. Standard response codes for both are defined here: https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go

AdminService service methods

Method Name	Request Type	Response Type	Description
CreateTask	TaskCreateRequest	TaskCreateResponse	Create and upload a Task definition
GetTask	ObjectGetRequest	Task	Fetch a Task definition.
ListTaskIds	NamedEntityIdentifierListRequest	NamedEntityIdentifierList	Fetch a list of NamedEntityIdentifier of task objects.
ListTasks	ResourceListRequest	TaskList	Fetch a list of Task definitions.
CreateWorkflow	WorkflowCreateRequest	WorkflowCreateResponse	Create and upload a Workflow definition
GetWorkflow	ObjectGetRequest	Workflow	Fetch a Workflow definition.
ListWorkflowIds	NamedEntityIdentifierListRequest	NamedEntityIdentifierList	Fetch a list of NamedEntityIdentifier of workflow objects.
ListWorkflows	ResourceListRequest	WorkflowList	Fetch a list of Workflow definitions.
CreateLaunchPlan	LaunchPlanCreateRequest	LaunchPlanCreateResponse	Create and upload a LaunchPlan definition
GetLaunchPlan	ObjectGetRequest	LaunchPlan	Fetch a LaunchPlan definition.
GetActiveLaunchPlan	ActiveLaunchPlanRequest	LaunchPlan	Fetch the active version of a LaunchPlan.
ListActiveLaunchPlans	ActiveLaunchPlanListRequest	LaunchPlanList	List active versions of LaunchPlan.
ListLaunchPlanIds	NamedEntityIdentifierListRequest	NamedEntityIdentifierList	Fetch a list of NamedEntityIdentifier of launch plan objects.
ListLaunchPlans	ResourceListRequest	LaunchPlanList	Fetch a list of LaunchPlan definitions.
UpdateLaunchPlan	LaunchPlanUpdateRequest	LaunchPlanUpdateResponse	Updates the status of a registered LaunchPlan.
CreateExecution	ExecutionCreateRequest	ExecutionCreateResponse	Triggers the creation of a Execution
RelaunchExecution	ExecutionRelaunchRequest	ExecutionCreateResponse	Triggers the creation of an identical Execution
RecoverExecution	ExecutionRecoverRequest	ExecutionCreateResponse	Recreates a previously-run workflow execution that will only start executing from the last known failure point. In Recover mode, users cannot change any input parameters or update the version of the execution. This is extremely useful to recover from system errors and byzantine faults like - Loss of K8s cluster, bugs in platform or instability, machine failures, downstream system failures (downstream services), or simply to recover executions that failed because of retry exhaustion and should complete if tried again. See ExecutionRecoverRequest for more details.
GetExecution	WorkflowExecutionGetRequest	Execution	Fetches a Execution.
UpdateExecution	ExecutionUpdateRequest	ExecutionUpdateResponse	Update execution belonging to project domain Execution.
GetExecutionData	WorkflowExecutionGetDataRequest	WorkflowExecutionGetDataResponse	Fetches input and output data for a Execution.
ListExecutions	ResourceListRequest	ExecutionList	Fetch a list of Execution.
TerminateExecution	ExecutionTerminateRequest	ExecutionTerminateResponse	Terminates an in-progress Execution.
GetNodeExecution	NodeExecutionGetRequest	NodeExecution	Fetches a NodeExecution.
ListNodeExecutions	NodeExecutionListRequest	NodeExecutionList	Fetch a list of NodeExecution.
ListNodeExecutionsForTask	NodeExecutionForTaskListRequest	NodeExecutionList	Fetch a list of NodeExecution launched by the reference TaskExecution.
GetNodeExecutionData	NodeExecutionGetDataRequest	NodeExecutionGetDataResponse	Fetches input and output data for a NodeExecution.
RegisterProject	ProjectRegisterRequest	ProjectRegisterResponse	Registers a Project with the Flyte deployment.
UpdateProject	Project	ProjectUpdateResponse	Updates an existing Project flyteidl.admin.Project should be passed but the domains property should be empty; it will be ignored in the handler as domains cannot be updated via this API.
ListProjects	ProjectListRequest	Projects	Fetches a list of Project
CreateWorkflowEvent	WorkflowExecutionEventRequest	WorkflowExecutionEventResponse	Indicates a WorkflowExecutionEvent has occurred.
CreateNodeEvent	NodeExecutionEventRequest	NodeExecutionEventResponse	Indicates a NodeExecutionEvent has occurred.
CreateTaskEvent	TaskExecutionEventRequest	TaskExecutionEventResponse	Indicates a TaskExecutionEvent has occurred.
GetTaskExecution	TaskExecutionGetRequest	TaskExecution	Fetches a TaskExecution.
ListTaskExecutions	TaskExecutionListRequest	TaskExecutionList	Fetches a list of TaskExecution.
GetTaskExecutionData	TaskExecutionGetDataRequest	TaskExecutionGetDataResponse	Fetches input and output data for a TaskExecution.
UpdateProjectDomainAttributes	ProjectDomainAttributesUpdateRequest	ProjectDomainAttributesUpdateResponse	Creates or updates custom MatchableAttributesConfiguration for a project and domain.
GetProjectDomainAttributes	ProjectDomainAttributesGetRequest	ProjectDomainAttributesGetResponse	Fetches custom MatchableAttributesConfiguration for a project and domain.
DeleteProjectDomainAttributes	ProjectDomainAttributesDeleteRequest	ProjectDomainAttributesDeleteResponse	Deletes custom MatchableAttributesConfiguration for a project and domain.
UpdateProjectAttributes	ProjectAttributesUpdateRequest	ProjectAttributesUpdateResponse	Creates or updates custom MatchableAttributesConfiguration at the project level
GetProjectAttributes	ProjectAttributesGetRequest	ProjectAttributesGetResponse	Fetches custom MatchableAttributesConfiguration for a project and domain.
DeleteProjectAttributes	ProjectAttributesDeleteRequest	ProjectAttributesDeleteResponse	Deletes custom MatchableAttributesConfiguration for a project and domain.
UpdateWorkflowAttributes	WorkflowAttributesUpdateRequest	WorkflowAttributesUpdateResponse	Creates or updates custom MatchableAttributesConfiguration for a project, domain and workflow.
GetWorkflowAttributes	WorkflowAttributesGetRequest	WorkflowAttributesGetResponse	Fetches custom MatchableAttributesConfiguration for a project, domain and workflow.
DeleteWorkflowAttributes	WorkflowAttributesDeleteRequest	WorkflowAttributesDeleteResponse	Deletes custom MatchableAttributesConfiguration for a project, domain and workflow.
ListMatchableAttributes	ListMatchableAttributesRequest	ListMatchableAttributesResponse	Lists custom MatchableAttributesConfiguration for a specific resource type.
ListNamedEntities	NamedEntityListRequest	NamedEntityList	Returns a list of NamedEntity objects.
GetNamedEntity	NamedEntityGetRequest	NamedEntity	Returns a NamedEntity object.
UpdateNamedEntity	NamedEntityUpdateRequest	NamedEntityUpdateResponse	Updates a NamedEntity object.
GetVersion	GetVersionRequest	GetVersionResponse
GetDescriptionEntity	ObjectGetRequest	DescriptionEntity	Fetch a DescriptionEntity object.
ListDescriptionEntities	DescriptionEntityListRequest	DescriptionEntityList	Fetch a list of DescriptionEntity definitions.

flyteidl/service/auth.proto

OAuth2MetadataRequest

OAuth2MetadataResponse

OAuth2MetadataResponse defines an RFC-Compliant response for /.well-known/oauth-authorization-server metadata as defined in https://tools.ietf.org/html/rfc8414

OAuth2MetadataResponse type fields

Field	Type	Label	Description
issuer	string		Defines the issuer string in all JWT tokens this server issues. The issuer can be admin itself or an external issuer.
authorization_endpoint	string		URL of the authorization server’s authorization endpoint [RFC6749]. This is REQUIRED unless no grant types are supported that use the authorization endpoint.
token_endpoint	string		URL of the authorization server’s token endpoint [RFC6749].
response_types_supported	string	repeated	Array containing a list of the OAuth 2.0 response_type values that this authorization server supports.
scopes_supported	string	repeated	JSON array containing a list of the OAuth 2.0 [RFC6749] scope values that this authorization server supports.
token_endpoint_auth_methods_supported	string	repeated	JSON array containing a list of client authentication methods supported by this token endpoint.
jwks_uri	string		URL of the authorization server’s JWK Set [JWK] document. The referenced document contains the signing key(s) the client uses to validate signatures from the authorization server.
code_challenge_methods_supported	string	repeated	JSON array containing a list of Proof Key for Code Exchange (PKCE) [RFC7636] code challenge methods supported by this authorization server.
grant_types_supported	string	repeated	JSON array containing a list of the OAuth 2.0 grant type values that this authorization server supports.
device_authorization_endpoint	string		URL of the authorization server’s device authorization endpoint, as defined in Section 3.1 of [RFC8628]

PublicClientAuthConfigRequest

PublicClientAuthConfigResponse

FlyteClientResponse encapsulates public information that flyte clients (CLIs… etc.) can use to authenticate users.

PublicClientAuthConfigResponse type fields

Field	Type	Label	Description
client_id	string		client_id to use when initiating OAuth2 authorization requests.
redirect_uri	string		redirect uri to use when initiating OAuth2 authorization requests.
scopes	string	repeated	scopes to request when initiating OAuth2 authorization requests.
authorization_metadata_key	string		Authorization Header to use when passing Access Tokens to the server. If not provided, the client should use the default http Authorization header.
service_http_endpoint	string		ServiceHttpEndpoint points to the http endpoint for the backend. If empty, clients can assume the endpoint used to configure the gRPC connection can be used for the http one respecting the insecure flag to choose between SSL or no SSL connections.
audience	string		audience to use when initiating OAuth2 authorization requests.

AuthMetadataService

The following defines an RPC service that is also served over HTTP via grpc-gateway. Standard response codes for both are defined here: https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go RPCs defined in this service must be anonymously accessible.

AuthMetadataService service methods

Method Name	Request Type	Response Type	Description
GetOAuth2Metadata	OAuth2MetadataRequest	OAuth2MetadataResponse	Anonymously accessible. Retrieves local or external oauth authorization server metadata.
GetPublicClientConfig	PublicClientAuthConfigRequest	PublicClientAuthConfigResponse	Anonymously accessible. Retrieves the client information clients should use when initiating OAuth2 authorization requests.

flyteidl/service/dataproxy.proto

CreateDownloadLinkRequest

CreateDownloadLinkRequest defines the request parameters to create a download link (signed url)

CreateDownloadLinkRequest type fields

Field	Type	Label	Description
artifact_type	ArtifactType		ArtifactType of the artifact requested.
expires_in	Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.
node_execution_id	NodeExecutionIdentifier		NodeId is the unique identifier for the node execution. For a task node, this will retrieve the output of the most recent attempt of the task.

CreateDownloadLinkResponse

CreateDownloadLinkResponse defines the response for the generated links

CreateDownloadLinkResponse type fields

Field	Type	Label	Description
signed_url	string	repeated	SignedUrl specifies the url to use to download content from (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
expires_at	Timestamp		ExpiresAt defines when will the signed URL expire.

CreateDownloadLocationRequest

CreateDownloadLocationRequest specified request for the CreateDownloadLocation API.

CreateDownloadLocationRequest type fields

Field	Type	Label	Description
native_url	string		NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
expires_in	Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.

CreateDownloadLocationResponse

CreateDownloadLocationResponse type fields

Field	Type	Label	Description
signed_url	string		SignedUrl specifies the url to use to download content from (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
expires_at	Timestamp		ExpiresAt defines when will the signed URL expires.

CreateUploadLocationRequest

CreateUploadLocationRequest specified request for the CreateUploadLocation API.

CreateUploadLocationRequest type fields

Field	Type	Label	Description
project	string		Project to create the upload location for +required
domain	string		Domain to create the upload location for. +required
filename	string		Filename specifies a desired suffix for the generated location. E.g. file.py or pre/fix/file.zip. +optional. By default, the service will generate a consistent name based on the provided parameters.
expires_in	Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.
content_md5	bytes		ContentMD5 restricts the upload location to the specific MD5 provided. The ContentMD5 will also appear in the generated path. +required

CreateUploadLocationResponse

CreateUploadLocationResponse type fields

Field	Type	Label	Description
signed_url	string		SignedUrl specifies the url to use to upload content to (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
native_url	string		NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
expires_at	Timestamp		ExpiresAt defines when will the signed URL expires.

ArtifactType

ArtifactType

Enum ArtifactType values

Name	Number	Description
ARTIFACT_TYPE_UNDEFINED	0	ARTIFACT_TYPE_UNDEFINED is the default, often invalid, value for the enum.
ARTIFACT_TYPE_DECK	1	ARTIFACT_TYPE_DECK refers to the deck html file optionally generated after a task, a workflow or a launch plan finishes executing.

DataProxyService

DataProxyService defines an RPC Service that allows access to user-data in a controlled manner.

DataProxyService service methods

Method Name	Request Type	Response Type	Description
CreateUploadLocation	CreateUploadLocationRequest	CreateUploadLocationResponse	CreateUploadLocation creates a signed url to upload artifacts to for a given project/domain.
CreateDownloadLocation	CreateDownloadLocationRequest	CreateDownloadLocationResponse	CreateDownloadLocation creates a signed url to download artifacts.
CreateDownloadLink	CreateDownloadLinkRequest	CreateDownloadLinkResponse	CreateDownloadLocation creates a signed url to download artifacts.

flyteidl/service/identity.proto

UserInfoRequest

UserInfoResponse

See the OpenID Connect spec at https://openid.net/specs/openid-connect-core-1_0.html#UserInfoResponse for more information.

UserInfoResponse type fields

Field	Type	Label	Description
subject	string		Locally unique and never reassigned identifier within the Issuer for the End-User, which is intended to be consumed by the Client.
name	string		Full name
preferred_username	string		Shorthand name by which the End-User wishes to be referred to
given_name	string		Given name(s) or first name(s)
family_name	string		Surname(s) or last name(s)
email	string		Preferred e-mail address
picture	string		Profile picture URL

IdentityService

IdentityService defines an RPC Service that interacts with user/app identities.

IdentityService service methods

Method Name	Request Type	Response Type	Description
UserInfo	UserInfoRequest	UserInfoResponse	Retrieves user information about the currently logged in user.

flyteidl/service/signal.proto

SignalService

SignalService defines an RPC Service that may create, update, and retrieve signal(s).

SignalService service methods

Method Name	Request Type	Response Type	Description
GetOrCreateSignal	SignalGetOrCreateRequest	Signal	Fetches or creates a Signal.
ListSignals	SignalListRequest	SignalList	Fetch a list of Signal definitions.
SetSignal	SignalSetRequest	SignalSetResponse	Sets the value on a Signal definition