Protocol Documentation

flyteidl/admin/cluster_assignment.proto

ClusterAssignment

Encapsulates specifications for routing an execution onto a specific cluster.

ClusterAssignment type fields

Field	Type	Label	Description
cluster_pool_name	string

flyteidl/admin/common.proto

Annotations

Annotation values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge annotations defined at registration and execution time.

Annotations type fields

Field	Type	Label	Description
values	Annotations.ValuesEntry	repeated	Map of custom annotations to be applied to the execution resource.

Annotations.ValuesEntry

Annotations.ValuesEntry type fields

Field	Type	Label	Description
key	string
value	string

AuthRole

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

AuthRole type fields

Field	Type	Label	Description
assumable_iam_role	string		Defines an optional iam role which will be used for tasks run in executions created with this launch plan.
kubernetes_service_account	string		Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

EmailNotification

Defines an email notification specification.

EmailNotification type fields

Field	Type	Label	Description
recipients_email	string	repeated	The list of email addresses recipients for this notification. +required

Labels

Label values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge labels defined at registration and execution time.

Labels type fields

Field	Type	Label	Description
values	Labels.ValuesEntry	repeated	Map of custom labels to be applied to the execution resource.

Labels.ValuesEntry

Labels.ValuesEntry type fields

Field	Type	Label	Description
key	string
value	string

NamedEntity

Encapsulates information common to a NamedEntity, a Flyte resource such as a task, workflow or launch plan. A NamedEntity is exclusively identified by its resource type and identifier.

NamedEntity type fields

Field	Type	Label	Description
resource_type	ResourceType		Resource type of the named entity. One of Task, Workflow or LaunchPlan.
id	NamedEntityIdentifier
metadata	NamedEntityMetadata		Additional metadata around a named entity.

NamedEntityGetRequest

A request to retrieve the metadata associated with a NamedEntityIdentifier

NamedEntityGetRequest type fields

Field	Type	Label	Description
resource_type	ResourceType		Resource type of the metadata to get. One of Task, Workflow or LaunchPlan. +required
id	NamedEntityIdentifier		The identifier for the named entity for which to fetch metadata. +required

NamedEntityIdentifier

Encapsulation of fields that identifies a Flyte resource. A Flyte resource can be a task, workflow or launch plan. A resource can internally have multiple versions and is uniquely identified by project, domain, and name.

NamedEntityIdentifier type fields

Field	Type	Label	Description
project	string		Name of the project the resource belongs to.
domain	string		Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project.
name	string		User provided value for the resource. The combination of project + domain + name uniquely identifies the resource. +optional - in certain contexts - like ‘List API’, ‘Launch plans’

NamedEntityIdentifierList

Represents a list of NamedEntityIdentifiers.

NamedEntityIdentifierList type fields

Field	Type	Label	Description
entities	NamedEntityIdentifier	repeated	A list of identifiers.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityIdentifierListRequest

Represents a request structure to list NamedEntityIdentifiers.

NamedEntityIdentifierListRequest type fields

Field	Type	Label	Description
project	string		Name of the project that contains the identifiers. +required
domain	string		Name of the domain the identifiers belongs to within the project. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Specifies how listed entities should be sorted in the response. +optional
filters	string		Indicates a list of filters passed as string. +optional

NamedEntityList

Represents a list of NamedEntityIdentifiers.

NamedEntityList type fields

Field	Type	Label	Description
entities	NamedEntity	repeated	A list of NamedEntity objects
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityListRequest

Represents a request structure to list NamedEntity objects

NamedEntityListRequest type fields

Field	Type	Label	Description
resource_type	ResourceType		Resource type of the metadata to query. One of Task, Workflow or LaunchPlan. +required
project	string		Name of the project that contains the identifiers. +required
domain	string		Name of the domain the identifiers belongs to within the project.
limit	uint32		Indicates the number of resources to be returned.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Specifies how listed entities should be sorted in the response. +optional
filters	string		Indicates a list of filters passed as string. +optional

NamedEntityMetadata

Additional metadata around a named entity.

NamedEntityMetadata type fields

Field	Type	Label	Description
description	string		Common description across all versions of the entity +optional
state	NamedEntityState		Shared state across all version of the entity At this point in time, only workflow entities can have their state archived.

NamedEntityUpdateRequest

Request to set the referenced named entity state to the configured value.

NamedEntityUpdateRequest type fields

Field	Type	Label	Description
resource_type	ResourceType		Resource type of the metadata to update +required
id	NamedEntityIdentifier		Identifier of the metadata to update +required
metadata	NamedEntityMetadata		Metadata object to set as the new value +required

NamedEntityUpdateResponse

Purposefully empty, may be populated in the future.

Notification

Represents a structure for notifications based on execution status. The notification content is configured within flyte admin but can be templatized. Future iterations could expose configuring notifications with custom content.

Notification type fields

Field	Type	Label	Description
phases	WorkflowExecution.Phase	repeated	A list of phases to which users can associate the notifications to. +required
email	EmailNotification
pager_duty	PagerDutyNotification
slack	SlackNotification

ObjectGetRequest

Shared request structure to fetch a single resource. Resources include: Task, Workflow, LaunchPlan

ObjectGetRequest type fields

Field	Type	Label	Description
id	Identifier		Indicates a unique version of resource. +required

PagerDutyNotification

Defines a pager duty notification specification.

PagerDutyNotification type fields

Field	Type	Label	Description
recipients_email	string	repeated	Currently, PagerDuty notifications leverage email to trigger a notification. +required

RawOutputDataConfig

Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). See https://github.com/flyteorg/flyte/issues/211 for more background information.

RawOutputDataConfig type fields

Field	Type	Label	Description
output_location_prefix	string		Prefix for where offloaded data from user workflows will be written e.g. s3://bucket/key or s3://bucket/

ResourceListRequest

Shared request structure to retrieve a list of resources. Resources include: Task, Workflow, LaunchPlan

ResourceListRequest type fields

Field	Type	Label	Description
id	NamedEntityIdentifier		id represents the unique identifier of the resource. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering. +optional

SlackNotification

Defines a slack notification specification.

SlackNotification type fields

Field	Type	Label	Description
recipients_email	string	repeated	Currently, Slack notifications leverage email to trigger a notification. +required

Sort

Specifies sort ordering in a list request.

Sort type fields

Field	Type	Label	Description
key	string		Indicates an attribute to sort the response values. +required
direction	Sort.Direction		Indicates the direction to apply sort key for response values. +optional

UrlBlob

Represents a string url and associated metadata used throughout the platform.

UrlBlob type fields

Field	Type	Label	Description
url	string		Actual url value.
bytes	int64		Represents the size of the file accessible at the above url.

NamedEntityState

The status of the named entity is used to control its visibility in the UI.

Enum NamedEntityState values

Name	Number	Description
NAMED_ENTITY_ACTIVE	0	By default, all named entities are considered active and under development.
NAMED_ENTITY_ARCHIVED	1	Archived named entities are no longer visible in the UI.
SYSTEM_GENERATED	2	System generated entities that aren’t explicitly created or managed by a user.

Sort.Direction

Enum Sort.Direction values

Name	Number	Description
DESCENDING	0	By default, fields are sorted in descending order.
ASCENDING	1

flyteidl/admin/description_entity.proto

Description

Full user description with formatting preserved. This can be rendered by clients, such as the console or command line tools with in-tact formatting.

Description type fields

Field	Type	Label	Description
value	string		long description - no more than 4KB
uri	string		if the description sizes exceed some threshold we can offload the entire description proto altogether to an external data store, like S3 rather than store inline in the db
format	DescriptionFormat		Format of the long description
icon_link	string		Optional link to an icon for the entity

DescriptionEntity

DescriptionEntity contains detailed description for the task/workflow. Documentation could provide insight into the algorithms, business use case, etc.

DescriptionEntity type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the description entity.
short_description	string		One-liner overview of the entity.
long_description	Description		Full user description with formatting preserved.
source_code	SourceCode		Optional link to source code used to define this entity.
tags	string	repeated	User-specified tags. These are arbitrary and can be used for searching filtering and discovering tasks.

DescriptionEntityList

Represents a list of DescriptionEntities returned from the admin. See DescriptionEntity for more details

DescriptionEntityList type fields

Field	Type	Label	Description
descriptionEntities	DescriptionEntity	repeated	A list of DescriptionEntities returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

DescriptionEntityListRequest

Represents a request structure to retrieve a list of DescriptionEntities. See DescriptionEntity for more details

DescriptionEntityListRequest type fields

Field	Type	Label	Description
resource_type	ResourceType		Identifies the specific type of resource that this identifier corresponds to.
id	NamedEntityIdentifier		The identifier for the description entity. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering for returned list. +optional

SourceCode

Link to source code used to define this entity

SourceCode type fields

Field	Type	Label	Description
link	string

DescriptionFormat

The format of the long description

Enum DescriptionFormat values

Name	Number	Description
DESCRIPTION_FORMAT_UNKNOWN	0
DESCRIPTION_FORMAT_MARKDOWN	1
DESCRIPTION_FORMAT_HTML	2
DESCRIPTION_FORMAT_RST	3	python default documentation - comments is rst

flyteidl/admin/event.proto

EventErrorAlreadyInTerminalState

Indicates that a sent event was not used to update execution state due to the referenced execution already being terminated (and therefore ineligible for further state transitions).

EventErrorAlreadyInTerminalState type fields

Field	Type	Label	Description
current_phase	string		+required

EventErrorIncompatibleCluster

Indicates an event was rejected because it came from a different cluster than is on record as running the execution.

EventErrorIncompatibleCluster type fields

Field	Type	Label	Description
cluster	string		The cluster which has been recorded as processing the execution. +required

EventFailureReason

Indicates why a sent event was not used to update execution.

EventFailureReason type fields

Field	Type	Label	Description
already_in_terminal_state	EventErrorAlreadyInTerminalState
incompatible_cluster	EventErrorIncompatibleCluster

NodeExecutionEventRequest

Request to send a notification that a node execution event has occurred.

NodeExecutionEventRequest type fields

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	NodeExecutionEvent		Details about the event that occurred.

NodeExecutionEventResponse

Purposefully empty, may be populated in the future.

TaskExecutionEventRequest

Request to send a notification that a task execution event has occurred.

TaskExecutionEventRequest type fields

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	TaskExecutionEvent		Details about the event that occurred.

TaskExecutionEventResponse

Purposefully empty, may be populated in the future.

WorkflowExecutionEventRequest

Request to send a notification that a workflow execution event has occurred.

WorkflowExecutionEventRequest type fields

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	WorkflowExecutionEvent		Details about the event that occurred.

WorkflowExecutionEventResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/execution.proto

AbortMetadata

Specifies metadata around an aborted workflow execution.

AbortMetadata type fields

Field	Type	Label	Description
cause	string		In the case of a user-specified abort, this will pass along the user-supplied cause.
principal	string		Identifies the entity (if any) responsible for terminating the execution

Execution

A workflow execution represents an instantiated workflow, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes. Used as a response object used in Get and List execution requests.

Execution type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Unique identifier of the workflow execution.
spec	ExecutionSpec		User-provided configuration and inputs for launching the execution.
closure	ExecutionClosure		Execution results.

ExecutionClosure

Encapsulates the results of the Execution

ExecutionClosure type fields

Field	Type	Label	Description
outputs	LiteralMapBlob		Deprecated. Output URI in the case of a successful execution. DEPRECATED. Use GetExecutionData to fetch output data instead.
error	ExecutionError		Error information in the case of a failed execution.
abort_cause	string		Deprecated. In the case of a user-specified abort, this will pass along the user-supplied cause.
abort_metadata	AbortMetadata		In the case of a user-specified abort, this will pass along the user and their supplied cause.
output_data	LiteralMap		Deprecated. Raw output data produced by this execution. DEPRECATED. Use GetExecutionData to fetch output data instead.
computed_inputs	LiteralMap		Deprecated. Inputs computed and passed for execution. computed_inputs depends on inputs in ExecutionSpec, fixed and default inputs in launch plan
phase	WorkflowExecution.Phase		Most recent recorded phase for the execution.
started_at	Timestamp		Reported time at which the execution began running.
duration	Duration		The amount of time the execution spent running.
created_at	Timestamp		Reported time at which the execution was created.
updated_at	Timestamp		Reported time at which the execution was last updated.
notifications	Notification	repeated	The notification settings to use after merging the CreateExecutionRequest and the launch plan notification settings. An execution launched with notifications will always prefer that definition to notifications defined statically in a launch plan.
workflow_id	Identifier		Identifies the workflow definition for this execution.
state_change_details	ExecutionStateChangeDetails		Provides the details of the last stage change

ExecutionCreateRequest

Request to launch an execution with the given project, domain and optionally-assigned name.

ExecutionCreateRequest type fields

Field	Type	Label	Description
project	string		Name of the project the execution belongs to. +required
domain	string		Name of the domain the execution belongs to. A domain can be considered as a subset within a specific project. +required
name	string		User provided value for the resource. If none is provided the system will generate a unique string. +optional
spec	ExecutionSpec		Additional fields necessary to launch the execution. +optional
inputs	LiteralMap		The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional

ExecutionCreateResponse

The unique identifier for a successfully created execution. If the name was not specified in the create request, this identifier will include a generated name.

ExecutionCreateResponse type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier

ExecutionList

Used as a response for request to list executions. See Execution for more details

ExecutionList type fields

Field	Type	Label	Description
executions	Execution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

ExecutionMetadata

Represents attributes about an execution which are not required to launch the execution but are useful to record. These attributes are assigned at launch time and do not change.

ExecutionMetadata type fields

Field	Type	Label	Description
mode	ExecutionMetadata.ExecutionMode
principal	string		Identifier of the entity that triggered this execution. For systems using back-end authentication any value set here will be discarded in favor of the authenticated user context.
nesting	uint32		Indicates the nestedness of this execution. If a user launches a workflow execution, the default nesting is 0. If this execution further launches a workflow (child workflow), the nesting level is incremented by 0 => 1 Generally, if workflow at nesting level k launches a workflow then the child workflow will have nesting = k + 1.
scheduled_at	Timestamp		For scheduled executions, the requested time for execution for this specific schedule invocation.
parent_node_execution	NodeExecutionIdentifier		Which subworkflow node (if any) launched this execution
reference_execution	WorkflowExecutionIdentifier		Optional, a reference workflow execution related to this execution. In the case of a relaunch, this references the original workflow execution.
system_metadata	SystemMetadata		Optional, platform-specific metadata about the execution. In this the future this may be gated behind an ACL or some sort of authorization.

ExecutionRecoverRequest

Request to recover the referenced execution.

ExecutionRecoverRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Identifier of the workflow execution to recover.
name	string		User provided value for the recovered execution. If none is provided the system will generate a unique string. +optional
metadata	ExecutionMetadata		Additional metadata which will be used to overwrite any metadata in the reference execution when triggering a recovery execution.

ExecutionRelaunchRequest

Request to relaunch the referenced execution.

ExecutionRelaunchRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Identifier of the workflow execution to relaunch. +required
name	string		User provided value for the relaunched execution. If none is provided the system will generate a unique string. +optional
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.

ExecutionSpec

An ExecutionSpec encompasses all data used to launch this execution. The Spec does not change over the lifetime of an execution as it progresses across phase changes.

ExecutionSpec type fields

Field	Type	Label	Description
launch_plan	Identifier		Launch plan to be executed
inputs	LiteralMap		Deprecated. Input values to be passed for the execution
metadata	ExecutionMetadata		Metadata for the execution
notifications	NotificationList		List of notifications based on Execution status transitions When this list is not empty it is used rather than any notifications defined in the referenced launch plan. When this list is empty, the notifications defined for the launch plan will be applied.
disable_all	bool		This should be set to true if all notifications are intended to be disabled for this execution.
labels	Labels		Labels to apply to the execution resource.
annotations	Annotations		Annotations to apply to the execution resource.
security_context	SecurityContext		Optional: security context override to apply this execution.
auth_role	AuthRole		Deprecated. Optional: auth override to apply this execution.
quality_of_service	QualityOfService		Indicates the runtime priority of the execution.
max_parallelism	int32		Controls the maximum number of task nodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.
raw_output_data_config	RawOutputDataConfig		User setting to configure where to store offloaded data (i.e. Blobs, structured datasets, query data, etc.). This should be a prefix like s3://my-bucket/my-data
cluster_assignment	ClusterAssignment		Controls how to select an available cluster on which this execution should run.
interruptible	BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.

ExecutionStateChangeDetails

ExecutionStateChangeDetails type fields

Field	Type	Label	Description
state	ExecutionState		The state of the execution is used to control its visibility in the UI/CLI.
occurred_at	Timestamp		This timestamp represents when the state changed.
principal	string		Identifies the entity (if any) responsible for causing the state change of the execution

ExecutionTerminateRequest

Request to terminate an in-progress execution. This action is irreversible. If an execution is already terminated, this request will simply be a no-op. This request will fail if it references a non-existent execution. If the request succeeds the phase “ABORTED” will be recorded for the termination with the optional cause added to the output_result.

ExecutionTerminateRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Uniquely identifies the individual workflow execution to be terminated.
cause	string		Optional reason for aborting.

ExecutionTerminateResponse

Purposefully empty, may be populated in the future.

ExecutionUpdateRequest

ExecutionUpdateRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Identifier of the execution to update
state	ExecutionState		State to set as the new value active/archive

ExecutionUpdateResponse

LiteralMapBlob

Input/output data can represented by actual values or a link to where values are stored

LiteralMapBlob type fields

Field	Type	Label	Description
values	LiteralMap		Deprecated. Data in LiteralMap format
uri	string		In the event that the map is too large, we return a uri to the data

NotificationList

NotificationList type fields

Field	Type	Label	Description
notifications	Notification	repeated

SystemMetadata

Represents system, rather than user-facing, metadata about an execution.

SystemMetadata type fields

Field	Type	Label	Description
execution_cluster	string		Which execution cluster this execution ran on.

WorkflowExecutionGetDataRequest

Request structure to fetch inputs, output and other data produced by an execution. By default this data is not returned inline in WorkflowExecutionGetRequest

WorkflowExecutionGetDataRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		The identifier of the execution for which to fetch inputs and outputs.

WorkflowExecutionGetDataResponse

Response structure for WorkflowExecutionGetDataRequest which contains inputs and outputs for an execution.

WorkflowExecutionGetDataResponse type fields

Field	Type	Label	Description
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of execution outputs. Deprecated: Please use full_outputs instead.
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of execution inputs. Deprecated: Please use full_inputs instead.
full_inputs	LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.

WorkflowExecutionGetRequest

A message used to fetch a single workflow execution entity. See Execution for more details

WorkflowExecutionGetRequest type fields

Field	Type	Label	Description
id	WorkflowExecutionIdentifier		Uniquely identifies an individual workflow execution.

ExecutionMetadata.ExecutionMode

The method by which this execution was launched.

Enum ExecutionMetadata.ExecutionMode values

Name	Number	Description
MANUAL	0	The default execution mode, MANUAL implies that an execution was launched by an individual.
SCHEDULED	1	A schedule triggered this execution launch.
SYSTEM	2	A system process was responsible for launching this execution rather an individual.
RELAUNCH	3	This execution was launched with identical inputs as a previous execution.
CHILD_WORKFLOW	4	This execution was triggered by another execution.
RECOVERED	5	This execution was recovered from another execution.

ExecutionState

The state of the execution is used to control its visibility in the UI/CLI.

Enum ExecutionState values

Name	Number	Description
EXECUTION_ACTIVE	0	By default, all executions are considered active.
EXECUTION_ARCHIVED	1	Archived executions are no longer visible in the UI.

flyteidl/admin/launch_plan.proto

ActiveLaunchPlanListRequest

Represents a request structure to list active launch plans within a project/domain. See LaunchPlan for more details

ActiveLaunchPlanListRequest type fields

Field	Type	Label	Description
project	string		Name of the project that contains the identifiers. +required.
domain	string		Name of the domain the identifiers belongs to within the project. +required.
limit	uint32		Indicates the number of resources to be returned. +required.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Sort ordering. +optional

ActiveLaunchPlanRequest

Represents a request struct for finding an active launch plan for a given NamedEntityIdentifier See LaunchPlan for more details

ActiveLaunchPlanRequest type fields

Field	Type	Label	Description
id	NamedEntityIdentifier		+required.

Auth

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

Auth type fields

Field	Type	Label	Description
assumable_iam_role	string		Defines an optional iam role which will be used for tasks run in executions created with this launch plan.
kubernetes_service_account	string		Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

LaunchPlan

A LaunchPlan provides the capability to templatize workflow executions. Launch plans simplify associating one or more schedules, inputs and notifications with your workflows. Launch plans can be shared and used to trigger executions with predefined inputs even when a workflow definition doesn’t necessarily have a default value for said input.

LaunchPlan type fields

Field	Type	Label	Description
id	Identifier		Uniquely identifies a launch plan entity.
spec	LaunchPlanSpec		User-provided launch plan details, including reference workflow, inputs and other metadata.
closure	LaunchPlanClosure		Values computed by the flyte platform after launch plan registration.

LaunchPlanClosure

Values computed by the flyte platform after launch plan registration. These include expected_inputs required to be present in a CreateExecutionRequest to launch the reference workflow as well timestamp values associated with the launch plan.

LaunchPlanClosure type fields

Field	Type	Label	Description
state	LaunchPlanState		Indicate the Launch plan state.
expected_inputs	ParameterMap		Indicates the set of inputs expected when creating an execution with the Launch plan
expected_outputs	VariableMap		Indicates the set of outputs expected to be produced by creating an execution with the Launch plan
created_at	Timestamp		Time at which the launch plan was created.
updated_at	Timestamp		Time at which the launch plan was last updated.

LaunchPlanCreateRequest

Request to register a launch plan. The included LaunchPlanSpec may have a complete or incomplete set of inputs required to launch a workflow execution. By default all launch plans are registered in state INACTIVE. If you wish to set the state to ACTIVE, you must submit a LaunchPlanUpdateRequest, after you have successfully created a launch plan.

LaunchPlanCreateRequest type fields

Field	Type	Label	Description
id	Identifier		Uniquely identifies a launch plan entity.
spec	LaunchPlanSpec		User-provided launch plan details, including reference workflow, inputs and other metadata.

LaunchPlanCreateResponse

Purposefully empty, may be populated in the future.

LaunchPlanList

Response object for list launch plan requests. See LaunchPlan for more details

LaunchPlanList type fields

Field	Type	Label	Description
launch_plans	LaunchPlan	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

LaunchPlanMetadata

Additional launch plan attributes included in the LaunchPlanSpec not strictly required to launch the reference workflow.

LaunchPlanMetadata type fields

Field	Type	Label	Description
schedule	Schedule		Schedule to execute the Launch Plan
notifications	Notification	repeated	List of notifications based on Execution status transitions

LaunchPlanSpec

User-provided launch plan definition and configuration values.

LaunchPlanSpec type fields

Field	Type	Label	Description
workflow_id	Identifier		Reference to the Workflow template that the launch plan references
entity_metadata	LaunchPlanMetadata		Metadata for the Launch Plan
default_inputs	ParameterMap		Input values to be passed for the execution. These can be overriden when an execution is created with this launch plan.
fixed_inputs	LiteralMap		Fixed, non-overridable inputs for the Launch Plan. These can not be overriden when an execution is created with this launch plan.
role	string		Deprecated. String to indicate the role to use to execute the workflow underneath
labels	Labels		Custom labels to be applied to the execution resource.
annotations	Annotations		Custom annotations to be applied to the execution resource.
auth	Auth		Deprecated. Indicates the permission associated with workflow executions triggered with this launch plan.
auth_role	AuthRole		Deprecated.
security_context	SecurityContext		Indicates security context for permissions triggered with this launch plan
quality_of_service	QualityOfService		Indicates the runtime priority of the execution.
raw_output_data_config	RawOutputDataConfig		Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).
max_parallelism	int32		Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.
interruptible	BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.

LaunchPlanUpdateRequest

Request to set the referenced launch plan state to the configured value. See LaunchPlan for more details

LaunchPlanUpdateRequest type fields

Field	Type	Label	Description
id	Identifier		Identifier of launch plan for which to change state. +required.
state	LaunchPlanState		Desired state to apply to the launch plan. +required.

LaunchPlanUpdateResponse

Purposefully empty, may be populated in the future.

LaunchPlanState

By default any launch plan regardless of state can be used to launch a workflow execution. However, at most one version of a launch plan (e.g. a NamedEntityIdentifier set of shared project, domain and name values) can be active at a time in regards to schedules. That is, at most one schedule in a NamedEntityIdentifier group will be observed and trigger executions at a defined cadence.

Enum LaunchPlanState values

Name	Number	Description
INACTIVE	0
ACTIVE	1

flyteidl/admin/matchable_resource.proto

ClusterResourceAttributes

ClusterResourceAttributes type fields

Field	Type	Label	Description
attributes	ClusterResourceAttributes.AttributesEntry	repeated	Custom resource attributes which will be applied in cluster resource creation (e.g. quotas). Map keys are the case-sensitive names of variables in templatized resource files. Map values should be the custom values which get substituted during resource creation.

ClusterResourceAttributes.AttributesEntry

ClusterResourceAttributes.AttributesEntry type fields

Field	Type	Label	Description
key	string
value	string

ExecutionClusterLabel

ExecutionClusterLabel type fields

Field	Type	Label	Description
value	string		Label value to determine where the execution will be run

ExecutionQueueAttributes

ExecutionQueueAttributes type fields

Field	Type	Label	Description
tags	string	repeated	Tags used for assigning execution queues for tasks defined within this project.

ListMatchableAttributesRequest

Request all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details

ListMatchableAttributesRequest type fields

Field	Type	Label	Description
resource_type	MatchableResource		+required

ListMatchableAttributesResponse

Response for a request for all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details

ListMatchableAttributesResponse type fields

Field	Type	Label	Description
configurations	MatchableAttributesConfiguration	repeated

MatchableAttributesConfiguration

Represents a custom set of attributes applied for either a domain; a domain and project; or domain, project and workflow name. These are used to override system level defaults for kubernetes cluster resource management, default execution values, and more all across different levels of specificity.

MatchableAttributesConfiguration type fields

Field	Type	Label	Description
attributes	MatchingAttributes
domain	string
project	string
workflow	string
launch_plan	string

MatchingAttributes

Generic container for encapsulating all types of the above attributes messages.

MatchingAttributes type fields

Field	Type	Label	Description
task_resource_attributes	TaskResourceAttributes
cluster_resource_attributes	ClusterResourceAttributes
execution_queue_attributes	ExecutionQueueAttributes
execution_cluster_label	ExecutionClusterLabel
quality_of_service	QualityOfService
plugin_overrides	PluginOverrides
workflow_execution_config	WorkflowExecutionConfig
cluster_assignment	ClusterAssignment

PluginOverride

This MatchableAttribute configures selecting alternate plugin implementations for a given task type. In addition to an override implementation a selection of fallbacks can be provided or other modes for handling cases where the desired plugin override is not enabled in a given Flyte deployment.

PluginOverride type fields

Field	Type	Label	Description
task_type	string		A predefined yet extensible Task type identifier.
plugin_id	string	repeated	A set of plugin ids which should handle tasks of this type instead of the default registered plugin. The list will be tried in order until a plugin is found with that id.
missing_plugin_behavior	PluginOverride.MissingPluginBehavior		Defines the behavior when no plugin from the plugin_id list is not found.

PluginOverrides

PluginOverrides type fields

Field	Type	Label	Description
overrides	PluginOverride	repeated

TaskResourceAttributes

Defines task resource defaults and limits that will be applied at task registration.

TaskResourceAttributes type fields

Field	Type	Label	Description
defaults	TaskResourceSpec
limits	TaskResourceSpec

TaskResourceSpec

Defines a set of overridable task resource attributes set during task registration.

TaskResourceSpec type fields

Field	Type	Label	Description
cpu	string
gpu	string
memory	string
storage	string
ephemeral_storage	string

WorkflowExecutionConfig

Adds defaults for customizable workflow-execution specifications and overrides.

WorkflowExecutionConfig type fields

Field	Type	Label	Description
max_parallelism	int32		Can be used to control the number of parallel nodes to run within the workflow. This is useful to achieve fairness.
security_context	SecurityContext		Indicates security context permissions for executions triggered with this matchable attribute.
raw_output_data_config	RawOutputDataConfig		Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).
labels	Labels		Custom labels to be applied to a triggered execution resource.
annotations	Annotations		Custom annotations to be applied to a triggered execution resource.
interruptible	BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.

MatchableResource

Defines a resource that can be configured by customizable Project-, ProjectDomain- or WorkflowAttributes based on matching tags.

Enum MatchableResource values

Name	Number	Description
TASK_RESOURCE	0	Applies to customizable task resource requests and limits.
CLUSTER_RESOURCE	1	Applies to configuring templated kubernetes cluster resources.
EXECUTION_QUEUE	2	Configures task and dynamic task execution queue assignment.
EXECUTION_CLUSTER_LABEL	3	Configures the K8s cluster label to be used for execution to be run
QUALITY_OF_SERVICE_SPECIFICATION	4	Configures default quality of service when undefined in an execution spec.
PLUGIN_OVERRIDE	5	Selects configurable plugin implementation behavior for a given task type.
WORKFLOW_EXECUTION_CONFIG	6	Adds defaults for customizable workflow-execution specifications and overrides.
CLUSTER_ASSIGNMENT	7	Controls how to select an available cluster on which this execution should run.

PluginOverride.MissingPluginBehavior

Enum PluginOverride.MissingPluginBehavior values

Name	Number	Description
FAIL	0	By default, if this plugin is not enabled for a Flyte deployment then execution will fail.
USE_DEFAULT	1	Uses the system-configured default implementation.

flyteidl/admin/node_execution.proto

DynamicWorkflowNodeMetadata

For dynamic workflow nodes we capture information about the dynamic workflow definition that gets generated.

DynamicWorkflowNodeMetadata type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the workflow.
compiled_workflow	CompiledWorkflowClosure		Represents the compiled representation of the embedded dynamic workflow.

NodeExecution

Encapsulates all details for a single node execution entity. A node represents a component in the overall workflow graph. A node launch a task, multiple tasks, an entire nested sub-workflow, or even a separate child-workflow execution. The same task can be called repeatedly in a single workflow but each node is unique.

NodeExecution type fields

Field	Type	Label	Description
id	NodeExecutionIdentifier		Uniquely identifies an individual node execution.
input_uri	string		Path to remote data store where input blob is stored.
closure	NodeExecutionClosure		Computed results associated with this node execution.
metadata	NodeExecutionMetaData		Metadata for Node Execution

NodeExecutionClosure

Container for node execution details and results.

NodeExecutionClosure type fields

Field	Type	Label	Description
output_uri	string		Deprecated. Links to a remotely stored, serialized core.LiteralMap of node execution outputs. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.
error	ExecutionError		Error information for the Node
output_data	LiteralMap		Deprecated. Raw output data produced by this node execution. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.
phase	NodeExecution.Phase		The last recorded phase for this node execution.
started_at	Timestamp		Time at which the node execution began running.
duration	Duration		The amount of time the node execution spent running.
created_at	Timestamp		Time at which the node execution was created.
updated_at	Timestamp		Time at which the node execution was last updated.
workflow_node_metadata	WorkflowNodeMetadata
task_node_metadata	TaskNodeMetadata
deck_uri	string		String location uniquely identifying where the deck HTML file is. NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)

NodeExecutionForTaskListRequest

Represents a request structure to retrieve a list of node execution entities launched by a specific task. This can arise when a task yields a subworkflow.

NodeExecutionForTaskListRequest type fields

Field	Type	Label	Description
task_execution_id	TaskExecutionIdentifier		Indicates the node execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering. +optional

NodeExecutionGetDataRequest

Request structure to fetch inputs and output for a node execution. By default, these are not returned in NodeExecutionGetRequest

NodeExecutionGetDataRequest type fields

Field	Type	Label	Description
id	NodeExecutionIdentifier		The identifier of the node execution for which to fetch inputs and outputs.

NodeExecutionGetDataResponse

Response structure for NodeExecutionGetDataRequest which contains inputs and outputs for a node execution.

NodeExecutionGetDataResponse type fields

Field	Type	Label	Description
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of node execution inputs. Deprecated: Please use full_inputs instead.
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of node execution outputs. Deprecated: Please use full_outputs instead.
full_inputs	LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.
dynamic_workflow	DynamicWorkflowNodeMetadata		Optional Workflow closure for a dynamically generated workflow, in the case this node yields a dynamic workflow we return its structure here.

NodeExecutionGetRequest

A message used to fetch a single node execution entity. See NodeExecution for more details

NodeExecutionGetRequest type fields

Field	Type	Label	Description
id	NodeExecutionIdentifier		Uniquely identifies an individual node execution. +required

NodeExecutionList

Request structure to retrieve a list of node execution entities. See NodeExecution for more details

NodeExecutionList type fields

Field	Type	Label	Description
node_executions	NodeExecution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NodeExecutionListRequest

Represents a request structure to retrieve a list of node execution entities. See NodeExecution for more details

NodeExecutionListRequest type fields

Field	Type	Label	Description
workflow_execution_id	WorkflowExecutionIdentifier		Indicates the workflow execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering. +optional
unique_parent_id	string		Unique identifier of the parent node in the execution +optional

NodeExecutionMetaData

Represents additional attributes related to a Node Execution

NodeExecutionMetaData type fields

Field	Type	Label	Description
retry_group	string		Node executions are grouped depending on retries of the parent Retry group is unique within the context of a parent node.
is_parent_node	bool		Boolean flag indicating if the node has child nodes under it This can be true when a node contains a dynamic workflow which then produces child nodes.
spec_node_id	string		Node id of the node in the original workflow This maps to value of WorkflowTemplate.nodes[X].id
is_dynamic	bool		Boolean flag indicating if the node has contains a dynamic workflow which then produces child nodes. This is to distinguish between subworkflows and dynamic workflows which can both have is_parent_node as true.

TaskNodeMetadata

Metadata for the case in which the node is a TaskNode

TaskNodeMetadata type fields

Field	Type	Label	Description
cache_status	CatalogCacheStatus		Captures the status of caching for this execution.
catalog_key	CatalogMetadata		This structure carries the catalog artifact information
checkpoint_uri	string		The latest checkpoint location

WorkflowNodeMetadata

Metadata for a WorkflowNode

WorkflowNodeMetadata type fields

Field	Type	Label	Description
executionId	WorkflowExecutionIdentifier		The identifier for a workflow execution launched by a node.

flyteidl/admin/notification.proto

EmailMessage

Represents the Email object that is sent to a publisher/subscriber to forward the notification. Note: This is internal to Admin and doesn’t need to be exposed to other components.

EmailMessage type fields

Field	Type	Label	Description
recipients_email	string	repeated	The list of email addresses to receive an email with the content populated in the other fields. Currently, each email recipient will receive its own email. This populates the TO field.
sender_email	string		The email of the sender. This populates the FROM field.
subject_line	string		The content of the subject line. This populates the SUBJECT field.
body	string		The content of the email body. This populates the BODY field.

flyteidl/admin/project.proto

Domain

Namespace within a project commonly used to differentiate between different service instances. e.g. “production”, “development”, etc.

Domain type fields

Field	Type	Label	Description
id	string		Globally unique domain name.
name	string		Display name.

Project

Top-level namespace used to classify different entities like workflows and executions.

Project type fields

Field	Type	Label	Description
id	string		Globally unique project name.
name	string		Display name.
domains	Domain	repeated
description	string
labels	Labels		Leverage Labels from flyteidl.admin.common.proto to tag projects with ownership information.
state	Project.ProjectState

ProjectListRequest

Request to retrieve a list of projects matching specified filters. See Project for more details

ProjectListRequest type fields

Field	Type	Label	Description
limit	uint32		Indicates the number of projects to be returned. +required
token	string		In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering. +optional

ProjectRegisterRequest

Adds a new user-project within the Flyte deployment. See Project for more details

ProjectRegisterRequest type fields

Field	Type	Label	Description
project	Project		+required

ProjectRegisterResponse

Purposefully empty, may be updated in the future.

ProjectUpdateResponse

Purposefully empty, may be updated in the future.

Projects

Represents a list of projects. See Project for more details

Projects type fields

Field	Type	Label	Description
projects	Project	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

Project.ProjectState

The state of the project is used to control its visibility in the UI and validity.

Enum Project.ProjectState values

Name	Number	Description
ACTIVE	0	By default, all projects are considered active.
ARCHIVED	1	Archived projects are no longer visible in the UI and no longer valid.
SYSTEM_GENERATED	2	System generated projects that aren’t explicitly created or managed by a user.

flyteidl/admin/project_attributes.proto

ProjectAttributes

Defines a set of custom matching attributes at the project level. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectAttributes type fields

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
matching_attributes	MatchingAttributes

ProjectAttributesDeleteRequest

Request to delete a set matchable project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectAttributesDeleteRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required

ProjectAttributesDeleteResponse

Purposefully empty, may be populated in the future.

ProjectAttributesGetRequest

Request to get an individual project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectAttributesGetRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required

ProjectAttributesGetResponse

Response to get an individual project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectAttributesGetResponse type fields

Field	Type	Label	Description
attributes	ProjectAttributes

ProjectAttributesUpdateRequest

Sets custom attributes for a project For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectAttributesUpdateRequest type fields

Field	Type	Label	Description
attributes	ProjectAttributes		+required

ProjectAttributesUpdateResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/project_domain_attributes.proto

ProjectDomainAttributes

Defines a set of custom matching attributes which defines resource defaults for a project and domain. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributes type fields

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
domain	string		Unique domain id for which this set of attributes will be applied.
matching_attributes	MatchingAttributes

ProjectDomainAttributesDeleteRequest

Request to delete a set matchable project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesDeleteRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required

ProjectDomainAttributesDeleteResponse

Purposefully empty, may be populated in the future.

ProjectDomainAttributesGetRequest

Request to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesGetRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required

ProjectDomainAttributesGetResponse

Response to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesGetResponse type fields

Field	Type	Label	Description
attributes	ProjectDomainAttributes

ProjectDomainAttributesUpdateRequest

Sets custom attributes for a project-domain combination. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesUpdateRequest type fields

Field	Type	Label	Description
attributes	ProjectDomainAttributes		+required

ProjectDomainAttributesUpdateResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/schedule.proto

CronSchedule

Options for schedules to run according to a cron expression.

CronSchedule type fields

Field	Type	Label	Description
schedule	string		Standard/default cron implementation as described by https://en.wikipedia.org/wiki/Cron#CRON_expression; Also supports nonstandard predefined scheduling definitions as described by https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#CronExpressions except @reboot
offset	string		ISO 8601 duration as described by https://en.wikipedia.org/wiki/ISO_8601#Durations

FixedRate

Option for schedules run at a certain frequency e.g. every 2 minutes.

FixedRate type fields

Field	Type	Label	Description
value	uint32
unit	FixedRateUnit

Schedule

Defines complete set of information required to trigger an execution on a schedule.

Schedule type fields

Field	Type	Label	Description
cron_expression	string		Deprecated. Uses AWS syntax: Minutes Hours Day-of-month Month Day-of-week Year e.g. for a schedule that runs every 15 minutes: 0/15 * * * ? *
rate	FixedRate
cron_schedule	CronSchedule
kickoff_time_input_arg	string		Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off.

FixedRateUnit

Represents a frequency at which to run a schedule.

Enum FixedRateUnit values

Name	Number	Description
MINUTE	0
HOUR	1
DAY	2

flyteidl/admin/signal.proto

Signal

Signal encapsulates a unique identifier, associated metadata, and a value for a single Flyte signal. Signals may exist either without a set value (representing a signal request) or with a populated value (indicating the signal has been given).

SignalGetOrCreateRequest

SignalGetOrCreateRequest represents a request structure to retrive or create a signal. See Signal for more details

SignalGetOrCreateRequest type fields

Field	Type	Label	Description
id	SignalIdentifier		A unique identifier for the requested signal.
type	LiteralType		A type denoting the required value type for this signal.

SignalList

SignalList represents collection of signals along with the token of the last result. See Signal for more details

SignalList type fields

Field	Type	Label	Description
signals	Signal	repeated	A list of signals matching the input filters.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

SignalListRequest

SignalListRequest represents a request structure to retrieve a collection of signals. See Signal for more details

SignalListRequest type fields

Field	Type	Label	Description
workflow_execution_id	WorkflowExecutionIdentifier		Indicates the workflow execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. +optional
sort_by	Sort		Sort ordering. +optional

SignalSetRequest

SignalSetRequest represents a request structure to set the value on a signal. Setting a signal effetively satisfies the signal condition within a Flyte workflow. See Signal for more details

SignalSetRequest type fields

Field	Type	Label	Description
id	SignalIdentifier		A unique identifier for the requested signal.
value	Literal		The value of this signal, must match the defining signal type.

SignalSetResponse

SignalSetResponse represents a response structure if signal setting succeeds.

Purposefully empty, may be populated in the future.

flyteidl/admin/task.proto

Task

Flyte workflows are composed of many ordered tasks. That is small, reusable, self-contained logical blocks arranged to process workflow inputs and produce a deterministic set of outputs. Tasks can come in many varieties tuned for specialized behavior.

Task type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the task.
closure	TaskClosure		closure encapsulates all the fields that maps to a compiled version of the task.
short_description	string		One-liner overview of the entity.

TaskClosure

Compute task attributes which include values derived from the TaskSpec, as well as plugin-specific data and task metadata.

TaskClosure type fields

Field	Type	Label	Description
compiled_task	CompiledTask		Represents the compiled representation of the task from the specification provided.
created_at	Timestamp		Time at which the task was created.

TaskCreateRequest

Represents a request structure to create a revision of a task. See Task for more details

TaskCreateRequest type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the task. +required
spec	TaskSpec		Represents the specification for task. +required

TaskCreateResponse

Represents a response structure if task creation succeeds.

Purposefully empty, may be populated in the future.

TaskList

Represents a list of tasks returned from the admin. See Task for more details

TaskList type fields

Field	Type	Label	Description
tasks	Task	repeated	A list of tasks returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskSpec

Represents a structure that encapsulates the user-configured specification of the task.

TaskSpec type fields

Field	Type	Label	Description
template	TaskTemplate		Template of the task that encapsulates all the metadata of the task.
description	DescriptionEntity		Represents the specification for description entity.

flyteidl/admin/task_execution.proto

TaskExecution

Encapsulates all details for a single task execution entity. A task execution represents an instantiated task, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes.

TaskExecution type fields

Field	Type	Label	Description
id	TaskExecutionIdentifier		Unique identifier for the task execution.
input_uri	string		Path to remote data store where input blob is stored.
closure	TaskExecutionClosure		Task execution details and results.
is_parent	bool		Whether this task spawned nodes.

TaskExecutionClosure

Container for task execution details and results.

TaskExecutionClosure type fields

Field	Type	Label	Description
output_uri	string		Deprecated. Path to remote data store where output blob is stored if the execution succeeded (and produced outputs). DEPRECATED. Use GetTaskExecutionData to fetch output data instead.
error	ExecutionError		Error information for the task execution. Populated if the execution failed.
output_data	LiteralMap		Deprecated. Raw output data produced by this task execution. DEPRECATED. Use GetTaskExecutionData to fetch output data instead.
phase	TaskExecution.Phase		The last recorded phase for this task execution.
logs	TaskLog	repeated	Detailed log information output by the task execution.
started_at	Timestamp		Time at which the task execution began running.
duration	Duration		The amount of time the task execution spent running.
created_at	Timestamp		Time at which the task execution was created.
updated_at	Timestamp		Time at which the task execution was last updated.
custom_info	Struct		Custom data specific to the task plugin.
reason	string		If there is an explanation for the most recent phase transition, the reason will capture it.
task_type	string		A predefined yet extensible Task type identifier.
metadata	TaskExecutionMetadata		Metadata around how a task was executed.
event_version	int32		The event version is used to indicate versioned changes in how data is maintained using this proto message. For example, event_verison > 0 means that maps tasks logs use the TaskExecutionMetadata ExternalResourceInfo fields for each subtask rather than the TaskLog in this message.

TaskExecutionGetDataRequest

Request structure to fetch inputs and output for a task execution. By default this data is not returned inline in TaskExecutionGetRequest

TaskExecutionGetDataRequest type fields

Field	Type	Label	Description
id	TaskExecutionIdentifier		The identifier of the task execution for which to fetch inputs and outputs. +required

TaskExecutionGetDataResponse

Response structure for TaskExecutionGetDataRequest which contains inputs and outputs for a task execution.

TaskExecutionGetDataResponse type fields

Field	Type	Label	Description
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of task execution inputs. Deprecated: Please use full_inputs instead.
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of task execution outputs. Deprecated: Please use full_outputs instead.
full_inputs	LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.

TaskExecutionGetRequest

A message used to fetch a single task execution entity. See TaskExecution for more details

TaskExecutionGetRequest type fields

Field	Type	Label	Description
id	TaskExecutionIdentifier		Unique identifier for the task execution. +required

TaskExecutionList

Response structure for a query to list of task execution entities. See TaskExecution for more details

TaskExecutionList type fields

Field	Type	Label	Description
task_executions	TaskExecution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskExecutionListRequest

Represents a request structure to retrieve a list of task execution entities yielded by a specific node execution. See TaskExecution for more details

TaskExecutionListRequest type fields

Field	Type	Label	Description
node_execution_id	NodeExecutionIdentifier		Indicates the node execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional
sort_by	Sort		Sort ordering for returned list. +optional

flyteidl/admin/version.proto

GetVersionRequest

Empty request for GetVersion

GetVersionResponse

Response for the GetVersion API

GetVersionResponse type fields

Field	Type	Label	Description
control_plane_version	Version		The control plane version information. FlyteAdmin and related components form the control plane of Flyte

Version

Provides Version information for a component

Version type fields

Field	Type	Label	Description
Build	string		Specifies the GIT sha of the build
Version	string		Version for the build, should follow a semver
BuildTime	string		Build timestamp

flyteidl/admin/workflow.proto

CreateWorkflowFailureReason

When a CreateWorkflowRequest failes due to matching id

CreateWorkflowFailureReason type fields

Field	Type	Label	Description
exists_different_structure	WorkflowErrorExistsDifferentStructure
exists_identical_structure	WorkflowErrorExistsIdenticalStructure

Workflow

Represents the workflow structure stored in the Admin A workflow is created by ordering tasks and associating outputs to inputs in order to produce a directed-acyclic execution graph.

Workflow type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the workflow.
closure	WorkflowClosure		closure encapsulates all the fields that maps to a compiled version of the workflow.
short_description	string		One-liner overview of the entity.

WorkflowClosure

A container holding the compiled workflow produced from the WorkflowSpec and additional metadata.

WorkflowClosure type fields

Field	Type	Label	Description
compiled_workflow	CompiledWorkflowClosure		Represents the compiled representation of the workflow from the specification provided.
created_at	Timestamp		Time at which the workflow was created.

WorkflowCreateRequest

Represents a request structure to create a revision of a workflow. See Workflow for more details

WorkflowCreateRequest type fields

Field	Type	Label	Description
id	Identifier		id represents the unique identifier of the workflow. +required
spec	WorkflowSpec		Represents the specification for workflow. +required

WorkflowCreateResponse

Purposefully empty, may be populated in the future.

WorkflowErrorExistsDifferentStructure

The workflow id is already used and the structure is different

WorkflowErrorExistsDifferentStructure type fields

Field	Type	Label	Description
id	Identifier

WorkflowErrorExistsIdenticalStructure

The workflow id is already used with an identical sctructure

WorkflowErrorExistsIdenticalStructure type fields

Field	Type	Label	Description
id	Identifier

WorkflowList

Represents a list of workflows returned from the admin. See Workflow for more details

WorkflowList type fields

Field	Type	Label	Description
workflows	Workflow	repeated	A list of workflows returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

WorkflowSpec

Represents a structure that encapsulates the specification of the workflow.

WorkflowSpec type fields

Field	Type	Label	Description
template	WorkflowTemplate		Template of the task that encapsulates all the metadata of the workflow.
sub_workflows	WorkflowTemplate	repeated	Workflows that are embedded into other workflows need to be passed alongside the parent workflow to the propeller compiler (since the compiler doesn’t have any knowledge of other workflows - ie, it doesn’t reach out to Admin to see other registered workflows). In fact, subworkflows do not even need to be registered.
description	DescriptionEntity		Represents the specification for description entity.

flyteidl/admin/workflow_attributes.proto

WorkflowAttributes

Defines a set of custom matching attributes which defines resource defaults for a project, domain and workflow. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributes type fields

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
domain	string		Unique domain id for which this set of attributes will be applied.
workflow	string		Workflow name for which this set of attributes will be applied.
matching_attributes	MatchingAttributes

WorkflowAttributesDeleteRequest

Request to delete a set matchable workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesDeleteRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
workflow	string		Workflow name which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required

WorkflowAttributesDeleteResponse

Purposefully empty, may be populated in the future.

WorkflowAttributesGetRequest

Request to get an individual workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesGetRequest type fields

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
workflow	string		Workflow name which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required

WorkflowAttributesGetResponse

Response to get an individual workflow attribute override.

WorkflowAttributesGetResponse type fields

Field	Type	Label	Description
attributes	WorkflowAttributes

WorkflowAttributesUpdateRequest

Sets custom attributes for a project, domain and workflow combination. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesUpdateRequest type fields

Field	Type	Label	Description
attributes	WorkflowAttributes

WorkflowAttributesUpdateResponse

Purposefully empty, may be populated in the future.

google/protobuf/duration.proto

Duration

A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like “day” or “month”. It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

# Examples

Example 1: Compute Duration from two Timestamps in pseudo code.

Timestamp start = …; Timestamp end = …; Duration duration = …;

duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

if (duration.seconds < 0 && duration.nanos > 0) {

duration.seconds += 1; duration.nanos -= 1000000000;

} else if (duration.seconds > 0 && duration.nanos < 0) {

duration.seconds -= 1; duration.nanos += 1000000000;

}

Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

Timestamp start = …; Duration duration = …; Timestamp end = …;

end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

if (end.nanos < 0) {

end.seconds -= 1; end.nanos += 1000000000;

} else if (end.nanos >= 1000000000) {

end.seconds += 1; end.nanos -= 1000000000;

}

Example 3: Compute Duration from datetime.timedelta in Python.

td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

# JSON Mapping

In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix “s” (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as “3s”, while 3 seconds and 1 nanosecond should be expressed in JSON format as “3.000000001s”, and 3 seconds and 1 microsecond should be expressed in JSON format as “3.000001s”.

Duration type fields

Field	Type	Label	Description
seconds	int64		Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years
nanos	int32		Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 seconds field and a positive or negative nanos field. For durations of one second or more, a non-zero value for the nanos field must be of the same sign as the seconds field. Must be from -999,999,999 to +999,999,999 inclusive.

google/protobuf/wrappers.proto

BoolValue

Wrapper message for bool.

The JSON representation for BoolValue is JSON true and false.

BoolValue type fields

Field	Type	Label	Description
value	bool		The bool value.

BytesValue

Wrapper message for bytes.

The JSON representation for BytesValue is JSON string.

BytesValue type fields

Field	Type	Label	Description
value	bytes		The bytes value.

DoubleValue

Wrapper message for double.

The JSON representation for DoubleValue is JSON number.

DoubleValue type fields

Field	Type	Label	Description
value	double		The double value.

FloatValue

Wrapper message for float.

The JSON representation for FloatValue is JSON number.

FloatValue type fields

Field	Type	Label	Description
value	float		The float value.

Int32Value

Wrapper message for int32.

The JSON representation for Int32Value is JSON number.

Int32Value type fields

Field	Type	Label	Description
value	int32		The int32 value.

Int64Value

Wrapper message for int64.

The JSON representation for Int64Value is JSON string.

Int64Value type fields

Field	Type	Label	Description
value	int64		The int64 value.

StringValue

Wrapper message for string.

The JSON representation for StringValue is JSON string.

StringValue type fields

Field	Type	Label	Description
value	string		The string value.

UInt32Value

Wrapper message for uint32.

The JSON representation for UInt32Value is JSON number.

UInt32Value type fields

Field	Type	Label	Description
value	uint32		The uint32 value.

UInt64Value

Wrapper message for uint64.

The JSON representation for UInt64Value is JSON string.

UInt64Value type fields

Field	Type	Label	Description
value	uint64		The uint64 value.