Protocol Documentation#

flyteidl/admin/cluster_assignment.proto#

Affinity#

Defines a set of constraints used to select eligible objects based on labels they possess.

Affinity type fields#

Field

Type

Label

Description

selectors

Selector

repeated

Multiples selectors are ‘and’-ed together to produce the list of matching, eligible objects.

ClusterAssignment#

Encapsulates specifications for routing an execution onto a specific cluster.

ClusterAssignment type fields#

Field

Type

Label

Description

affinity

Affinity

toleration

Toleration

Selector#

A Selector is a specification for identifying a set of objects with corresponding labels.

Selector type fields#

Field

Type

Label

Description

key

string

The label key.

value

string

repeated

One or more values used to match labels. For equality (or inequality) requirements, values must contain a single element. For set-based requirements, values may contain one or more elements.

operator

Selector.Operator

Toleration#

Defines a set of specific label selectors that the execution can tolerate on a cluster.

Toleration type fields#

Field

Type

Label

Description

selectors

Selector

repeated

A toleration selector is similar to that of an affinity but the only valid operators are EQUALS AND EXISTS.

Selector.Operator#

Defines how a label with a corresponding key and value is selected or excluded.

Enum Selector.Operator values#

Name

Number

Description

EQUALS

0

NOT_EQUALS

1

IN

2

NOT_IN

3

EXISTS

4

A label key with any value

flyteidl/admin/common.proto#

Annotations#

Annotation values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge annotations defined at registration and execution time.

Annotations type fields#

Field

Type

Label

Description

values

Annotations.ValuesEntry

repeated

Map of custom annotations to be applied to the execution resource.

Annotations.ValuesEntry#

Annotations.ValuesEntry type fields#

Field

Type

Label

Description

key

string

value

string

AuthRole#

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

AuthRole type fields#

Field

Type

Label

Description

assumable_iam_role

string

Defines an optional iam role which will be used for tasks run in executions created with this launch plan.

kubernetes_service_account

string

Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

EmailNotification#

Defines an email notification specification.

EmailNotification type fields#

Field

Type

Label

Description

recipients_email

string

repeated

The list of email addresses recipients for this notification. +required

Labels#

Label values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge labels defined at registration and execution time.

Labels type fields#

Field

Type

Label

Description

values

Labels.ValuesEntry

repeated

Map of custom labels to be applied to the execution resource.

Labels.ValuesEntry#

Labels.ValuesEntry type fields#

Field

Type

Label

Description

key

string

value

string

NamedEntity#

Encapsulates information common to a NamedEntity, a Flyte resource such as a task, workflow or launch plan. A NamedEntity is exclusively identified by its resource type and identifier.

NamedEntity type fields#

Field

Type

Label

Description

resource_type

ResourceType

Resource type of the named entity. One of Task, Workflow or LaunchPlan.

id

NamedEntityIdentifier

metadata

NamedEntityMetadata

Additional metadata around a named entity.

NamedEntityGetRequest#

A request to retrieve the metadata associated with a NamedEntityIdentifier

NamedEntityGetRequest type fields#

Field

Type

Label

Description

resource_type

ResourceType

Resource type of the metadata to get. One of Task, Workflow or LaunchPlan. +required

id

NamedEntityIdentifier

The identifier for the named entity for which to fetch metadata. +required

NamedEntityIdentifier#

Encapsulation of fields that identifies a Flyte resource. A Flyte resource can be a task, workflow or launch plan. A resource can internally have multiple versions and is uniquely identified by project, domain, and name.

NamedEntityIdentifier type fields#

Field

Type

Label

Description

project

string

Name of the project the resource belongs to.

domain

string

Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project.

name

string

User provided value for the resource. The combination of project + domain + name uniquely identifies the resource. +optional - in certain contexts - like ‘List API’, ‘Launch plans’

NamedEntityIdentifierList#

Represents a list of NamedEntityIdentifiers.

NamedEntityIdentifierList type fields#

Field

Type

Label

Description

entities

NamedEntityIdentifier

repeated

A list of identifiers.

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityIdentifierListRequest#

Represents a request structure to list NamedEntityIdentifiers.

NamedEntityIdentifierListRequest type fields#

Field

Type

Label

Description

project

string

Name of the project that contains the identifiers. +required

domain

string

Name of the domain the identifiers belongs to within the project. +required

limit

uint32

Indicates the number of resources to be returned. +required

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional

sort_by

Sort

Specifies how listed entities should be sorted in the response. +optional

filters

string

Indicates a list of filters passed as string. +optional

NamedEntityList#

Represents a list of NamedEntityIdentifiers.

NamedEntityList type fields#

Field

Type

Label

Description

entities

NamedEntity

repeated

A list of NamedEntity objects

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityListRequest#

Represents a request structure to list NamedEntity objects

NamedEntityListRequest type fields#

Field

Type

Label

Description

resource_type

ResourceType

Resource type of the metadata to query. One of Task, Workflow or LaunchPlan. +required

project

string

Name of the project that contains the identifiers. +required

domain

string

Name of the domain the identifiers belongs to within the project.

limit

uint32

Indicates the number of resources to be returned.

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional

sort_by

Sort

Specifies how listed entities should be sorted in the response. +optional

filters

string

Indicates a list of filters passed as string. +optional

NamedEntityMetadata#

Additional metadata around a named entity.

NamedEntityMetadata type fields#

Field

Type

Label

Description

description

string

Common description across all versions of the entity +optional

state

NamedEntityState

Shared state across all version of the entity At this point in time, only workflow entities can have their state archived.

NamedEntityUpdateRequest#

Request to set the referenced named entity state to the configured value.

NamedEntityUpdateRequest type fields#

Field

Type

Label

Description

resource_type

ResourceType

Resource type of the metadata to update +required

id

NamedEntityIdentifier

Identifier of the metadata to update +required

metadata

NamedEntityMetadata

Metadata object to set as the new value +required

NamedEntityUpdateResponse#

Purposefully empty, may be populated in the future.

Notification#

Represents a structure for notifications based on execution status. The notification content is configured within flyte admin but can be templatized. Future iterations could expose configuring notifications with custom content.

Notification type fields#

Field

Type

Label

Description

phases

WorkflowExecution.Phase

repeated

A list of phases to which users can associate the notifications to. +required

email

EmailNotification

pager_duty

PagerDutyNotification

slack

SlackNotification

ObjectGetRequest#

Shared request structure to fetch a single resource. Resources include: Task, Workflow, LaunchPlan

ObjectGetRequest type fields#

Field

Type

Label

Description

id

Identifier

Indicates a unique version of resource. +required

PagerDutyNotification#

Defines a pager duty notification specification.

PagerDutyNotification type fields#

Field

Type

Label

Description

recipients_email

string

repeated

Currently, PagerDuty notifications leverage email to trigger a notification. +required

RawOutputDataConfig#

Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). See https://github.com/flyteorg/flyte/issues/211 for more background information.

RawOutputDataConfig type fields#

Field

Type

Label

Description

output_location_prefix

string

Prefix for where offloaded data from user workflows will be written e.g. s3://bucket/key or s3://bucket/

ResourceListRequest#

Shared request structure to retrieve a list of resources. Resources include: Task, Workflow, LaunchPlan

ResourceListRequest type fields#

Field

Type

Label

Description

id

NamedEntityIdentifier

id represents the unique identifier of the resource. +required

limit

uint32

Indicates the number of resources to be returned. +required

token

string

In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional

filters

string

Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional

sort_by

Sort

Sort ordering. +optional

SlackNotification#

Defines a slack notification specification.

SlackNotification type fields#

Field

Type

Label

Description

recipients_email

string

repeated

Currently, Slack notifications leverage email to trigger a notification. +required

Sort#

Specifies sort ordering in a list request.

Sort type fields#

Field

Type

Label

Description

key

string

Indicates an attribute to sort the response values. +required

direction

Sort.Direction

Indicates the direction to apply sort key for response values. +optional

UrlBlob#

Represents a string url and associated metadata used throughout the platform.

UrlBlob type fields#

Field

Type

Label

Description

url

string

Actual url value.

bytes

int64

Represents the size of the file accessible at the above url.

NamedEntityState#

The status of the named entity is used to control its visibility in the UI.

Enum NamedEntityState values#

Name

Number

Description

NAMED_ENTITY_ACTIVE

0

By default, all named entities are considered active and under development.

NAMED_ENTITY_ARCHIVED

1

Archived named entities are no longer visible in the UI.

SYSTEM_GENERATED

2

System generated entities that aren’t explicitly created or managed by a user.

Sort.Direction#

Enum Sort.Direction values#

Name

Number

Description

DESCENDING

0

By default, fields are sorted in descending order.

ASCENDING

1

flyteidl/admin/event.proto#

EventErrorAlreadyInTerminalState#

Indicates that a sent event was not used to update execution state due to the referenced execution already being terminated (and therefore ineligible for further state transitions).

EventErrorAlreadyInTerminalState type fields#

Field

Type

Label

Description

current_phase

string

+required

EventErrorIncompatibleCluster#

Indicates an event was rejected because it came from a different cluster than is on record as running the execution.

EventErrorIncompatibleCluster type fields#

Field

Type

Label

Description

cluster

string

The cluster which has been recorded as processing the execution. +required

EventFailureReason#

Indicates why a sent event was not used to update execution.

EventFailureReason type fields#

Field

Type

Label

Description

already_in_terminal_state

EventErrorAlreadyInTerminalState

incompatible_cluster

EventErrorIncompatibleCluster

NodeExecutionEventRequest#

Request to send a notification that a node execution event has occurred.

NodeExecutionEventRequest type fields#

Field

Type

Label

Description

request_id

string

Unique ID for this request that can be traced between services

event

NodeExecutionEvent

Details about the event that occurred.

NodeExecutionEventResponse#

Purposefully empty, may be populated in the future.

TaskExecutionEventRequest#

Request to send a notification that a task execution event has occurred.

TaskExecutionEventRequest type fields#

Field

Type

Label

Description

request_id

string

Unique ID for this request that can be traced between services

event

TaskExecutionEvent

Details about the event that occurred.

TaskExecutionEventResponse#

Purposefully empty, may be populated in the future.

WorkflowExecutionEventRequest#

Request to send a notification that a workflow execution event has occurred.

WorkflowExecutionEventRequest type fields#

Field

Type

Label

Description

request_id

string

Unique ID for this request that can be traced between services

event

WorkflowExecutionEvent

Details about the event that occurred.

WorkflowExecutionEventResponse#

Purposefully empty, may be populated in the future.

flyteidl/admin/execution.proto#

AbortMetadata#

Specifies metadata around an aborted workflow execution.

AbortMetadata type fields#

Field

Type

Label

Description

cause

string

In the case of a user-specified abort, this will pass along the user-supplied cause.

principal

string

Identifies the entity (if any) responsible for terminating the execution

Execution#

A workflow execution represents an instantiated workflow, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes. Used as a response object used in Get and List execution requests.

Execution type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Unique identifier of the workflow execution.

spec

ExecutionSpec

User-provided configuration and inputs for launching the execution.

closure

ExecutionClosure

Execution results.

ExecutionClosure#

Encapsulates the results of the Execution

ExecutionClosure type fields#

Field

Type

Label

Description

outputs

LiteralMapBlob

Deprecated. Output URI in the case of a successful execution. DEPRECATED. Use GetExecutionData to fetch output data instead.

error

ExecutionError

Error information in the case of a failed execution.

abort_cause

string

Deprecated. In the case of a user-specified abort, this will pass along the user-supplied cause.

abort_metadata

AbortMetadata

In the case of a user-specified abort, this will pass along the user and their supplied cause.

output_data

LiteralMap

Deprecated. Raw output data produced by this execution. DEPRECATED. Use GetExecutionData to fetch output data instead.

computed_inputs

LiteralMap

Deprecated. Inputs computed and passed for execution. computed_inputs depends on inputs in ExecutionSpec, fixed and default inputs in launch plan

phase

WorkflowExecution.Phase

Most recent recorded phase for the execution.

started_at

Timestamp

Reported time at which the execution began running.

duration

Duration

The amount of time the execution spent running.

created_at

Timestamp

Reported time at which the execution was created.

updated_at

Timestamp

Reported time at which the execution was last updated.

notifications

Notification

repeated

The notification settings to use after merging the CreateExecutionRequest and the launch plan notification settings. An execution launched with notifications will always prefer that definition to notifications defined statically in a launch plan.

workflow_id

Identifier

Identifies the workflow definition for this execution.

state_change_details

ExecutionStateChangeDetails

Provides the details of the last stage change

ExecutionCreateRequest#

Request to launch an execution with the given project, domain and optionally-assigned name.

ExecutionCreateRequest type fields#

Field

Type

Label

Description

project

string

Name of the project the execution belongs to. +required

domain

string

Name of the domain the execution belongs to. A domain can be considered as a subset within a specific project. +required

name

string

User provided value for the resource. If none is provided the system will generate a unique string. +optional

spec

ExecutionSpec

Additional fields necessary to launch the execution. +optional

inputs

LiteralMap

The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional

ExecutionCreateResponse#

The unique identifier for a successfully created execution. If the name was not specified in the create request, this identifier will include a generated name.

ExecutionCreateResponse type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

ExecutionList#

Used as a response for request to list executions. See Execution for more details

ExecutionList type fields#

Field

Type

Label

Description

executions

Execution

repeated

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

ExecutionMetadata#

Represents attributes about an execution which are not required to launch the execution but are useful to record. These attributes are assigned at launch time and do not change.

ExecutionMetadata type fields#

Field

Type

Label

Description

mode

ExecutionMetadata.ExecutionMode

principal

string

Identifier of the entity that triggered this execution. For systems using back-end authentication any value set here will be discarded in favor of the authenticated user context.

nesting

uint32

Indicates the nestedness of this execution. If a user launches a workflow execution, the default nesting is 0. If this execution further launches a workflow (child workflow), the nesting level is incremented by 0 => 1 Generally, if workflow at nesting level k launches a workflow then the child workflow will have nesting = k + 1.

scheduled_at

Timestamp

For scheduled executions, the requested time for execution for this specific schedule invocation.

parent_node_execution

NodeExecutionIdentifier

Which subworkflow node (if any) launched this execution

reference_execution

WorkflowExecutionIdentifier

Optional, a reference workflow execution related to this execution. In the case of a relaunch, this references the original workflow execution.

system_metadata

SystemMetadata

Optional, platform-specific metadata about the execution. In this the future this may be gated behind an ACL or some sort of authorization.

ExecutionRecoverRequest#

Request to recover the referenced execution.

ExecutionRecoverRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Identifier of the workflow execution to recover.

name

string

User provided value for the recovered execution. If none is provided the system will generate a unique string. +optional

metadata

ExecutionMetadata

Additional metadata which will be used to overwrite any metadata in the reference execution when triggering a recovery execution.

ExecutionRelaunchRequest#

Request to relaunch the referenced execution.

ExecutionRelaunchRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Identifier of the workflow execution to relaunch. +required

name

string

User provided value for the relaunched execution. If none is provided the system will generate a unique string. +optional

ExecutionSpec#

An ExecutionSpec encompasses all data used to launch this execution. The Spec does not change over the lifetime of an execution as it progresses across phase changes.

ExecutionSpec type fields#

Field

Type

Label

Description

launch_plan

Identifier

Launch plan to be executed

inputs

LiteralMap

Deprecated. Input values to be passed for the execution

metadata

ExecutionMetadata

Metadata for the execution

notifications

NotificationList

List of notifications based on Execution status transitions When this list is not empty it is used rather than any notifications defined in the referenced launch plan. When this list is empty, the notifications defined for the launch plan will be applied.

disable_all

bool

This should be set to true if all notifications are intended to be disabled for this execution.

labels

Labels

Labels to apply to the execution resource.

annotations

Annotations

Annotations to apply to the execution resource.

security_context

SecurityContext

Optional: security context override to apply this execution.

auth_role

AuthRole

Deprecated. Optional: auth override to apply this execution.

quality_of_service

QualityOfService

Indicates the runtime priority of the execution.

max_parallelism

int32

Controls the maximum number of task nodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.

raw_output_data_config

RawOutputDataConfig

User setting to configure where to store offloaded data (i.e. Blobs, structured datasets, query data, etc.). This should be a prefix like s3://my-bucket/my-data

cluster_assignment

ClusterAssignment

Controls how to select an available cluster on which this execution should run.

interruptible

BoolValue

Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.

ExecutionStateChangeDetails#

ExecutionStateChangeDetails type fields#

Field

Type

Label

Description

state

ExecutionState

The state of the execution is used to control its visibility in the UI/CLI.

occurred_at

Timestamp

This timestamp represents when the state changed.

principal

string

Identifies the entity (if any) responsible for causing the state change of the execution

ExecutionTerminateRequest#

Request to terminate an in-progress execution. This action is irreversible. If an execution is already terminated, this request will simply be a no-op. This request will fail if it references a non-existent execution. If the request succeeds the phase “ABORTED” will be recorded for the termination with the optional cause added to the output_result.

ExecutionTerminateRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Uniquely identifies the individual workflow execution to be terminated.

cause

string

Optional reason for aborting.

ExecutionTerminateResponse#

Purposefully empty, may be populated in the future.

ExecutionUpdateRequest#

ExecutionUpdateRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Identifier of the execution to update

state

ExecutionState

State to set as the new value active/archive

ExecutionUpdateResponse#

LiteralMapBlob#

Input/output data can represented by actual values or a link to where values are stored

LiteralMapBlob type fields#

Field

Type

Label

Description

values

LiteralMap

Deprecated. Data in LiteralMap format

uri

string

In the event that the map is too large, we return a uri to the data

NotificationList#

NotificationList type fields#

Field

Type

Label

Description

notifications

Notification

repeated

SystemMetadata#

Represents system, rather than user-facing, metadata about an execution.

SystemMetadata type fields#

Field

Type

Label

Description

execution_cluster

string

Which execution cluster this execution ran on.

WorkflowExecutionGetDataRequest#

Request structure to fetch inputs, output and other data produced by an execution. By default this data is not returned inline in WorkflowExecutionGetRequest

WorkflowExecutionGetDataRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

The identifier of the execution for which to fetch inputs and outputs.

WorkflowExecutionGetDataResponse#

Response structure for WorkflowExecutionGetDataRequest which contains inputs and outputs for an execution.

WorkflowExecutionGetDataResponse type fields#

Field

Type

Label

Description

outputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of execution outputs. Deprecated: Please use full_outputs instead.

inputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of execution inputs. Deprecated: Please use full_inputs instead.

full_inputs

LiteralMap

Full_inputs will only be populated if they are under a configured size threshold.

full_outputs

LiteralMap

Full_outputs will only be populated if they are under a configured size threshold.

WorkflowExecutionGetRequest#

A message used to fetch a single workflow execution entity. See Execution for more details

WorkflowExecutionGetRequest type fields#

Field

Type

Label

Description

id

WorkflowExecutionIdentifier

Uniquely identifies an individual workflow execution.

ExecutionMetadata.ExecutionMode#

The method by which this execution was launched.

Enum ExecutionMetadata.ExecutionMode values#

Name

Number

Description

MANUAL

0

The default execution mode, MANUAL implies that an execution was launched by an individual.

SCHEDULED

1

A schedule triggered this execution launch.

SYSTEM

2

A system process was responsible for launching this execution rather an individual.

RELAUNCH

3

This execution was launched with identical inputs as a previous execution.

CHILD_WORKFLOW

4

This execution was triggered by another execution.

RECOVERED

5

This execution was recovered from another execution.

ExecutionState#

The state of the execution is used to control its visibility in the UI/CLI.

Enum ExecutionState values#

Name

Number

Description

EXECUTION_ACTIVE

0

By default, all executions are considered active.

EXECUTION_ARCHIVED

1

Archived executions are no longer visible in the UI.

flyteidl/admin/launch_plan.proto#

ActiveLaunchPlanListRequest#

Represents a request structure to list active launch plans within a project/domain. See LaunchPlan for more details

ActiveLaunchPlanListRequest type fields#

Field

Type

Label

Description

project

string

Name of the project that contains the identifiers. +required.

domain

string

Name of the domain the identifiers belongs to within the project. +required.

limit

uint32

Indicates the number of resources to be returned. +required.

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional

sort_by

Sort

Sort ordering. +optional

ActiveLaunchPlanRequest#

Represents a request struct for finding an active launch plan for a given NamedEntityIdentifier See LaunchPlan for more details

ActiveLaunchPlanRequest type fields#

Field

Type

Label

Description

id

NamedEntityIdentifier

+required.

Auth#

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

Auth type fields#

Field

Type

Label

Description

assumable_iam_role

string

Defines an optional iam role which will be used for tasks run in executions created with this launch plan.

kubernetes_service_account

string

Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

LaunchPlan#

A LaunchPlan provides the capability to templatize workflow executions. Launch plans simplify associating one or more schedules, inputs and notifications with your workflows. Launch plans can be shared and used to trigger executions with predefined inputs even when a workflow definition doesn’t necessarily have a default value for said input.

LaunchPlan type fields#

Field

Type

Label

Description

id

Identifier

Uniquely identifies a launch plan entity.

spec

LaunchPlanSpec

User-provided launch plan details, including reference workflow, inputs and other metadata.

closure

LaunchPlanClosure

Values computed by the flyte platform after launch plan registration.

LaunchPlanClosure#

Values computed by the flyte platform after launch plan registration. These include expected_inputs required to be present in a CreateExecutionRequest to launch the reference workflow as well timestamp values associated with the launch plan.

LaunchPlanClosure type fields#

Field

Type

Label

Description

state

LaunchPlanState

Indicate the Launch plan state.

expected_inputs

ParameterMap

Indicates the set of inputs expected when creating an execution with the Launch plan

expected_outputs

VariableMap

Indicates the set of outputs expected to be produced by creating an execution with the Launch plan

created_at

Timestamp

Time at which the launch plan was created.

updated_at

Timestamp

Time at which the launch plan was last updated.

LaunchPlanCreateRequest#

Request to register a launch plan. The included LaunchPlanSpec may have a complete or incomplete set of inputs required to launch a workflow execution. By default all launch plans are registered in state INACTIVE. If you wish to set the state to ACTIVE, you must submit a LaunchPlanUpdateRequest, after you have successfully created a launch plan.

LaunchPlanCreateRequest type fields#

Field

Type

Label

Description

id

Identifier

Uniquely identifies a launch plan entity.

spec

LaunchPlanSpec

User-provided launch plan details, including reference workflow, inputs and other metadata.

LaunchPlanCreateResponse#

Purposefully empty, may be populated in the future.

LaunchPlanList#

Response object for list launch plan requests. See LaunchPlan for more details

LaunchPlanList type fields#

Field

Type

Label

Description

launch_plans

LaunchPlan

repeated

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

LaunchPlanMetadata#

Additional launch plan attributes included in the LaunchPlanSpec not strictly required to launch the reference workflow.

LaunchPlanMetadata type fields#

Field

Type

Label

Description

schedule

Schedule

Schedule to execute the Launch Plan

notifications

Notification

repeated

List of notifications based on Execution status transitions

LaunchPlanSpec#

User-provided launch plan definition and configuration values.

LaunchPlanSpec type fields#

Field

Type

Label

Description

workflow_id

Identifier

Reference to the Workflow template that the launch plan references

entity_metadata

LaunchPlanMetadata

Metadata for the Launch Plan

default_inputs

ParameterMap

Input values to be passed for the execution. These can be overriden when an execution is created with this launch plan.

fixed_inputs

LiteralMap

Fixed, non-overridable inputs for the Launch Plan. These can not be overriden when an execution is created with this launch plan.

role

string

Deprecated. String to indicate the role to use to execute the workflow underneath

labels

Labels

Custom labels to be applied to the execution resource.

annotations

Annotations

Custom annotations to be applied to the execution resource.

auth

Auth

Deprecated. Indicates the permission associated with workflow executions triggered with this launch plan.

auth_role

AuthRole

Deprecated.

security_context

SecurityContext

Indicates security context for permissions triggered with this launch plan

quality_of_service

QualityOfService

Indicates the runtime priority of the execution.

raw_output_data_config

RawOutputDataConfig

Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).

max_parallelism

int32

Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.

interruptible

BoolValue

Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.

LaunchPlanUpdateRequest#

Request to set the referenced launch plan state to the configured value. See LaunchPlan for more details

LaunchPlanUpdateRequest type fields#

Field

Type

Label

Description

id

Identifier

Identifier of launch plan for which to change state. +required.

state

LaunchPlanState

Desired state to apply to the launch plan. +required.

LaunchPlanUpdateResponse#

Purposefully empty, may be populated in the future.

LaunchPlanState#

By default any launch plan regardless of state can be used to launch a workflow execution. However, at most one version of a launch plan (e.g. a NamedEntityIdentifier set of shared project, domain and name values) can be active at a time in regards to schedules. That is, at most one schedule in a NamedEntityIdentifier group will be observed and trigger executions at a defined cadence.

Enum LaunchPlanState values#

Name

Number

Description

INACTIVE

0

ACTIVE

1

flyteidl/admin/matchable_resource.proto#

ClusterResourceAttributes#

ClusterResourceAttributes type fields#

Field

Type

Label

Description

attributes

ClusterResourceAttributes.AttributesEntry

repeated

Custom resource attributes which will be applied in cluster resource creation (e.g. quotas). Map keys are the case-sensitive names of variables in templatized resource files. Map values should be the custom values which get substituted during resource creation.

ClusterResourceAttributes.AttributesEntry#

ClusterResourceAttributes.AttributesEntry type fields#

Field

Type

Label

Description

key

string

value

string

ExecutionClusterLabel#

ExecutionClusterLabel type fields#

Field

Type

Label

Description

value

string

Label value to determine where the execution will be run

ExecutionQueueAttributes#

ExecutionQueueAttributes type fields#

Field

Type

Label

Description

tags

string

repeated

Tags used for assigning execution queues for tasks defined within this project.

ListMatchableAttributesRequest#

Request all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details

ListMatchableAttributesRequest type fields#

Field

Type

Label

Description

resource_type

MatchableResource

+required

ListMatchableAttributesResponse#

Response for a request for all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details

ListMatchableAttributesResponse type fields#

Field

Type

Label

Description

configurations

MatchableAttributesConfiguration

repeated

MatchableAttributesConfiguration#

Represents a custom set of attributes applied for either a domain; a domain and project; or domain, project and workflow name. These are used to override system level defaults for kubernetes cluster resource management, default execution values, and more all across different levels of specificity.

MatchableAttributesConfiguration type fields#

Field

Type

Label

Description

attributes

MatchingAttributes

domain

string

project

string

workflow

string

launch_plan

string

MatchingAttributes#

Generic container for encapsulating all types of the above attributes messages.

MatchingAttributes type fields#

Field

Type

Label

Description

task_resource_attributes

TaskResourceAttributes

cluster_resource_attributes

ClusterResourceAttributes

execution_queue_attributes

ExecutionQueueAttributes

execution_cluster_label

ExecutionClusterLabel

quality_of_service

QualityOfService

plugin_overrides

PluginOverrides

workflow_execution_config

WorkflowExecutionConfig

cluster_assignment

ClusterAssignment

PluginOverride#

This MatchableAttribute configures selecting alternate plugin implementations for a given task type. In addition to an override implementation a selection of fallbacks can be provided or other modes for handling cases where the desired plugin override is not enabled in a given Flyte deployment.

PluginOverride type fields#

Field

Type

Label

Description

task_type

string

A predefined yet extensible Task type identifier.

plugin_id

string

repeated

A set of plugin ids which should handle tasks of this type instead of the default registered plugin. The list will be tried in order until a plugin is found with that id.

missing_plugin_behavior

PluginOverride.MissingPluginBehavior

Defines the behavior when no plugin from the plugin_id list is not found.

PluginOverrides#

PluginOverrides type fields#

Field

Type

Label

Description

overrides

PluginOverride

repeated

TaskResourceAttributes#

Defines task resource defaults and limits that will be applied at task registration.

TaskResourceAttributes type fields#

Field

Type

Label

Description

defaults

TaskResourceSpec

limits

TaskResourceSpec

TaskResourceSpec#

Defines a set of overridable task resource attributes set during task registration.

TaskResourceSpec type fields#

Field

Type

Label

Description

cpu

string

gpu

string

memory

string

storage

string

ephemeral_storage

string

WorkflowExecutionConfig#

Adds defaults for customizable workflow-execution specifications and overrides.

WorkflowExecutionConfig type fields#

Field

Type

Label

Description

max_parallelism

int32

Can be used to control the number of parallel nodes to run within the workflow. This is useful to achieve fairness.

security_context

SecurityContext

Indicates security context permissions for executions triggered with this matchable attribute.

raw_output_data_config

RawOutputDataConfig

Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).

labels

Labels

Custom labels to be applied to a triggered execution resource.

annotations

Annotations

Custom annotations to be applied to a triggered execution resource.

interruptible

BoolValue

Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.

MatchableResource#

Defines a resource that can be configured by customizable Project-, ProjectDomain- or WorkflowAttributes based on matching tags.

Enum MatchableResource values#

Name

Number

Description

TASK_RESOURCE

0

Applies to customizable task resource requests and limits.

CLUSTER_RESOURCE

1

Applies to configuring templated kubernetes cluster resources.

EXECUTION_QUEUE

2

Configures task and dynamic task execution queue assignment.

EXECUTION_CLUSTER_LABEL

3

Configures the K8s cluster label to be used for execution to be run

QUALITY_OF_SERVICE_SPECIFICATION

4

Configures default quality of service when undefined in an execution spec.

PLUGIN_OVERRIDE

5

Selects configurable plugin implementation behavior for a given task type.

WORKFLOW_EXECUTION_CONFIG

6

Adds defaults for customizable workflow-execution specifications and overrides.

CLUSTER_ASSIGNMENT

7

Controls how to select an available cluster on which this execution should run.

PluginOverride.MissingPluginBehavior#

Enum PluginOverride.MissingPluginBehavior values#

Name

Number

Description

FAIL

0

By default, if this plugin is not enabled for a Flyte deployment then execution will fail.

USE_DEFAULT

1

Uses the system-configured default implementation.

flyteidl/admin/node_execution.proto#

DynamicWorkflowNodeMetadata#

For dynamic workflow nodes we capture information about the dynamic workflow definition that gets generated.

DynamicWorkflowNodeMetadata type fields#

Field

Type

Label

Description

id

Identifier

id represents the unique identifier of the workflow.

compiled_workflow

CompiledWorkflowClosure

Represents the compiled representation of the embedded dynamic workflow.

NodeExecution#

Encapsulates all details for a single node execution entity. A node represents a component in the overall workflow graph. A node launch a task, multiple tasks, an entire nested sub-workflow, or even a separate child-workflow execution. The same task can be called repeatedly in a single workflow but each node is unique.

NodeExecution type fields#

Field

Type

Label

Description

id

NodeExecutionIdentifier

Uniquely identifies an individual node execution.

input_uri

string

Path to remote data store where input blob is stored.

closure

NodeExecutionClosure

Computed results associated with this node execution.

metadata

NodeExecutionMetaData

Metadata for Node Execution

NodeExecutionClosure#

Container for node execution details and results.

NodeExecutionClosure type fields#

Field

Type

Label

Description

output_uri

string

Deprecated. Links to a remotely stored, serialized core.LiteralMap of node execution outputs. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.

error

ExecutionError

Error information for the Node

output_data

LiteralMap

Deprecated. Raw output data produced by this node execution. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.

phase

NodeExecution.Phase

The last recorded phase for this node execution.

started_at

Timestamp

Time at which the node execution began running.

duration

Duration

The amount of time the node execution spent running.

created_at

Timestamp

Time at which the node execution was created.

updated_at

Timestamp

Time at which the node execution was last updated.

workflow_node_metadata

WorkflowNodeMetadata

task_node_metadata

TaskNodeMetadata

deck_uri

string

String location uniquely identifying where the deck HTML file is. NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)

NodeExecutionForTaskListRequest#

Represents a request structure to retrieve a list of node execution entities launched by a specific task. This can arise when a task yields a subworkflow.

NodeExecutionForTaskListRequest type fields#

Field

Type

Label

Description

task_execution_id

TaskExecutionIdentifier

Indicates the node execution to filter by. +required

limit

uint32

Indicates the number of resources to be returned. +required

token

string

In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional

filters

string

Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional

sort_by

Sort

Sort ordering. +optional

NodeExecutionGetDataRequest#

Request structure to fetch inputs and output for a node execution. By default, these are not returned in NodeExecutionGetRequest

NodeExecutionGetDataRequest type fields#

Field

Type

Label

Description

id

NodeExecutionIdentifier

The identifier of the node execution for which to fetch inputs and outputs.

NodeExecutionGetDataResponse#

Response structure for NodeExecutionGetDataRequest which contains inputs and outputs for a node execution.

NodeExecutionGetDataResponse type fields#

Field

Type

Label

Description

inputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of node execution inputs. Deprecated: Please use full_inputs instead.

outputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of node execution outputs. Deprecated: Please use full_outputs instead.

full_inputs

LiteralMap

Full_inputs will only be populated if they are under a configured size threshold.

full_outputs

LiteralMap

Full_outputs will only be populated if they are under a configured size threshold.

dynamic_workflow

DynamicWorkflowNodeMetadata

Optional Workflow closure for a dynamically generated workflow, in the case this node yields a dynamic workflow we return its structure here.

NodeExecutionGetRequest#

A message used to fetch a single node execution entity. See NodeExecution for more details

NodeExecutionGetRequest type fields#

Field

Type

Label

Description

id

NodeExecutionIdentifier

Uniquely identifies an individual node execution. +required

NodeExecutionList#

Request structure to retrieve a list of node execution entities. See NodeExecution for more details

NodeExecutionList type fields#

Field

Type

Label

Description

node_executions

NodeExecution

repeated

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NodeExecutionListRequest#

Represents a request structure to retrieve a list of node execution entities. See NodeExecution for more details

NodeExecutionListRequest type fields#

Field

Type

Label

Description

workflow_execution_id

WorkflowExecutionIdentifier

Indicates the workflow execution to filter by. +required

limit

uint32

Indicates the number of resources to be returned. +required

token

string

filters

string

Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional

sort_by

Sort

Sort ordering. +optional

unique_parent_id

string

Unique identifier of the parent node in the execution +optional

NodeExecutionMetaData#

Represents additional attributes related to a Node Execution

NodeExecutionMetaData type fields#

Field

Type

Label

Description

retry_group

string

Node executions are grouped depending on retries of the parent Retry group is unique within the context of a parent node.

is_parent_node

bool

Boolean flag indicating if the node has child nodes under it This can be true when a node contains a dynamic workflow which then produces child nodes.

spec_node_id

string

Node id of the node in the original workflow This maps to value of WorkflowTemplate.nodes[X].id

is_dynamic

bool

Boolean flag indicating if the node has contains a dynamic workflow which then produces child nodes. This is to distinguish between subworkflows and dynamic workflows which can both have is_parent_node as true.

TaskNodeMetadata#

Metadata for the case in which the node is a TaskNode

TaskNodeMetadata type fields#

Field

Type

Label

Description

cache_status

CatalogCacheStatus

Captures the status of caching for this execution.

catalog_key

CatalogMetadata

This structure carries the catalog artifact information

WorkflowNodeMetadata#

Metadata for a WorkflowNode

WorkflowNodeMetadata type fields#

Field

Type

Label

Description

executionId

WorkflowExecutionIdentifier

The identifier for a workflow execution launched by a node.

flyteidl/admin/notification.proto#

EmailMessage#

Represents the Email object that is sent to a publisher/subscriber to forward the notification. Note: This is internal to Admin and doesn’t need to be exposed to other components.

EmailMessage type fields#

Field

Type

Label

Description

recipients_email

string

repeated

The list of email addresses to receive an email with the content populated in the other fields. Currently, each email recipient will receive its own email. This populates the TO field.

sender_email

string

The email of the sender. This populates the FROM field.

subject_line

string

The content of the subject line. This populates the SUBJECT field.

body

string

The content of the email body. This populates the BODY field.

flyteidl/admin/project.proto#

Domain#

Namespace within a project commonly used to differentiate between different service instances. e.g. “production”, “development”, etc.

Domain type fields#

Field

Type

Label

Description

id

string

Globally unique domain name.

name

string

Display name.

Project#

Top-level namespace used to classify different entities like workflows and executions.

Project type fields#

Field

Type

Label

Description

id

string

Globally unique project name.

name

string

Display name.

domains

Domain

repeated

description

string

labels

Labels

Leverage Labels from flyteidel.admin.common.proto to tag projects with ownership information.

state

Project.ProjectState

ProjectListRequest#

Request to retrieve a list of projects matching specified filters. See Project for more details

ProjectListRequest type fields#

Field

Type

Label

Description

limit

uint32

Indicates the number of projects to be returned. +required

token

string

In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional

filters

string

Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional

sort_by

Sort

Sort ordering. +optional

ProjectRegisterRequest#

Adds a new user-project within the Flyte deployment. See Project for more details

ProjectRegisterRequest type fields#

Field

Type

Label

Description

project

Project

+required

ProjectRegisterResponse#

Purposefully empty, may be updated in the future.

ProjectUpdateResponse#

Purposefully empty, may be updated in the future.

Projects#

Represents a list of projects. See Project for more details

Projects type fields#

Field

Type

Label

Description

projects

Project

repeated

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

Project.ProjectState#

The state of the project is used to control its visibility in the UI and validity.

Enum Project.ProjectState values#

Name

Number

Description

ACTIVE

0

By default, all projects are considered active.

ARCHIVED

1

Archived projects are no longer visible in the UI and no longer valid.

SYSTEM_GENERATED

2

System generated projects that aren’t explicitly created or managed by a user.

flyteidl/admin/project_domain_attributes.proto#

ProjectDomainAttributes#

Defines a set of custom matching attributes which defines resource defaults for a project and domain. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributes type fields#

Field

Type

Label

Description

project

string

Unique project id for which this set of attributes will be applied.

domain

string

Unique domain id for which this set of attributes will be applied.

matching_attributes

MatchingAttributes

ProjectDomainAttributesDeleteRequest#

Request to delete a set matchable project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesDeleteRequest type fields#

Field

Type

Label

Description

project

string

Unique project id which this set of attributes references. +required

domain

string

Unique domain id which this set of attributes references. +required

resource_type

MatchableResource

Which type of matchable attributes to delete. +required

ProjectDomainAttributesDeleteResponse#

Purposefully empty, may be populated in the future.

ProjectDomainAttributesGetRequest#

Request to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesGetRequest type fields#

Field

Type

Label

Description

project

string

Unique project id which this set of attributes references. +required

domain

string

Unique domain id which this set of attributes references. +required

resource_type

MatchableResource

Which type of matchable attributes to return. +required

ProjectDomainAttributesGetResponse#

Response to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesGetResponse type fields#

Field

Type

Label

Description

attributes

ProjectDomainAttributes

ProjectDomainAttributesUpdateRequest#

Sets custom attributes for a project-domain combination. For more info on matchable attributes, see MatchableAttributesConfiguration

ProjectDomainAttributesUpdateRequest type fields#

Field

Type

Label

Description

attributes

ProjectDomainAttributes

+required

ProjectDomainAttributesUpdateResponse#

Purposefully empty, may be populated in the future.

flyteidl/admin/schedule.proto#

CronSchedule#

Options for schedules to run according to a cron expression.

CronSchedule type fields#

Field

Type

Label

Description

schedule

string

Standard/default cron implementation as described by https://en.wikipedia.org/wiki/Cron#CRON_expression; Also supports nonstandard predefined scheduling definitions as described by https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#CronExpressions except @reboot

offset

string

ISO 8601 duration as described by https://en.wikipedia.org/wiki/ISO_8601#Durations

FixedRate#

Option for schedules run at a certain frequency e.g. every 2 minutes.

FixedRate type fields#

Field

Type

Label

Description

value

uint32

unit

FixedRateUnit

Schedule#

Defines complete set of information required to trigger an execution on a schedule.

Schedule type fields#

Field

Type

Label

Description

cron_expression

string

Deprecated. Uses AWS syntax: Minutes Hours Day-of-month Month Day-of-week Year e.g. for a schedule that runs every 15 minutes: 0/15 * * * ? *

rate

FixedRate

cron_schedule

CronSchedule

kickoff_time_input_arg

string

Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off.

FixedRateUnit#

Represents a frequency at which to run a schedule.

Enum FixedRateUnit values#

Name

Number

Description

MINUTE

0

HOUR

1

DAY

2

flyteidl/admin/task.proto#

Task#

Flyte workflows are composed of many ordered tasks. That is small, reusable, self-contained logical blocks arranged to process workflow inputs and produce a deterministic set of outputs. Tasks can come in many varieties tuned for specialized behavior.

Task type fields#

Field

Type

Label

Description

id

Identifier

id represents the unique identifier of the task.

closure

TaskClosure

closure encapsulates all the fields that maps to a compiled version of the task.

TaskClosure#

Compute task attributes which include values derived from the TaskSpec, as well as plugin-specific data and task metadata.

TaskClosure type fields#

Field

Type

Label

Description

compiled_task

CompiledTask

Represents the compiled representation of the task from the specification provided.

created_at

Timestamp

Time at which the task was created.

TaskCreateRequest#

Represents a request structure to create a revision of a task. See Task for more details

TaskCreateRequest type fields#

Field

Type

Label

Description

id

Identifier

id represents the unique identifier of the task. +required

spec

TaskSpec

Represents the specification for task. +required

TaskCreateResponse#

Represents a response structure if task creation succeeds.

Purposefully empty, may be populated in the future.

TaskList#

Represents a list of tasks returned from the admin. See Task for more details

TaskList type fields#

Field

Type

Label

Description

tasks

Task

repeated

A list of tasks returned based on the request.

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskSpec#

Represents a structure that encapsulates the user-configured specification of the task.

TaskSpec type fields#

Field

Type

Label

Description

template

TaskTemplate

Template of the task that encapsulates all the metadata of the task.

flyteidl/admin/task_execution.proto#

TaskExecution#

Encapsulates all details for a single task execution entity. A task execution represents an instantiated task, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes.

TaskExecution type fields#

Field

Type

Label

Description

id

TaskExecutionIdentifier

Unique identifier for the task execution.

input_uri

string

Path to remote data store where input blob is stored.

closure

TaskExecutionClosure

Task execution details and results.

is_parent

bool

Whether this task spawned nodes.

TaskExecutionClosure#

Container for task execution details and results.

TaskExecutionClosure type fields#

Field

Type

Label

Description

output_uri

string

Deprecated. Path to remote data store where output blob is stored if the execution succeeded (and produced outputs). DEPRECATED. Use GetTaskExecutionData to fetch output data instead.

error

ExecutionError

Error information for the task execution. Populated if the execution failed.

output_data

LiteralMap

Deprecated. Raw output data produced by this task execution. DEPRECATED. Use GetTaskExecutionData to fetch output data instead.

phase

TaskExecution.Phase

The last recorded phase for this task execution.

logs

TaskLog

repeated

Detailed log information output by the task execution.

started_at

Timestamp

Time at which the task execution began running.

duration

Duration

The amount of time the task execution spent running.

created_at

Timestamp

Time at which the task execution was created.

updated_at

Timestamp

Time at which the task execution was last updated.

custom_info

Struct

Custom data specific to the task plugin.

reason

string

If there is an explanation for the most recent phase transition, the reason will capture it.

task_type

string

A predefined yet extensible Task type identifier.

metadata

TaskExecutionMetadata

Metadata around how a task was executed.

event_version

int32

The event version is used to indicate versioned changes in how data is maintained using this proto message. For example, event_verison > 0 means that maps tasks logs use the TaskExecutionMetadata ExternalResourceInfo fields for each subtask rather than the TaskLog in this message.

TaskExecutionGetDataRequest#

Request structure to fetch inputs and output for a task execution. By default this data is not returned inline in TaskExecutionGetRequest

TaskExecutionGetDataRequest type fields#

Field

Type

Label

Description

id

TaskExecutionIdentifier

The identifier of the task execution for which to fetch inputs and outputs. +required

TaskExecutionGetDataResponse#

Response structure for TaskExecutionGetDataRequest which contains inputs and outputs for a task execution.

TaskExecutionGetDataResponse type fields#

Field

Type

Label

Description

inputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of task execution inputs. Deprecated: Please use full_inputs instead.

outputs

UrlBlob

Deprecated. Signed url to fetch a core.LiteralMap of task execution outputs. Deprecated: Please use full_outputs instead.

full_inputs

LiteralMap

Full_inputs will only be populated if they are under a configured size threshold.

full_outputs

LiteralMap

Full_outputs will only be populated if they are under a configured size threshold.

TaskExecutionGetRequest#

A message used to fetch a single task execution entity. See TaskExecution for more details

TaskExecutionGetRequest type fields#

Field

Type

Label

Description

id

TaskExecutionIdentifier

Unique identifier for the task execution. +required

TaskExecutionList#

Response structure for a query to list of task execution entities. See TaskExecution for more details

TaskExecutionList type fields#

Field

Type

Label

Description

task_executions

TaskExecution

repeated

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskExecutionListRequest#

Represents a request structure to retrieve a list of task execution entities yielded by a specific node execution. See TaskExecution for more details

TaskExecutionListRequest type fields#

Field

Type

Label

Description

node_execution_id

NodeExecutionIdentifier

Indicates the node execution to filter by. +required

limit

uint32

Indicates the number of resources to be returned. +required

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional

filters

string

Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional

sort_by

Sort

Sort ordering for returned list. +optional

flyteidl/admin/version.proto#

GetVersionRequest#

Empty request for GetVersion

GetVersionResponse#

Response for the GetVersion API

GetVersionResponse type fields#

Field

Type

Label

Description

control_plane_version

Version

The control plane version information. FlyteAdmin and related components form the control plane of Flyte

Version#

Provides Version information for a component

Version type fields#

Field

Type

Label

Description

Build

string

Specifies the GIT sha of the build

Version

string

Version for the build, should follow a semver

BuildTime

string

Build timestamp

flyteidl/admin/workflow.proto#

Workflow#

Represents the workflow structure stored in the Admin A workflow is created by ordering tasks and associating outputs to inputs in order to produce a directed-acyclic execution graph.

Workflow type fields#

Field

Type

Label

Description

id

Identifier

id represents the unique identifier of the workflow.

closure

WorkflowClosure

closure encapsulates all the fields that maps to a compiled version of the workflow.

WorkflowClosure#

A container holding the compiled workflow produced from the WorkflowSpec and additional metadata.

WorkflowClosure type fields#

Field

Type

Label

Description

compiled_workflow

CompiledWorkflowClosure

Represents the compiled representation of the workflow from the specification provided.

created_at

Timestamp

Time at which the workflow was created.

WorkflowCreateRequest#

Represents a request structure to create a revision of a workflow. See Workflow for more details

WorkflowCreateRequest type fields#

Field

Type

Label

Description

id

Identifier

id represents the unique identifier of the workflow. +required

spec

WorkflowSpec

Represents the specification for workflow. +required

WorkflowCreateResponse#

Purposefully empty, may be populated in the future.

WorkflowList#

Represents a list of workflows returned from the admin. See Workflow for more details

WorkflowList type fields#

Field

Type

Label

Description

workflows

Workflow

repeated

A list of workflows returned based on the request.

token

string

In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

WorkflowSpec#

Represents a structure that encapsulates the specification of the workflow.

WorkflowSpec type fields#

Field

Type

Label

Description

template

WorkflowTemplate

Template of the task that encapsulates all the metadata of the workflow.

sub_workflows

WorkflowTemplate

repeated

Workflows that are embedded into other workflows need to be passed alongside the parent workflow to the propeller compiler (since the compiler doesn’t have any knowledge of other workflows - ie, it doesn’t reach out to Admin to see other registered workflows). In fact, subworkflows do not even need to be registered.

flyteidl/admin/workflow_attributes.proto#

WorkflowAttributes#

Defines a set of custom matching attributes which defines resource defaults for a project, domain and workflow. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributes type fields#

Field

Type

Label

Description

project

string

Unique project id for which this set of attributes will be applied.

domain

string

Unique domain id for which this set of attributes will be applied.

workflow

string

Workflow name for which this set of attributes will be applied.

matching_attributes

MatchingAttributes

WorkflowAttributesDeleteRequest#

Request to delete a set matchable workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesDeleteRequest type fields#

Field

Type

Label

Description

project

string

Unique project id which this set of attributes references. +required

domain

string

Unique domain id which this set of attributes references. +required

workflow

string

Workflow name which this set of attributes references. +required

resource_type

MatchableResource

Which type of matchable attributes to delete. +required

WorkflowAttributesDeleteResponse#

Purposefully empty, may be populated in the future.

WorkflowAttributesGetRequest#

Request to get an individual workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesGetRequest type fields#

Field

Type

Label

Description

project

string

Unique project id which this set of attributes references. +required

domain

string

Unique domain id which this set of attributes references. +required

workflow

string

Workflow name which this set of attributes references. +required

resource_type

MatchableResource

Which type of matchable attributes to return. +required

WorkflowAttributesGetResponse#

Response to get an individual workflow attribute override.

WorkflowAttributesGetResponse type fields#

Field

Type

Label

Description

attributes

WorkflowAttributes

WorkflowAttributesUpdateRequest#

Sets custom attributes for a project, domain and workflow combination. For more info on matchable attributes, see MatchableAttributesConfiguration

WorkflowAttributesUpdateRequest type fields#

Field

Type

Label

Description

attributes

WorkflowAttributes

WorkflowAttributesUpdateResponse#

Purposefully empty, may be populated in the future.

google/protobuf/duration.proto#

Duration#

A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like “day” or “month”. It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

# Examples

Example 1: Compute Duration from two Timestamps in pseudo code.

Timestamp start = …; Timestamp end = …; Duration duration = …;

duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

if (duration.seconds < 0 && duration.nanos > 0) {

duration.seconds += 1; duration.nanos -= 1000000000;

} else if (duration.seconds > 0 && duration.nanos < 0) {

duration.seconds -= 1; duration.nanos += 1000000000;

}

Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

Timestamp start = …; Duration duration = …; Timestamp end = …;

end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

if (end.nanos < 0) {

end.seconds -= 1; end.nanos += 1000000000;

} else if (end.nanos >= 1000000000) {

end.seconds += 1; end.nanos -= 1000000000;

}

Example 3: Compute Duration from datetime.timedelta in Python.

td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

# JSON Mapping

In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix “s” (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as “3s”, while 3 seconds and 1 nanosecond should be expressed in JSON format as “3.000000001s”, and 3 seconds and 1 microsecond should be expressed in JSON format as “3.000001s”.

Duration type fields#

Field

Type

Label

Description

seconds

int64

Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years

nanos

int32

Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 seconds field and a positive or negative nanos field. For durations of one second or more, a non-zero value for the nanos field must be of the same sign as the seconds field. Must be from -999,999,999 to +999,999,999 inclusive.

google/protobuf/wrappers.proto#

BoolValue#

Wrapper message for bool.

The JSON representation for BoolValue is JSON true and false.

BoolValue type fields#

Field

Type

Label

Description

value

bool

The bool value.

BytesValue#

Wrapper message for bytes.

The JSON representation for BytesValue is JSON string.

BytesValue type fields#

Field

Type

Label

Description

value

bytes

The bytes value.

DoubleValue#

Wrapper message for double.

The JSON representation for DoubleValue is JSON number.

DoubleValue type fields#

Field

Type

Label

Description

value

double

The double value.

FloatValue#

Wrapper message for float.

The JSON representation for FloatValue is JSON number.

FloatValue type fields#

Field

Type

Label

Description

value

float

The float value.

Int32Value#

Wrapper message for int32.

The JSON representation for Int32Value is JSON number.

Int32Value type fields#

Field

Type

Label

Description

value

int32

The int32 value.

Int64Value#

Wrapper message for int64.

The JSON representation for Int64Value is JSON string.

Int64Value type fields#

Field

Type

Label

Description

value

int64

The int64 value.

StringValue#

Wrapper message for string.

The JSON representation for StringValue is JSON string.

StringValue type fields#

Field

Type

Label

Description

value

string

The string value.

UInt32Value#

Wrapper message for uint32.

The JSON representation for UInt32Value is JSON number.

UInt32Value type fields#

Field

Type

Label

Description

value

uint32

The uint32 value.

UInt64Value#

Wrapper message for uint64.

The JSON representation for UInt64Value is JSON string.

UInt64Value type fields#

Field

Type

Label

Description

value

uint64

The uint64 value.