Protocol Documentation#
flyteidl/admin/cluster_assignment.proto#
ClusterAssignment#
Encapsulates specifications for routing an execution onto a specific cluster.
flyteidl/admin/common.proto#
Annotations#
Annotation values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge annotations defined at registration and execution time.
Field |
Type |
Label |
Description |
---|---|---|---|
values |
repeated |
Map of custom annotations to be applied to the execution resource. |
Annotations.ValuesEntry#
AuthRole#
Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.
Field |
Type |
Label |
Description |
---|---|---|---|
assumable_iam_role |
Defines an optional iam role which will be used for tasks run in executions created with this launch plan. |
||
kubernetes_service_account |
Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan. |
EmailNotification#
Defines an email notification specification.
Labels#
Label values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge labels defined at registration and execution time.
Field |
Type |
Label |
Description |
---|---|---|---|
values |
repeated |
Map of custom labels to be applied to the execution resource. |
Labels.ValuesEntry#
NamedEntity#
Encapsulates information common to a NamedEntity, a Flyte resource such as a task, workflow or launch plan. A NamedEntity is exclusively identified by its resource type and identifier.
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
Resource type of the named entity. One of Task, Workflow or LaunchPlan. |
||
id |
|||
metadata |
Additional metadata around a named entity. |
NamedEntityGetRequest#
A request to retrieve the metadata associated with a NamedEntityIdentifier
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
Resource type of the metadata to get. One of Task, Workflow or LaunchPlan. +required |
||
id |
The identifier for the named entity for which to fetch metadata. +required |
NamedEntityIdentifier#
Encapsulation of fields that identifies a Flyte resource. A Flyte resource can be a task, workflow or launch plan. A resource can internally have multiple versions and is uniquely identified by project, domain, and name.
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Name of the project the resource belongs to. |
||
domain |
Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project. |
||
name |
User provided value for the resource. The combination of project + domain + name uniquely identifies the resource. +optional - in certain contexts - like ‘List API’, ‘Launch plans’ |
NamedEntityIdentifierList#
Represents a list of NamedEntityIdentifiers.
Field |
Type |
Label |
Description |
---|---|---|---|
entities |
repeated |
A list of identifiers. |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
NamedEntityIdentifierListRequest#
Represents a request structure to list NamedEntityIdentifiers.
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Name of the project that contains the identifiers. +required |
||
domain |
Name of the domain the identifiers belongs to within the project. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional |
||
sort_by |
Specifies how listed entities should be sorted in the response. +optional |
||
filters |
Indicates a list of filters passed as string. +optional |
NamedEntityList#
Represents a list of NamedEntityIdentifiers.
Field |
Type |
Label |
Description |
---|---|---|---|
entities |
repeated |
A list of NamedEntity objects |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
NamedEntityListRequest#
Represents a request structure to list NamedEntity objects
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
Resource type of the metadata to query. One of Task, Workflow or LaunchPlan. +required |
||
project |
Name of the project that contains the identifiers. +required |
||
domain |
Name of the domain the identifiers belongs to within the project. |
||
limit |
Indicates the number of resources to be returned. |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional |
||
sort_by |
Specifies how listed entities should be sorted in the response. +optional |
||
filters |
Indicates a list of filters passed as string. +optional |
NamedEntityMetadata#
Additional metadata around a named entity.
Field |
Type |
Label |
Description |
---|---|---|---|
description |
Common description across all versions of the entity +optional |
||
state |
Shared state across all version of the entity At this point in time, only workflow entities can have their state archived. |
NamedEntityUpdateRequest#
Request to set the referenced named entity state to the configured value.
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
Resource type of the metadata to update +required |
||
id |
Identifier of the metadata to update +required |
||
metadata |
Metadata object to set as the new value +required |
NamedEntityUpdateResponse#
Purposefully empty, may be populated in the future.
Notification#
Represents a structure for notifications based on execution status. The notification content is configured within flyte admin but can be templatized. Future iterations could expose configuring notifications with custom content.
Field |
Type |
Label |
Description |
---|---|---|---|
phases |
repeated |
A list of phases to which users can associate the notifications to. +required |
|
pager_duty |
|||
slack |
ObjectGetRequest#
Shared request structure to fetch a single resource. Resources include: Task, Workflow, LaunchPlan
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Indicates a unique version of resource. +required |
PagerDutyNotification#
Defines a pager duty notification specification.
RawOutputDataConfig#
Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). See flyteorg/flyte#211 for more background information.
ResourceListRequest#
Shared request structure to retrieve a list of resources. Resources include: Task, Workflow, LaunchPlan
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the resource. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering. +optional |
SlackNotification#
Defines a slack notification specification.
Sort#
Specifies sort ordering in a list request.
Field |
Type |
Label |
Description |
---|---|---|---|
key |
Indicates an attribute to sort the response values. +required |
||
direction |
Indicates the direction to apply sort key for response values. +optional |
UrlBlob#
Represents a string url and associated metadata used throughout the platform.
NamedEntityState#
The status of the named entity is used to control its visibility in the UI.
Name |
Number |
Description |
---|---|---|
NAMED_ENTITY_ACTIVE |
0 |
By default, all named entities are considered active and under development. |
NAMED_ENTITY_ARCHIVED |
1 |
Archived named entities are no longer visible in the UI. |
SYSTEM_GENERATED |
2 |
System generated entities that aren’t explicitly created or managed by a user. |
Sort.Direction#
Name |
Number |
Description |
---|---|---|
DESCENDING |
0 |
By default, fields are sorted in descending order. |
ASCENDING |
1 |
flyteidl/admin/description_entity.proto#
Description#
Full user description with formatting preserved. This can be rendered by clients, such as the console or command line tools with in-tact formatting.
Field |
Type |
Label |
Description |
---|---|---|---|
value |
long description - no more than 4KB |
||
uri |
if the description sizes exceed some threshold we can offload the entire description proto altogether to an external data store, like S3 rather than store inline in the db |
||
format |
Format of the long description |
||
icon_link |
Optional link to an icon for the entity |
DescriptionEntity#
DescriptionEntity contains detailed description for the task/workflow. Documentation could provide insight into the algorithms, business use case, etc.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the description entity. |
||
short_description |
One-liner overview of the entity. |
||
long_description |
Full user description with formatting preserved. |
||
source_code |
Optional link to source code used to define this entity. |
||
tags |
repeated |
User-specified tags. These are arbitrary and can be used for searching filtering and discovering tasks. |
DescriptionEntityList#
Represents a list of DescriptionEntities returned from the admin. See DescriptionEntity for more details
Field |
Type |
Label |
Description |
---|---|---|---|
descriptionEntities |
repeated |
A list of DescriptionEntities returned based on the request. |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
DescriptionEntityListRequest#
Represents a request structure to retrieve a list of DescriptionEntities. See DescriptionEntity for more details
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
Identifies the specific type of resource that this identifier corresponds to. |
||
id |
The identifier for the description entity. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering for returned list. +optional |
SourceCode#
Link to source code used to define this entity
DescriptionFormat#
The format of the long description
Name |
Number |
Description |
---|---|---|
DESCRIPTION_FORMAT_UNKNOWN |
0 |
|
DESCRIPTION_FORMAT_MARKDOWN |
1 |
|
DESCRIPTION_FORMAT_HTML |
2 |
|
DESCRIPTION_FORMAT_RST |
3 |
python default documentation - comments is rst |
flyteidl/admin/event.proto#
EventErrorAlreadyInTerminalState#
Indicates that a sent event was not used to update execution state due to the referenced execution already being terminated (and therefore ineligible for further state transitions).
EventErrorIncompatibleCluster#
Indicates an event was rejected because it came from a different cluster than is on record as running the execution.
EventFailureReason#
Indicates why a sent event was not used to update execution.
Field |
Type |
Label |
Description |
---|---|---|---|
already_in_terminal_state |
|||
incompatible_cluster |
NodeExecutionEventRequest#
Request to send a notification that a node execution event has occurred.
Field |
Type |
Label |
Description |
---|---|---|---|
request_id |
Unique ID for this request that can be traced between services |
||
event |
Details about the event that occurred. |
NodeExecutionEventResponse#
Purposefully empty, may be populated in the future.
TaskExecutionEventRequest#
Request to send a notification that a task execution event has occurred.
Field |
Type |
Label |
Description |
---|---|---|---|
request_id |
Unique ID for this request that can be traced between services |
||
event |
Details about the event that occurred. |
TaskExecutionEventResponse#
Purposefully empty, may be populated in the future.
WorkflowExecutionEventRequest#
Request to send a notification that a workflow execution event has occurred.
Field |
Type |
Label |
Description |
---|---|---|---|
request_id |
Unique ID for this request that can be traced between services |
||
event |
Details about the event that occurred. |
WorkflowExecutionEventResponse#
Purposefully empty, may be populated in the future.
flyteidl/admin/execution.proto#
AbortMetadata#
Specifies metadata around an aborted workflow execution.
Execution#
A workflow execution represents an instantiated workflow, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes. Used as a response object used in Get and List execution requests.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Unique identifier of the workflow execution. |
||
spec |
User-provided configuration and inputs for launching the execution. |
||
closure |
Execution results. |
ExecutionClosure#
Encapsulates the results of the Execution
Field |
Type |
Label |
Description |
---|---|---|---|
outputs |
Deprecated. Output URI in the case of a successful execution. DEPRECATED. Use GetExecutionData to fetch output data instead. |
||
error |
Error information in the case of a failed execution. |
||
abort_cause |
Deprecated. In the case of a user-specified abort, this will pass along the user-supplied cause. |
||
abort_metadata |
In the case of a user-specified abort, this will pass along the user and their supplied cause. |
||
output_data |
Deprecated. Raw output data produced by this execution. DEPRECATED. Use GetExecutionData to fetch output data instead. |
||
computed_inputs |
Deprecated. Inputs computed and passed for execution. computed_inputs depends on inputs in ExecutionSpec, fixed and default inputs in launch plan |
||
phase |
Most recent recorded phase for the execution. |
||
started_at |
Reported time at which the execution began running. |
||
duration |
The amount of time the execution spent running. |
||
created_at |
Reported time at which the execution was created. |
||
updated_at |
Reported time at which the execution was last updated. |
||
notifications |
repeated |
The notification settings to use after merging the CreateExecutionRequest and the launch plan notification settings. An execution launched with notifications will always prefer that definition to notifications defined statically in a launch plan. |
|
workflow_id |
Identifies the workflow definition for this execution. |
||
state_change_details |
Provides the details of the last stage change |
ExecutionCreateRequest#
Request to launch an execution with the given project, domain and optionally-assigned name.
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Name of the project the execution belongs to. +required |
||
domain |
Name of the domain the execution belongs to. A domain can be considered as a subset within a specific project. +required |
||
name |
User provided value for the resource. If none is provided the system will generate a unique string. +optional |
||
spec |
Additional fields necessary to launch the execution. +optional |
||
inputs |
The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional |
ExecutionCreateResponse#
The unique identifier for a successfully created execution. If the name was not specified in the create request, this identifier will include a generated name.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
ExecutionList#
Used as a response for request to list executions. See Execution for more details
ExecutionMetadata#
Represents attributes about an execution which are not required to launch the execution but are useful to record. These attributes are assigned at launch time and do not change.
Field |
Type |
Label |
Description |
---|---|---|---|
mode |
|||
principal |
Identifier of the entity that triggered this execution. For systems using back-end authentication any value set here will be discarded in favor of the authenticated user context. |
||
nesting |
Indicates the nestedness of this execution. If a user launches a workflow execution, the default nesting is 0. If this execution further launches a workflow (child workflow), the nesting level is incremented by 0 => 1 Generally, if workflow at nesting level k launches a workflow then the child workflow will have nesting = k + 1. |
||
scheduled_at |
For scheduled executions, the requested time for execution for this specific schedule invocation. |
||
parent_node_execution |
Which subworkflow node (if any) launched this execution |
||
reference_execution |
Optional, a reference workflow execution related to this execution. In the case of a relaunch, this references the original workflow execution. |
||
system_metadata |
Optional, platform-specific metadata about the execution. In this the future this may be gated behind an ACL or some sort of authorization. |
ExecutionRecoverRequest#
Request to recover the referenced execution.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Identifier of the workflow execution to recover. |
||
name |
User provided value for the recovered execution. If none is provided the system will generate a unique string. +optional |
||
metadata |
Additional metadata which will be used to overwrite any metadata in the reference execution when triggering a recovery execution. |
ExecutionRelaunchRequest#
Request to relaunch the referenced execution.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Identifier of the workflow execution to relaunch. +required |
||
name |
User provided value for the relaunched execution. If none is provided the system will generate a unique string. +optional |
||
overwrite_cache |
Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully. |
ExecutionSpec#
An ExecutionSpec encompasses all data used to launch this execution. The Spec does not change over the lifetime of an execution as it progresses across phase changes.
Field |
Type |
Label |
Description |
---|---|---|---|
launch_plan |
Launch plan to be executed |
||
inputs |
Deprecated. Input values to be passed for the execution |
||
metadata |
Metadata for the execution |
||
notifications |
List of notifications based on Execution status transitions When this list is not empty it is used rather than any notifications defined in the referenced launch plan. When this list is empty, the notifications defined for the launch plan will be applied. |
||
disable_all |
This should be set to true if all notifications are intended to be disabled for this execution. |
||
labels |
Labels to apply to the execution resource. |
||
annotations |
Annotations to apply to the execution resource. |
||
security_context |
Optional: security context override to apply this execution. |
||
auth_role |
Deprecated. Optional: auth override to apply this execution. |
||
quality_of_service |
Indicates the runtime priority of the execution. |
||
max_parallelism |
Controls the maximum number of task nodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this. |
||
raw_output_data_config |
User setting to configure where to store offloaded data (i.e. Blobs, structured datasets, query data, etc.). This should be a prefix like s3://my-bucket/my-data |
||
cluster_assignment |
Controls how to select an available cluster on which this execution should run. |
||
interruptible |
Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field. |
||
overwrite_cache |
Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully. |
ExecutionStateChangeDetails#
Field |
Type |
Label |
Description |
---|---|---|---|
state |
The state of the execution is used to control its visibility in the UI/CLI. |
||
occurred_at |
This timestamp represents when the state changed. |
||
principal |
Identifies the entity (if any) responsible for causing the state change of the execution |
ExecutionTerminateRequest#
Request to terminate an in-progress execution. This action is irreversible. If an execution is already terminated, this request will simply be a no-op. This request will fail if it references a non-existent execution. If the request succeeds the phase “ABORTED” will be recorded for the termination with the optional cause added to the output_result.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies the individual workflow execution to be terminated. |
||
cause |
Optional reason for aborting. |
ExecutionTerminateResponse#
Purposefully empty, may be populated in the future.
ExecutionUpdateRequest#
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Identifier of the execution to update |
||
state |
State to set as the new value active/archive |
ExecutionUpdateResponse#
LiteralMapBlob#
Input/output data can represented by actual values or a link to where values are stored
Field |
Type |
Label |
Description |
---|---|---|---|
values |
Deprecated. Data in LiteralMap format |
||
uri |
In the event that the map is too large, we return a uri to the data |
NotificationList#
Field |
Type |
Label |
Description |
---|---|---|---|
notifications |
repeated |
SystemMetadata#
Represents system, rather than user-facing, metadata about an execution.
WorkflowExecutionGetDataRequest#
Request structure to fetch inputs, output and other data produced by an execution. By default this data is not returned inline in WorkflowExecutionGetRequest
Field |
Type |
Label |
Description |
---|---|---|---|
id |
The identifier of the execution for which to fetch inputs and outputs. |
WorkflowExecutionGetDataResponse#
Response structure for WorkflowExecutionGetDataRequest which contains inputs and outputs for an execution.
Field |
Type |
Label |
Description |
---|---|---|---|
outputs |
Deprecated. Signed url to fetch a core.LiteralMap of execution outputs. Deprecated: Please use full_outputs instead. |
||
inputs |
Deprecated. Signed url to fetch a core.LiteralMap of execution inputs. Deprecated: Please use full_inputs instead. |
||
full_inputs |
Full_inputs will only be populated if they are under a configured size threshold. |
||
full_outputs |
Full_outputs will only be populated if they are under a configured size threshold. |
WorkflowExecutionGetRequest#
A message used to fetch a single workflow execution entity. See Execution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies an individual workflow execution. |
ExecutionMetadata.ExecutionMode#
The method by which this execution was launched.
Name |
Number |
Description |
---|---|---|
MANUAL |
0 |
The default execution mode, MANUAL implies that an execution was launched by an individual. |
SCHEDULED |
1 |
A schedule triggered this execution launch. |
SYSTEM |
2 |
A system process was responsible for launching this execution rather an individual. |
RELAUNCH |
3 |
This execution was launched with identical inputs as a previous execution. |
CHILD_WORKFLOW |
4 |
This execution was triggered by another execution. |
RECOVERED |
5 |
This execution was recovered from another execution. |
ExecutionState#
The state of the execution is used to control its visibility in the UI/CLI.
Name |
Number |
Description |
---|---|---|
EXECUTION_ACTIVE |
0 |
By default, all executions are considered active. |
EXECUTION_ARCHIVED |
1 |
Archived executions are no longer visible in the UI. |
flyteidl/admin/launch_plan.proto#
ActiveLaunchPlanListRequest#
Represents a request structure to list active launch plans within a project/domain. See LaunchPlan for more details
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Name of the project that contains the identifiers. +required. |
||
domain |
Name of the domain the identifiers belongs to within the project. +required. |
||
limit |
Indicates the number of resources to be returned. +required. |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional |
||
sort_by |
Sort ordering. +optional |
ActiveLaunchPlanRequest#
Represents a request struct for finding an active launch plan for a given NamedEntityIdentifier See LaunchPlan for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
+required. |
Auth#
Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.
Field |
Type |
Label |
Description |
---|---|---|---|
assumable_iam_role |
Defines an optional iam role which will be used for tasks run in executions created with this launch plan. |
||
kubernetes_service_account |
Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan. |
LaunchPlan#
A LaunchPlan provides the capability to templatize workflow executions. Launch plans simplify associating one or more schedules, inputs and notifications with your workflows. Launch plans can be shared and used to trigger executions with predefined inputs even when a workflow definition doesn’t necessarily have a default value for said input.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies a launch plan entity. |
||
spec |
User-provided launch plan details, including reference workflow, inputs and other metadata. |
||
closure |
Values computed by the flyte platform after launch plan registration. |
LaunchPlanClosure#
Values computed by the flyte platform after launch plan registration. These include expected_inputs required to be present in a CreateExecutionRequest to launch the reference workflow as well timestamp values associated with the launch plan.
Field |
Type |
Label |
Description |
---|---|---|---|
state |
Indicate the Launch plan state. |
||
expected_inputs |
Indicates the set of inputs expected when creating an execution with the Launch plan |
||
expected_outputs |
Indicates the set of outputs expected to be produced by creating an execution with the Launch plan |
||
created_at |
Time at which the launch plan was created. |
||
updated_at |
Time at which the launch plan was last updated. |
LaunchPlanCreateRequest#
Request to register a launch plan. The included LaunchPlanSpec may have a complete or incomplete set of inputs required to launch a workflow execution. By default all launch plans are registered in state INACTIVE. If you wish to set the state to ACTIVE, you must submit a LaunchPlanUpdateRequest, after you have successfully created a launch plan.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies a launch plan entity. |
||
spec |
User-provided launch plan details, including reference workflow, inputs and other metadata. |
LaunchPlanCreateResponse#
Purposefully empty, may be populated in the future.
LaunchPlanList#
Response object for list launch plan requests. See LaunchPlan for more details
Field |
Type |
Label |
Description |
---|---|---|---|
launch_plans |
repeated |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
LaunchPlanMetadata#
Additional launch plan attributes included in the LaunchPlanSpec not strictly required to launch the reference workflow.
Field |
Type |
Label |
Description |
---|---|---|---|
schedule |
Schedule to execute the Launch Plan |
||
notifications |
repeated |
List of notifications based on Execution status transitions |
LaunchPlanSpec#
User-provided launch plan definition and configuration values.
Field |
Type |
Label |
Description |
---|---|---|---|
workflow_id |
Reference to the Workflow template that the launch plan references |
||
entity_metadata |
Metadata for the Launch Plan |
||
default_inputs |
Input values to be passed for the execution. These can be overridden when an execution is created with this launch plan. |
||
fixed_inputs |
Fixed, non-overridable inputs for the Launch Plan. These can not be overridden when an execution is created with this launch plan. |
||
role |
Deprecated. String to indicate the role to use to execute the workflow underneath |
||
labels |
Custom labels to be applied to the execution resource. |
||
annotations |
Custom annotations to be applied to the execution resource. |
||
auth |
Deprecated. Indicates the permission associated with workflow executions triggered with this launch plan. |
||
auth_role |
Deprecated. |
||
security_context |
Indicates security context for permissions triggered with this launch plan |
||
quality_of_service |
Indicates the runtime priority of the execution. |
||
raw_output_data_config |
Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). |
||
max_parallelism |
Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this. |
||
interruptible |
Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field. |
||
overwrite_cache |
Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully. |
LaunchPlanUpdateRequest#
Request to set the referenced launch plan state to the configured value. See LaunchPlan for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Identifier of launch plan for which to change state. +required. |
||
state |
Desired state to apply to the launch plan. +required. |
LaunchPlanUpdateResponse#
Purposefully empty, may be populated in the future.
LaunchPlanState#
By default any launch plan regardless of state can be used to launch a workflow execution. However, at most one version of a launch plan (e.g. a NamedEntityIdentifier set of shared project, domain and name values) can be active at a time in regards to schedules. That is, at most one schedule in a NamedEntityIdentifier group will be observed and trigger executions at a defined cadence.
Name |
Number |
Description |
---|---|---|
INACTIVE |
0 |
|
ACTIVE |
1 |
flyteidl/admin/matchable_resource.proto#
ClusterResourceAttributes#
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
repeated |
Custom resource attributes which will be applied in cluster resource creation (e.g. quotas). Map keys are the case-sensitive names of variables in templatized resource files. Map values should be the custom values which get substituted during resource creation. |
ClusterResourceAttributes.AttributesEntry#
ExecutionClusterLabel#
ExecutionQueueAttributes#
ListMatchableAttributesRequest#
Request all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details
Field |
Type |
Label |
Description |
---|---|---|---|
resource_type |
+required |
ListMatchableAttributesResponse#
Response for a request for all matching resource attributes for a resource type. See MatchableAttributesConfiguration for more details
Field |
Type |
Label |
Description |
---|---|---|---|
configurations |
repeated |
MatchableAttributesConfiguration#
Represents a custom set of attributes applied for either a domain; a domain and project; or domain, project and workflow name. These are used to override system level defaults for kubernetes cluster resource management, default execution values, and more all across different levels of specificity.
MatchingAttributes#
Generic container for encapsulating all types of the above attributes messages.
Field |
Type |
Label |
Description |
---|---|---|---|
task_resource_attributes |
|||
cluster_resource_attributes |
|||
execution_queue_attributes |
|||
execution_cluster_label |
|||
quality_of_service |
|||
plugin_overrides |
|||
workflow_execution_config |
|||
cluster_assignment |
PluginOverride#
This MatchableAttribute configures selecting alternate plugin implementations for a given task type. In addition to an override implementation a selection of fallbacks can be provided or other modes for handling cases where the desired plugin override is not enabled in a given Flyte deployment.
Field |
Type |
Label |
Description |
---|---|---|---|
task_type |
A predefined yet extensible Task type identifier. |
||
plugin_id |
repeated |
A set of plugin ids which should handle tasks of this type instead of the default registered plugin. The list will be tried in order until a plugin is found with that id. |
|
missing_plugin_behavior |
Defines the behavior when no plugin from the plugin_id list is not found. |
PluginOverrides#
Field |
Type |
Label |
Description |
---|---|---|---|
overrides |
repeated |
TaskResourceAttributes#
Defines task resource defaults and limits that will be applied at task registration.
Field |
Type |
Label |
Description |
---|---|---|---|
defaults |
|||
limits |
TaskResourceSpec#
Defines a set of overridable task resource attributes set during task registration.
WorkflowExecutionConfig#
Adds defaults for customizable workflow-execution specifications and overrides.
Field |
Type |
Label |
Description |
---|---|---|---|
max_parallelism |
Can be used to control the number of parallel nodes to run within the workflow. This is useful to achieve fairness. |
||
security_context |
Indicates security context permissions for executions triggered with this matchable attribute. |
||
raw_output_data_config |
Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). |
||
labels |
Custom labels to be applied to a triggered execution resource. |
||
annotations |
Custom annotations to be applied to a triggered execution resource. |
||
interruptible |
Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field. |
||
overwrite_cache |
Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully. |
MatchableResource#
Defines a resource that can be configured by customizable Project-, ProjectDomain- or WorkflowAttributes based on matching tags.
Name |
Number |
Description |
---|---|---|
TASK_RESOURCE |
0 |
Applies to customizable task resource requests and limits. |
CLUSTER_RESOURCE |
1 |
Applies to configuring templated kubernetes cluster resources. |
EXECUTION_QUEUE |
2 |
Configures task and dynamic task execution queue assignment. |
EXECUTION_CLUSTER_LABEL |
3 |
Configures the K8s cluster label to be used for execution to be run |
QUALITY_OF_SERVICE_SPECIFICATION |
4 |
Configures default quality of service when undefined in an execution spec. |
PLUGIN_OVERRIDE |
5 |
Selects configurable plugin implementation behavior for a given task type. |
WORKFLOW_EXECUTION_CONFIG |
6 |
Adds defaults for customizable workflow-execution specifications and overrides. |
CLUSTER_ASSIGNMENT |
7 |
Controls how to select an available cluster on which this execution should run. |
PluginOverride.MissingPluginBehavior#
Name |
Number |
Description |
---|---|---|
FAIL |
0 |
By default, if this plugin is not enabled for a Flyte deployment then execution will fail. |
USE_DEFAULT |
1 |
Uses the system-configured default implementation. |
flyteidl/admin/node_execution.proto#
DynamicWorkflowNodeMetadata#
For dynamic workflow nodes we capture information about the dynamic workflow definition that gets generated.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the workflow. |
||
compiled_workflow |
Represents the compiled representation of the embedded dynamic workflow. |
NodeExecution#
Encapsulates all details for a single node execution entity. A node represents a component in the overall workflow graph. A node launch a task, multiple tasks, an entire nested sub-workflow, or even a separate child-workflow execution. The same task can be called repeatedly in a single workflow but each node is unique.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies an individual node execution. |
||
input_uri |
Path to remote data store where input blob is stored. |
||
closure |
Computed results associated with this node execution. |
||
metadata |
Metadata for Node Execution |
NodeExecutionClosure#
Container for node execution details and results.
Field |
Type |
Label |
Description |
---|---|---|---|
output_uri |
Deprecated. Links to a remotely stored, serialized core.LiteralMap of node execution outputs. DEPRECATED. Use GetNodeExecutionData to fetch output data instead. |
||
error |
Error information for the Node |
||
output_data |
Deprecated. Raw output data produced by this node execution. DEPRECATED. Use GetNodeExecutionData to fetch output data instead. |
||
phase |
The last recorded phase for this node execution. |
||
started_at |
Time at which the node execution began running. |
||
duration |
The amount of time the node execution spent running. |
||
created_at |
Time at which the node execution was created. |
||
updated_at |
Time at which the node execution was last updated. |
||
workflow_node_metadata |
|||
task_node_metadata |
|||
deck_uri |
String location uniquely identifying where the deck HTML file is. NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar) |
NodeExecutionForTaskListRequest#
Represents a request structure to retrieve a list of node execution entities launched by a specific task. This can arise when a task yields a subworkflow.
Field |
Type |
Label |
Description |
---|---|---|---|
task_execution_id |
Indicates the node execution to filter by. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering. +optional |
NodeExecutionGetDataRequest#
Request structure to fetch inputs and output for a node execution. By default, these are not returned in NodeExecutionGetRequest
Field |
Type |
Label |
Description |
---|---|---|---|
id |
The identifier of the node execution for which to fetch inputs and outputs. |
NodeExecutionGetDataResponse#
Response structure for NodeExecutionGetDataRequest which contains inputs and outputs for a node execution.
Field |
Type |
Label |
Description |
---|---|---|---|
inputs |
Deprecated. Signed url to fetch a core.LiteralMap of node execution inputs. Deprecated: Please use full_inputs instead. |
||
outputs |
Deprecated. Signed url to fetch a core.LiteralMap of node execution outputs. Deprecated: Please use full_outputs instead. |
||
full_inputs |
Full_inputs will only be populated if they are under a configured size threshold. |
||
full_outputs |
Full_outputs will only be populated if they are under a configured size threshold. |
||
dynamic_workflow |
Optional Workflow closure for a dynamically generated workflow, in the case this node yields a dynamic workflow we return its structure here. |
NodeExecutionGetRequest#
A message used to fetch a single node execution entity. See NodeExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Uniquely identifies an individual node execution. +required |
NodeExecutionList#
Request structure to retrieve a list of node execution entities. See NodeExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
node_executions |
repeated |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
NodeExecutionListRequest#
Represents a request structure to retrieve a list of node execution entities. See NodeExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
workflow_execution_id |
Indicates the workflow execution to filter by. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
|||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering. +optional |
||
unique_parent_id |
Unique identifier of the parent node in the execution +optional |
NodeExecutionMetaData#
Represents additional attributes related to a Node Execution
Field |
Type |
Label |
Description |
---|---|---|---|
retry_group |
Node executions are grouped depending on retries of the parent Retry group is unique within the context of a parent node. |
||
is_parent_node |
Boolean flag indicating if the node has child nodes under it This can be true when a node contains a dynamic workflow which then produces child nodes. |
||
spec_node_id |
Node id of the node in the original workflow This maps to value of WorkflowTemplate.nodes[X].id |
||
is_dynamic |
Boolean flag indicating if the node has contains a dynamic workflow which then produces child nodes. This is to distinguish between subworkflows and dynamic workflows which can both have is_parent_node as true. |
TaskNodeMetadata#
Metadata for the case in which the node is a TaskNode
Field |
Type |
Label |
Description |
---|---|---|---|
cache_status |
Captures the status of caching for this execution. |
||
catalog_key |
This structure carries the catalog artifact information |
||
checkpoint_uri |
The latest checkpoint location |
WorkflowNodeMetadata#
Metadata for a WorkflowNode
Field |
Type |
Label |
Description |
---|---|---|---|
executionId |
The identifier for a workflow execution launched by a node. |
flyteidl/admin/notification.proto#
EmailMessage#
Represents the Email object that is sent to a publisher/subscriber to forward the notification. Note: This is internal to Admin and doesn’t need to be exposed to other components.
Field |
Type |
Label |
Description |
---|---|---|---|
recipients_email |
repeated |
The list of email addresses to receive an email with the content populated in the other fields. Currently, each email recipient will receive its own email. This populates the TO field. |
|
sender_email |
The email of the sender. This populates the FROM field. |
||
subject_line |
The content of the subject line. This populates the SUBJECT field. |
||
body |
The content of the email body. This populates the BODY field. |
flyteidl/admin/project.proto#
Domain#
Namespace within a project commonly used to differentiate between different service instances. e.g. “production”, “development”, etc.
Project#
Top-level namespace used to classify different entities like workflows and executions.
ProjectListRequest#
Request to retrieve a list of projects matching specified filters. See Project for more details
Field |
Type |
Label |
Description |
---|---|---|---|
limit |
Indicates the number of projects to be returned. +required |
||
token |
In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering. +optional |
ProjectRegisterRequest#
Adds a new user-project within the Flyte deployment. See Project for more details
ProjectRegisterResponse#
Purposefully empty, may be updated in the future.
ProjectUpdateResponse#
Purposefully empty, may be updated in the future.
Projects#
Represents a list of projects. See Project for more details
Project.ProjectState#
The state of the project is used to control its visibility in the UI and validity.
Name |
Number |
Description |
---|---|---|
ACTIVE |
0 |
By default, all projects are considered active. |
ARCHIVED |
1 |
Archived projects are no longer visible in the UI and no longer valid. |
SYSTEM_GENERATED |
2 |
System generated projects that aren’t explicitly created or managed by a user. |
flyteidl/admin/project_attributes.proto#
ProjectAttributes#
Defines a set of custom matching attributes at the project level. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id for which this set of attributes will be applied. |
||
matching_attributes |
ProjectAttributesDeleteRequest#
Request to delete a set matchable project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to delete. +required |
ProjectAttributesDeleteResponse#
Purposefully empty, may be populated in the future.
ProjectAttributesGetRequest#
Request to get an individual project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to return. +required |
ProjectAttributesGetResponse#
Response to get an individual project level attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
ProjectAttributesUpdateRequest#
Sets custom attributes for a project For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
+required |
ProjectAttributesUpdateResponse#
Purposefully empty, may be populated in the future.
flyteidl/admin/project_domain_attributes.proto#
ProjectDomainAttributes#
Defines a set of custom matching attributes which defines resource defaults for a project and domain. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id for which this set of attributes will be applied. |
||
domain |
Unique domain id for which this set of attributes will be applied. |
||
matching_attributes |
ProjectDomainAttributesDeleteRequest#
Request to delete a set matchable project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
domain |
Unique domain id which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to delete. +required |
ProjectDomainAttributesDeleteResponse#
Purposefully empty, may be populated in the future.
ProjectDomainAttributesGetRequest#
Request to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
domain |
Unique domain id which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to return. +required |
ProjectDomainAttributesGetResponse#
Response to get an individual project domain attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
ProjectDomainAttributesUpdateRequest#
Sets custom attributes for a project-domain combination. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
+required |
ProjectDomainAttributesUpdateResponse#
Purposefully empty, may be populated in the future.
flyteidl/admin/schedule.proto#
CronSchedule#
Options for schedules to run according to a cron expression.
Field |
Type |
Label |
Description |
---|---|---|---|
schedule |
Standard/default cron implementation as described by https://en.wikipedia.org/wiki/Cron#CRON_expression; Also supports nonstandard predefined scheduling definitions as described by https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#CronExpressions except @reboot |
||
offset |
ISO 8601 duration as described by https://en.wikipedia.org/wiki/ISO_8601#Durations |
FixedRate#
Option for schedules run at a certain frequency e.g. every 2 minutes.
Field |
Type |
Label |
Description |
---|---|---|---|
value |
|||
unit |
Schedule#
Defines complete set of information required to trigger an execution on a schedule.
Field |
Type |
Label |
Description |
---|---|---|---|
cron_expression |
Deprecated. Uses AWS syntax: Minutes Hours Day-of-month Month Day-of-week Year e.g. for a schedule that runs every 15 minutes: 0/15 * * * ? * |
||
rate |
|||
cron_schedule |
|||
kickoff_time_input_arg |
Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off. |
FixedRateUnit#
Represents a frequency at which to run a schedule.
Name |
Number |
Description |
---|---|---|
MINUTE |
0 |
|
HOUR |
1 |
|
DAY |
2 |
flyteidl/admin/signal.proto#
Signal#
Signal encapsulates a unique identifier, associated metadata, and a value for a single Flyte signal. Signals may exist either without a set value (representing a signal request) or with a populated value (indicating the signal has been given).
SignalGetOrCreateRequest#
SignalGetOrCreateRequest represents a request structure to retrieve or create a signal. See Signal for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
A unique identifier for the requested signal. |
||
type |
A type denoting the required value type for this signal. |
SignalList#
SignalList represents collection of signals along with the token of the last result. See Signal for more details
Field |
Type |
Label |
Description |
---|---|---|---|
signals |
repeated |
A list of signals matching the input filters. |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
SignalListRequest#
SignalListRequest represents a request structure to retrieve a collection of signals. See Signal for more details
Field |
Type |
Label |
Description |
---|---|---|---|
workflow_execution_id |
Indicates the workflow execution to filter by. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. +optional |
||
sort_by |
Sort ordering. +optional |
SignalSetRequest#
SignalSetRequest represents a request structure to set the value on a signal. Setting a signal effetively satisfies the signal condition within a Flyte workflow. See Signal for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
A unique identifier for the requested signal. |
||
value |
The value of this signal, must match the defining signal type. |
SignalSetResponse#
SignalSetResponse represents a response structure if signal setting succeeds.
Purposefully empty, may be populated in the future.
flyteidl/admin/task.proto#
Task#
Flyte workflows are composed of many ordered tasks. That is small, reusable, self-contained logical blocks arranged to process workflow inputs and produce a deterministic set of outputs. Tasks can come in many varieties tuned for specialized behavior.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the task. |
||
closure |
closure encapsulates all the fields that maps to a compiled version of the task. |
||
short_description |
One-liner overview of the entity. |
TaskClosure#
Compute task attributes which include values derived from the TaskSpec, as well as plugin-specific data and task metadata.
Field |
Type |
Label |
Description |
---|---|---|---|
compiled_task |
Represents the compiled representation of the task from the specification provided. |
||
created_at |
Time at which the task was created. |
TaskCreateRequest#
Represents a request structure to create a revision of a task. See Task for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the task. +required |
||
spec |
Represents the specification for task. +required |
TaskCreateResponse#
Represents a response structure if task creation succeeds.
Purposefully empty, may be populated in the future.
TaskList#
Represents a list of tasks returned from the admin. See Task for more details
Field |
Type |
Label |
Description |
---|---|---|---|
tasks |
repeated |
A list of tasks returned based on the request. |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
TaskSpec#
Represents a structure that encapsulates the user-configured specification of the task.
Field |
Type |
Label |
Description |
---|---|---|---|
template |
Template of the task that encapsulates all the metadata of the task. |
||
description |
Represents the specification for description entity. |
flyteidl/admin/task_execution.proto#
TaskExecution#
Encapsulates all details for a single task execution entity. A task execution represents an instantiated task, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Unique identifier for the task execution. |
||
input_uri |
Path to remote data store where input blob is stored. |
||
closure |
Task execution details and results. |
||
is_parent |
Whether this task spawned nodes. |
TaskExecutionClosure#
Container for task execution details and results.
Field |
Type |
Label |
Description |
---|---|---|---|
output_uri |
Deprecated. Path to remote data store where output blob is stored if the execution succeeded (and produced outputs). DEPRECATED. Use GetTaskExecutionData to fetch output data instead. |
||
error |
Error information for the task execution. Populated if the execution failed. |
||
output_data |
Deprecated. Raw output data produced by this task execution. DEPRECATED. Use GetTaskExecutionData to fetch output data instead. |
||
phase |
The last recorded phase for this task execution. |
||
logs |
repeated |
Detailed log information output by the task execution. |
|
started_at |
Time at which the task execution began running. |
||
duration |
The amount of time the task execution spent running. |
||
created_at |
Time at which the task execution was created. |
||
updated_at |
Time at which the task execution was last updated. |
||
custom_info |
Custom data specific to the task plugin. |
||
reason |
If there is an explanation for the most recent phase transition, the reason will capture it. |
||
task_type |
A predefined yet extensible Task type identifier. |
||
metadata |
Metadata around how a task was executed. |
||
event_version |
The event version is used to indicate versioned changes in how data is maintained using this proto message. For example, event_verison > 0 means that maps tasks logs use the TaskExecutionMetadata ExternalResourceInfo fields for each subtask rather than the TaskLog in this message. |
TaskExecutionGetDataRequest#
Request structure to fetch inputs and output for a task execution. By default this data is not returned inline in TaskExecutionGetRequest
Field |
Type |
Label |
Description |
---|---|---|---|
id |
The identifier of the task execution for which to fetch inputs and outputs. +required |
TaskExecutionGetDataResponse#
Response structure for TaskExecutionGetDataRequest which contains inputs and outputs for a task execution.
Field |
Type |
Label |
Description |
---|---|---|---|
inputs |
Deprecated. Signed url to fetch a core.LiteralMap of task execution inputs. Deprecated: Please use full_inputs instead. |
||
outputs |
Deprecated. Signed url to fetch a core.LiteralMap of task execution outputs. Deprecated: Please use full_outputs instead. |
||
full_inputs |
Full_inputs will only be populated if they are under a configured size threshold. |
||
full_outputs |
Full_outputs will only be populated if they are under a configured size threshold. |
TaskExecutionGetRequest#
A message used to fetch a single task execution entity. See TaskExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
Unique identifier for the task execution. +required |
TaskExecutionList#
Response structure for a query to list of task execution entities. See TaskExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
task_executions |
repeated |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
TaskExecutionListRequest#
Represents a request structure to retrieve a list of task execution entities yielded by a specific node execution. See TaskExecution for more details
Field |
Type |
Label |
Description |
---|---|---|---|
node_execution_id |
Indicates the node execution to filter by. +required |
||
limit |
Indicates the number of resources to be returned. +required |
||
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional |
||
filters |
Indicates a list of filters passed as string. More info on constructing filters : <Link> +optional |
||
sort_by |
Sort ordering for returned list. +optional |
flyteidl/admin/version.proto#
GetVersionRequest#
Empty request for GetVersion
GetVersionResponse#
Response for the GetVersion API
Version#
Provides Version information for a component
flyteidl/admin/workflow.proto#
CreateWorkflowFailureReason#
When a CreateWorkflowRequest fails due to matching id
Field |
Type |
Label |
Description |
---|---|---|---|
exists_different_structure |
|||
exists_identical_structure |
Workflow#
Represents the workflow structure stored in the Admin A workflow is created by ordering tasks and associating outputs to inputs in order to produce a directed-acyclic execution graph.
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the workflow. |
||
closure |
closure encapsulates all the fields that maps to a compiled version of the workflow. |
||
short_description |
One-liner overview of the entity. |
WorkflowClosure#
A container holding the compiled workflow produced from the WorkflowSpec and additional metadata.
Field |
Type |
Label |
Description |
---|---|---|---|
compiled_workflow |
Represents the compiled representation of the workflow from the specification provided. |
||
created_at |
Time at which the workflow was created. |
WorkflowCreateRequest#
Represents a request structure to create a revision of a workflow. See Workflow for more details
Field |
Type |
Label |
Description |
---|---|---|---|
id |
id represents the unique identifier of the workflow. +required |
||
spec |
Represents the specification for workflow. +required |
WorkflowCreateResponse#
Purposefully empty, may be populated in the future.
WorkflowErrorExistsDifferentStructure#
The workflow id is already used and the structure is different
Field |
Type |
Label |
Description |
---|---|---|---|
id |
WorkflowErrorExistsIdenticalStructure#
The workflow id is already used with an identical sctructure
Field |
Type |
Label |
Description |
---|---|---|---|
id |
WorkflowList#
Represents a list of workflows returned from the admin. See Workflow for more details
Field |
Type |
Label |
Description |
---|---|---|---|
workflows |
repeated |
A list of workflows returned based on the request. |
|
token |
In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty. |
WorkflowSpec#
Represents a structure that encapsulates the specification of the workflow.
Field |
Type |
Label |
Description |
---|---|---|---|
template |
Template of the task that encapsulates all the metadata of the workflow. |
||
sub_workflows |
repeated |
Workflows that are embedded into other workflows need to be passed alongside the parent workflow to the propeller compiler (since the compiler doesn’t have any knowledge of other workflows - ie, it doesn’t reach out to Admin to see other registered workflows). In fact, subworkflows do not even need to be registered. |
|
description |
Represents the specification for description entity. |
flyteidl/admin/workflow_attributes.proto#
WorkflowAttributes#
Defines a set of custom matching attributes which defines resource defaults for a project, domain and workflow. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id for which this set of attributes will be applied. |
||
domain |
Unique domain id for which this set of attributes will be applied. |
||
workflow |
Workflow name for which this set of attributes will be applied. |
||
matching_attributes |
WorkflowAttributesDeleteRequest#
Request to delete a set matchable workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
domain |
Unique domain id which this set of attributes references. +required |
||
workflow |
Workflow name which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to delete. +required |
WorkflowAttributesDeleteResponse#
Purposefully empty, may be populated in the future.
WorkflowAttributesGetRequest#
Request to get an individual workflow attribute override. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
project |
Unique project id which this set of attributes references. +required |
||
domain |
Unique domain id which this set of attributes references. +required |
||
workflow |
Workflow name which this set of attributes references. +required |
||
resource_type |
Which type of matchable attributes to return. +required |
WorkflowAttributesGetResponse#
Response to get an individual workflow attribute override.
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
WorkflowAttributesUpdateRequest#
Sets custom attributes for a project, domain and workflow combination. For more info on matchable attributes, see MatchableAttributesConfiguration
Field |
Type |
Label |
Description |
---|---|---|---|
attributes |
WorkflowAttributesUpdateResponse#
Purposefully empty, may be populated in the future.
google/protobuf/duration.proto#
Duration#
A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like “day” or “month”. It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.
# Examples
Example 1: Compute Duration from two Timestamps in pseudo code.
Timestamp start = …; Timestamp end = …; Duration duration = …;
duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;
- if (duration.seconds < 0 && duration.nanos > 0) {
duration.seconds += 1; duration.nanos -= 1000000000;
- } else if (duration.seconds > 0 && duration.nanos < 0) {
duration.seconds -= 1; duration.nanos += 1000000000;
}
Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.
Timestamp start = …; Duration duration = …; Timestamp end = …;
end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;
- if (end.nanos < 0) {
end.seconds -= 1; end.nanos += 1000000000;
- } else if (end.nanos >= 1000000000) {
end.seconds += 1; end.nanos -= 1000000000;
}
Example 3: Compute Duration from datetime.timedelta in Python.
td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)
# JSON Mapping
In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix “s” (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as “3s”, while 3 seconds and 1 nanosecond should be expressed in JSON format as “3.000000001s”, and 3 seconds and 1 microsecond should be expressed in JSON format as “3.000001s”.
Field |
Type |
Label |
Description |
---|---|---|---|
seconds |
Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years |
||
nanos |
Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 seconds field and a positive or negative nanos field. For durations of one second or more, a non-zero value for the nanos field must be of the same sign as the seconds field. Must be from -999,999,999 to +999,999,999 inclusive. |
google/protobuf/wrappers.proto#
BoolValue#
Wrapper message for bool.
The JSON representation for BoolValue is JSON true and false.
BytesValue#
Wrapper message for bytes.
The JSON representation for BytesValue is JSON string.
DoubleValue#
Wrapper message for double.
The JSON representation for DoubleValue is JSON number.
FloatValue#
Wrapper message for float.
The JSON representation for FloatValue is JSON number.
Int32Value#
Wrapper message for int32.
The JSON representation for Int32Value is JSON number.
Int64Value#
Wrapper message for int64.
The JSON representation for Int64Value is JSON string.
StringValue#
Wrapper message for string.
The JSON representation for StringValue is JSON string.
UInt32Value#
Wrapper message for uint32.
The JSON representation for UInt32Value is JSON number.
UInt64Value#
Wrapper message for uint64.
The JSON representation for UInt64Value is JSON string.