Understanding the State Transition in a workflow

High Level Overview of how a Workflow progresses to Success

Happy case for a workflow with one node and one task.

This State diagram illustrates an extremely high level, simplistic view of the state transitions that a Workflow, with a single node and one task will go through as the observer observes success.

The following section explains in detail the various observable (and some hidden) states for a workflow, node and tasks state transitions.

Workflow States

The State diagram above illustrates the various states through which a workflow transitions. This is the core finite state machine (FSM) of a Workflow.

The State diagram above illustrates the various states through which a Workflow transitions. This is the core Finite state machine of a Workflow.

A Workflow always starts in the Ready State and ends either in Failed, Succeeded or Aborted state. Any system error within a state causes a retry on that state. These retries are capped by system retries and will eventually lead to an Aborted state.

Every transition between states is recorded in Flyteadmin using workflowexecutionevent

The phases in the above state diagram are captured in the Admin database as specified here workflowexecution.phase and are sent as part of the Execution Event.

The state machine specification for the illustration can be found here

Node States

The State diagram above illustrates the various states through which a Node transitions. This is the core FSM for a Node.

The state diagram above illustrates the various states through which a Node transitions. This is the core FSM for a Node. From a user’s point of view, a Workflow simply consists of a sequence of tasks. But to Flyte, a Workflow internally creates a meta entity called a

Once a Workflow enters a Running state, it triggers the phantom start node of the workflow. The Start node is always the entry node of any workflow. The start node starts executing all its child-nodes using a modified DepthFirst Search algorithm recursively.

Nodes can be of different types, as follows, but all the nodes traverse through the same transitions

  1. Start Node - Only exists during the execution and is not modeled in the core spec

  2. tasknode

  3. branchnode

  4. workflownode

  5. Dynamic node - which is just a task node that does not return outputs, but a dynamic workflow. When the task runs, it stays in a RUNNING state. Once the task completes and Flyte starts executing the dynamic workflow, the overarching node that contains both the original task and the dynamic workflow enters DYNAMIC_RUNNING state.

  6. End Node - only exists during the execution and is not modeled in the core spec

Every transition between states is recorded in Flyteadmin using nodeexecutionevent

Every NodeExecutionEvent can have one of the nodeexecution.phase

Note

TODO add explanation for each phase

The state machine specification for the illustration can be found here

Task States

The State diagram above illustrates the various states through which a Task transitions. This is the core FSM for any Task in Flyte.

The State diagram above illustrates the various states through which a Task transitions.

Every transition between states is recorded in Flyteadmin using taskexecutionevent

Every TaskExecutionEvent can have one of the taskexecution.phase

Note

TODO add explanation for each phase

The state machine specification for the illustration can be found here