Concepts

A Task is any independent unit of processing. Tasks can be pure functions or functions with side-effects. Each definition of a task also has associated configurations and requirements specifications.

Workflows are programs that are guaranteed to eventually reach a terminal state and are represented as Directed Acyclic Graphs (DAGs) expressed in protobuf.

A Node is an encapsulation of an instance of a Task. Nodes represent the unit of work, where multiple Nodes are interconnected via workflows.

Launch Plans provide a mechanism to specialize input parameters for workflows associated with different schedules.

Scheduling is critical to data and ML jobs. Flyte provides a native Cron-style scheduler.

Registration is the process of uploading a workflow and its task definitions to the FlyteAdmin service. Registration creates an inventory of available tasks, workflows and launch plans, declared per project and domain.

Executions are instances of workflows, nodes or tasks created in the system as a result of a user-requested execution or a scheduled execution.

The various states an Execution passes through.

How an Execution progresses through the FlytePropeller execution engine and the timeline.

What is metadata? How are large amounts of raw data handled? How does data flow between tasks?

A quick overview of the FlyteConsole.

A deeper dive into memoization and the mechanics of memoization in Flyte.

A deeper dive into one of Flyte’s most important features: versioning of workflows and tasks.

The diagram below shows how inputs flow through tasks and workflows to produce outputs.

https://raw.githubusercontent.com/flyteorg/static-resources/main/flyte/concepts/basics/flyte_wf_tasks_high_level.png