Core Flytekit

This package contains all of the most common abstractions you’ll need to write Flyte workflows, and extend Flytekit.

Basic Authoring

These are the essentials needed to get started writing tasks and workflows. The elements here correspond well with Basic section of the cookbook.

task

This is the core decorator to use for any task type in flytekit.

workflow

This decorator declares a function to be a Flyte workflow.

kwtypes

Converts the keyword arguments to typed dictionary

current_context

Use this method to get a handle of specific parameters available in a flyte task.

ExecutionParameters

This is the context object that is accessible to every @task method.

FlyteContext

Running Locally

Tasks and Workflows can both be locally run (assuming the relevant tasks are capable of local execution). This is useful for unit testing.

Branching and Conditionals

Branches and conditionals can be expressed explicitly in Flyte. These conditions are evaluated in the flyte engine and hence should be used for control flow. dynamic workflows can be used to perform custom conditional logic not supported by flytekit

Customizing Tasks & Workflows

TaskMetadata

Create Metadata to be associated with this Task

Resources

This class is used to specify both resource requests and resource limits. ::.

WorkflowFailurePolicy

An enumeration.

Dynamic and Nested Workflows

See the Dynamic module for more information.

dynamic

Please first see the comments for flytekit.task() and flytekit.workflow().

Scheduling and Notifications

See the Notifications Module and Schedules Module for more information.

CronSchedule

Use this when you have a launch plan that you want to run on a cron expression.

FixedRate

Email

This notification should be used when sending regular emails to people.

PagerDuty

This notification should be used when sending emails to the PagerDuty service.

Slack

This notification should be used when sending emails to the Slack.

Reference Entities

get_reference_entity

LaunchPlanReference

TaskReference

WorkflowReference

reference_task

A reference task is a pointer to a task that already exists on your Flyte installation.

reference_workflow

A reference workflow is a pointer to a workflow that already exists on your Flyte installation.

Core Task Types

SQLTask

Base task types for all SQL tasks

ContainerTask

PythonFunctionTask

A Python Function task should be used as the base for all extensions that have a python function.

PythonInstanceTask

This class should be used as the base class for all Tasks that do not have a user defined function body, but have a platform defined execute method.

LaunchPlan

Secrets and SecurityContext

Secret

param group is the Name of the secret. For example in kubernetes secrets is the name of the secret

SecurityContext

Core Modules

core.dynamic_workflow_task

Dynamic Workflows

core.notification

Notifications are primarily used when defining Launch Plans (also can be used when launching executions) and will trigger the Flyte platform to send emails when a workflow run reaches certain stages (fails or suceeds, etc.).

core.schedule

class flytekit.core.schedule.CronSchedule(cron_expression: Optional[str] = None, schedule: Optional[str] = None, offset: Optional[str] = None, kickoff_time_input_arg: Optional[str] = None)[source]

Use this when you have a launch plan that you want to run on a cron expression. The syntax currently used for this follows the AWS convention

CronSchedule(
    cron_expression="0 10 * * ? *",
)

See cookbook for further examples.