flytekit.core.base_task.TaskResolverMixin#
- class flytekit.core.base_task.TaskResolverMixin[source]#
Flytekit tasks interact with the Flyte platform very, very broadly in two steps. They need to be uploaded to Admin, and then they are run by the user upon request (either as a single task execution or as part of a workflow). In any case, at execution time, for most tasks (that is those that generate a container target) the container image containing the task needs to be spun up again at which point the container needs to know which task it’s supposed to run and how to rehydrate the task object.
For example, the serialization of a simple task
# in repo_root/workflows/example.py @task def t1(...) -> ...: ...
might result in a container with arguments like
pyflyte-execute --inputs s3://path/inputs.pb --output-prefix s3://outputs/location --raw-output-data-prefix /tmp/data --resolver flytekit.core.python_auto_container.default_task_resolver -- task-module repo_root.workflows.example task-name t1
At serialization time, the container created for the task will start out automatically with the
pyflyte-execute
bit, along with the requisite input/output args and the offloaded data prefix. Appended to that will be two things,the
location
of the task’s task resolver, followed by two dashes, followed bythe arguments provided by calling the
loader_args
function below.
The
default_task_resolver
declared below knows thatWhen
loader_args
is called on a task, to look up the module the task is in, and the name of the task (the key of the task in the module, either the function name, or the variable it was assigned to).When
load_task
is called, it interprets the first part of the command as the module to callimportlib.import_module
on, and then looks for a keyt1
.
This is just the default behavior. Users should feel free to implement their own resolvers.
Methods
- abstract get_all_tasks()[source]#
Future proof method. Just making it easy to access all tasks (Not required today as we auto register them)
- abstract load_task(loader_args)[source]#
Given the set of identifier keys, should return one Python Task or raise an error if not found
- abstract loader_args(settings, t)[source]#
Return a list of strings that can help identify the parameter Task
- Parameters:
settings (SerializationSettings)
t (Task)
- Return type:
- task_name(t)[source]#
Overridable function that can optionally return a custom name for a given task
Attributes
- location