flytekit.map_task

flytekit.map_task(task_function, concurrency=None, min_success_ratio=None, **kwargs)[source]

Use a map task for parallelizable tasks that are run across a List of an input type. A map task can be composed of any individual flytekit.PythonFunctionTask.

Invoke a map task with arguments using the list version of the expected input.

Usage:

@task
def my_mappable_task(a: int) -> str:
    return str(a)

@workflow
def my_wf(x: typing.List[int]) -> typing.List[str]:
    return map_task(
        my_mappable_task,
        metadata=TaskMetadata(retries=1),
        requests=Resources(cpu="10M"),
        concurrency=10,
        min_success_ratio=0.75,
    )(a=x)

At run time, the underlying map task will be run for every value in the input collection. Task-specific attributes such as flytekit.TaskMetadata and flytekit.Resources are applied to individual instances of the mapped task.

Parameters
  • task_function (flytekit.core.python_function_task.PythonFunctionTask) – This argument is implicitly passed and represents the repeatable function

  • concurrency (Optional[int]) – If specified, this limits the number of mapped tasks than can run in parallel to the given batch size. If the size of the input exceeds the concurrency value, then multiple batches will be run serially until all inputs are processed.

  • min_success_ratio (Optional[float]) – If specified, this determines the minimum fraction of total jobs which can complete successfully before terminating this task and marking it successful.