flytekitplugins.dask.WorkerGroup¶

class flytekitplugins.dask.WorkerGroup(number_of_workers=1, image=None, requests=None, limits=None)[source]¶

Configuration for a group of dask worker pods

Parameters:

number_of_workers (int | None) – Number of workers to use. Optional, defaults to 1.
image (str | None) – Custom image to use. If None, will use the same image the task was registered with. Optional, defaults to None. The image must have dask[distributed] installed. The provided image should have the same Python environment as the job runner/driver as well as the scheduler.
requests (Resources | None) – Resources to request for the worker pods. If None, the requests passed into the task will be used. Optional, defaults to None.
limits (Resources | None) – Resource limits for the worker pods. If None, the limits passed into the task will be used. Optional, defaults to None.

Methods

Attributes