flytekitplugins.dask.WorkerGroup#

class flytekitplugins.dask.WorkerGroup(number_of_workers=1, image=None, requests=None, limits=None)[source]#

Configuration for a group of dask worker pods

Parameters

number_of_workers (Optional[int]) – Number of workers to use. Optional, defaults to 1.
image (Optional[str]) – Custom image to use. If None, will use the same image the task was registered with. Optional, defaults to None. The image must have dask[distributed] installed. The provided image should have the same Python environment as the job runner/driver as well as the scheduler.
requests (Optional[flytekit.core.resources.Resources]) – Resources to request for the worker pods. If None, the requests passed into the task will be used. Optional, defaults to None.
limits (Optional[flytekit.core.resources.Resources]) – Resource limits for the worker pods. If None, the limits passed into the task will be used. Optional, defaults to None.

Return type

None

Methods

Attributes