flytekit.types.structured.StructuredDatasetDecoder

class flytekit.types.structured.StructuredDatasetDecoder(python_type, protocol=None, supported_format=None, additional_protocols=None)

Methods

Parameters:

• python_type (Type[DF])

• protocol (Optional[str])

• supported_format (Optional[str])

• additional_protocols (Optional[List[str]])

abstract decode(ctx, flyte_value, current_task_metadata)

This is code that will be called by the dataset transformer engine to ultimately translate from a Flyte Literal value into a Python instance.

Parameters:

• ctx (FlyteContext) – A FlyteContext, useful in accessing the filesystem and other attributes

• flyte_value (StructuredDataset) – This will be a Flyte IDL StructuredDataset Literal - do not confuse this with the StructuredDataset class defined also in this module.

• current_task_metadata (StructuredDatasetMetadata) – Metadata object containing the type (and columns if any) for the currently executing task. This type may have more or less information than the type information bundled inside the incoming flyte_value.

Returns:

This function can either return an instance of the dataframe that this decoder handles, or an iterator of those dataframes.

Return type:

DF | Iterator[DF]

Attributes

protocol

python_type

supported_format