flytekitplugins.spark.SparkDataFrameSchemaReader¶

class flytekitplugins.spark.SparkDataFrameSchemaReader(from_path, cols, fmt)[source]¶

Implements how SparkDataFrame should be read using the open method of FlyteSchema

Methods

Parameters:

from_path (str)
cols (Dict[str, type] | None)
fmt (SchemaFormat)

all(**kwargs)[source]¶

Return type:: DataFrame

iter(**kwargs)[source]¶

Return type:: Generator[T, None, None]

Attributes

column_names

from_path