flytekitplugins.spark.SparkDataFrameSchemaReader#

class flytekitplugins.spark.SparkDataFrameSchemaReader(from_path, cols, fmt)[source]#

Implements how SparkDataFrame should be read using the open method of FlyteSchema

Methods

all(**kwargs)[source]#
Return type

pyspark.sql.dataframe.DataFrame

iter(**kwargs)[source]#
Return type

Generator[flytekit.core.type_engine.T, None, None]

Attributes

column_names#
from_path#