flytekitplugins.spark.SparkDataFrameSchemaWriter

class flytekitplugins.spark.SparkDataFrameSchemaWriter(to_path, cols, fmt)

Implements how SparkDataFrame should be written to using open method of FlyteSchema

Methods

Parameters:

• to_path (str)

• cols (Dict[str, type] | None)

• fmt (SchemaFormat)

write(*dfs, **kwargs)

Parameters:

dfs (DataFrame)

Attributes

column_names

to_path