FS Spec API reference

Tags: Data, AWS, GCP

This package contains things that are useful when extending Flytekit.

ArrowToParquetEncodingHandler(protocol)	Extend this abstract class, implement the encode function, and register your concrete class with the StructuredDatasetTransformerEngine class in order for the core flytekit type engine to handle dataframe libraries.
FSSpecPersistence([default_prefix, data_config])	This DataPersistence plugin uses fsspec to perform the IO.
PandasToParquetEncodingHandler(protocol)	Extend this abstract class, implement the encode function, and register your concrete class with the StructuredDatasetTransformerEngine class in order for the core flytekit type engine to handle dataframe libraries.
ParquetToArrowDecodingHandler(protocol)	Extend this abstract class, implement the decode function, and register your concrete class with the StructuredDatasetTransformerEngine class in order for the core flytekit type engine to handle dataframe libraries.
ParquetToPandasDecodingHandler(protocol)	Extend this abstract class, implement the decode function, and register your concrete class with the StructuredDatasetTransformerEngine class in order for the core flytekit type engine to handle dataframe libraries.