flytekit.extend.DataPersistence

class flytekit.extend.DataPersistence(name, default_prefix=None, **kwargs)[source]

Base abstract type for all DataPersistence operations. This can be extended using the flytekitplugins architecture

Parameters

  • name (str) –

  • default_prefix (Optional[str]) –

__init__(name, default_prefix=None, **kwargs)[source]
Parameters

  • name (str) –

  • default_prefix (Optional[str]) –

Methods

__init__(name[, default_prefix])

construct_path(add_protocol, add_prefix, *paths)

if add_protocol is true then <protocol> is prefixed else Constructs a path in the format <base><delim>*args delim is dependent on the storage medium.

exists(path)

Returns true if the given path exists, else false

get(from_path, to_path[, recursive])

Retrieves data from from_path and writes to the given to_path (to_path is locally accessible)

listdir(path[, recursive])

Returns true if the given path exists, else false

put(from_path, to_path[, recursive])

Stores data from from_path and writes to the given to_path (from_path is locally accessible)

Attributes

default_prefix

name