flytekit.core.data_persistence.DiskPersistence

class flytekit.core.data_persistence.DiskPersistence(default_prefix=None, **kwargs)

The simplest form of persistence that is available with default flytekit - Disk-based persistence. This will store all data locally and retrieve the data from local. This is helpful for local execution and simulating runs.

Methods

Parameters

default_prefix (Optional[str]) –

construct_path(_, add_prefix, *args)

if add_protocol is true then <protocol> is prefixed else Constructs a path in the format <base><delim>*args delim is dependent on the storage medium. each of the args is joined with the delim

Parameters

• _ (bool) –

• add_prefix (bool) –

• args (str) –

Return type

str

copy_tree(from_path, to_path)

Parameters

• from_path (str) –

• to_path (str) –

exists(path)

Returns true if the given path exists, else false

Parameters

path (str) –

get(from_path, to_path, recursive=False)

Retrieves data from from_path and writes to the given to_path (to_path is locally accessible)

Parameters

• from_path (str) –

• to_path (str) –

• recursive (bool) –

listdir(path, recursive=False)

Returns true if the given path exists, else false

Parameters

• path (str) –

• recursive (bool) –

Return type

Generator[str, None, None]

put(from_path, to_path, recursive=False)

Stores data from from_path and writes to the given to_path (from_path is locally accessible)

Parameters

• from_path (str) –

• to_path (str) –

• recursive (bool) –

static strip_file_header(path)

Drops file:// if it exists from the file

Parameters

path (str) –

Return type

str

Attributes

PROTOCOL = 'file://'

default_prefix

name