flytekit.configuration.DataConfig#

class flytekit.configuration.DataConfig(s3=S3Config(enable_debug=False, endpoint=None, retries=3, backoff=datetime.timedelta(seconds=5), access_key_id=None, secret_access_key=None), gcs=GCSConfig(gsutil_parallelism=False))[source]#

Any data storage specific configuration. Please do not use this to store secrets, in S3 case, as it is used in Flyte sandbox environment we store the access key id and secret. All DataPersistence plugins are passed all DataConfig and the plugin should correctly use the right config

Methods

Parameters

s3 (flytekit.configuration.S3Config) –
gcs (flytekit.configuration.GCSConfig) –

Return type

None

classmethod auto(config_file=None)[source]#

Parameters: config_file (Optional[Union[str, flytekit.configuration.file.ConfigFile]]) –
Return type: flytekit.configuration.DataConfig

Attributes

gcs: flytekit.configuration.GCSConfig = GCSConfig(gsutil_parallelism=False)

s3: flytekit.configuration.S3Config = S3Config(enable_debug=False, endpoint=None, retries=3, backoff=datetime.timedelta(seconds=5), access_key_id=None, secret_access_key=None)