flytekitplugins.great_expectations.GreatExpectationsFlyteConfig¶

class flytekitplugins.great_expectations.GreatExpectationsFlyteConfig(datasource_name, expectation_suite_name, data_connector_name, data_asset_name=None, local_file_path=None, checkpoint_params=None, batch_request_config=None, context_root_dir='./great_expectations')[source]¶

Use this configuration to configure GreatExpectations Plugin.

Parameters:

datasource_name (str) – tell where your data lives and how to get it
expectation_suite_name (str) – suite which consists of the data expectations
data_connector_name (str) – connector to identify data batches
data_asset_name (str | None) – name of the data asset (to be used for RuntimeBatchRequest)
local_file_path (str | None) – dataset file path useful for FlyteFile and FlyteSchema
checkpoint_params (Dict[str, str | List[str]] | None) – optional SimpleCheckpoint parameters
batch_request_config (BatchRequestConfig | None) – batchrequest config
context_root_dir (str) – directory in which GreatExpectations’ configuration resides

Methods

classmethod from_dict(kvs, *, infer_missing=False)¶

Parameters:: kvs (dict | list | str | int | float | bool | None)
Return type:: A

classmethod from_json(s, *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw)¶

Parameters:: s (str | bytes | bytearray)
Return type:: A

classmethod schema(*, infer_missing=False, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)¶

Parameters:

infer_missing (bool)
many (bool)
partial (bool)

Return type:

SchemaF[A]

to_dict(encode_json=False)¶

Return type:: Dict[str, dict | list | str | int | float | bool | None]

to_json(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, indent=None, separators=None, default=None, sort_keys=False, **kw)¶

Parameters:

skipkeys (bool)
ensure_ascii (bool)
check_circular (bool)
allow_nan (bool)
indent (int | str | None)
separators (Tuple[str, str] | None)
default (Callable | None)
sort_keys (bool)

Return type:

str

Attributes

batch_request_config: BatchRequestConfig | None = None

checkpoint_params: Dict[str, str | List[str]] | None = None

context_root_dir: str = './great_expectations'

data_asset_name: str | None = None: local_file_path is a must in two scenrios: * When using FlyteSchema * When using FlyteFile for remote paths This is because base directory which has the dataset file ‘must’ be given in GreatExpectations’ config file

dataclass_json_config = None

local_file_path: str | None = None

datasource_name: str

expectation_suite_name: str

data_connector_name: str