flytekitplugins.great_expectations.GreatExpectationsFlyteConfig#

class flytekitplugins.great_expectations.GreatExpectationsFlyteConfig(datasource_name, expectation_suite_name, data_connector_name, data_asset_name=None, local_file_path=None, checkpoint_params=None, batch_request_config=None, context_root_dir='./great_expectations')[source]#

Use this configuration to configure GreatExpectations Plugin.

Parameters

  • datasource_name (str) – tell where your data lives and how to get it

  • expectation_suite_name (str) – suite which consists of the data expectations

  • data_connector_name (str) – connector to identify data batches

  • data_asset_name (Optional[str]) – name of the data asset (to be used for RuntimeBatchRequest)

  • local_file_path (Optional[str]) – dataset file path useful for FlyteFile and FlyteSchema

  • checkpoint_params (Optional[Dict[str, Union[str, List[str]]]]) – optional SimpleCheckpoint parameters

  • batch_request_config (Optional[flytekitplugins.great_expectations.task.BatchRequestConfig]) – batchrequest config

  • context_root_dir (str) – directory in which GreatExpectations’ configuration resides

Return type

None

Methods

classmethod from_dict(kvs, *, infer_missing=False)#
Parameters

kvs (Optional[Union[dict, list, str, int, float, bool]]) –

Return type

dataclasses_json.api.A

classmethod from_json(s, *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw)#
Parameters

s (Union[str, bytes, bytearray]) –

Return type

dataclasses_json.api.A

classmethod schema(*, infer_missing=False, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)#
Parameters

  • infer_missing (bool) –

  • many (bool) –

  • partial (bool) –

Return type

dataclasses_json.mm.SchemaF[dataclasses_json.api.A]

to_dict(encode_json=False)#
Return type

Dict[str, Optional[Union[dict, list, str, int, float, bool]]]

to_json(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, indent=None, separators=None, default=None, sort_keys=False, **kw)#
Parameters
Return type

str

Attributes

batch_request_config: Optional[flytekitplugins.great_expectations.task.BatchRequestConfig] = None
checkpoint_params: Optional[Dict[str, Union[str, List[str]]]] = None
context_root_dir: str = './great_expectations'
data_asset_name: Optional[str] = None

local_file_path is a must in two scenrios: * When using FlyteSchema * When using FlyteFile for remote paths This is because base directory which has the dataset file ‘must’ be given in GreatExpectations’ config file

dataclass_json_config = None
local_file_path: Optional[str] = None
datasource_name: str
expectation_suite_name: str
data_connector_name: str