flytekit.extend.TypeEngine

class flytekit.extend.TypeEngine[source]

Core Extensible TypeEngine of Flytekit. This should be used to extend the capabilities of FlyteKits type system. Users can implement their own TypeTransformers and register them with the TypeEngine. This will allow special handling of user objects

Methods

classmethod dict_to_literal_map(ctx, d, type_hints=None)[source]

Given a dictionary mapping string keys to python values and a dictionary containing guessed types for such string keys, convert to a LiteralMap.

Parameters:
Return type:

LiteralMap

classmethod dict_to_literal_map_pb(ctx, d, type_hints=None)[source]
Parameters:
Return type:

LiteralMap | None

classmethod get_available_transformers()[source]

Returns all python types for which transformers are available

Return type:

KeysView[Type]

classmethod get_transformer(python_type)[source]

The TypeEngine hierarchy for flyteKit. This method looksup and selects the type transformer. The algorithm is as follows

d = dictionary of registered transformers, where is a python type v = lookup type

Step 1:

If the type is annotated with a TypeTransformer instance, use that.

Step 2:

find a transformer that matches v exactly

Step 3:

find a transformer that matches the generic type of v. e.g List[int], Dict[str, int] etc

Step 4:

Walk the inheritance hierarchy of v and find a transformer that matches the first base class. This is potentially non-deterministic - will depend on the registration pattern.

Special case:

If v inherits from Enum, use the Enum transformer even if Enum is not the first base class.

TODO lets make this deterministic by using an ordered dict

Step 5:

if v is of type data class, use the dataclass transformer

Step 6:

Pickle transformer is used

Parameters:

python_type (Type)

Return type:

TypeTransformer[T]

classmethod guess_python_type(flyte_type)[source]

Transforms a flyte-specific LiteralType to a regular python value.

Parameters:

flyte_type (LiteralType)

Return type:

type

classmethod guess_python_types(flyte_variable_dict)[source]

Transforms a dictionary of flyte-specific Variable objects to a dictionary of regular python values.

Parameters:

flyte_variable_dict (Dict[str, Variable])

Return type:

Dict[str, type]

classmethod lazy_import_transformers()[source]

Only load the transformers if needed.

classmethod literal_map_to_kwargs(ctx, lm, python_types=None, literal_types=None)[source]

Given a LiteralMap (usually an input into a task - intermediate), convert to kwargs for the task

Parameters:
Return type:

Dict[str, Any]

classmethod named_tuple_to_variable_map(t)[source]

Converts a python-native NamedTuple to a flyte-specific VariableMap of named literals.

Parameters:

t (NamedTuple)

Return type:

VariableMap

classmethod register(transformer, additional_types=None)[source]

This should be used for all types that respond with the right type annotation when you use type(…) function

Parameters:
classmethod register_additional_type(transformer, additional_type, override=False)[source]
Parameters:
classmethod register_restricted_type(name, type)[source]
Parameters:
classmethod to_html(ctx, python_val, expected_python_type)[source]
Parameters:
Return type:

str

classmethod to_literal(ctx, python_val, python_type, expected)[source]

Converts a python value of a given type and expected LiteralType into a resolved Literal value.

Parameters:
Return type:

Literal

classmethod to_literal_type(python_type)[source]

Converts a python type into a flyte specific LiteralType

Parameters:

python_type (Type)

Return type:

LiteralType

classmethod to_python_value(ctx, lv, expected_python_type)[source]

Converts a Literal value with an expected python type into a python value.

Parameters:
Return type:

Any

Attributes

has_lazy_import = False