Complex types in Tierkreis Python workers

In general Tierkreis allows a task to write arbitrary bytes to its output files. This allows Tierkreis graphs to more easily amalgamate tasks that do not share a common set of type definitions. (For example a command line tool that defines its own serialization format.)

However workers written using the Tierkreis Python library should use only a specific subset of all the possible classes that the Python language can produce.

This page lists the main types a worker author can use (as inputs or outputs) in their functions.

Warning

This page only talks about the types describing a single input or output. Therefore it does not talk about the portmapping decorator used by graph builder code to group together multiple outputs. Now each of the attributes of a port mapping is itself a single output and so the remarks in this page do apply to the attributes individually.

JSON style types

Any type satisfying the following recursive definition is allowed as an input or output.

type Jsonable = (
    bool
    | int
    | float
    | str
    | NoneType
    | list[Jsonable]
    | Sequence[Jsonable]
    | tuple[Jsonable, ...]
    | dict[str, Jsonable]
    | Mapping[str, Jsonable]
)

After the stub generation process is run these type appear in graph builder code wrapped in TKR. E.g. TKR[int], TKR[str], TKR[tuple[dict[str, list[int]], int]].

Struct using NamedTuple

Given a sequence T_0, T_1, …, T_n of allowed types then a NamedTuple wrapping tuple[T_0, T_1, ..., T_n] is allowed as a type. For example

from typing import NamedTuple

class MyStruct(NamedTuple):
    a: int
    b: str
    c: tuple[dict[str, list[int]], int]

The stub generation process will duplicate this type into the stubs file and it can then be used in graph builder code as TKR[MyStruct].

Tip

The class generated in the stubs file will additionally inherit from Protocol. Therefore if a struct is used as an input to a task then the graph builder code will accept any class with the appropriate fields. This makes it easier to pass data between workers that contain similar class definitions but where there is not a shared model library between the workers. For nominal typing please use BaseModel, DictConvertible or ListConvertible as below.

bytes

The Python type bytes is allowed as an input or output. The exact behavior will depend on whether the bytes are at the ‘top level’ (e.g. the type of a whole output is bytes) or whether the bytes are nested within an output.

If the type of an output is bytes then no processing will be applied. This is to enable smooth interop with tasks not produced by the Tierkreis Python library, which might be using an arbitrary serialization format.

If the bytes are nested inside an output (e.g. an output is of type dict[str, bytes]) then a custom JSON encoder is used. The bytes o will appear nested in a JSON object as:

{"__tkr_bytes__": True, "bytes": b64encode(o).decode()}

The bytes type is indicated by TKR[bytes] in graph builder code.

DictConvertible and ListConvertible

In some cases we want to use complex classes that we nevertheless know how to serialize and deserialize. In this case we can use the DictConvertible and ListConvertible protocols. Specifically, any Python class that implements to_dict and from_dict methods are allowed.

from typing import Protocol, runtime_checkable

@runtime_checkable
class DictConvertible(Protocol):
    def to_dict(self) -> dict: ...
    @classmethod
    def from_dict(cls, arg: dict, /) -> "Self": ...

similarly for classes that implement to_list and from_list

@runtime_checkable
class ListConvertible(Protocol):
    def to_list(self) -> list: ...
    @classmethod
    def from_list(cls, arg: list, /) -> "Self": ...

Caution

The Tierkreis Python library will attempt to serialize the resulting dict or list as JSON. The worker author should ensure that this will not result in errors.

The stub generation process does not provide any introspection for these types but instead considers them ‘opaque’ and identifies them only by their fully qualified name. For instance if one wants to use a pytket Circuit as an input or an output then the resulting type will look as follows:

TKR[OpaqueType["pytket._tket.circuit.Circuit"]]

and a list of Circuits would be typed as:

TKR[list[OpaqueType["pytket._tket.circuit.Circuit"]]]

Pydantic BaseModels

We can also use pydantic.BaseModel as an input or output. The behavior of BaseModels is very similar to DictConvertible. For serialization the method model_dump(mode="json") will be used instead of to_dict and the stub generation process will create types using OpaqueType as above.