4.7. Shared Concepts#

Module containing shared constants and utility functions for managing and processing structs.

This module defines a set of constants and functions used across different operations involving structs, bitfields, and related concepts. It includes key attributes for managing struct definitions, mode constants for packing /unpacking operations, and action handling for custom processing.

The module introduces the concept of “Actions”, which are used to define custom behavior that should be executed before parsing a struct field (e.g., installing IO hooks for checksums or hash calculations). Actions are represented by classes with __action_XX methods.

Usage Example:

To define a struct that executes a custom action before parsing a field:

>>> class MyAction:
>>>     def __action_unpack__(self, context: _ContextLike) -> None:
>>>         ...
>>>
>>> @struct
>>> class MyStruct:
>>>     any_name: MyAction()

In this case, the action will be executed before parsing any subsequent fields and won’t be stored as part of the struct model.

caterpillar.shared.MODE_PACK: Literal[0] = 0#

Constant representing packing mode.

caterpillar.shared.MODE_UNPACK: Literal[1] = 1#

Constant representing unpacking mode.

caterpillar.shared.ATTR_STRUCT: Final[str] = '__struct__'#

All models annotated with either @struct or @bitfield are struct containers. Thus, they store the additional class attribute __struct__.

Internally, any types utilizing this attribute can be employed within a struct, bitfield, or sequence definition. The type of the stored value must be conforming to the _StructLike protocol.

Changed in version 2.5.0: This attribute is now used when callung getstruct() or hasstruct().

caterpillar.shared.ATTR_BYTEORDER: Final[str] = '__byteorder__'#

Attribute representing the byte order of a struct.

Added in version 2.5.0: Moved from caterpillar.byteorder.

caterpillar.shared.ATTR_TYPE: Final[str] = '__type__'#

Attribute that stores the type of the struct.

Added in version 2.5.0.

caterpillar.shared.ATTR_BITS: Final[str] = '__bits__'#

Attribute representing the number of bits in a struct.

Added in version 2.5.0: Moved from caterpillar.model._bitfield.

caterpillar.shared.ATTR_SIGNED: Final[str] = '__signed__'#

Attribute that marks a struct as signed.

Added in version 2.5.0: Moved from caterpillar.model._bitfield.

caterpillar.shared.ATTR_TEMPLATE: Final[str] = '__template__'#

Attribute that provides the template for the struct.

Added in version 2.5.0: Moved from caterpillar.model._template.

caterpillar.shared.ATTR_PACK: Final[str] = '__pack__'#

Attribute that defines packing behavior for the struct.

Added in version 2.8.0.

caterpillar.shared.ATTR_UNPACK: Final[str] = '__unpack__'#

Attribute that defines unpacking behavior for the struct.

Added in version 2.8.0.

caterpillar.shared.ATTR_ACTION_PACK: Final[str] = '__action_pack__'#

Attribute indicating an action to be executed during packing.

Added in version 2.4.0.

caterpillar.shared.ATTR_ACTION_UNPACK: Final[str] = '__action_unpack__'#

Attribute indicating an action to be executed during unpacking.

Added in version 2.4.0.

caterpillar.shared.constval(value: _OT) → _ContextLambda[_OT][source]#

Returns a lambda that returns a constant value when invoked.

Parameters:

value (_OT) – The constant value to be returned by the lambda.

Returns:

A lambda function that, when called, returns the given constant value.

Return type:

_ContextLambda[_OT]

class caterpillar.shared.Action(pack: _ContextLambda[None] | None = None, unpack: _ContextLambda[None] | None = None, both: _ContextLambda[None] | None = None)[source]#

A class representing an action to be executed during the parsing or processing of a struct. This is used for cases where a field requires an operation to be performed instead of directly storing data.

An action can be used to execute custom logic (such as modifying the IO stream) before the next field is processed. For example, it could be used to trigger checksum or hash calculations before reading a struct field.

There are two types of actions:

  1. Packing actions: These actions are triggered before packing the data into the struct (i.e., before serializing or encoding).

  2. Unpacking actions: These actions are triggered before unpacking the data from the struct (i.e., before deserializing or decoding).

Example:

>>> def checksum_action(context: _ContextLike) -> None:
...     # This action could perform some checksum or logging logic
...     pass
...
>>> @struct
... class MyStruct:
...     some_field: Bytes(10)
...     checksum: Action(checksum_action) # runs only when packing

In this example, the checksum field is an action, and the checksum_action will be invoked before the some_field field is parsed.

The action itself is not stored as part of the struct’s model; it merely runs during struct processing.

Parameters:

  • pack (_ContextLambda | None) – The callable that will be executed before packing the struct (optional).

  • unpack (_ContextLambda | None) – The callable that will be executed before unpacking the struct (optional).

static is_action(obj: object) → TypeIs[_SupportsActionPack | _SupportsActionUnpack][source]#

Checks if the given object is an instance of an Action class, based on the presence of the __action_pack__ or __action_unpack__ attributes.

Parameters:

obj (Any) – The object to check.

Returns:

True if the object has either a __action_pack__ or __action_unpack__ attribute, indicating it is an action.

Return type:

bool

caterpillar.shared.hasstruct(obj: object) → TypeIs[_ContainsStruct][source]#

Check if the given object has a structure attribute.

Parameters:

obj – The object to check.

Returns:

True if the object has a structure attribute, else False.

caterpillar.shared.getstruct(obj: type[_IT], /, __default: None = None) → FieldStruct[_IT, _IT][source]#
caterpillar.shared.getstruct(obj: _ContainsStruct[_IT, _OT], /, __default: None = None) → _StructLike[_IT, _OT]
caterpillar.shared.getstruct(obj: object, /, __default: _StructLike | None = None) → _StructLike | FieldStruct | None

Get the structure attribute of the given object.

Parameters:

obj – The object to get the structure attribute from.

Returns:

The structure attribute of the object.

caterpillar.shared.typeof(struct: _StructLike | _SupportsType | _ContainsStruct | object) → type[source]#

Returns the type of a given struct or object, checking for a custom type annotation.

If the given object has a __type__ attribute, this function returns that type. If not, it returns the object’s default type. If the type cannot be resolved, the function defaults to returning object.

Parameters:

struct (_StructLike | _SupportsType | _ContainsStruct | object) – The struct or object whose type is to be resolved.

Returns:

The resolved type of the struct or object.

Return type:

type

class caterpillar.shared.PackMixin[source]#

Mixin providing serialization helpers for packing objects into binary representations.

This mixin exposes a unified to_bytes interface that can either return serialized bytes directly or write them into a provided writable stream.

Added in version 2.8.1.

to_bytes(obj: _IT, *, fp: IOBase, order: _EndianLike | None = None, arch: _ArchLike | None = None, use_tempfile: bool = False, **kwargs: Any) → None[source]#
to_bytes(obj: _IT, *, fp: None = None, order: _EndianLike | None = None, arch: _ArchLike | None = None, use_tempfile: bool = False, **kwargs: Any) → bytes

Serialize the object into its binary representation.

If fp is provided, the binary data is written directly into the given stream using pack_into. Otherwise, the binary data is returned as a bytes object using pack.

Parameters:

  • obj (_IT) – Object instance to serialize

  • fp (_StreamType | None, optional) – Writable stream to receive the serialized data. If None, the serialized bytes are returned, defaults to None

  • order (_EndianLike | None, optional) – Endianness override for serialization, defaults to None

  • arch (_ArchLike | None, optional) – Architecture override for serialization, defaults to None

  • use_tempfile (bool, optional) – Whether to use a temporary file during packing, defaults to False

Raises:

Exception – Propagates any exception raised by the underlying packing implementation

Returns:

Serialized bytes if fp is None, otherwise None

Return type:

bytes | None

class caterpillar.shared.UnpackMixin[source]#

Mixin providing deserialization helpers for constructing objects from binary data sources.

This mixin supports unpacking from raw bytes, streams, or files, and also defines a convenience left-shift operator for inline unpacking.

Added in version 2.8.1.

from_bytes(data: Buffer | IOBase, *, order: _EndianLike | None = None, arch: _ArchLike | None = None, **kwargs: Any) → _OT[source]#

Construct an object from raw binary data or a stream.

This is a convenience wrapper around the underlying unpack function, allowing associated types to be instantiated directly from bytes without importing parsing utilities.

Parameters:

  • data (Buffer | _StreamType) – Raw bytes or a readable stream containing serialized data

  • order (_EndianLike | None, optional) – Endianness override for parsing, defaults to None

  • arch (_ArchLike | None, optional) – Architecture override for parsing, defaults to None

Raises:

Exception – Propagates any exception raised by the underlying unpacking implementation

Returns:

Parsed associated type instance

Return type:

_OT

from_file(filename: str, *, order: _EndianLike | None = None, arch: _ArchLike | None = None, **kwargs: Any) → _OT[source]#

Construct an instance from a binary file on disk.

This is a convenience wrapper around unpack_file for reading and parsing structured data directly from a file path.

Parameters:

  • filename (str) – Path to the file containing serialized data

  • order (_EndianLike | None, optional) – Endianness override for parsing, defaults to None

  • arch (_ArchLike | None, optional) – Architecture override for parsing, defaults to None

Raises:

Exception – Propagates any exception raised by the underlying file-unpacking implementation

Returns:

Parsed associated type instance

Return type:

_OT