4.8.4. Cryptographic Structs#

4.8.4.1. Digests#

class caterpillar.fields.Algorithm(create: _ContextLambda[_AlgoObjT] | None = None, update: Callable[[_AlgoObjT, bytes, _ContextLike], _AlgoObjT | None] | None = None, digest: Callable[[_AlgoObjT, _ContextLike], _AlgoReturnT] | None = None, name: str | None = None)[source]#

Added in version 2.4.0.

A class representing a cryptographic or checksum algorithm.

This class allows for the creation, updating, and finalization of values using a specified algorithm. It abstracts the necessary methods to interact with hash algorithms or checksums like SHA256 or CRC32.

Example usage with SHA256:

>>> SHA256 = Algorithm(
...     create=lambda context: hashlib.sha256(),
...     update=lambda algo_obj, data, context: algo_obj.update(data),
...     digest=lambda algo_obj, context: algo_obj.digest(),
... )

Example usage with CRC32:

>>> CRC32 = Algorithm(
...     create=lambda context: zlib.crc32(b""),
...     update=lambda crc_value, data, context: crc32(data, crc_value),
...     digest=lambda crc_value, context: crc_value,
... )

This class can be used to handle both cryptographic algorithms and non-cryptographic checksums.

Parameters:

  • create (Callable[[_ContextLike], Any]) – A callable that creates an instance of the algorithm or checksum.

  • update (Callable[[Any, bytes, _ContextLike], Any]) – A callable that updates the algorithm or checksum with new data.

  • digest (Callable[[Any, _ContextLike], bytes]) – A callable that returns the digest or checksum value.

  • name (Optional[str]) – An optional name for the algorithm.

create(context: _ContextLike) → _AlgoObjT[source]#

Create an instance of the algorithm or checksum using the provided context.

This method invokes the create callable passed during initialization.

Parameters:

context (_ContextLike) – The context in which to create the algorithm instance.

Returns:

An instance of the algorithm or checksum.

Return type:

Any

Raises:

NotImplementedError – If the create method is not implemented for this algorithm.

update(algo_obj: _AlgoObjT, data: bytes, context: _ContextLike) → _AlgoObjT | None[source]#

Update the algorithm or checksum with the given data.

This method invokes the update callable passed during initialization.

Parameters:

  • algo_obj (Any) – The algorithm instance to update (e.g., a SHA256 object).

  • data (bytes) – The data to be used for updating the algorithm.

  • context (_ContextLike) – The context in which the update takes place.

Returns:

The updated algorithm instance or checksum value.

Return type:

Any

Raises:

NotImplementedError – If the update method is not implemented for this algorithm.

digest(algo_obj: _AlgoObjT, context: _ContextLike) → _AlgoReturnT[source]#

Compute the digest or checksum value from the algorithm instance.

This method invokes the digest callable passed during initialization.

Parameters:

  • algo_obj (Any) – The algorithm instance from which to compute the digest.

  • context (_ContextLike) – The context in which the digest operation takes place.

Returns:

The computed digest or checksum value.

Return type:

bytes

Raises:

NotImplementedError – If the digest method is not implemented for this algorithm.

class caterpillar.fields.Digest(algorithm: Algorithm[_AlgoObjT, _AlgoReturnT], struct: _StructLike[_AlgoReturnT, _AlgoReturnT], name: str | None = None, verify: bool = False, path: str | None = None)[source]#

A class to handle the creation, updating, and verification of digests using a specified algorithm.

Added in version 2.4.0.

Changed in version 2.4.3: Python 3.14 is not supported, use DigestField instead.

The Digest class allows you to integrate hash or checksum algorithms into a struct by installing hooks that manage the digest state during packing and unpacking. It supports automatic calculation and validation of digests during serialization (packing) and deserialization (unpacking).

Example usage:

>>> @struct
... class Format:
...     with Digest(SHA256, Bytes(32), "sha256", verify=True):
...         user_data: Bytes(10)
...

This example defines a struct with a user_data field and a sha256 digest field. The Digest class automatically calculates the SHA256 checksum of user_data during packing and validates the checksum during unpacking.

The resulting struct has the following fields:

>>> Format.__struct__.fields
[
    Action(Digest.begin),
    Field('user_data', struct=<Bytes>),
    Action(Digest.end),
    Field('sha256', struct=<Bytes>),
    Action(Digest.verify)
]

The Action objects install hooks into the current stream for digest calculation and validation. When packing an object, the digest will be updated automatically:

>>> obj = Format(user_data=b"helloworld" * 10)
Format(user_data=b"helloworld", sha256=<_DigestValue>)

On packing, the digest is calculated and included in the serialized output:

>>> pack(obj, Format)
b"hello world\xb9M'\xb9" ...

If verification is enabled, unpacking will check the validity of the digest:

>>> unpack(Format, b"invalid data with sha256")
Traceback (most recent call last):
...
caterpillar.exception.ValidationError: Failed to verify digest of 'sha256'!
- Expected: <expected_digest_value>
- Actual: <actual_digest_value>
Context-Path: <root>.sha256
Parameters:

  • algorithm (Algorithm) – The checksum or cryptographic algorithm used to generate the digest.

  • struct (_StructLike) – The struct to which the digest will be attached.

  • name (Optional[str]) – An optional name for the digest field (defaults to ‘sha256’ or another default name).

  • verify (bool) – Whether to enable verification of the digest upon unpacking. Default is False.

  • path (Optional[str]) – Optional path to the digest field in the context, used for accessing and storing the digest.

begin(context: _ContextLike) → None[source]#

Initialize the digest calculation at the beginning of packing/unpacking.

This method sets up the algorithm and prepares the context for checksum calculation.

Parameters:

context (_ContextLike) – The current context during packing/unpacking.

end_pack(context: _ContextLike) → None[source]#

Finalize the digest calculation at the end of packing/unpacking.

This method calculates the digest and stores it in the context for later use.

Parameters:

context (_ContextLike) – The current context during packing/unpacking.

end_unpack(context: _ContextLike) → None[source]#

Finalize the digest calculation at the end of unpacking.

This method retrieves the digest from the context and updates the checksum.

Parameters:

context (_ContextLike) – The current context during unpacking.

update(data: bytes, context: _ContextLike) → None[source]#

Update the checksum with new data during packing/unpacking.

This method feeds new data into the algorithm to update the digest.

Parameters:

  • data (bytes) – The data to update the checksum with.

  • context (_ContextLike) – The current context during packing/unpacking.

verfiy(context: _ContextLike) → None[source]#

Verify the checksum upon unpacking.

This method checks if the digest calculated during unpacking matches the expected digest. If verification fails, a ValidationError is raised.

Parameters:

context (_ContextLike) – The current context during unpacking.

Raises:

ValidationError – If the digest verification fails.

class caterpillar.fields.DigestField(target: str, struct: _StructLike[_AlgoReturnT, _AlgoReturnT], verify: bool = False)[source]#

Added in version 2.4.5.

Represents a field in a struct that stores or verifies a digest.

This struct field computes a digest over all preceding fields and then either:

  • Packs the resulting digest value.

  • Unpacks a stored digest and optionally verifies it.

Example Usage:

@struct
class Format:
    _hash_begin: DigestField.begin("hash", Sha2_256_Algo)
    user_data: Bytes(10)
    hash: Sha2_256_Field("hash", verify=True) = None
Parameters:

  • target (str) – The unique name shared with the corresponding DigestFieldAction.

  • struct (_StructLike) – The struct used to pack/unpack the digest value (e.g., Bytes(32))

  • verify (bool) – Whether to verify the unpacked digest against the computed one.

static begin(target: str, algo: Algorithm[_AlgoObjT, _AlgoReturnT]) → DigestFieldAction[_AlgoObjT, _AlgoReturnT][source]#

Factory method to create a DigestFieldAction used at the start of a struct to set up hashing for the named digest field.

Parameters:

  • target (str) – Shared identifier between DigestField and its corresponding action.

  • algo (Algorithm) – Digest algorithm (e.g., Sha2_256_Algo)

Returns:

An action to initialize hashing in the struct context.

Return type:

_Action

class caterpillar.fields.DigestFieldAction(target: str, algorithm: Algorithm[_AlgoObjT, _AlgoReturnT])[source]#

Added in version 2.4.5.

Represents an Action used to initialize a digest computation before struct processing begins.

This action is intended to be used as a pre-processing step before serializing or parsing struct fields that are to be hashed. It injects a digest hook into the IO stream to intercept and update digest state as data is read or written.

The following context variables will be populated based on the given target: - _digest_obj__<target>: The current state of the digest computation. (hash object) - _digest_hook__<target>: The IO hook associated with the digest computation. - _digest_algo__<target>: The algorithm used for the digest computation.

Parameters:

  • target (str) – The unique name identifying this digest field.

  • algorithm (Algorithm) – The digest algorithm implementation (must support create/update/digest).

update(data: bytes, context: _ContextLike) → None[source]#

Updates the digest object with new data.

This method is registered with an IO hook so it gets triggered automatically during reads/writes to accumulate digest state.

begin(context: _ContextLike) → None[source]#

Initializes the digest algorithm and attaches an IO hook to track data.

class caterpillar.fields.HMACAlgorithm(key: bytes | _ContextLambda[bytes], algorithm: HashAlgorithm)[source]#

HMAC (Hash-based Message Authentication Code) algorithm implementation.

This class wraps an HMAC algorithm using a specified hash function and key.

create(context: _ContextLike) → hmac.HMAC[source]#

Creates an HMAC object with the provided key and algorithm.

update(algo_obj: hmac.HMAC, data: bytes, context: _ContextLike) → None[source]#

Updates the HMAC object with new data.

digest(algo_obj: hmac.HMAC, context: _ContextLike) → bytes[source]#

Finalizes the HMAC object and returns the computed digest.

class caterpillar.fields.HMAC(key: bytes | _ContextLambda[bytes], algorithm: HashAlgorithm, name: str | None = None, verify: bool = False, path: str | None = None)[source]#

HMAC Digest handler, used to create and verify HMACs based on a provided key and algorithm.

4.8.4.2. Ciphers#

class caterpillar.fields.Encrypted(length: int | EllipsisType | _ContextLambda[int], algorithm: type[CipherAlgorithm], mode: type[modes.Mode] | modes.Mode, padding: Padding | type[Padding] | None = None, algo_args: Iterable[_ContextLambda[Any] | Any] | None = None, mode_args: Iterable[_ContextLambda[Any] | Any] | None = None, padding_args: Iterable[_ContextLambda[Any] | Any] | None = None, post: _StructLike | None = None)[source]#

Struct that is able to encrypt/decrypt blocks of memory.

Parameters:

  • length (Union[int, _GreedyType, _ContextLambda]) – Length of the encrypted data.

  • algorithm (Type[algorithms.CipherAlgorithm]) – Encryption algorithm.

  • mode (Union[Type[modes.Mode], modes.Mode]) – Encryption mode.

  • padding (Union[Padding, Type[Padding]], optional) – Padding scheme for encryption.

  • algo_args (Optional[Iterable[_ArgType]], optional) – Additional arguments for the encryption algorithm.

  • mode_args (Optional[Iterable[_ArgType]], optional) – Additional arguments for the encryption mode.

  • padding_args (Optional[Iterable[_ArgType]], optional) – Additional arguments for the padding scheme.

  • post – Post-processing structure.

algorithm(context: _ContextLike) → CipherAlgorithm[source]#

Get the encryption algorithm instance.

Parameters:

context (_ContextLike) – The current operation context.

Returns:

An instance of the encryption algorithm.

Return type:

algorithms.CipherAlgorithm

mode(context: _ContextLike) → modes.Mode | None[source]#

Get the encryption mode instance.

Parameters:

context (_ContextLike) – The current operation context.

Returns:

An instance of the encryption mode.

Return type:

modes.Mode

padding(context: _ContextLike) → Padding | None[source]#

Get the padding scheme instance.

Parameters:

context (_ContextLike) – The current operation context.

Returns:

An instance of the padding scheme.

Return type:

Padding

get_instance(type_: type[_IT], field: _IT | Any | None, args: dict[str, _ContextLambda[Any] | Any] | Iterable[_ContextLambda[Any] | Any] | None, context: _ContextLike) → _IT | None[source]#

Get an instance of a specified type.

Parameters:

  • type (type) – The desired type of the instance.

  • field (Any) – The field or instance.

  • args (Any) – Additional arguments for the instance.

  • context (_ContextLambda) – The current operation context.

Returns:

An instance of the specified type.

Return type:

Any

pack_single(obj: bytes | memoryview | bytearray, context: _ContextLike) → None[source]#

Pack a single element.

Parameters:

  • obj (Any) – The element to pack.

  • context (_ContextLike) – The current operation context.

unpack_single(context: _ContextLike) → memoryview[source]#

Unpack a single element.

Parameters:

context (_ContextLike) – The current operation context.

Returns:

The unpacked element as a memoryview.

Return type:

memoryview

4.8.4.3. Standard interface#

TODO default functions