Options#

TODO

Options by type#

Global options#

caterpillar.options.O_ARRAY_FACTORY#

To control the returned array type, a factory class or method can be set in this option using its attached value. For instance, we can incorporate the numpy.ndarray into our unpacked objects:

from caterpillar.options import O_ARRAY_FACTORY
from numpy import array

# just set the option's value
O_ARRAY_FACTORY.value = array

With the new configuration applied, your unpacked objects will occupy less memory space. The following table shows the size of unpacked objects in bytes:

Object sizes between different configuration options#
Configuration	`formats/nibarchive`	`formats/caf` [*]
Default configuration	26520	10608
`__slots__` classes	14240	3848
Default configuration and `numpy.ndarray`	7520	1232
`__slots__` classes and `numpy.ndarray`	6152	384
Original filesize	1157	5433

Sequence options#

Note

All sequence-related configuration options are applied to structs as well.

caterpillar.options.S_DISCARD_UNNAMED#

Using this option, all unnamed fields will be discarded and won’t be visible in the final result object. An unnamed field must follow the following naming convention:

<unnamed> := '_' [0-9]*

Therefore, it is possible to include more than one unnamed field, for example:

Simple sequence with an unnamed field#

>>> schema = Seq({
...     "a": uint8,
...     "_": padding[10]
... }, options={opt.S_DISCARD_UNNAMED})
>>> data = b"\xFF" + bytes(10)
>>> unpack(schema, data)
{'a': 255}
>>> pack(_, schema)
b'\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

caterpillar.options.S_DISCARD_CONST#: This option will only discard constant fields.

Struct options#

caterpillar.options.S_SLOTS#

Feature option that will create a new class with the __slots__ attribute to lower required space. Take the following two structs into consideration:

@struct
class DictClass:
    a: uint8
    b: uint8
    c: uint8
    d: uint8
    e: uint8
    f: uint8
    g: uint8

@struct(options={opt.S_SLOTS})
class SlotsClass:
    a: uint8
    b: uint8
    c: uint8
    d: uint8
    e: uint8
    f: uint8
    g: uint8

Each struct stores seven fields in total, whereby Format2 uses __slots__ to store each member. We used pympler to retrieve the amount of occupied bytes per object:

Size difference between a __slots__ class and a normal Python class#

>>> o1 = DictClass(*[0xFF]*7)
>>> asizeof.asizeof(o1)
712
>>> o2 = SlotsClass(*[0xFF]*7)
>>> asizeof.asizeof(o2)
120

In addition, the overall used memory will be reduced, because the defined type will also occupy less memory:

>>> from sys import getsizeof
>>> getsizeof(DictClass)
1704
>>> getsizeof(SlotsClass)
936

caterpillar.options.S_REPLACE_TYPES#

This option was designed for documentation purposes only and should be used in that context only. It will alter the class’ annotations and remove all caterpillar.fields.Field instances.

Consider the following struct:

@struct
class Format:
    a: uint8
    b: String(10)
    c: uuid

# use the following line to enable type replacement globally
opt.set_struct_flags(opt.S_REPLACE_TYPES)
# otherwise, just add options={opt.S_REPLACE_TYPES} to the
# @struct call.

You will notice the difference in the following output on disabled type replacement (1) and enabled replacement (2):

>>> Format.__annotations__ # (1)
{'a': <FormatField(int) 'B'>, 'b': <String>, 'c': <uuid>}
>>> Format.__annotations__ # (2)
{'a': <class 'int'>, 'b': <class 'str'>, 'c': <class 'uuid.UUID'>}

caterpillar.options.S_EVAL_ANNOTATIONS#: If you decide to use from __future__ import annotations, you have to set this option for each struct in the scope of this import, because it will stringify all placed annotations. Therefore, they need to be executed before analyzed.

Caution

Use this option with caution! It may result in execution of untrusted code, be aware!

caterpillar.options.S_UNION#: Internal option to add union behaviour to the caterpillar.model.Struct class.

Field options#

caterpillar.options.F_KEEP_POSITION#

caterpillar.options.F_DYNAMIC#

caterpillar.options.F_SEQUENTIAL#

caterpillar.options.F_OFFSET_OVERRIDE#

Interface#

class caterpillar.options.Flag(name: str, value: Any | None = None)[source]#

Simple customizable user-flag.

name: str#: The name of this flag

value: Any | None = None#: Optional configuration value.

caterpillar.options.configure(base: Set[Flag], *flags: Flag) → None[source]#

Update the base set of flags with additional flags.

Parameters:

base – The base set of flags.
flags – Additional flags to be added.

caterpillar.options.set_struct_flags(*flags: Flag, with_union: bool = False) → None[source]#

Set default flags for structs.

Parameters:

flags – Flags to be set.
with_union – Whether to include the flags for unions as well.

caterpillar.options.set_field_flags(*flags: Flag) → None[source]#

Set default flags for fields.

Parameters:: flags – Flags to be set.

caterpillar.options.set_union_flags(*flags: Flag) → None[source]#

Set default flags for unions.

Parameters:: flags – Flags to be set.

caterpillar.options.get_flags(obj: Any, attr: str | None = None) → Set[Flag] | None[source]#

Get the flags associated with an object.

Parameters:

obj – The object.
attr – The attribute name containing the flags (default is “flags”).

Returns:

The set of flags, or None if no flags are present.

caterpillar.options.has_flag(flag: str | Flag, obj: Any, attr: str | None = None) → bool[source]#

Check if an object has a specific flag.

Parameters:

flag – The flag to check, either as a string or a Flag instance.
obj – The object.
attr – The attribute name containing the flags (default is “flags”).

Returns:

True if the flag is present, False otherwise.

caterpillar.options.get_flag(name: str, obj: Any, attr: str | None = None) → Flag | None[source]#

Get a specific flag associated with an object.

Parameters:

name – The name of the flag.
obj – The object.
attr – The attribute name containing the flags (default is “flags”).

Returns:

The Flag instance if found, otherwise None.