Options

TODO

Options by type

Global options

caterpillar.options.O_ARRAY_FACTORY

To control the returned array type, a factory class or method can be set in this option using its attached value. For instance, we can incorporate the numpy.ndarray into our unpacked objects:

from caterpillar.options import O_ARRAY_FACTORY
from numpy import array

# just set the option's value
O_ARRAY_FACTORY.value = array

With the new configuration applied, your unpacked objects will occupy less memory space. The following table shows the size of unpacked objects in bytes:

Object sizes between different configuration options

Configuration	formats/nibarchive	formats/caf [*]
Default configuration	26520	10608
__slots__ classes	14240	3848
Default configuration and numpy.ndarray	7520	1232
__slots__ classes and numpy.ndarray	6152	384
Original filesize	1157	5433

[*]

A CAF audio comes with a special chunk type that stores only zeros. By ingoring the data in this chunk, we can achieve less bytes in memory than the file originally used.

Sequence options

Note

All sequence-related configuration options are applied to structs as well.

caterpillar.options.S_DISCARD_UNNAMED

Using this option, all unnamed fields will be discarded and won’t be visible in the final result object. An unnamed field must follow the following naming convention:

<unnamed> := '_' [0-9]*

Therefore, it is possible to include more than one unnamed field, for example:

Simple sequence with an unnamed field

>>> schema = Seq({
...     "a": uint8,
...     "_": padding[10]
... }, options={opt.S_DISCARD_UNNAMED})
>>> data = b"\xFF" + bytes(10)
>>> unpack(schema, data)
{'a': 255}
>>> pack(_, schema)
b'\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

caterpillar.options.S_DISCARD_CONST

This option will only discard constant fields.

Struct options

caterpillar.options.S_SLOTS

Feature option that will create a new class with the __slots__ attribute to lower required space. Take the following two structs into consideration:

@struct
class DictClass:
    a: uint8
    b: uint8
    c: uint8
    d: uint8
    e: uint8
    f: uint8
    g: uint8

@struct(options={opt.S_SLOTS})
class SlotsClass:
    a: uint8
    b: uint8
    c: uint8
    d: uint8
    e: uint8
    f: uint8
    g: uint8

Each struct stores seven fields in total, whereby Format2 uses __slots__ to store each member. We used pympler to retrieve the amount of occupied bytes per object:

Size difference between a __slots__ class and a normal Python class

>>> o1 = DictClass(*[0xFF]*7)
>>> asizeof.asizeof(o1)
712
>>> o2 = SlotsClass(*[0xFF]*7)
>>> asizeof.asizeof(o2)
120

In addition, the overall used memory will be reduced, because the defined type will also occupy less memory:

>>> from sys import getsizeof
>>> getsizeof(DictClass)
1704
>>> getsizeof(SlotsClass)
936

caterpillar.options.S_REPLACE_TYPES

This option was designed for documentation purposes only and should be used in that context only. It will alter the class’ annotations and remove all caterpillar.fields.Field instances.

Consider the following struct:

@struct
class Format:
    a: uint8
    b: String(10)
    c: uuid

# use the following line to enable type replacement globally
opt.set_struct_flags(opt.S_REPLACE_TYPES)
# otherwise, just add options={opt.S_REPLACE_TYPES} to the
# @struct call.

You will notice the difference in the following output on disabled type replacement (1) and enabled replacement (2):

>>> Format.__annotations__ # (1)
{'a': <FormatField(int) 'B'>, 'b': <String>, 'c': <uuid>}
>>> Format.__annotations__ # (2)
{'a': <class 'int'>, 'b': <class 'str'>, 'c': <class 'uuid.UUID'>}

caterpillar.options.S_EVAL_ANNOTATIONS

If you decide to use from __future__ import annotations, you have to set this option for each struct in the scope of this import, because it will stringify all placed annotations. Therefore, they need to be executed before analyzed.

Caution

Use this option with caution! It may result in execution of untrusted code, be aware!

caterpillar.options.S_UNION

Internal option to add union behaviour to the caterpillar.model.Struct class.

Field options

caterpillar.options.F_KEEP_POSITION

caterpillar.options.F_DYNAMIC

caterpillar.options.F_SEQUENTIAL

caterpillar.options.F_OFFSET_OVERRIDE

Interface

class caterpillar.options.Flag(name: str, value: Any | None = None)

Simple customizable user-flag.

name: str

The name of this flag

value: Any | None = None

Optional configuration value.

caterpillar.options.configure(base: Set[Flag], *flags: Flag) → None

Update the base set of flags with additional flags.

Parameters:

• base – The base set of flags.

• flags – Additional flags to be added.

caterpillar.options.set_struct_flags(*flags: Flag, with_union: bool = False) → None

Set default flags for structs.

Parameters:

• flags – Flags to be set.

• with_union – Whether to include the flags for unions as well.

caterpillar.options.set_field_flags(*flags: Flag) → None

Set default flags for fields.

Parameters:

flags – Flags to be set.

caterpillar.options.set_union_flags(*flags: Flag) → None

Set default flags for unions.

Parameters:

flags – Flags to be set.

caterpillar.options.get_flags(obj: Any, attr: str | None = None) → Set[Flag] | None

Get the flags associated with an object.

Parameters:

• obj – The object.

• attr – The attribute name containing the flags (default is “flags”).

Returns:

The set of flags, or None if no flags are present.

caterpillar.options.has_flag(flag: str | Flag, obj: Any, attr: str | None = None) → bool

Check if an object has a specific flag.

Parameters:

• flag – The flag to check, either as a string or a Flag instance.

• obj – The object.

• attr – The attribute name containing the flags (default is “flags”).

Returns:

True if the flag is present, False otherwise.

caterpillar.options.get_flag(name: str, obj: Any, attr: str | None = None) → Flag | None

Get a specific flag associated with an object.

Parameters:

• name – The name of the flag.

• obj – The object.

• attr – The attribute name containing the flags (default is “flags”).

Returns:

The Flag instance if found, otherwise None.