FFI Memory Contract

NautilusTrader exposes several C-compatible types so that compiled Rust code can be consumed from C-extensions generated by Cython or by other native languages. The most important of these is CVec – a thin wrapper around a Rust Vec<T> that is passed across the FFI boundary by value.

The rules below are strict; violating them results in undefined behaviour (usually a double-free or a memory leak).

CVec lifecycle

Step	Owner	Action
1	Rust	Build a Vec<T> and convert it with into() – this leaks the vector and transfers ownership of the raw allocation to foreign code.
2	Foreign (Python / Cython / C)	Use the data while the CVec value is in scope. Do not modify the fields ptr, len, cap.
3	Foreign	Exactly once, call the type-specific drop helper exported by Rust (for example vec_drop_book_levels, vec_drop_book_orders, vec_time_event_handlers_drop). The helper reconstructs the original Vec<T> with Vec::from_raw_parts and lets it drop, freeing the memory.

warning

If step 3 is forgotten the allocation is leaked for the remainder of the process; if it is performed twice the program will double-free and likely crash.

Capsules created on the Python side

Several Cython helpers allocate temporary C buffers with PyMem_Malloc, wrap them into a CVec, and return the address inside a PyCapsule. Every such capsule is created with a destructor (capsule_destructor or capsule_destructor_deltas) that frees both the buffer and the CVec. Callers must therefore not free the memory manually – doing so would double free.

Capsules created on the Rust side (PyO3 bindings)

When Rust code pushes a heap-allocated value into Python it must use PyCapsule::new_with_destructor so that Python knows how to free the allocation once the capsule becomes unreachable. The closure/destructor is responsible for reconstructing the original Box<T> or Vec<T> and letting it drop.

Python::with_gil(|py| {
    // allocate the value on the heap
    let my_data = MyStruct::new();

    // move it into the capsule and register a destructor
    let capsule = pyo3::types::PyCapsule::new_with_destructor(py, my_data, None, |_, _| {})
        .expect("capsule creation failed");

    // ... pass `capsule` back to Python ...
});

Do not use PyCapsule::new(…, None); that variant registers no destructor and will leak memory unless the recipient manually extracts and frees the pointer (something we never rely on). The codebase has been updated to follow this rule everywhere – adding new FFI modules must follow the same pattern.

Why there is no generic cvec_drop anymore

Earlier versions of the codebase shipped a generic cvec_drop function that always treated the buffer as Vec<u8>. Using it with any other element type causes a size-mismatch during deallocation and corrupts the allocator’s bookkeeping. Because the helper was not referenced anywhere inside the project it has been removed to avoid accidental misuse.

Box-backed *_API wrappers (owned Rust objects)

When the Rust core needs to hand a complex value (for example an OrderBook, SyntheticInstrument, or TimeEventAccumulator) to foreign code it allocates the value on the heap with Box::new and returns a small repr(C) wrapper whose only field is that Box.

#[repr(C)]
pub struct OrderBook_API(Box<OrderBook>);

#[unsafe(no_mangle)]
pub extern "C" fn orderbook_new(id: InstrumentId, book_type: BookType) -> OrderBook_API {
    OrderBook_API(Box::new(OrderBook::new(id, book_type)))
}

#[unsafe(no_mangle)]
pub extern "C" fn orderbook_drop(book: OrderBook_API) {
    drop(book); // frees the heap allocation
}

Memory-safety requirements are therefore:

1. Every constructor (*_new) must have a matching *_drop exported next to it.

2. The Python/Cython binding must guarantee that *_drop is invoked exactly once. Two approaches are accepted:

   • Wrap the pointer in a PyCapsule created with PyCapsule::new_with_destructor, passing a destructor that calls the drop helper.

   • Call the helper explicitly in __del__/__dealloc__ on the Python side. This is the historical pattern for most v1 Cython modules:

   cdef class OrderBook:
       cdef OrderBook_API _mem
   
       def __cinit__(self, ...):
           self._mem = orderbook_new(...)
   
       def __del__(self):
           if self._mem._0 != NULL:
               orderbook_drop(self._mem)

Whichever style is used, remember: forgetting the drop call leaks the entire structure, while calling it twice will double-free and crash.

New FFI code must follow this template before it can be merged.