Skip to main content

Crate nautilus_serialization

Crate nautilus_serialization 

Source
Expand description

Data serialization and format conversion for NautilusTrader.

The nautilus-serialization crate provides data serialization capabilities for converting trading data between different formats including Apache Arrow, JSON, MessagePack, and SBE. This enables efficient data storage, retrieval, and interoperability across different systems:

  • Apache Arrow integration: Schema definitions and encoding/decoding for market data types.
  • Parquet file operations: High-performance columnar storage for historical data analysis.
  • Record batch processing: Efficient batch operations for time-series data.
  • Schema management: Type-safe schema definitions with metadata preservation.
  • Cross-format conversion: Data interchange between Arrow and native types.
  • SBE decode utilities: Zero-copy cursor and shared decode errors for SBE parsers (requires sbe feature).

§NautilusTrader

NautilusTrader is an open-source, production-grade, Rust-native engine for multi-asset, multi-venue trading systems.

The system spans research, deterministic simulation, and live execution within a single event-driven architecture, providing research-to-live semantic parity.

§Feature Flags

This crate provides feature flags to control source code inclusion during compilation, depending on the intended use case, i.e. whether to provide Python bindings for the nautilus_trader Python package, or as part of a Rust only build.

  • arrow: Enables Apache Arrow schema definitions and RecordBatch encoding/decoding.
  • arrow-display: Enables display-friendly Arrow encoders for market data and requires arrow.
  • extension-module: Builds as a Python extension module.
  • high-precision: Enables high-precision mode to use 128-bit value types.
  • python: Enables Python bindings from PyO3.
  • sbe: Enables generic SBE (Simple Binary Encoding) decode utilities.

Warning: SBE schemas are not yet stable and may break between releases.

Re-exports§

pub use arrow::custom::ensure_custom_data_registered;

Modules§

arrow
Defines the Apache Arrow schema for Nautilus types.
msgpack
Re-exports MsgPack codecs for consumers configuring serialization through this crate. Provides MsgPack serialization support for types implementing [Serializable].
python
Python bindings from PyO3.
sbe
Generic SBE (Simple Binary Encoding) codec utilities.

Attribute Macros§

arrow_custom_data
Generates typed Arrow encoding for custom data. Adds Arrow schema, encoding, and decoding implementations to a custom data struct.