# NautilusTrader Documentation Source: https://nautilustrader.io/docs/latest/ # NautilusTrader Documentation NautilusTrader is an open-source, production-grade, Rust-native trading engine infrastructure for multi-asset, multi-venue trading systems. Its Rust-native core spans research, deterministic simulation, portfolio and risk modeling, and live execution under one event-driven architecture. Python serves as the control plane for strategy logic, configuration, and orchestration, while entire systems can also be built in Rust. NautilusTrader is engineered to minimize the gap between research and production. The same execution semantics and time model apply in both backtests and live systems, reducing deployment risk and unexpected behavior. Modular adapters keep venue integration neutral. Current integrations span crypto venues, traditional markets, betting exchanges, and data providers, with custom adapters available for REST, WebSocket, and provider-specific APIs. ## Evaluation areas - [Architecture](concepts/architecture): event-driven design, core components, environment contexts, fail-fast behavior, and recovery model. - [Backtesting](concepts/backtesting): historical simulation APIs, data-loading contracts, streaming workflows, and post-run analysis. - [Live trading](concepts/live): live command outcomes, execution reconciliation, startup recovery, and operational caveats. - [Integrations](integrations): supported venues and data providers, adapter status, and adapter implementation goals. - [Developer guide](developer_guide): Rust and Python internals, testing standards, adapter specs, and release process. }> Install NautilusTrader and run your first backtests. }> Architecture, data, execution, backtesting, live trading, and core domain model guides. }> Goal-oriented recipes for data workflows, backtesting, live trading, and common operational tasks. }> Runnable walkthroughs for backtesting, data workflows, strategy patterns, options, and Rust. }> Supported venue and data-provider adapters, status, and setup guides. }> Rust and Python internals, testing standards, adapter specs, and release process. # Accounting Source: https://nautilustrader.io/docs/latest/concepts/accounting/ The accounting subsystem tracks balances, margins, and PnL for every account the platform interacts with. This guide covers the data model, the query API that strategies use, and the conventions adapter authors must follow to stay consistent across venues. It applies equally to backtest and live trading. For backtest-specific configuration (starting balances, margin-model selection per venue), see [Backtesting](backtesting.md). ## Account types When you attach a venue to the engine for either live trading or a backtest, you pick one of three accounting modes via `account_type`: | Account type | Typical use case | What the engine locks | | ------------ | ------------------------------------------------ | ------------------------------------------------------------------------- | | Cash | Spot trading (e.g., BTC/USDT, stocks) | Notional value for every position a pending order would open. | | Margin | Derivatives or any product that allows leverage | Initial margin for each order plus maintenance margin for open positions. | | Betting | Sports betting, bookmaking | Stake required by the venue; no leverage. | ### Cash accounts Cash accounts settle trades in full; there is no leverage and therefore no concept of margin. Locked balances reflect the notional reserved for pending orders. ### Margin accounts Margin accounts support instruments that require collateral, such as futures or leveraged crypto perps. They track account balances, reserve margin for open orders and positions, and apply a configurable leverage per instrument. Margin is tracked in two scopes; see [Margin scopes](#margin-scopes) below. **Key terms**: - **Leverage**: amplifies exposure relative to account equity. Higher leverage raises both potential returns and risk. - **Initial margin**: collateral reserved when an order is submitted. - **Maintenance margin**: minimum collateral required to keep an open position. - **Locked balance**: funds reserved as collateral, not available for new orders. :::note Reduce-only orders do not contribute to `balance_locked` on cash accounts and do not add to initial margin on margin accounts, since they can only decrease exposure. ::: ### Betting accounts Betting accounts are specialised for venues where you stake an amount to win or lose a fixed payout (prediction markets, sports books). The engine locks only the stake required by the venue; leverage and margin do not apply. ## Balance model An `AccountBalance` holds three values in the same currency: - `total`: the venue-reported total balance figure (wallet, net liquidation, or margin balance, depending on the venue). - `locked`: amount reserved against open orders and positions. - `free`: amount available for new orders (`total - locked`). The invariant `total == locked + free` must always hold at currency precision. The Python `AccountBalance(total, locked, free)` constructor requires all three fields up front. Adapter code written in Rust has two additional derived constructors that enforce the invariant centrally; prefer them over `AccountBalance::new` whenever the venue reports only two of the three values: | Rust helper | When to use | | --------------------------------------- | ------------------------------------------------------------------------------ | | `AccountBalance::from_total_and_locked` | Venue reports total and locked; `free` is derived and clamped to `[0, total]`. | | `AccountBalance::from_total_and_free` | Venue reports total and free; `locked` is derived and clamped. | | `AccountBalance::new` | All three values are already known and consistent (tests, pass‑through). | The helpers clamp the derived field to `[0, total]` when `total >= 0`, so transient overshoots from venue rounding never leave the account in a broken state. ## Margin scopes A `MarginBalance` has four fields: `initial`, `maintenance`, `currency`, and an `Optional[InstrumentId]` that selects one of two scopes. ### Per-instrument scope `MarginBalance.instrument_id` is set to a concrete instrument. Use this for: - Isolated margin (per-position collateral). - Backtest or calculated margin, where the `AccountsManager` derives margin locally from open orders and positions per instrument. ### Account-wide scope `MarginBalance.instrument_id` is `None`. The entry is keyed by its `currency` (the collateral currency). Use this for cross-margin venues that report a single aggregate per collateral currency. A venue may emit one account-wide entry (single-collateral cross margin) or several (one per collateral coin). Both scopes coexist on the same `MarginAccount` in separate internal stores. An `AccountState` event may carry entries in either or both scopes, and `MarginAccount.apply()` routes each entry to the correct store based on whether `instrument_id` is set. :::note `MarginAccount.apply()` **replaces** both stores from the incoming event. It does not merge with prior state. Adapters that emit partial snapshots must include every live margin entry on each update or those entries will be dropped until the next full snapshot. The balances list is likewise replaced. ::: ## Strategy query API Use the query that matches the venue's reporting shape. If a venue reports per-instrument margins, ask by `InstrumentId`. If it reports account-wide margins, ask by `Currency`. | Scope of the value you want | Use | | -------------------------------------- | ----------------------------------------------------------------------------------------------- | | Per‑instrument margin (isolated) | `margin(id)` / `margin_init(id)` / `margin_maint(id)` | | Account‑wide margin for one collateral | `margin_for_currency(ccy)` / `margin_init_for_currency(ccy)` / `margin_maint_for_currency(ccy)` | | Combined total across both scopes | `total_margin_init(ccy)` / `total_margin_maint(ccy)` | Point queries return `None` when the entry is absent; total queries always return a `Money` (zero for the currency if nothing matches). :::note The names below are the Python / Cython API on `MarginAccount`. Rust strategies using the `nautilus-model` crate call `account_margin(¤cy)`, `account_initial_margin(¤cy)`, `account_maintenance_margin(¤cy)`, `total_initial_margin(currency)`, and `total_maintenance_margin(currency)`: the same split by `Option`, with different method names. ::: ### Per-instrument queries (`MarginAccount`) - `margin(instrument_id) -> MarginBalance | None` - `margin_init(instrument_id) -> Money | None` - `margin_maint(instrument_id) -> Money | None` - `margins() -> dict[InstrumentId, MarginBalance]` (all per-instrument entries) - `margins_init() -> dict[InstrumentId, Money]` - `margins_maint() -> dict[InstrumentId, Money]` These methods only see the per-instrument store. On a cross-margin venue they return empty dicts or `None`. Use the account-wide queries below. ### Account-wide queries (`MarginAccount`) - `margin_for_currency(currency) -> MarginBalance | None` - `margin_init_for_currency(currency) -> Money | None` - `margin_maint_for_currency(currency) -> Money | None` - `account_margins() -> dict[Currency, MarginBalance]` (all account-wide entries) - `account_margins_init() -> dict[Currency, Money]` - `account_margins_maint() -> dict[Currency, Money]` ### Totals (`MarginAccount`) These sum across per-instrument and account-wide entries for a given currency: - `total_margin_init(currency) -> Money` - `total_margin_maint(currency) -> Money` Useful when a strategy trades on a venue where both scopes may appear (for example, isolated positions alongside cross-margin collateral). ### Clearing account-wide entries - `clear_account_margin(currency)` removes the account-wide entry for a given collateral currency and triggers a balance recalculation. The counterpart for per-instrument entries is `clear_margin(instrument_id)`. These are system methods; adapter code calls them implicitly via `MarginAccount.apply()`. Strategies should not need them directly. ### Portfolio-level queries Margin queries: - `portfolio.margins_init(venue=..., account_id=...) -> dict[InstrumentId, Money]` - `portfolio.margins_maint(venue=..., account_id=...) -> dict[InstrumentId, Money]` These mirror `MarginAccount.margins_init` / `margins_maint` and return only the per-instrument entries. For account-wide data on cross-margin venues, query the account directly via `portfolio.account(venue).margin_init_for_currency(ccy)`. PnL, exposure, mark-to-market, and equity queries all accept `venue` and an optional `account_id` to scope multi-account venues: - `portfolio.unrealized_pnls(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.realized_pnls(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.total_pnls(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.net_exposures(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.mark_values(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.equity(venue=..., account_id=...) -> dict[Currency, Money]` - `portfolio.missing_price_instruments(venue) -> list[InstrumentId]` See the [Portfolio guide](portfolio.md#equity-and-mark-to-market) for the equity formula, price fallback chain, base-currency conversion behavior, and the warn-once missing-price tracker. ### Worked examples Single-collateral cross margin (one account-wide entry): ```python usdc_margin = margin_account.margin_init_for_currency(USDC) usdc_total = margin_account.total_margin_init(USDC) ``` Per-coin cross margin (one entry per collateral currency): ```python for ccy, margin_balance in margin_account.account_margins().items(): print(ccy, margin_balance.initial, margin_balance.maintenance) ``` ## Margin models NautilusTrader provides flexible margin calculation models for the calculated path (backtests, and live strategies running with `calculate_account_state=True` for reconciliation). Reported margins from a venue flow straight into `_account_margins` or `_margins` without going through a model. ### Overview Different venues treat leverage differently: - **Traditional brokers** (Interactive Brokers, TD Ameritrade): fixed margin percentages regardless of leverage. - **Crypto exchanges** (Binance, others): leverage may reduce margin requirements. Both built-in models compute margin as a percentage of notional using the instrument's `margin_init` and `margin_maint` fields. They differ only in whether leverage reduces the reservation. For venues with true per-contract fixed margin (CME / ICE), set `instrument.margin_init` and `margin_maint` so the percentage recovers the desired dollar amount, or implement a [custom model](#custom-models). ### HEDGING-mode netting Under `OmsType.HEDGING` each fill opens its own `Position`, so an account can hold many open sub-positions for the same instrument. The accounts manager nets those sub-positions onto a hypothetical NETTING position in `ts_opened` order, then runs the margin model once on the resulting net signed quantity and average open price. The replay follows the same rules as `Position.apply`: same-side fills produce a quantity-weighted average open price, opposite-side fills partial-close at the existing average, and a fill that crosses zero makes the residual take the flipping fill's price. Sub-positions sharing a `ts_opened` fold in `(ts_opened, position_id)` order so the result does not depend on cache iteration order. HEDGING and NETTING accounts compute the same maintenance margin for the same fill sequence; the requirement scales with net economic exposure. ### Available models #### `StandardMarginModel` Uses fixed percentages without leverage division, matching traditional broker behavior. ```python # Fixed percentages - leverage ignored margin = notional * instrument.margin_init ``` - Initial margin: `notional_value * instrument.margin_init` - Maintenance margin: `notional_value * instrument.margin_maint` **Use cases**: traditional brokers (Interactive Brokers), forex brokers with fixed margin requirements. #### `LeveragedMarginModel` Divides margin requirements by leverage. ```python # Leverage reduces margin requirements adjusted_notional = notional / leverage margin = adjusted_notional * instrument.margin_init ``` - Initial margin: `(notional_value / leverage) * instrument.margin_init` - Maintenance margin: `(notional_value / leverage) * instrument.margin_maint` **Use cases**: crypto exchanges that reduce margin with leverage, venues where leverage affects margin requirements. ### Default behavior `MarginAccount` uses `LeveragedMarginModel` by default. Override programmatically: ```python from nautilus_trader.backtest.models import LeveragedMarginModel from nautilus_trader.backtest.models import StandardMarginModel from nautilus_trader.test_kit.stubs.execution import TestExecStubs account = TestExecStubs.margin_account() # Traditional broker behavior account.set_margin_model(StandardMarginModel()) # Or the leveraged model (default) account.set_margin_model(LeveragedMarginModel()) ``` ### Worked example: EUR/USD - **Instrument**: EUR/USD - **Quantity**: 100,000 EUR - **Price**: 1.10000 - **Notional**: $110,000 - **Leverage**: 50x - **`instrument.margin_init`**: 3% | Model | Calculation | Result | Percentage | | --------- | ---------------------- | ------ | ---------- | | Standard | $110,000 × 0.03 | $3,300 | 3.00% | | Leveraged | ($110,000 ÷ 50) × 0.03 | $66 | 0.06% | On a $10,000 account: the standard model blocks the trade; the leveraged model allows it. ### Custom models Subclass `MarginModel` and receive configuration through `MarginModelConfig`: ```python from decimal import Decimal from nautilus_trader.backtest.config import MarginModelConfig from nautilus_trader.backtest.models import MarginModel from nautilus_trader.model.objects import Money class RiskAdjustedMarginModel(MarginModel): def __init__(self, config: MarginModelConfig) -> None: self.risk_multiplier = Decimal(str(config.config.get("risk_multiplier", 1.0))) self.use_leverage = config.config.get("use_leverage", False) def calculate_margin_init(self, instrument, quantity, price, leverage, use_quote_for_inverse=False): notional = instrument.notional_value(quantity, price, use_quote_for_inverse) if self.use_leverage: adjusted = notional.as_decimal() / leverage else: adjusted = notional.as_decimal() margin = adjusted * instrument.margin_init * self.risk_multiplier return Money(margin, instrument.quote_currency) def calculate_margin_maint(self, instrument, side, quantity, price, leverage, use_quote_for_inverse=False): return self.calculate_margin_init(instrument, quantity, price, leverage, use_quote_for_inverse) ``` For backtest-wide configuration of the margin model via `BacktestVenueConfig` and `MarginModelConfig`, see the margin-models section of [Backtesting](backtesting.md#margin-models). ## Adapter convention Live adapters translate venue responses into `AccountBalance` and `MarginBalance` instances. The convention that adapter authors must follow: ### Building `AccountBalance` Prefer the derived helpers so that clamping and the `total == locked + free` invariant are enforced centrally. Hand-computing three fields and passing them to `AccountBalance::new` is only appropriate for pass-through paths where all three values are already authoritative (e.g., tests). ### Building `MarginBalance` Pick the scope that matches what the venue reports: | Venue reports | Scope | Emit with | | ---------------------------------------------- | -------------- | ---------------------------------------------------------- | | Per‑instrument (isolated positions) | Per‑instrument | `MarginBalance::new(initial, maint, Some(id))` | | Single aggregate per collateral (cross margin) | Account‑wide | `MarginBalance::new(initial, maint, None)` | | Multiple aggregates, one per collateral | Account‑wide | One `MarginBalance` per currency with `instrument_id=None` | :::note Synthetic `ACCOUNT.{VENUE}` or `ACCOUNT-{COIN}.{VENUE}` `InstrumentId` placeholders are not used. Account-wide entries carry `instrument_id=None` and are keyed by `currency`. ::: ## Related guides - [Backtesting](backtesting.md): starting balances, `MarginModelConfig`, and backtest-specific account setup. - [Portfolio](portfolio.md): portfolio-level PnL, exposures, and currency conversion. - [Positions](positions.md): position lifecycle, aggregation, and PnL. - [Adapters](adapters.md): requirements and best practices for adapter authors. # Actors Source: https://nautilustrader.io/docs/latest/concepts/actors/ An `Actor` receives data, handles events, and manages state. The `Strategy` class extends Actor with order management capabilities. **Key capabilities**: - Data subscription and requests (market data, custom data). - Event handling and publishing. - Timers and alerts. - Cache and portfolio access. - Logging. ## Basic example Actors support configuration through a pattern similar to strategies. ```python from nautilus_trader.config import ActorConfig from nautilus_trader.model import InstrumentId from nautilus_trader.model import Bar, BarType from nautilus_trader.common.actor import Actor class MyActorConfig(ActorConfig): instrument_id: InstrumentId # example value: "ETHUSDT-PERP.BINANCE" bar_type: BarType # example value: "ETHUSDT-PERP.BINANCE-15-MINUTE[LAST]-INTERNAL" lookback_period: int = 10 class MyActor(Actor): def __init__(self, config: MyActorConfig) -> None: super().__init__(config) # Custom state variables self.count_of_processed_bars: int = 0 def on_start(self) -> None: # Subscribe to bars matching the configured bar type self.subscribe_bars(self.config.bar_type) def on_bar(self, bar: Bar) -> None: self.count_of_processed_bars += 1 ``` ## Actor configuration and IDs Actors can receive an `ActorConfig` subclass. The base config may include an `actor_id`; if supplied, the actor registers with that ID. If omitted, the system derives a runtime actor ID. Treat configuration as construction data for the actor. Read user-supplied settings through `self.config`, and keep runtime state on the actor itself. :::info Rust implementation For Rust actors, generated or assigned runtime IDs live on the actor core rather than being written back into `DataActorConfig`. This differs from Python bridge paths which may copy inherited config fields into runtime state when a Python object is created from an importable config. Rust authors implement `DataActor` and use the facade methods on `self`. `DataActorNative` is native-only access for runtime wiring and borrowed core state. Import it only for same-binary performance paths or internal runtime wiring. ::: ## Lifecycle Actors follow a defined state machine through their lifecycle: ```mermaid stateDiagram-v2 [*] --> PRE_INITIALIZED PRE_INITIALIZED --> READY : register() READY --> STARTING : start() STARTING --> RUNNING : on_start() RUNNING --> STOPPING : stop() STOPPING --> STOPPED : on_stop() STOPPED --> RUNNING : resume() RUNNING --> DEGRADING : degrade() DEGRADING --> DEGRADED : on_degrade() DEGRADED --> RUNNING : resume() RUNNING --> FAULTING : fault() FAULTING --> FAULTED : on_fault() RUNNING --> DISPOSED : dispose() ``` Override these methods to hook into lifecycle events: | Method | When called | |-----------------|---------------------------------------------------------------------| | `on_start()` | Actor is starting (subscribe to data here). | | `on_stop()` | Actor is stopping (cancel timers, clean up resources). | | `on_resume()` | Actor is resuming from a stopped state. | | `on_reset()` | Reset indicators and internal state (called between backtest runs). | | `on_degrade()` | Actor is entering a degraded state (partial functionality). | | `on_fault()` | Actor has encountered a fault. | | `on_dispose()` | Actor is being disposed (final cleanup). | ## Timers and alerts Actors have access to a clock for scheduling: ```python def on_start(self) -> None: # Set a recurring timer with a callback (fires every 5 seconds) self.clock.set_timer( "my_timer", timedelta(seconds=5), callback=self._on_timer, ) # Set a one-time alert with a callback self.clock.set_time_alert( "my_alert", self.clock.utc_now() + timedelta(minutes=1), callback=self._on_alert, ) def on_stop(self) -> None: # Cancel timers to prevent resource leaks across stop/resume cycles self.clock.cancel_timer("my_timer") def _on_timer(self, event: TimeEvent) -> None: self.log.info("Timer fired!") def _on_alert(self, event: TimeEvent) -> None: self.log.info("Alert triggered!") ``` Pass a `callback` to direct `TimeEvent` objects to your own method. If you omit the callback, the event is delivered to `on_event` instead. ## System access Actors have access to core system components: | Property | Description | |-------------------|----------------------------------------------------------| | `self.cache` | Shared state for instruments, orders, positions, etc. | | `self.portfolio` | Portfolio state and calculations. | | `self.clock` | Current time and timer/alert scheduling. | | `self.log` | Structured logging. | | `self.msgbus` | Publish/subscribe to custom messages. | For custom messaging between components, see the [Message Bus](message_bus.md) guide. ## Data handling and callbacks The system uses different callback handlers depending on whether data is historical or real-time. Understanding the relationship between data *requests/subscriptions* and their handlers is key. ### Historical vs real-time data The system distinguishes between two data flows: 1. **Historical data** (from *requests*): - Obtained through methods like `request_bars()`, `request_quote_ticks()`, etc. - Processed through the `on_historical_data()` handler. - Used for initial data loading and historical analysis. 2. **Real-time data** (from *subscriptions*): - Obtained through methods like `subscribe_bars()`, `subscribe_quote_ticks()`, etc. - Processed through specific handlers like `on_bar()`, `on_quote_tick()`, etc. - Used for live data processing. ### Callback handlers Different data operations map to these handlers: | Operation | Category | Handler | Purpose | |--------------------------------------|------------|--------------------------|---------------------------------------------------| | `subscribe_data()` | Real‑time | `on_data()` | Live data updates. | | `subscribe_instrument()` | Real‑time | `on_instrument()` | Live instrument definition updates. | | `subscribe_instruments()` | Real‑time | `on_instrument()` | Live instrument definition updates (for venue). | | `subscribe_order_book_deltas()` | Real‑time | `on_order_book_deltas()` | Live order book deltas. | | `subscribe_order_book_depth()` | Real‑time | `on_order_book_depth()` | Live order book depth snapshots. | | `subscribe_order_book_at_interval()` | Real‑time | `on_order_book()` | Live order book snapshots at intervals. | | `subscribe_quote_ticks()` | Real‑time | `on_quote_tick()` | Live quote updates. | | `subscribe_trade_ticks()` | Real‑time | `on_trade_tick()` | Live trade updates. | | `subscribe_mark_prices()` | Real‑time | `on_mark_price()` | Live mark price updates. | | `subscribe_index_prices()` | Real‑time | `on_index_price()` | Live index price updates. | | `subscribe_bars()` | Real‑time | `on_bar()` | Live bar updates. | | `subscribe_funding_rates()` | Real‑time | `on_funding_rate()` | Live funding rate updates. | | `subscribe_instrument_status()` | Real‑time | `on_instrument_status()` | Live instrument status updates. | | `subscribe_instrument_close()` | Real‑time | `on_instrument_close()` | Live instrument close updates. | | `subscribe_option_greeks()` | Real‑time | `on_option_greeks()` | Live option greeks updates. | | `subscribe_option_chain()` | Real‑time | `on_option_chain()` | Live option chain slice snapshots. | | `subscribe_order_fills()` | Real‑time | `on_order_filled()` | Live order fill events for an instrument. | | `subscribe_order_cancels()` | Real‑time | `on_order_canceled()` | Live order cancel events for an instrument. | | `request_data()` | Historical | `on_historical_data()` | Historical data processing. | | `request_order_book_deltas()` | Historical | `on_historical_data()` | Historical order book deltas. | | `request_order_book_depth()` | Historical | `on_historical_data()` | Historical order book depth. | | `request_order_book_snapshot()` | Historical | `on_historical_data()` | Historical order book snapshot. | | `request_instrument()` | Historical | `on_instrument()` | Instrument definition. | | `request_instruments()` | Historical | `on_instrument()` | Instrument definitions. | | `request_quote_ticks()` | Historical | `on_historical_data()` | Historical quotes processing. | | `request_trade_ticks()` | Historical | `on_historical_data()` | Historical trades processing. | | `request_bars()` | Historical | `on_historical_data()` | Historical bars processing. | | `request_aggregated_bars()` | Historical | `on_historical_data()` | Historical aggregated bars (on‑the‑fly). | | `request_funding_rates()` | Historical | `on_historical_data()` | Historical funding rates processing. | ### Example This example shows both historical and real-time data handling: ```python from nautilus_trader.common.actor import Actor from nautilus_trader.config import ActorConfig from nautilus_trader.core.data import Data from nautilus_trader.model import Bar, BarType from nautilus_trader.model import ClientId, InstrumentId class MyActorConfig(ActorConfig): instrument_id: InstrumentId # example value: "AAPL.XNAS" bar_type: BarType # example value: "AAPL.XNAS-1-MINUTE-LAST-EXTERNAL" class MyActor(Actor): def __init__(self, config: MyActorConfig) -> None: super().__init__(config) self.bar_type = config.bar_type def on_start(self) -> None: # Request historical data - will be processed by on_historical_data() handler self.request_bars( bar_type=self.bar_type, # Many optional parameters start=None, # pd.Timestamp | None end=None, # pd.Timestamp | None callback=None, # Callable[[UUID4], None] | None update_catalog_mode=None, # UpdateCatalogMode | None params=None, # dict[str, Any] | None ) # Subscribe to real-time data - will be processed by on_bar() handler self.subscribe_bars( bar_type=self.bar_type, # Many optional parameters client_id=None, # ClientId, optional params=None, # dict[str, Any], optional ) def on_historical_data(self, data: Data) -> None: # Handle historical data (from requests) if isinstance(data, Bar): self.log.info(f"Received historical bar: {data}") def on_bar(self, bar: Bar) -> None: # Handle real-time bar updates (from subscriptions) self.log.info(f"Received real-time bar: {bar}") ``` When `validate_data_sequence=True`, subscribe to live bars via the `request_bars()` `callback` (rather than a separate `subscribe_bars()` call) so the stream starts only after history has loaded; see [Working with bars: request vs. subscribe](data.md#working-with-bars-request-vs-subscribe). Separating historical and real-time handlers lets you apply different processing logic based on context. For example: - Use historical data to initialize indicators or establish baseline metrics. - Process real-time data differently for live trading decisions. - Apply different validation or logging for historical vs real-time data. :::tip When debugging data flow issues, check that you're looking at the correct handler for your data source. If you're not seeing data in `on_bar()` but see log messages about receiving bars, check `on_historical_data()` as the data might be coming from a request rather than a subscription. ::: ## Order fill subscriptions Actors can subscribe to order fill events for specific instruments using `subscribe_order_fills()`. This is useful for monitoring trading activity, fill analysis, or tracking execution quality. When subscribed, the handler `on_order_filled()` receives all fills for the specified instrument, regardless of which strategy or component generated the original order. ### Example ```python from nautilus_trader.common.actor import Actor from nautilus_trader.config import ActorConfig from nautilus_trader.model import InstrumentId from nautilus_trader.model.events import OrderFilled class MyActorConfig(ActorConfig): instrument_id: InstrumentId # example value: "ETHUSDT-PERP.BINANCE" class FillMonitorActor(Actor): def __init__(self, config: MyActorConfig) -> None: super().__init__(config) self.fill_count = 0 self.total_volume = 0.0 def on_start(self) -> None: # Subscribe to all fills for the instrument self.subscribe_order_fills(self.config.instrument_id) def on_order_filled(self, event: OrderFilled) -> None: # Handle order fill events self.fill_count += 1 self.total_volume += float(event.last_qty) self.log.info( f"Fill received: {event.order_side} {event.last_qty} @ {event.last_px}, " f"Total fills: {self.fill_count}, Volume: {self.total_volume}" ) def on_stop(self) -> None: # Unsubscribe from fills self.unsubscribe_order_fills(self.config.instrument_id) ``` :::note Order fill subscriptions use the message bus only and do not involve the data engine. The `on_order_filled()` handler receives events only while the actor is running. ::: ## Order cancel subscriptions Actors can subscribe to order cancel events for specific instruments using `subscribe_order_cancels()`. This is useful for monitoring cancellations or tracking order lifecycle events. When subscribed, the handler `on_order_canceled()` receives all cancels for the specified instrument, regardless of which strategy or component generated the original order. ### Example ```python from nautilus_trader.common.actor import Actor from nautilus_trader.config import ActorConfig from nautilus_trader.model import InstrumentId from nautilus_trader.model.events import OrderCanceled class MyActorConfig(ActorConfig): instrument_id: InstrumentId # example value: "ETHUSDT-PERP.BINANCE" class CancelMonitorActor(Actor): def __init__(self, config: MyActorConfig) -> None: super().__init__(config) self.cancel_count = 0 def on_start(self) -> None: # Subscribe to all cancels for the instrument self.subscribe_order_cancels(self.config.instrument_id) def on_order_canceled(self, event: OrderCanceled) -> None: # Handle order cancel events self.cancel_count += 1 self.log.info( f"Cancel received: {event.client_order_id}, " f"Total cancels: {self.cancel_count}" ) def on_stop(self) -> None: # Unsubscribe from cancels self.unsubscribe_order_cancels(self.config.instrument_id) ``` :::note Order cancel subscriptions use the message bus only and do not involve the data engine. The `on_order_canceled()` handler receives events only while the actor is running. ::: ## Related guides - [Strategies](strategies.md) - Strategies extend actors with order management capabilities. - [Data](data.md) - Data types and subscriptions available to actors. - [Message Bus](message_bus.md) - The messaging system actors use for communication. # Adapters Source: https://nautilustrader.io/docs/latest/concepts/adapters/ Adapters integrate data providers and trading venues into NautilusTrader. They can be found in the top-level `adapters` subpackage. An adapter typically comprises these components: ```mermaid flowchart LR subgraph Venue ["Trading Venue"] API[REST API] WS[WebSocket] end subgraph Adapter ["Adapter"] HTTP[HttpClient] WSC[WebSocketClient] IP[InstrumentProvider] DC[DataClient] EC[ExecutionClient] end subgraph Core ["Nautilus Core"] DE[DataEngine] EE[ExecutionEngine] end API <--> HTTP WS <--> WSC HTTP --> IP HTTP --> DC HTTP --> EC WSC --> DC WSC --> EC DC <--> DE EC <--> EE ``` | Component | Purpose | |----------------------|------------------------------------------------------------| | `HttpClient` | REST API communication. | | `WebSocketClient` | Real‑time streaming connection. | | `InstrumentProvider` | Loads and parses instrument definitions from the venue. | | `DataClient` | Handles market data subscriptions and requests. | | `ExecutionClient` | Handles order submission, modification, and cancellation. | ## Instrument providers Instrument providers parse venue API responses into Nautilus `Instrument` objects. An `InstrumentProvider` serves two use cases: - Standalone discovery of available instruments for research or backtesting - Runtime loading in a `sandbox` or `live` [environment context](architecture.md#environment-contexts) for actors and strategies ### Research and backtesting Here is an example of discovering the current instruments for the Binance Futures testnet: ```python import asyncio import os from nautilus_trader.adapters.binance.common.enums import BinanceAccountType from nautilus_trader.adapters.binance.common.enums import BinanceEnvironment from nautilus_trader.adapters.binance import get_cached_binance_http_client from nautilus_trader.adapters.binance.futures.providers import BinanceFuturesInstrumentProvider from nautilus_trader.common.component import LiveClock async def main(): clock = LiveClock() client = get_cached_binance_http_client( clock=clock, account_type=BinanceAccountType.USDT_FUTURES, api_key=os.getenv("BINANCE_FUTURES_TESTNET_API_KEY"), api_secret=os.getenv("BINANCE_FUTURES_TESTNET_API_SECRET"), environment=BinanceEnvironment.TESTNET, ) provider = BinanceFuturesInstrumentProvider( client=client, account_type=BinanceAccountType.USDT_FUTURES, ) await provider.load_all_async() # Access loaded instruments instruments = provider.list_all() print(f"Loaded {len(instruments)} instruments") if __name__ == "__main__": asyncio.run(main()) ``` ### Live trading Each integration handles this differently. An `InstrumentProvider` within a `TradingNode` generally offers two loading behaviors: - Load all instruments on start: ```python from nautilus_trader.config import InstrumentProviderConfig InstrumentProviderConfig(load_all=True) ``` - Load only the instruments specified in configuration: ```python InstrumentProviderConfig(load_ids=["BTCUSDT-PERP.BINANCE", "ETHUSDT-PERP.BINANCE"]) ``` Subscriptions do not load instruments by themselves. Before a strategy subscribes to live data, configure the provider to load the instrument at startup or request the instrument explicitly and wait until it reaches the cache. ## Data clients Data clients handle market data subscriptions and requests for a venue. They connect to venue APIs and normalize incoming data into Nautilus types. ### Requesting data Actors and strategies can request data using built-in methods. Data returns via callbacks: ```python from nautilus_trader.model import Instrument, InstrumentId from nautilus_trader.trading.strategy import Strategy class MyStrategy(Strategy): def on_start(self) -> None: # Request an instrument definition self.request_instrument(InstrumentId.from_str("BTCUSDT-PERP.BINANCE")) # Request historical bars self.request_bars(BarType.from_str("BTCUSDT-PERP.BINANCE-1-HOUR-LAST-EXTERNAL")) def on_instrument(self, instrument: Instrument) -> None: self.log.info(f"Received instrument: {instrument.id}") def on_historical_data(self, data) -> None: self.log.info(f"Received historical data: {data}") ``` ### Subscribing to data For real-time data, use subscription methods: ```python def on_start(self) -> None: # Assumes the instrument has already been loaded into the cache # Subscribe to live trade updates self.subscribe_trade_ticks(InstrumentId.from_str("BTCUSDT-PERP.BINANCE")) # Subscribe to live bars self.subscribe_bars(BarType.from_str("BTCUSDT-PERP.BINANCE-1-MINUTE-LAST-EXTERNAL")) def on_trade_tick(self, tick: TradeTick) -> None: self.log.info(f"Trade: {tick}") def on_bar(self, bar: Bar) -> None: self.log.info(f"Bar: {bar}") ``` :::tip See the [Actors](actors.md) documentation for a complete reference of available request and subscription methods with their corresponding callbacks. ::: ## Execution clients Execution clients handle order management for a venue. They translate Nautilus order commands into venue-specific API calls and process execution reports back into Nautilus events. Key responsibilities: - Submit, modify, and cancel orders. - Process fills and execution reports. - Reconcile order state with the venue. - Handle account and position updates. The `ExecutionEngine` routes commands to the appropriate execution client based on the order's venue. See the [Execution](execution.md) guide for details on order management from a strategy perspective. :::tip For building a custom adapter, see the [Adapter Developer Guide](../developer_guide/adapters.md). ::: ## Related guides - [Live Trading](live.md) - Configure and run live trading with adapters. - [Execution](execution.md) - Order execution through adapters. - [Data](data.md) - Market data provided by adapters. # Architecture Source: https://nautilustrader.io/docs/latest/concepts/architecture/ This guide covers the architectural principles and structure of NautilusTrader: - Design philosophy and quality attributes. - Core components and how they interact. - Environment contexts (backtest, sandbox, live). - Framework organization and code structure. :::note Throughout the documentation, the term *"Nautilus system boundary"* refers to operations within the runtime of a single Nautilus node (also known as a "trader instance"). ::: ## Design philosophy The major architectural techniques and design patterns employed by NautilusTrader are: - [Domain driven design (DDD)](https://en.wikipedia.org/wiki/Domain-driven_design) - [Event-driven architecture](https://en.wikipedia.org/wiki/Event-driven_programming) - [Messaging patterns](https://en.wikipedia.org/wiki/Messaging_pattern) (Pub/Sub, Req/Rep, point-to-point) - [Ports and adapters](https://en.wikipedia.org/wiki/Hexagonal_architecture_(software)) - [Crash-only design](#crash-only-design) These techniques help achieve certain architectural quality attributes. ### Quality attributes Architectural decisions are often a trade-off between competing priorities. The following quality attributes guide design and architectural decisions, roughly in order of weighting. - Reliability - Performance - Modularity - Testability - Maintainability - Deployability ### Assurance-driven engineering NautilusTrader is incrementally adopting a high-assurance mindset: critical code paths should carry executable invariants that verify behaviour matches the business requirements. Practically this means we: - Identify the components whose failure has the highest blast radius (core domain types, risk and execution flows) and write down their invariants in plain language. - Codify those invariants as executable checks (unit tests, property tests, fuzzers, static assertions) that run in CI, keeping the feedback loop light. - Prefer zero-cost safety techniques built into Rust (ownership, `Result` surfaces, `panic = abort`) and add targeted formal tools only where they pay for themselves. - Track “assurance debt” alongside feature work so new integrations extend the safety net rather than bypass it. This approach preserves the platform’s delivery cadence while giving high-stakes flows the additional scrutiny they need. Further reading: [High Assurance Rust](https://highassurance.rs/). ### Crash-only design NautilusTrader draws inspiration from [crash-only design](https://en.wikipedia.org/wiki/Crash-only_software) principles, particularly for handling unrecoverable faults. The core insight is that systems which can recover cleanly from crashes are more robust than those with separate (and rarely tested) graceful shutdown paths. Key principles: - **Unified recovery path** - Startup and crash recovery share the same code path, ensuring it is well-tested. - **Externalized state** - Critical state is meant to be persisted externally when configured, reducing data-loss risk; durability depends on the backing store. - **Fast restart** - The system is designed to restart quickly after a crash, minimizing downtime. - **Idempotent operations** - Operations are designed to be safely retried after restart. - **Fail-fast for unrecoverable errors** - Data corruption or invariant violations trigger immediate termination rather than attempting to continue in a compromised state. :::note The system does provide graceful shutdown flows (`stop`, `dispose`) for normal operation. These tear down clients, persist state, and flush writers. The crash-only philosophy applies specifically to *unrecoverable faults* where attempting graceful cleanup could cause further damage. ::: This design complements the [fail-fast policy](#data-integrity-and-fail-fast-policy), where unrecoverable errors result in immediate process termination. **References:** - [Crash-Only Software](https://www.usenix.org/conference/hotos-ix/crash-only-software) - Candea & Fox, HotOS 2003 (original research paper) - [Microreboot: A technique for cheap recovery](https://www.usenix.org/events/osdi04/tech/candea.html) - Candea et al., OSDI 2004 - [The properties of crash-only software](https://brooker.co.za/blog/2012/01/22/crash-only.html) - Marc Brooker's blog - [Crash-only software: More than meets the eye](https://lwn.net/Articles/191059/) - LWN.net article - [Recovery-Oriented Computing (ROC) Project](http://roc.cs.berkeley.edu/) - UC Berkeley/Stanford research ### Data integrity and fail-fast policy NautilusTrader prioritizes data integrity over availability for trading operations. The system employs a strict fail-fast policy for arithmetic operations and data handling to prevent silent data corruption that could lead to incorrect trading decisions. #### Fail-fast principles The system will fail fast (panic or return an error) when encountering: - Arithmetic overflow or underflow in operations on timestamps, prices, or quantities that exceed valid ranges. - Invalid data during deserialization including NaN, Infinity, or out-of-range values in market data or configuration. - Type conversion failures such as negative values where only positive values are valid (timestamps, quantities). - Malformed input parsing for prices, timestamps, or precision values. Rationale: In trading systems, corrupt data is worse than no data. A single incorrect price, timestamp, or quantity can cascade through the system, resulting in: - Incorrect position sizing or risk calculations. - Orders placed at wrong prices. - Backtests producing misleading results. - Silent financial losses. By crashing immediately on invalid data, NautilusTrader aims to provide: 1. **No silent corruption** - The fail-fast policy is intended to prevent invalid data from propagating; this relies on checks covering the inputs. 2. **Immediate feedback** - Issues are discovered during development and testing, not in production. 3. **Audit trail** - Crash logs clearly identify the source of invalid data. 4. **Deterministic behavior** - With deterministic ordering and configuration, the same invalid input should trigger the same failure; nondeterministic sources can vary outcomes. #### When fail-fast applies Panics are used for: - Programmer errors (logic bugs, incorrect API usage). - Data that violates fundamental invariants (negative timestamps, NaN prices). - Arithmetic that would silently produce incorrect results. Results or Options are used for: - Expected runtime failures (network errors, file I/O). - Business logic validation (order constraints, risk limits). - User input validation. - Library APIs exposed to downstream crates where callers need explicit error handling without relying on panics for control flow. #### Example scenarios ```rust // CORRECT: Panics on overflow - prevents data corruption let total_ns = timestamp1 + timestamp2; // Panics if result > u64::MAX // CORRECT: Rejects NaN during deserialization let price = serde_json::from_str("NaN"); // Error: "must be finite" // CORRECT: Explicit overflow handling when needed let total_ns = timestamp1.checked_add(timestamp2)?; // Returns Option ``` This policy is implemented throughout the core types (`UnixNanos`, `Price`, `Quantity`, etc.) and helps NautilusTrader maintain strong data correctness for production trading. In production deployments, the system is typically configured with `panic = abort` in release builds, ensuring that any panic results in a clean process termination that can be handled by process supervisors or orchestration systems. This aligns with the [crash-only design](#crash-only-design) principle, where unrecoverable errors lead to immediate restart rather than attempting to continue in a potentially corrupted state. ## System architecture The NautilusTrader codebase is actually both a framework for composing trading systems, and a set of default system implementations which can operate in various [environment contexts](#environment-contexts). ![Architecture](https://github.com/nautechsystems/nautilus_trader/blob/develop/assets/architecture-overview.png?raw=true "architecture") ### Core components Several core components work together to form the trading system: #### `NautilusKernel` The central orchestration component responsible for: - Initializing and managing all system components. - Configuring the messaging infrastructure. - Maintaining environment-specific behaviors. - Coordinating shared resources and lifecycle management. - Providing a unified entry point for system operations. #### `MessageBus` The backbone of inter-component communication, implementing: - **Publish/Subscribe patterns**: For broadcasting events and data to multiple consumers. - **Request/Response communication**: For operations requiring acknowledgment. - **Command/Event messaging**: For triggering actions and notifying state changes. - **Optional state persistence**: Using Redis for durability and restart capabilities. #### `Cache` High-performance in-memory storage system that: - Stores instruments, accounts, orders, positions, and more. - Provides performant fetching capabilities for trading components. - Maintains consistent state across the system. - Supports both read and write operations with optimized access patterns. #### `DataEngine` Processes and routes market data throughout the system: - Handles multiple data types (quotes, trades, bars, order books, custom data, and more). - Routes data to appropriate consumers based on subscriptions. - Manages data flow from external sources to internal components. #### `ExecutionEngine` Manages order lifecycle and execution: - Routes trading commands to the appropriate adapter clients. - Tracks order and position states. - Coordinates with risk management systems. - Handles execution reports and fills from venues. - Handles reconciliation of external execution state. #### `RiskEngine` Provides risk management: - Pre-trade risk checks and validation. - Position and exposure monitoring. - Real-time risk calculations. - Configurable risk rules and limits. ### Environment contexts An environment context in NautilusTrader defines the type of data and trading venue you work with. Understanding these contexts matters for backtesting, development, and live trading. Here are the available environments you can work with: - `Backtest`: Historical data with simulated venues. - `Sandbox`: Real-time data with simulated venues. - `Live`: Real-time data with live venues (paper trading or real accounts). ### Common core The platform has been designed to share as much common code between backtest, sandbox and live trading systems as possible. This is formalized in the `system` subpackage, where you will find the `NautilusKernel` class, providing a common core system 'kernel'. The *ports and adapters* architectural style enables modular components to be integrated into the core system, providing various hooks for user-defined or custom component implementations. ### Data and execution flow patterns Understanding how data and execution flow through the system helps when working with the platform. #### Data flow: life of a quote tick The following trace shows every step a `QuoteTick` takes from the network to your strategy. Trades and bars follow the same cache-then-publish path with different handler names. Order book deltas and depth snapshots take a different route (see the tip below the steps). ```mermaid sequenceDiagram participant Adapter as DataClient adapter participant Channel as MPSC channel participant DE as DataEngine participant Cache as Cache participant MB as MessageBus participant Strategy as Strategy Adapter->>Channel: DataEvent::Data(Data::Quote(quote)) Channel->>DE: process_data(Data::Quote) DE->>DE: handle_quote(quote) DE->>Cache: add_quote(quote) DE->>MB: publish_quote(topic, quote) MB->>Strategy: on_quote_tick(quote) ``` **Step by step:** 1. **Adapter receives raw data.** A venue-specific `DataClient` (e.g. Binance, Bybit) receives a WebSocket message, parses it, and constructs a `QuoteTick`. 2. **Adapter sends a data event.** The adapter sends `DataEvent::Data(Data::Quote(quote))` through an MPSC channel. In live mode this is an async unbounded channel; in backtests the engine feeds data directly. 3. **DataEngine processes the event.** The channel receiver routes the event to `DataEngine::process_data`, which dispatches to `handle_quote`. 4. **Cache stores the quote.** `handle_quote` writes the quote into the `Cache` via `cache.add_quote(quote)`, making it available to any component through `self.cache.quote_tick(instrument_id)`. 5. **MessageBus publishes.** The engine publishes the quote on a topic derived from the instrument ID (e.g. `data.quotes.BINANCE.BTCUSDT-PERP`). The `MessageBus` finds all handlers subscribed to that topic. 6. **Strategy handler fires.** Each subscribed strategy's `on_quote_tick(quote)` runs on the single-threaded kernel. The quote is already in the cache before the handler executes, so `self.cache.quote_tick(instrument_id)` returns the same quote. :::tip For quotes, trades, and bars the cache-then-publish order means your strategy handler can always read the latest value from the cache. Order book deltas and depth snapshots are published directly; book state is maintained separately through `BookUpdater` subscriptions. ::: #### Execution flow: life of an order When a strategy submits an order, it flows through validation, routing, and back again as execution events: ```mermaid sequenceDiagram participant Strategy as Strategy participant RE as RiskEngine participant EE as ExecutionEngine participant EC as ExecutionClient participant Venue as Venue Strategy->>RE: submit_order(command) RE->>RE: pre-trade risk checks RE->>EE: route command EE->>EC: submit_order EC->>Venue: place order (REST/WS) Venue-->>EC: OrderAccepted EC->>EE: OrderAccepted event EE->>Strategy: on_order_accepted(event) Venue-->>EC: OrderFilled EC->>EE: OrderFilled event EE->>Strategy: on_order_filled(event) ``` 1. **Strategy creates a command.** The strategy calls `self.submit_order(order)`. 2. **RiskEngine validates.** Pre-trade checks run (position limits, notional limits, order rate). If a check fails the strategy receives `OrderDenied` and the order never reaches the venue. 3. **ExecutionEngine routes.** The command is routed to the `ExecutionClient` for the target venue. 4. **ExecutionClient submits.** The adapter sends the order to the venue over REST or WebSocket. 5. **Events flow back.** The venue responds with acknowledgments and fills. Each event (Accepted, Filled, Canceled, Rejected, Expired) flows back through the `ExecutionEngine`, which updates order state in the `Cache` and delivers the event to the strategy's handler. Fill events also trigger position and portfolio updates. #### Component state management All components follow a finite state machine pattern. The `ComponentState` enum defines both stable states and transitional states: ```mermaid stateDiagram-v2 [*] --> PRE_INITIALIZED PRE_INITIALIZED --> READY : register() READY --> STARTING : start() STARTING --> RUNNING RUNNING --> STOPPING : stop() STOPPING --> STOPPED STOPPED --> STARTING : start() STOPPED --> RESETTING : reset() RESETTING --> READY RUNNING --> RESUMING : resume() RESUMING --> RUNNING RUNNING --> DEGRADING : degrade() DEGRADING --> DEGRADED DEGRADED --> STOPPING : stop() DEGRADED --> FAULTING : fault() RUNNING --> FAULTING : fault() FAULTING --> FAULTED STOPPED --> DISPOSING : dispose() FAULTED --> DISPOSING : dispose() DISPOSING --> DISPOSED DISPOSED --> [*] ``` **Stable states:** - **PRE_INITIALIZED**: Component is instantiated but not yet ready to fulfill its specification. - **READY**: Component is configured and able to be started. - **RUNNING**: Component is operating normally and can fulfill its specification. - **STOPPED**: Component has successfully stopped. - **DEGRADED**: Component has degraded and may not meet its full specification. - **FAULTED**: Component has shut down due to a detected fault. - **DISPOSED**: Component has shut down and released all of its resources. **Transitional states:** - **STARTING**: Component is executing its actions on `start`. - **STOPPING**: Component is executing its actions on `stop`. - **RESUMING**: Component is being started again after its initial start. - **RESETTING**: Component is executing its actions on `reset`. - **DISPOSING**: Component is executing its actions on `dispose`. - **DEGRADING**: Component is executing its actions on `degrade`. - **FAULTING**: Component is executing its actions on `fault`. Transitional states are brief intermediate states that occur during state transitions. Components should not remain in transitional states for extended periods. #### Actor vs Component traits At the Rust implementation level, the system distinguishes between two complementary traits: ```mermaid classDiagram class Actor { <> +id() Ustr +handle(message) } class Component { <> +component_id() ComponentId +state() ComponentState +register() +start() +stop() +reset() +dispose() } class ActorRegistry { +insert(actor) +get(id) ActorRef } class ComponentRegistry { +insert(component) +get(id) ComponentRef } Actor <|.. Throttler : implements Actor <|.. Strategy : implements Component <|.. Strategy : implements Component <|.. DataEngine : implements Component <|.. ExecutionEngine : implements ActorRegistry --> Actor : manages ComponentRegistry --> Component : manages class Throttler { Actor only } class Strategy { Actor + Component } class DataEngine { Component only } class ExecutionEngine { Component only } ``` **`Actor` trait** - Message dispatch: - Provides the `handle` method for receiving messages dispatched through the actor registry. - Enables type-safe lookup and message dispatch by actor ID. - Used by components that need to receive targeted messages (strategies, throttlers). **`Component` trait** - Lifecycle management: - Manages state transitions (`start`, `stop`, `reset`, `dispose`). - Provides registration with the system kernel (`register`). - Tracks component state via the finite state machine described above. - Used by all system components that need lifecycle management. :::note All components can publish and subscribe to messages via the `MessageBus` directly - this is independent of the `Actor` trait. The `Actor` trait specifically enables the registry-based message dispatch pattern where messages are routed to a specific actor by ID. ::: This separation allows: - **Actor-only**: Lightweight message handlers without lifecycle (e.g., `Throttler`). - **Component-only**: System infrastructure with lifecycle but using direct MessageBus pub/sub (e.g., `DataEngine`, `ExecutionEngine`). - **Both traits**: Trading strategies that need lifecycle management AND targeted message dispatch. The traits are managed by separate registries to support their different access patterns - lifecycle methods are called sequentially, while message handlers may be invoked re-entrantly during callbacks. ### Messaging For modularity and loose coupling, an efficient `MessageBus` passes messages (data, commands, and events) between components. #### Threading model Within a node, the *kernel* consumes and dispatches messages on a single thread. The kernel encompasses: - The `MessageBus` and actor callback dispatch. - Strategy logic and order management. - Risk engine checks and execution coordination. - Cache reads and writes. This single-threaded core provides deterministic event ordering and helps maintain backtest-live parity, though live inputs and latency can still cause behavioral differences. Components consume messages synchronously in a pattern *similar* to the [actor model](https://en.wikipedia.org/wiki/Actor_model). :::note Of interest is the LMAX exchange architecture, which achieves award winning performance running on a single thread. You can read about their *disruptor* pattern based architecture in [this interesting article](https://martinfowler.com/articles/lmax.html) by Martin Fowler. ::: Background services use separate threads or async runtimes: - **Network I/O** - WebSocket connections, REST clients, and async data feeds. - **Persistence** - DataFusion queries and database operations via multi-threaded Tokio runtime. - **Adapters** - Async adapter operations via thread pool executors. These services communicate results back to the kernel via the `MessageBus`. The bus itself is thread-local, so each thread has its own instance, with cross-thread communication occurring through channels that ultimately deliver events to the single-threaded core. ## Framework organization The codebase organizes into layers of abstraction, grouped into logical subpackages of cohesive concepts. You can navigate to the documentation for each subpackage from the left nav menu. ### Core / low-level - `core`: Constants, functions and low-level components used throughout the framework. - `common`: Common parts for assembling the frameworks various components. - `network`: Low-level base components for networking clients. - `serialization`: Serialization base components and serializer implementations. - `model`: Defines a rich trading domain model. ### Components - `accounting`: Different account types and account management machinery. - `adapters`: Integration adapters for the platform including brokers and exchanges. - `analysis`: Components relating to trading performance statistics and analysis. - `cache`: Provides common caching infrastructure. - `data`: The data stack and data tooling for the platform. - `execution`: The execution stack for the platform. - `indicators`: A set of efficient indicators and analyzers. - `persistence`: Data storage, cataloging and retrieval, mainly to support backtesting. - `portfolio`: Portfolio management functionality. - `risk`: Risk specific components and tooling. - `trading`: Trading domain specific components and tooling. ### System implementations - `backtest`: Backtesting componentry as well as a backtest engine and node implementations. - `live`: Live engine and client implementations as well as a node for live trading. - `system`: The core system kernel common between `backtest`, `sandbox`, `live` [environment contexts](#environment-contexts). ## Code structure The foundation of the codebase is the `crates` directory, containing a collection of Rust crates including a C foreign function interface (FFI) generated by `cbindgen`. The bulk of the production code resides in the `nautilus_trader` directory, which contains a collection of Python/Cython subpackages and modules. Python bindings for the Rust core are provided by statically linking the Rust libraries to the C extension modules generated by Cython at compile time (effectively extending the CPython API). ### Dependency flow ```mermaid flowchart TB subgraph trader["nautilus_trader
Python / Cython"] end subgraph core["crates
Rust"] end trader -->|"C API"| core ``` ### Rust crates The `crates/` directory contains the Rust implementation organized into focused crates with clear dependency boundaries. Feature flags control optional functionality - for example, `streaming` enables persistence for catalog-based data streaming, and `cloud` enables cloud storage backends (S3, Azure, GCP). Dependency flow (arrows point to dependencies): ```mermaid flowchart BT subgraph Foundation core model common system trading end subgraph Infrastructure serialization network cryptography persistence end subgraph Engines data execution portfolio risk end subgraph Runtime live backtest end adapters pyo3 model --> core common --> core common --> model system --> common trading --> common serialization --> model network --> common network --> cryptography persistence --> serialization data --> common execution --> common portfolio --> common risk --> portfolio live --> system live --> trading backtest --> system backtest --> persistence adapters --> live adapters --> network pyo3 --> adapters ``` **Crate categories:** | Category | Crates | Purpose | |----------------|-----------------------------------------------------------|----------------------------------------------------------| | Foundation | `core`, `model`, `common`, `system`, `trading` | Primitives, domain model, kernel, actor & strategy base. | | Engines | `data`, `execution`, `portfolio`, `risk` | Core trading engine components. | | Infrastructure | `serialization`, `network`, `cryptography`, `persistence` | Encoding, networking, signing, storage. | | Runtime | `live`, `backtest` | Environment‑specific node implementations. | | External | `adapters/*` | Venue and data integrations. | | Bindings | `pyo3` | Python bindings. | **Feature flags:** | Feature | Crates | Effect | |-------------|----------------------------|------------------------------------------------------------| | `streaming` | `data`, `system`, `live` | Enables `persistence` dependency for catalog streaming. | | `cloud` | `persistence` | Enables cloud storage backends (S3, Azure, GCP, HTTP). | | `python` | most crates | Enables PyO3 bindings (auto‑enables `streaming`, `cloud`). | | `defi` | `common`, `model`, `data` | Enables DeFi/blockchain data types. | :::note Both Rust and Cython are build dependencies. The binary wheels produced from a build do not require Rust or Cython to be installed at runtime. ::: ### Type safety The platform design prioritizes software correctness and safety. The Rust codebase under `crates/` relies on the `rustc` compiler's guarantees for safe code. Any `unsafe` blocks are explicit opt-outs where we must uphold the required invariants ourselves (see the Rust section of the [Developer Guide](../developer_guide/rust.md)); overall memory and type safety depend on those invariants holding. Cython provides type safety at the C level at both compile time, and runtime: :::info If you pass an argument with an invalid type to a Cython implemented module with typed parameters, then you will receive a `TypeError` at runtime. ::: If a function or method's parameter is not explicitly typed to accept `None`, passing `None` as an argument will result in a `ValueError` at runtime. :::warning The above exceptions are not explicitly documented to prevent excessive bloating of the docstrings. ::: ### Errors and exceptions The documentation aims to cover all possible exceptions that NautilusTrader code can raise, and the conditions that trigger them. :::warning There may be other undocumented exceptions which can be raised by Python's standard library, or from third party library dependencies. ::: ### Processes and threads :::warning[One node per process] Running multiple `TradingNode` or `BacktestNode` instances **concurrently** in the same process is not supported due to global singleton state: - **Backtest force-stop flag** - The `_FORCE_STOP` global flag is shared across all engines in the process. - **Logger mode and timestamps** - The logging subsystem uses global state; backtests flip between static and real-time modes. - **Runtime singletons** - Global Tokio runtime, callback registries, and other `OnceLock` instances are process-wide. **Sequential execution** of multiple nodes (one after another with proper disposal between runs) is fully supported and used in the test suite. For production deployments, add multiple strategies to a **single TradingNode** within a process. For parallel execution or workload isolation, run each node in its own separate process. ::: ## Related guides - [Overview](overview.md) - High-level introduction to NautilusTrader. - [Message Bus](message_bus.md) - Core messaging infrastructure. # Backtesting Source: https://nautilustrader.io/docs/latest/concepts/backtesting/ Backtesting simulates trading using a specific system implementation. The system comprises the built-in engines, `Cache`, [MessageBus](message_bus.md), `Portfolio`, [Actors](actors.md), [Strategies](strategies.md), [Execution Algorithms](execution.md), and user-defined modules. A `BacktestEngine` processes a stream of historical data. When the stream is exhausted, the engine produces results and performance metrics for analysis. NautilusTrader offers two API levels for backtesting: - **High-level API**: Uses a `BacktestNode` and configuration objects (`BacktestEngine`s are used internally). - **Low-level API**: Uses a `BacktestEngine` directly with more "manual" setup. ## Choosing an API level Consider using the **low-level** API when: - Your entire data stream can be processed within the available machine resources (e.g., RAM). - You prefer not to store data in the Nautilus-specific Parquet format. - You have a specific need or preference to retain raw data in its original format (e.g., CSV, binary, etc.). - You require fine-grained control over the `BacktestEngine`, such as the ability to re-run backtests on identical datasets while swapping out components (e.g., actors or strategies) or adjusting parameter configurations. Consider using the **high-level** API when: - Your data stream exceeds available memory, requiring streaming data in batches. - You want the performance and convenience of the `ParquetDataCatalog` for storing data in the Nautilus-specific Parquet format. - You value the flexibility and functionality of passing configuration objects to define and manage multiple backtest runs across various engines simultaneously. ## Low-level API The low-level API centers around a `BacktestEngine`, where inputs are initialized and added manually via a Python script. An instantiated `BacktestEngine` can accept the following: - Lists of `Data` objects, which are automatically sorted into monotonic order based on `ts_init`. - Multiple venues, manually initialized. - Multiple actors, manually initialized and added. - Multiple execution algorithms, manually initialized and added. This approach offers detailed control over the backtesting process, allowing you to manually configure each component. ### Loading large datasets efficiently When working with large amounts of data across multiple instruments, the way you load data can significantly impact performance. #### The performance consideration By default, `BacktestEngine.add_data()` sorts the entire data stream (existing data + newly added data) on each call when `sort=True` (the default). This means: - First call with 1M bars: sorts 1M bars. - Second call with 1M bars: sorts 2M bars. - Third call with 1M bars: sorts 3M bars. - And so on... This repeated sorting of increasingly large datasets can become a bottleneck when loading data for multiple instruments. #### Optimization strategies **Strategy 1: Defer sorting until the end (recommended for multiple instruments)** ```python from nautilus_trader.backtest.engine import BacktestEngine engine = BacktestEngine() # Setup venue and instruments engine.add_venue(...) engine.add_instrument(instrument1) engine.add_instrument(instrument2) engine.add_instrument(instrument3) # Load all data WITHOUT sorting on each call engine.add_data(instrument1_bars, sort=False) engine.add_data(instrument2_bars, sort=False) engine.add_data(instrument3_bars, sort=False) # Sort once at the end - much more efficient! engine.sort_data() # Now run your backtest engine.add_strategy(strategy) engine.run() ``` **Strategy 2: Collect and add in a single batch** ```python # Collect all data first all_bars = [] all_bars.extend(instrument1_bars) all_bars.extend(instrument2_bars) all_bars.extend(instrument3_bars) # Add once with sorting engine.add_data(all_bars, sort=True) ``` **Strategy 3: Use streaming API for very large datasets** For datasets that don't fit in memory, there are two streaming approaches: **Automatic chunking** - supply a generator that yields batches. The engine pulls chunks lazily during a single `run()` call: ```python def data_generator(): # Yield chunks of data (each chunk is a list of Data objects) yield load_chunk_1() yield load_chunk_2() yield load_chunk_3() engine.add_data_iterator( data_name="my_data_stream", generator=data_generator(), ) engine.run() # Chunks are consumed on-demand ``` **Manual chunking** - load and run each batch yourself. This is the pattern used internally by `BacktestNode` and gives full control over batch boundaries: ```python engine.add_strategy(strategy) for batch in data_batches: engine.add_data(batch) engine.run(streaming=True) engine.clear_data() engine.end() # Finalize: flushes remaining timers, stops engines, produces results ``` :::note In streaming mode, timer advancement stops when data exhausts for each batch. Timers scheduled past the last data point (e.g. bar aggregation intervals) are deferred until more data arrives or `end()` is called, which flushes up to the `end` boundary from the last `run()` call. ::: :::tip[Performance impact] For a backtest with 10 instruments, each with 1M bars: - Sorting on each call: ~10 sorts of increasing size (1M, 2M, 3M, ... 10M bars). - Sorting once at the end: 1 sort of 10M bars. The deferred sorting approach can be **significantly faster** for large datasets. ::: ### Data loading contract The `BacktestEngine` enforces important invariants to ensure data integrity: **Requirements:** - All data must be sorted before calling `run()`. - When using `sort=False`, you **must** call `sort_data()` before running. - The engine validates this and raises `RuntimeError` if unsorted data is detected. - Calling `sort_data()` multiple times is safe (idempotent). **Safety guarantees:** - Data lists are always copied internally to prevent external mutations from affecting engine state. - You can safely clear or modify data lists after passing them to `add_data()`. - Adding data with `sort=True` makes it immediately available for backtesting. This design ensures data integrity while enabling performance optimizations for large datasets. ## Funding Backtests settle perpetual funding at funding boundaries from `FundingRateUpdate` data. When an update has `next_funding_ns`, the simulated exchange stores the latest rate and the backtest clock emits one `FundingSettlement` at that timestamp. Without `next_funding_ns`, the exchange settles only when `ts_event` lands on the `interval` boundary. Updates without a boundary remain strategy data and do not create funding payments. ```mermaid flowchart LR A[FundingRateUpdate] --> B[SimulatedExchange stores latest rate] B --> C[Backtest clock reaches funding boundary] C --> D[FundingSettlement] D --> E[Open positions] E --> F[PositionAdjusted: Funding] E --> G[AccountState] F --> H[Portfolio] G --> H ``` `PositionAdjusted` remains the position accounting event. A positive funding rate debits long positions and credits short positions. The resulting adjustment changes realized PnL, and the matching account balance update records the cash movement. ## High-level API The high-level API centers around a `BacktestNode`, which orchestrates the management of multiple `BacktestEngine` instances, each defined by a `BacktestRunConfig`. Multiple configurations can be bundled into a list and processed by the node in one run. Each `BacktestRunConfig` object consists of the following: - A list of `BacktestDataConfig` objects. - A list of `BacktestVenueConfig` objects. - A list of `ImportableActorConfig` objects. - A list of `ImportableStrategyConfig` objects. - A list of `ImportableExecAlgorithmConfig` objects. - An optional `ImportableControllerConfig` object. - An optional `BacktestEngineConfig` object, with a default configuration if not specified. ## Shutdown on error Set `BacktestEngineConfig.shutdown_on_error=True` so that a Rust error log ends the backtest run. The Rust logger records the first `log::error!` emitted after the kernel starts, and the kernel converts that trigger into a `ShutdownSystem` command the next time the backtest loop checks for shutdown. The shutdown request follows the normal backtest stop path. It stops the trader and engines, then returns the backtest results collected up to the shutdown point. It does not abort the process. ```python from nautilus_trader.backtest import BacktestEngineConfig config = BacktestEngineConfig(shutdown_on_error=True) ``` Error logs suppressed by component filters or `bypass_logging=True` still request shutdown. The trigger is cleared and re-armed when a new kernel run starts, so a process can run another backtest without reinitializing the logging system. Shutdown-on-error observes Rust `log` records, not Python `logging.error(...)` calls. ## Repeated runs When conducting multiple backtest runs, it's important to understand how components reset to avoid unexpected behavior. ### BacktestEngine.reset() The `.reset()` method returns engine state and loaded component state to their **initial value**. It keeps loaded components, data, instruments, and venues registered. **What gets reset:** - All trading state (orders, positions, account balances). - Loaded actors, strategies, and execution algorithms are reset in place. - Engine counters and timestamps. **What persists:** - Data added via `.add_data()` (use `.clear_data()` to remove). - Instruments (must match the persisted data). - Venue configurations. - Loaded actors, strategies, and execution algorithms. **Instrument handling:** For `BacktestEngine`, instruments persist across resets by default (because data persists and instruments must match data). This is configured via `CacheConfig.drop_instruments_on_reset=False` in the default `BacktestEngineConfig`. ### Approaches for multiple backtest runs There are two main approaches for running multiple backtests: #### 1. Use BacktestNode (recommended for production) The high-level API is designed for multiple backtest runs with different configurations: ```python from nautilus_trader.backtest.node import BacktestNode from nautilus_trader.config import BacktestRunConfig # Define multiple run configurations configs = [ BacktestRunConfig(...), # Run 1 BacktestRunConfig(...), # Run 2 BacktestRunConfig(...), # Run 3 ] # Execute all runs node = BacktestNode(configs=configs) results = node.run() ``` Each run gets a fresh engine with clean state - no reset() needed. #### 2. Use BacktestEngine.reset() For fine-grained control with the low-level API: ```python from nautilus_trader.backtest.engine import BacktestEngine engine = BacktestEngine() # Setup once engine.add_venue(...) engine.add_instrument(ETHUSDT) engine.add_data(data) # Run 1 engine.add_strategy(strategy1) engine.run() # Reset and run 2 with the same loaded strategy engine.reset() engine.run() # Reset and run 3 with a different strategy engine.reset() engine.clear_strategies() engine.add_strategy(strategy2) engine.run() ``` :::note Instruments and data persist across resets by default for `BacktestEngine`, making parameter optimizations straightforward. ::: :::tip[Best practices] - **For production backtesting:** Use `BacktestNode` with configuration objects. - **For parameter optimizations:** Use `BacktestEngine.reset()` to keep data and instruments, then call `clear_strategies()` before adding a replacement strategy instance. - **For quick experiments:** Either approach works - choose based on individual use case. ::: ## Data Data provided for backtesting drives the execution flow. Since a variety of data types can be used, it's crucial that your venue configurations align with the data being provided for backtesting. Mismatches between data and configuration can lead to unexpected behavior during execution. NautilusTrader is primarily designed and optimized for order book data, which provides a complete representation of every price level or order in the market, reflecting the real-time behavior of a trading venue. This provides the greatest execution granularity and realism. However, if granular order book data is either not available or necessary, then the platform has the capability of processing market data in the following descending order of detail: ```mermaid flowchart LR L3["L3 Order Book
(market-by-order)"] L2["L2 Order Book
(market-by-price)"] L1["L1 Quotes
(top of book)"] T["Trades"] B["Bars"] L3 --> L2 --> L1 --> T --> B style L3 fill:#2d5a3d,color:#fff style L2 fill:#3d6a4d,color:#fff style L1 fill:#4d7a5d,color:#fff style T fill:#5d8a6d,color:#fff style B fill:#6d9a7d,color:#fff ``` 1. **Order Book Data/Deltas (L3 market-by-order)**: - Full market depth with visibility of all individual orders. 2. **Order Book Data/Deltas (L2 market-by-price)**: - Market depth visibility across all price levels. 3. **Quote Ticks (L1 market-by-price)**: - Top of book only - best bid and ask prices and sizes. 4. **Trade Ticks**: - Actual executed trades. 5. **Bars**: - Aggregated trading activity over fixed time intervals (e.g., 1-minute, 1-hour, 1-day). ### Choosing data: cost vs. accuracy For many trading strategies, bar data (e.g., 1-minute) can be sufficient for backtesting and strategy development. This is particularly important because bar data is typically much more accessible and cost-effective compared to tick or order book data. Given this practical reality, Nautilus is designed to support bar-based backtesting with advanced features that maximize simulation accuracy, even when working with lower granularity data. :::tip For some trading strategies, it can be practical to start development with bar data to validate core trading ideas. If the strategy looks promising, but is more sensitive to precise execution timing (e.g., requires fills at specific prices between OHLC levels, or uses tight take-profit/stop-loss levels), you can then invest in higher granularity data for more accurate validation. ::: ## Venues When initializing a venue for backtesting, you must specify its internal order `book_type` for execution processing from the following options: - `L1_MBP`: Level 1 market-by-price (default). Only the top level of the order book is maintained. - `L2_MBP`: Level 2 market-by-price. Order book depth is maintained, with a single order aggregated per price level. - `L3_MBO`: Level 3 market-by-order. Order book depth is maintained, with all individual orders tracked as provided by the data. The `book_type` determines which data types the matching engine uses to update book state and drive execution. Data types not applicable for a given `book_type` are ignored for book and price updates, though precision validation still applies and the engine clock still advances. Strategies always receive all subscribed data via the data engine regardless of `book_type`. | Data Type | L1_MBP | L2_MBP | L3_MBO | | ------------------ | ----------------- | ----------------- | ----------------- | | `QuoteTick` | Updates book | *Ignored* | *Ignored* | | `TradeTick` | Triggers matching | Triggers matching | Triggers matching | | `Bar` | Updates book | *Ignored* | *Ignored* | | `OrderBookDelta` | *Ignored* | Updates book | Updates book | | `OrderBookDeltas` | *Ignored* | Updates book | Updates book | | `OrderBookDepth10` | Updates book | Updates book | Updates book | :::note The granularity of the data must match the specified order `book_type`. Nautilus cannot generate higher granularity data (L2 or L3) from lower-level data such as quotes, trades, or bars. ::: :::warning If you specify `L2_MBP` or `L3_MBO` as the venue’s `book_type`, quotes and bars will not update the book. Ensure you provide order book delta data, otherwise orders may appear as though they are never filled. ::: :::warning When using `L1_MBP` (the default), order book deltas are ignored by the matching engine. If you subscribe to order book deltas, set the venue `book_type` to `L2_MBP` or `L3_MBO`. This also applies to sandbox execution, where the matching engine uses the same `book_type` configuration. ::: ## Execution ### Data and message sequencing In the main backtesting loop, new market data is processed for order execution before being dispatched to actors/strategies via the data engine. #### Main loop flow For each data point the engine runs three phases: - **Exchange processes data.** The simulated exchange updates its order book from the incoming market data and iterates the matching engine. This fills any existing orders that now match against the new market state. - **Strategy receives data.** The data engine dispatches the data point to actors and strategies via their callbacks (e.g. `on_quote_tick`, `on_bar`). Strategies may submit, cancel, or modify orders during these callbacks. - **Settle venues.** The engine drains all queued venue commands and then iterates matching engines to fill newly submitted orders. This loop repeats until no pending commands remain, so cascading orders (e.g. a hedge submitted from `on_order_filled`) settle within the same timestamp. ```mermaid sequenceDiagram participant BL as Backtest Loop participant Exch as SimulatedExchange participant ME as MatchingEngine participant DE as DataEngine participant Stgy as Strategy BL->>BL: next data point (ts=T) rect rgb(240, 248, 255) note right of BL: Phase 1 - Exchange processes data BL->>Exch: process_quote_tick / process_bar Exch->>ME: update book + iterate() note right of ME: Matches existing orders
against new market state end rect rgb(245, 255, 245) note right of BL: Phase 2 - Strategy receives data BL->>DE: process(data) DE->>Stgy: on_quote_tick() / on_bar() Stgy-->>Exch: submit_order (queued or immediate) end rect rgb(255, 248, 240) note right of BL: Phase 3 - Settle venues BL->>BL: _process_and_settle_venues(T) BL->>Exch: _drain_commands(T) note right of Exch: Processes queued commands,
adds orders to matching core BL->>ME: _core.iterate(T) note right of ME: Matches newly added orders
against current market state note right of ME: Fills may trigger strategy callbacks
that enqueue further commands,
repeats until no pending commands BL->>Exch: run simulation modules BL->>Exch: check instrument expirations end ``` Timer events use the same settle mechanism but batch by timestamp: all callbacks at timestamp T execute first, then venues are settled for T before advancing to T+1. #### Command settling When an order fill triggers a strategy callback that submits additional orders (e.g., a stop-loss submitted in `on_order_filled`), those cascading commands are settled within the same timestamp/event cycle. The engine repeatedly drains venue command queues and any newly generated commands until no commands remain pending for the current timestamp. Simulation modules are run only once per cycle, after all commands have settled. When a `LatencyModel` is configured, commands are placed in the venue's inflight queue with a future timestamp derived from the simulated latency. The settle loop considers inflight commands that are due at the current timestamp as pending, so zero-latency or same-tick latency configurations still settle correctly. Commands with future timestamps are deferred and processed when the engine reaches that time. #### Shutdown semantics `BacktestEngine::end()` invokes each strategy's `on_stop` handler, drains and settles any commands it emits (e.g. `close_all_positions`, `cancel_all_orders`), then stops the engines. - `on_stop` commands use normal venue queueing and latency. They do not get priority over earlier inflight commands. - If a pre-stop order reaches the venue before an `on_stop` cancel, it may still fill. A later reduce-only close can then reject if the fill changed net exposure. - Strategies that need deterministic flattening should enter an exit-only state before stopping and avoid new opening orders while cancel and close commands are in-flight. - Strategy event handlers do not fire for the resulting events: the strategy is already `Stopped`, so `OrderFilled` and similar events log but bypass `on_order_filled` and friends. Logic that reacts to fills must run before `on_stop` returns. - Simulation modules do not re-run at shutdown. `SimulationModule::process` is once per timestamp; re-invoking would double-apply side effects like FX rollover interest. - A `LatencyModel` adds its configured delay to trailing commands (those emitted on the final data tick or in `on_stop`). The shutdown path advances the engine clock to the latest inflight arrival timestamp so those commands still settle before the engines stop. ### Fill modeling philosophy NautilusTrader treats historical order book and trade data as **immutable** during backtesting. What happened in the market is preserved exactly as recorded. Fills never modify the underlying book state. This addresses a gap in academic literature: most research focuses on live market dynamics where the book actually evolves. Historical backtesting with frozen snapshots is a distinct engineering problem: how do we simulate realistic fills against data that doesn't change in response to our orders? **Design choices:** - **Immutable historical data**: Order book and trade data are never modified. - **Optional consumption tracking**: When `liquidity_consumption=True`, the engine tracks consumed liquidity per price level to prevent duplicate fills. See [Order book immutability](#order-book-immutability) for configuration. - **Reproducible results**: A fixed `random_seed` pins the probabilistic fill model's PRNG. Same-process reruns are expected to match; cross-process reruns may differ in rare cases due to hash-ordering effects outside the fill model. ### Fill price determination The matching engine determines fill prices based on order type, book type, and market state. #### L2/L3 order book data With full order book depth, fills are determined by actual book simulation: | Order Type | Fill Price | | ---------------------- | ----------------------------------------------------------- | | `MARKET` | Walks the book, filling at each price level (taker). | | `MARKET_TO_LIMIT` | Walks the book, filling at each price level (taker). | | `LIMIT` | Order's limit price when matched (maker). | | `STOP_MARKET` | Walks the book when triggered. | | `STOP_LIMIT` | Order's limit price when triggered and matched. | | `MARKET_IF_TOUCHED` | Walks the book when triggered. | | `LIMIT_IF_TOUCHED` | Order's limit price when triggered. | | `TRAILING_STOP_MARKET` | Walks the book when activated and triggered. | | `TRAILING_STOP_LIMIT` | Order's limit price when activated, triggered, and matched. | With L2/L3 data, market-type orders may partially fill across multiple price levels if insufficient liquidity exists at the top of book. Limit-type orders act as resting orders after triggering and may remain unfilled if the market doesn't reach the limit price. `MARKET_TO_LIMIT` fills as a taker first, then rests any remaining quantity as a limit order at its first fill price. #### L1 order book data (quotes, trades, bars) With only top-of-book data, the same book simulation is used with a single-level book: | Order Type | BUY Fill Price | SELL Fill Price | | ---------------------- | -------------- | --------------- | | `MARKET` | Best ask | Best bid | | `MARKET_TO_LIMIT` | Best ask | Best bid | | `LIMIT` | Limit price | Limit price | | `STOP_MARKET` | Best ask | Best bid | | `STOP_LIMIT` | Limit price | Limit price | | `MARKET_IF_TOUCHED` | Best ask | Best bid | | `LIMIT_IF_TOUCHED` | Limit price | Limit price | | `TRAILING_STOP_MARKET` | Best ask | Best bid | | `TRAILING_STOP_LIMIT` | Limit price | Limit price | With L1 data, the simulated book has a single price level. Orders fill against the available size at that level. If an order has remaining quantity after exhausting top-of-book liquidity, market and marketable limit-style orders will slip one tick to fill the residual. For bar data specifically, `STOP_MARKET` and `TRAILING_STOP_MARKET` orders may fill at the trigger price rather than best ask/bid when the bar moves through the trigger during its high/low processing. See [Stop order fill behavior with bar data](#stop-order-fill-behavior-with-bar-data) for details. :::note Fill models can alter these fill prices. See the [Fill models](#fill-models) section for details on configuring execution simulation. ::: #### Order type semantics - **Market execution**: Fill at current market price (bid/ask). This models real exchange behavior where these orders execute at the best available price after triggering. Exception: with bar data, `STOP_MARKET` and `TRAILING_STOP_MARKET` orders triggered during H/L processing fill at the trigger price (see below). - **Limit execution**: Fill at the order's limit price when matched. Provides price guarantee but may not fill if the market doesn't reach the limit. #### Stop order fill behavior with bar data When backtesting with bar data only (no tick data), the matching engine distinguishes between two scenarios for `STOP_MARKET` and `TRAILING_STOP_MARKET` orders: **Gap scenario** (bar opens past trigger): When a bar's open price gaps past the trigger price, the stop triggers immediately and fills at the market price (the open). This models real exchange behavior where stop-market orders provide no price guarantee during gaps. Example - SELL `STOP_MARKET` with trigger at 100: - Previous bar closes at 105. - Next bar opens at 90 (overnight gap down). - Stop triggers at open and fills at 90. **Move-through scenario** (bar moves through trigger): When a bar opens normally and then its high or low moves through the trigger price, the stop fills at the trigger price. Since we only have OHLC data, we assume the market moved smoothly through the trigger and the order would have filled there. Example - SELL `STOP_MARKET` with trigger at 100: - Bar opens at 102 (no gap). - Bar low reaches 98, moving through trigger at 100. - Stop fills at 100 (the trigger price). This behavior caps potential slippage during orderly market moves while still modeling gap slippage accurately. For tick-level precision, use quote or trade tick data instead of bars. ### Price protection Price protection defines an exchange-calculated price boundary that prevents marketable orders from executing at excessively aggressive prices. This models exchanges like Binance and CME that implement protection mechanisms for market and stop-market orders. **Configuration:** ```python from nautilus_trader.backtest.config import BacktestVenueConfig venue_config = BacktestVenueConfig( name="BINANCE", oms_type="NETTING", account_type="MARGIN", starting_balances=["100_000 USDT"], price_protection_points=100, # 100 points = 1.00 offset for 2-decimal instruments ) ``` **How it works:** The matching engine calculates the protection boundary from the current best bid/ask at fill time: - **BUY orders**: `protection_price = ask + (points × price_increment)` - **SELL orders**: `protection_price = bid - (points × price_increment)` The engine filters out fills beyond the protection boundary. For example, with `price_protection_points=100` on an instrument with `price_increment=0.01`: - Best ask is 1001.00. - Protection price = 1001.00 + (100 × 0.01) = 1002.00. - A BUY market order fills only at prices ≤ 1002.00. - Liquidity at 1003.00 or higher is filtered, leaving the order partially filled. **Trigger-time semantics:** The engine computes protection at fill time, not order submission time: - **Market orders**: Protection computed immediately when the order processes. - **Stop-market orders**: Protection computed when the stop triggers, using the bid/ask at that moment. This design allows stop orders to be submitted even when the opposite side of the book is empty, since the engine computes protection later when the stop triggers. **Order types affected:** - `MARKET` - `STOP_MARKET` Limit orders are unaffected since they already define a price boundary. :::note Set `price_protection_points=0` to disable price protection (default behavior). ::: ### Slippage and spread handling When backtesting with different types of data, Nautilus implements specific handling for slippage and spread simulation: For L2 (market-by-price) or L3 (market-by-order) data, slippage is simulated with high accuracy by: - Filling orders against actual order book levels. - Matching available size at each price level sequentially. - Maintaining realistic order book depth impact (per order fill). For L1 data types (e.g., L1 order book, trades, quotes, bars), slippage is handled through the `FillModel`: **Per-fill slippage** (`prob_slippage`): - Applies to each fill when using an L1 book with a configured `FillModel`. - Affects all order types (market, limit, stop, etc.). - When triggered, moves the fill price one tick against the order direction. - Example: With `prob_slippage=0.5`, a BUY order has 50% chance of filling one tick above the best ask. :::note When backtesting with bar data, be aware that the reduced granularity of price information affects the slippage mechanism. For the most realistic backtesting results, consider using higher granularity data sources such as L2 or L3 order book data when available. ::: #### How simulation varies by data type The behavior of the `FillModel` adapts based on the order book type being used: **L2/L3 order book data** With full order book depth, the `FillModel` focuses purely on simulating queue position for limit orders through `prob_fill_on_limit`. The order book itself handles slippage naturally based on available liquidity at each price level. - `prob_fill_on_limit` is active - simulates queue position. - `prob_slippage` is not used - real order book depth determines price impact. :::warning The historical order book is immutable during backtesting. Book depth is **not** decremented after fills. By default (`liquidity_consumption=False`), the same liquidity can be consumed repeatedly within an iteration. Enable `liquidity_consumption=True` to track consumed liquidity per price level. Consumption resets when fresh data arrives at that level. See [Order book immutability](#order-book-immutability) for details. ::: **L1 order book data** With only best bid/ask prices available, the `FillModel` provides additional simulation: - `prob_fill_on_limit` is active - simulates queue position. - `prob_slippage` is active - simulates basic price impact since we lack real depth information. **Bar/Quote/Trade data** When using less granular data, the same behaviors apply as L1: - `prob_fill_on_limit` is active - simulates queue position. - `prob_slippage` is active - simulates basic price impact. #### Important considerations - **Partial fills**: With L2/L3 data, fills are limited to available liquidity at each price level. With L1 data, the full order quantity fills at the single available level. - **Consumption tracking**: See [Order book immutability](#order-book-immutability) for details on preventing duplicate fills. ### Order book immutability Historical order book data is immutable during backtesting. When your order fills against book liquidity, the book state remains unchanged. This preserves historical data integrity. The matching engine can optionally use **per-level consumption tracking** to prevent duplicate fills while allowing fills when fresh liquidity arrives. This behavior is controlled by the `liquidity_consumption` configuration option. **Configuration:** ```python from nautilus_trader.backtest.config import BacktestVenueConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="CASH", starting_balances=["100_000 USD"], liquidity_consumption=True, # Enable consumption tracking (default: False) ) ``` - `liquidity_consumption=False` (default): Each iteration fills against the full book liquidity independently. Simpler behavior, assumes you're a small participant whose orders don't meaningfully impact available liquidity. - `liquidity_consumption=True`: Tracks consumed liquidity per price level. Prevents the same displayed liquidity from generating multiple fills. Resets when fresh data arrives at that level. **How consumption tracking works (when enabled):** For each price level, the engine maintains: - `original_size`: The book's quantity when tracking began. - `consumed`: How much has been filled against this level. When processing a fill: 1. Check if the book's current size at this level matches `original_size` 2. If different (fresh data arrived), reset the entry: `original_size = current_size`, `consumed = 0` 3. Calculate `available = original_size - consumed` 4. After filling, increment `consumed` by the fill quantity **Example:** 1. Order book shows 100 units at ask 100.00. Engine tracks: `(original=100, consumed=0)`. 2. Your BUY order fills 30 units. Engine updates: `(original=100, consumed=30)`. Available = 70. 3. Another BUY order attempts 50 units. Available = 70, so it fills 50. `(original=100, consumed=80)`. 4. A delta updates ask 100.00 to 120 units. Engine resets: `(original=120, consumed=0)`. 5. New orders can now fill against the fresh 120 units. **Passive limit order fills on L1 data:** With L1 data (quotes, trades, bars), the book has only a single price level per side. When the market moves through a passive (MAKER) limit order's price, the engine must decide how to handle remaining order quantity after exhausting displayed liquidity. | `liquidity_consumption` | Behavior when market moves through passive limit | | ----------------------- | ----------------------------------------------------------------------------------------------- | | `False` (default) | Fill entire order at limit price. Assumes market movement implies sufficient liquidity existed. | | `True` | Fill only against displayed liquidity. Order remains open for subsequent fills. | **Example scenario** (`liquidity_consumption=True`): 1. Quote shows ask 100.10 with 50 units. 2. You place BUY LIMIT at 100.05 for 1000 units (passive, resting below ask). 3. Next quote shows ask 100.00 with 30 units (market moved through your limit). 4. Order fills 30 units against displayed liquidity. 970 units remain open. 5. Next quote shows ask 99.95 with 200 units. 6. Order fills another 200 units. 770 units remain open. 7. Fills continue as fresh liquidity arrives at crossed price levels. This behavior provides conservative fill simulation: your order only fills against liquidity actually observed in the data, rather than inferring liquidity from price movements. **Trade tick liquidity:** Trade ticks provide evidence of executable liquidity at the trade price. When a trade occurs at a price level not reflected in the current book, the engine can use the trade quantity as available liquidity, subject to the same consumption tracking rules (when enabled). **Trade consumption seeding:** When using L2/L3 book data and a trade tick triggers order matching (e.g., triggering a resting stop order), the trade itself consumed liquidity from the book. Before simulating fills for triggered orders, the engine pre-seeds the consumption maps with the trade's consumed volume. This prevents triggered orders from filling against liquidity that the triggering trade already consumed. This seeding is skipped for L1 books, where the trade tick has already updated the single top-of-book level directly. For example, if the book has 10 units at the best ask and a BUY trade of size 8 triggers a stop market BUY for 5 units, the stop order sees only 2 units remaining at best ask (10 - 8) and must fill the remaining 3 units at the next price level. Without this seeding, the stop would incorrectly fill all 5 units at the best ask price. The engine uses a timestamp guard to avoid double-counting: if the book's most recent update (`ts_last`) is newer than the trade's event time (`ts_event`), seeding is skipped. This handles exchanges like Binance where depth deltas arrive before the corresponding trade tick, so the book already reflects the consumed liquidity, so additional seeding would over-penalize fills. :::note As the `FillModel` continues to evolve, future versions may introduce more sophisticated simulation of order execution dynamics, including: - Variable slippage based on order size. - More complex queue position modeling. ::: #### Known limitations **No queue position within a level**: Consumption tracking determines *how much* liquidity remains at a level, but doesn't model *where* your order sits in the queue relative to other participants. Use `prob_fill_on_limit` to simulate queue position probabilistically. **Trade-driven fills are opportunistic**: When trade ticks indicate liquidity at a price not in the book, the engine uses this as fill evidence. However, this represents liquidity that existed momentarily and may not reflect sustained availability. ### Trade-based execution Trade tick data triggers order fills by default (`trade_execution=True`). A trade tick indicates that liquidity was accessed at the trade price, allowing resting limit orders to match. This mirrors the default behavior for bar data (`bar_execution=True`). Advanced users who want to isolate execution to L1 book data only (quotes or order book updates) can disable trade-based execution: ```python venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="CASH", starting_balances=["100_000 USD"], trade_execution=False, # Disable trade-based fills ) ``` When `trade_execution=False` or `bar_execution=False`, the respective data types skip order matching and maintenance operations (GTD order expiry, trailing stop activation, instrument expiration checks). Quote ticks always trigger maintenance, so this is typically acceptable when using multiple data types. The matching engine uses a "transient override" mechanism: during the matching process, it temporarily adjusts the matching core's Best Bid (for BUYER trades) or Best Ask (for SELLER trades) toward the trade price. This allows resting orders on the passive side to cross the spread and fill. Note: the underlying order book data is never modified (it remains immutable); only the matching core's internal price references are adjusted. **Fill determination:** When a trade tick triggers order matching, the engine determines fills as follows: 1. **Book reflects trade price**: If the order book has liquidity at the trade price, fills use book depth (standard behavior). 2. **Book doesn't reflect trade price**: If the book's liquidity is at a different price, the engine uses a "trade-driven fill" at the order's limit price, capped to `min(order.leaves_qty, trade.size)`. This ensures that when a trade prints through the spread but the book hasn't updated, fills are bounded by what the trade tick actually evidences. When `liquidity_consumption=False` (default), the same trade size can fill multiple orders within an iteration. When `liquidity_consumption=True`, consumption tracking applies to trade-driven fills as well. Repeated fills at the same trade price will be bounded by consumed liquidity until fresh data arrives. **Restoration behavior:** After matching, the core's bid/ask are only restored to their original values if the trade price improved them (moved them away from the spread): - **SELLER trade**: Ask is restored only if trade price was below the original ask. - **BUYER trade**: Bid is restored only if trade price was above the original bid. If the trade price didn't improve the quote (e.g., a SELLER trade at or above the ask), the core retains the trade price. This means repeated trades at or beyond the spread can progressively move the core's bid/ask. **Fill price:** - **SELLER trade at P**: The engine sets the core's Best Ask to P (if P < current ask). Resting BUY LIMIT orders at P or higher will fill at their limit price (if book doesn't have that level) or at book prices (if book does). - **BUYER trade at P**: The engine sets the core's Best Bid to P (if P > current bid). Resting SELL LIMIT orders at P or lower will fill at their limit price (if book doesn't have that level) or at book prices (if book does). This conservative approach ensures fills occur at the order's limit price rather than potentially better trade prices. For example, a BUY LIMIT at 100.05 triggered by a SELLER trade at 100.00 will fill at 100.05, not 100.00. :::tip Combine trade data with book or quote data for best results: book/quote data establishes the baseline spread, while trade ticks trigger execution for orders that might be inside the spread or ahead of the quote updates. ::: #### Understanding trade tick aggressor sides A common source of confusion is the `aggressor_side` field on trade ticks: - **SELLER trade**: A seller aggressed, selling into the bid. This provides evidence of fill-able liquidity for **BUY** orders at the trade price. - **BUYER trade**: A buyer aggressed, buying from the ask. This provides evidence of fill-able liquidity for **SELL** orders at the trade price. In other words, trade ticks trigger fills for orders on the **opposite** side of the aggressor. A SELLER trade at 100.00 can fill your resting BUY LIMIT at 100.00, but cannot fill your SELL LIMIT, since the trade already represents someone else selling. #### Combining L2 book data with trade ticks When using L2 order book data (e.g., 100ms throttled depth snapshots) combined with trade tick data: 1. **Book updates establish the spread**: Each book delta/snapshot updates the matching engine's view of available liquidity at each price level. 2. **Trade ticks provide execution evidence**: Trade ticks indicate that liquidity was accessed at a specific price, potentially between book snapshots. 3. **Fill quantity determination**: When a trade triggers a fill: - If the book already reflects liquidity at the trade price, fills use book depth - If the trade price is inside the spread (not in the current book), fills are capped by `min(order.leaves_qty, trade.size)` 4. **Timing considerations**: With throttled book data (e.g., 100ms), the book may lag behind trades. A trade at a price not yet reflected in the book will use trade-driven fill logic. **Common misconception**: Users sometimes expect every trade tick to trigger fills. Remember: - Only trades on the **opposite** side can fill your orders. - SELLER trades -> potential BUY fills. - BUYER trades -> potential SELL fills. - Book UPDATE events move the market but only trigger fills if prices cross your order. #### Queue position tracking When `queue_position=True` is enabled alongside `trade_execution=True`, the matching engine simulates queue position for limit orders. This provides more realistic fill behavior by tracking how many orders are "ahead" of your order at a given price level. **How it works:** 1. **Order placement**: When a LIMIT order is accepted, the engine snapshots the current same-side book depth at the order's price level. This represents the orders ahead in the queue. 2. **Trade ticks**: When trade ticks occur at the order's price level, the "quantity ahead" is decremented by the trade size. Only trades on the correct side affect the queue (BUYER trades decrement queue for SELL orders, SELLER trades decrement queue for BUY orders). Trades with `NO_AGGRESSOR` (common in historical datasets lacking aggressor metadata) affect both sides. This is pessimistic but prevents orders from stalling indefinitely. 3. **Fill eligibility**: The order becomes eligible to fill only when the quantity ahead reaches zero. On the tick that clears the queue, only the excess volume (trade size minus queue ahead) is available for fill, preventing overfill. 4. **Price level DELETE**: If the order book level is deleted (BookAction.DELETE), the queue clears immediately, making the order fill-eligible. UPDATE actions are ignored (queue unchanged). 5. **Order modification**: If the order is modified (price or quantity change), the queue position resets. The order moves to the back of the queue at its new price level. **Configuration:** ```python from nautilus_trader.backtest.config import BacktestVenueConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="MARGIN", starting_balances=["100_000 USD"], trade_execution=True, # Required for queue_position queue_position=True, # Enable queue position tracking ) ``` **Example scenario:** 1. Order book shows 100 units at bid 100.00. 2. You place a BUY LIMIT at 100.00 for 50 units. Queue ahead = 100. 3. SELLER trade of 80 units at 100.00 -> queue ahead = 20. No fill yet. 4. SELLER trade of 30 units at 100.00 -> queue clears with 10 excess. Fill = 10 units. 5. Next SELLER trade of 50 units -> fill remaining 40 units. **Limitations:** - Only applies to `LIMIT` orders. Stop-limit and limit-if-touched orders are not tracked in this implementation. - Queue position is per-order, not shared across multiple orders at the same price. - The queue snapshot is based on book state at order acceptance time. - Trades with `NO_AGGRESSOR` decrement queue for both sides, which may cause orders to fill sooner than in reality (pessimistic for queue estimation, but prevents stalling). **L1 quote-based mode:** When using `BookType.L1_MBP` (top-of-book quotes only), queue position tracking uses trade ticks to decrement the queue (the same mechanism as L2/L3), while quote ticks handle price-move detection and deferred snapshot resolution. - **Trade ticks**: Trades at the order's price level decrement the queue ahead by the trade size, identical to L2/L3 behavior. Only trades on the correct aggressor side affect the queue (SELLER trades decrement queue for BUY orders, BUYER trades for SELL orders). - **Price moves away**: If the bid drops below a BUY order's price (or ask rises above a SELL order's price), the order's price level has been "crossed" and the queue clears to zero, making the order fill-eligible on the next matching trade. - **Price moves toward**: If the bid rises (or ask drops), the level at the order's price was not consumed, so queue positions are preserved. - **Price returns to a level**: When the price returns after moving away, the queue ahead is capped at the new displayed size if it was previously larger. - **Orders behind BBO (pending)**: When a limit order is placed behind the best bid/ask (e.g., BUY below best bid), the queue snapshot is deferred because L1 data has no visible depth at that level. Fills are blocked until the BBO reaches the order's price, at which point the queue is snapshotted from the displayed size. Pending orders are also resolved when trades cross through their price level. L1 mode uses the same configuration: set `queue_position=True` with `book_type=BookType.L1_MBP`. This provides a lightweight alternative to full L2/L3 data when only top-of-book quotes are available. :::note Queue position tracking provides a heuristic simulation of queue dynamics. Real exchange queue behavior depends on many factors (order priority rules, hidden orders, etc.) that cannot be perfectly reconstructed from historical data. ::: ### Bar-based execution Bar data provides a summary of market activity with four key prices for each time period (assuming bars are aggregated by trades): - **Open**: opening price (first trade) - **High**: highest price traded - **Low**: lowest price traded - **Close**: closing price (last trade) While this gives us an overview of price movement, we lose some important information that we'd have with more granular data: - We don't know in what order the market hit the high and low prices. - We can't see exactly when prices changed within the time period. - We don't know the actual sequence of trades that occurred. This is why Nautilus processes bar data through a system that attempts to maintain the most realistic yet conservative market behavior possible, despite these limitations. At its core, the platform always maintains an order book simulation - even when you provide less granular data such as quotes, trades, or bars (although the simulation will only have a top level book). :::warning When using bars for execution simulation (enabled by default with `bar_execution=True` in venue configurations), Nautilus strictly expects the initialization timestamp (`ts_init`) of each bar to represent its **closing time**. This ensures accurate chronological processing, prevents look-ahead bias, and aligns market updates (Open -> High -> Low -> Close) with the moment the bar is complete. The event timestamp (`ts_event`) can represent either the open or close time of the bar: - If `ts_event` is at the **close**, ensure `ts_init_delta=0` when processing bars (default). - If `ts_event` is at the **open**, set `ts_init_delta` equal to the bar's duration to shift `ts_init` to the close. ::: #### Bar timestamp convention If your data source provides bars timestamped at the **opening time** (common in some providers), you need to ensure `ts_init` is set to the closing time for correct execution simulation. There are two approaches: **Approach 1: Adjust data timestamps (recommended)** - Use adapter-specific configurations like `bars_timestamp_on_close=True` (e.g., for Bybit or Databento adapters) to handle this automatically during data ingestion. - For custom data, manually shift the timestamps by the bar duration before loading (e.g., add 1 minute for `1-MINUTE` bars). - This approach is clearest because the data itself reflects the close time. **Approach 2: Use `ts_init_delta` parameter** - When calling `BarDataWrangler.process()`, set `ts_init_delta` to the bar's duration in nanoseconds (e.g., `60_000_000_000` for 1-minute bars). - The wrangler computes `ts_init = ts_event + ts_init_delta`, shifting execution timing to the close. - Use this when you cannot or prefer not to modify source data timestamps. Always verify your data's timestamp convention with a small sample to avoid simulation inaccuracies. Incorrect timestamp handling can lead to look-ahead bias and unrealistic backtest results. #### Processing bar data Even when you provide bar data, Nautilus maintains an internal order book for each instrument, as a real venue would. 1. **Time processing**: - Nautilus has a specific way of handling the timing of bar data *for execution* that's crucial for accurate simulation. - The initialization timestamp (`ts_init`) is used for execution timing and must represent the close time of the bar. This approach is most logical because it represents the moment when the bar is fully formed and its aggregation is complete. - The event timestamp (`ts_event`) represents when the data event occurred and may differ from `ts_init` depending on your data source: - If your bars are timestamped at the **close** (the recommended default), use `ts_init_delta=0` in `BarDataWrangler` so that `ts_init = ts_event`. - If your bars are timestamped at the **open**, set `ts_init_delta` to the bar's duration in nanoseconds (e.g., 60_000_000_000 for 1-minute bars) to shift `ts_init` to the close time. - The platform ensures all events happen in the correct sequence based on `ts_init`, preventing any possibility of look-ahead bias in your backtests. :::note[Exceptions for bar execution] Bars will **not** be processed for execution (and will not update the order book) in the following cases: - **Internally aggregated bars**: Bars with `AggregationSource.INTERNAL` are skipped to avoid processing bars that are derived from already-processed tick data. - **Non-L1 book types**: When the venue's `book_type` is configured as `L2_MBP` or `L3_MBO`, bar data is ignored for execution processing, as bars are derived from top-of-book prices only. In these cases, bars will still be received by strategies for analytics and decision-making, but they won't trigger order matching or update the simulated order book. ::: 2. **Price processing**: - The platform converts each bar's OHLC prices into a sequence of market updates. - By default, updates follow the order: Open -> High -> Low -> Close (configurable via `bar_adaptive_high_low_ordering`). - If you provide multiple timeframes (like both 1-minute and 5-minute bars), the platform uses the more granular data for highest accuracy. 3. **Executions**: - When you place orders, they interact with the simulated order book as they would on a real venue. - For MARKET orders, execution happens at the current simulated market price plus any configured latency. - For LIMIT orders working in the market, they'll execute if any of the bar's prices reach or cross your limit price (see below). - The matching engine continuously processes orders as OHLC prices move, rather than waiting for complete bars. #### OHLC prices simulation During backtest execution, each bar is converted into a sequence of four price points: 1. Opening price 2. High price *(Order between High/Low is configurable. See `bar_adaptive_high_low_ordering` below.)* 3. Low price 4. Closing price The trading volume for that bar is **split evenly** among these four points (25% each), with any remainder added to the closing price trade to preserve total volume. In marginal cases, if the bar's volume divided by 4 is less than the instrument's minimum `size_increment`, we use the minimum `size_increment` per price point to ensure valid market activity (e.g., 1 contract for CME group exchanges). How these price points are sequenced can be controlled via the `bar_adaptive_high_low_ordering` parameter when configuring a venue. Nautilus supports two modes of bar processing: 1. **Fixed ordering** (`bar_adaptive_high_low_ordering=False`, default) - Processes every bar in a fixed sequence: `Open -> High -> Low -> Close`. - Simple and deterministic approach. 2. **Adaptive ordering** (`bar_adaptive_high_low_ordering=True`) - Uses bar structure to estimate likely price path: - If Open is closer to High: processes as `Open -> High -> Low -> Close`. - If Open is closer to Low: processes as `Open -> Low -> High -> Close`. - [Research](https://gist.github.com/stefansimik/d387e1d9ff784a8973feca0cde51e363) shows this approach achieves ~75-85% accuracy in predicting correct High/Low sequence (compared to statistical ~50% accuracy with fixed ordering). - This is particularly important when both take-profit and stop-loss levels occur within the same bar - as the sequence determines which order fills first. Here's how to configure adaptive bar ordering for a venue, including account setup: ```python from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.model.enums import OmsType, AccountType from nautilus_trader.model import Money, Currency # Initialize the backtest engine engine = BacktestEngine() # Add a venue with adaptive bar ordering and required account settings engine.add_venue( venue=venue, # Your Venue identifier, e.g., Venue("BINANCE") oms_type=OmsType.NETTING, account_type=AccountType.CASH, starting_balances=[Money(10_000, Currency.from_str("USDT"))], bar_adaptive_high_low_ordering=True, # Enable adaptive ordering of High/Low bar prices ) ``` #### Order submission timing Bar N's OHLC sequence processes before `on_bar(N)` fires. Without a `LatencyModel`, an order submitted from `on_bar` settles immediately and matches against the current book, whose top reflects bar N's close. Attach a `LatencyModel` to the venue to defer the order's effective arrival. With bar-only data and no intervening timer events, the order settles after the next bar's OHLC sweep, so the fill price is that bar's close (or a later bar's close if latency exceeds the bar interval). Finer-grained data (quotes, trades) or timer-driven settlement between bars can drain the order earlier, against the book as it stands at that point: ```python from nautilus_trader.backtest.models import LatencyModel engine.add_venue( venue=venue, oms_type=OmsType.NETTING, account_type=AccountType.CASH, starting_balances=[Money(10_000, Currency.from_str("USDT"))], latency_model=LatencyModel(base_latency_nanos=1_000_000_000), # 1 second ) ``` :::note A native "next-bar-open" execution mode is not provided. A bar's `ts_init` is its close timestamp, so the open price is only known once the bar arrives. Filling at that open from a signal generated on the prior bar would require look-ahead. ::: ### Internal bar aggregation timing When aggregating time bars internally from tick data, the data engine uses timers to close bars at interval boundaries. A timing edge case occurs when data arrives at the exact bar close timestamp: the timer may fire before processing boundary data. Configure `time_bars_build_delay` in `DataEngineConfig` to delay bar close timers: ```python from nautilus_trader.config import BacktestEngineConfig from nautilus_trader.data.config import DataEngineConfig config = BacktestEngineConfig( data_engine=DataEngineConfig( time_bars_build_delay=1, # Microseconds ), ) ``` :::tip A small delay (1 microsecond) ensures boundary data is processed before the bar closes. Useful when tick data clusters at round interval timestamps. ::: :::note Only affects internally aggregated bars (`AggregationSource.INTERNAL`). ::: ### Timer-only backtests The backtest engine supports running with timers but no market data. This is useful for scheduled operations or testing timer-based logic. Timers fire in chronological order, and timer callbacks can dynamically add data via `add_data_iterator()` which will be processed in sequence. :::warning Data added by timer callbacks at the exact start time should have timestamps **after** the start time. The engine reads the first data point before processing start-time timers, so dynamically added data with timestamps at or before the start time may not be processed in the expected order. ::: ### Fill models Fill models simulate order execution dynamics during backtesting. They address a fundamental challenge: *even with perfect historical market data, we can't fully simulate how orders may have interacted with other market participants in real-time*. The base `FillModel` provides probabilistic parameters for queue position and slippage simulation. Subclasses can override `get_orderbook_for_fill_simulation()` to generate synthetic order books for more sophisticated liquidity modeling. #### Available fill models | Model | Description | Use Case | | ---------------------------- | ------------------------------------------------------- | -------------------------------------------- | | `FillModel` | Base model with probabilistic fill/slippage parameters. | Simple queue position and slippage. | | `BestPriceFillModel` | Fills at best price with unlimited liquidity. | Testing basic strategy logic optimistically. | | `OneTickSlippageFillModel` | Forces exactly one tick of slippage on all orders. | Conservative slippage testing. | | `TwoTierFillModel` | 10 contracts at best price, remainder one tick worse. | Basic market depth simulation. | | `ThreeTierFillModel` | 50/30/20 contracts across three price levels. | More realistic depth simulation. | | `ProbabilisticFillModel` | 50% chance best price, 50% chance one tick slippage. | Randomized execution quality. | | `SizeAwareFillModel` | Different execution based on order size (≤10 vs >10). | Size‑dependent market impact. | | `LimitOrderPartialFillModel` | Max 5 contracts fill per price touch. | Queue position via partial fills. | | `MarketHoursFillModel` | Wider spreads during low liquidity periods. | Session‑aware execution. | | `VolumeSensitiveFillModel` | Liquidity based on recent trading volume. | Volume‑adaptive depth. | | `CompetitionAwareFillModel` | Only percentage of visible liquidity available. | Multi‑participant competition. | #### Configuring fill models **Using the base FillModel with probabilistic parameters:** ```python from nautilus_trader.backtest.config import BacktestVenueConfig from nautilus_trader.backtest.config import ImportableFillModelConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="CASH", starting_balances=["100_000 USD"], fill_model=ImportableFillModelConfig( fill_model_path="nautilus_trader.backtest.models:FillModel", config_path="nautilus_trader.backtest.config:FillModelConfig", config={ "prob_fill_on_limit": 0.2, # Chance a limit order fills when price matches "prob_slippage": 0.5, # Chance of 1-tick slippage (L1 data only) "random_seed": 42, # Optional: Set for reproducible results }, ), ) ``` **Using an order book simulation model:** ```python from nautilus_trader.backtest.config import BacktestVenueConfig from nautilus_trader.backtest.config import ImportableFillModelConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="CASH", starting_balances=["100_000 USD"], fill_model=ImportableFillModelConfig( fill_model_path="nautilus_trader.backtest.models:ThreeTierFillModel", ), ) ``` #### Probabilistic parameters (base FillModel) **prob_fill_on_limit** (default: `1.0`) Simulates queue position by controlling the probability of a limit order filling when its price level is touched (but not crossed). - `0.0`: Never fills at touch (back of queue). - `0.5`: 50% chance of filling (middle of queue). - `1.0`: Always fills at touch (front of queue). **prob_slippage** (default: `0.0`) Simulates price slippage on each fill. Only applies to L1 data types (quotes, trades, bars) where real depth is unavailable. Affects all order types when executing as takers. - `0.0`: No slippage (fills at best price). - `0.5`: 50% chance of one tick slippage per fill. - `1.0`: Always slips one tick. #### Order book simulation models These models override the `get_orderbook_for_fill_simulation()` method to generate synthetic order books representing expected market liquidity. The matching engine fills orders against this simulated book. **How it works:** 1. Before processing a fill, the matching engine calls `get_orderbook_for_fill_simulation()`. 2. If the model returns a synthetic order book, fills execute against that book's liquidity. 3. If the model returns `None`, standard fill logic applies. :::note When a custom fill model provides a simulated order book, the `liquidity_consumption` tracking is **not** applied. Custom fill models are expected to manage their own liquidity simulation within the returned order book. Liquidity consumption tracking only affects the built-in fill logic (when `get_orderbook_for_fill_simulation()` returns `None`). ::: **Example: ThreeTierFillModel** This model creates a book with liquidity distributed across three price levels: - 50 contracts at best price - 30 contracts one tick worse - 20 contracts two ticks worse A 100-contract market order would fill partially at each level, experiencing realistic price impact. **Creating custom fill models:** ```python from nautilus_trader.backtest.models import FillModel from nautilus_trader.model.book import OrderBook, BookOrder from nautilus_trader.model.enums import OrderSide from nautilus_trader.core.rust.model import BookType class MyCustomFillModel(FillModel): def get_orderbook_for_fill_simulation( self, instrument, order, best_bid, best_ask, ): book = OrderBook( instrument_id=instrument.id, book_type=BookType.L2_MBP, ) # Add custom liquidity based on your market model # ... return book ``` ### Precision requirements and invariants The matching engine enforces strict precision invariants to ensure data integrity throughout the fill pipeline. All prices and quantities must match the instrument's configured precision (`price_precision` and `size_precision`). Mismatches raise a `RuntimeError` immediately, preventing silent corruption of fill quantities. | Data/Operation | Field | Required Precision | Validation Location | | -------------- | ------------------------------ | ---------------------------- | --------------------------- | | `QuoteTick` | `bid_price`, `ask_price` | `instrument.price_precision` | `process_quote_tick` | | `QuoteTick` | `bid_size`, `ask_size` | `instrument.size_precision` | `process_quote_tick` | | `TradeTick` | `price` | `instrument.price_precision` | `process_trade_tick` | | `TradeTick` | `size` | `instrument.size_precision` | `process_trade_tick` | | `Bar` | `open`, `high`, `low`, `close` | `instrument.price_precision` | `process_bar` | | `Bar` | `volume` (base units) | `instrument.size_precision` | `process_bar` | | `Order` | `quantity` | `instrument.size_precision` | `process_order` | | `Order` | `price` | `instrument.price_precision` | `process_order` | | `Order` | `trigger_price` | `instrument.price_precision` | `process_order` | | `Order` | `activation_price`\* | `instrument.price_precision` | `process_order` | | Order update | `quantity` | `instrument.size_precision` | `update_order` | | Order update | `price`, `trigger_price` | `instrument.price_precision` | `update_order` | | Fill | `fill_qty` | `instrument.size_precision` | `apply_fills`, `fill_order` | | Fill | `fill_px` | `instrument.price_precision` | `apply_fills` | \*`activation_price` is immutable after order submission. :::warning `Bar.volume` must be in **base currency units**. Some data providers report quote-currency volume; convert to base units before loading (divide by price or use provider-specific fields). ::: :::tip If you encounter a precision mismatch error, align your data to the instrument: ```python # Align price/quantity to instrument precision price = instrument.make_price(raw_price) qty = instrument.make_qty(raw_qty) ``` Also verify that: 1. The instrument definition matches your data source's precision. 2. Data was not inadvertently rounded or truncated during loading. 3. Custom data loaders preserve the original precision metadata. ::: ## Accounts Every backtest venue is attached with one of three `account_type` values: `CASH`, `MARGIN`, or `BETTING`. For the full data model, query API, and margin model reference, see [Accounting](accounting.md). Example of adding a `CASH` account for a backtest venue: ```python from nautilus_trader.adapters.binance import BINANCE_VENUE from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.model.currencies import USDT from nautilus_trader.model.enums import OmsType, AccountType from nautilus_trader.model import Money, Currency # Initialize the backtest engine engine = BacktestEngine() # Add a CASH account for the venue engine.add_venue( venue=BINANCE_VENUE, # Create or reference a Venue identifier oms_type=OmsType.NETTING, account_type=AccountType.CASH, starting_balances=[Money(10_000, USDT)], ) ``` ## Margin models Margin models determine how the simulated exchange reserves collateral for orders and positions in backtest runs. The model types (`StandardMarginModel` vs `LeveragedMarginModel`), their formulas, the default behavior, and custom model authoring are covered in the dedicated [Accounting](accounting.md#margin-models) guide. This section covers only the backtest-specific configuration. ### Backtest venue configuration Specify the margin model on `BacktestVenueConfig` via `MarginModelConfig`: ```python from nautilus_trader.backtest.config import BacktestVenueConfig from nautilus_trader.backtest.config import MarginModelConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="MARGIN", starting_balances=["1_000_000 USD"], margin_model=MarginModelConfig(model_type="standard"), # Options: 'standard', 'leveraged' ) ``` Available `model_type` values: - `"leveraged"`: margin reduced by leverage (default). - `"standard"`: fixed percentages (traditional brokers). - Fully-qualified class path for a custom model: `"my_package.my_module:MyMarginModel"`. ### High-level backtest API When using the high-level API, attach the margin model in the same way: ```python from nautilus_trader.backtest.config import BacktestVenueConfig from nautilus_trader.backtest.config import MarginModelConfig from nautilus_trader.config import BacktestRunConfig venue_config = BacktestVenueConfig( name="SIM", oms_type="NETTING", account_type="MARGIN", starting_balances=["1_000_000 USD"], margin_model=MarginModelConfig( model_type="standard", # Traditional broker simulation ), ) config = BacktestRunConfig( venues=[venue_config], # ... other config ) ``` Custom model with parameters: ```python margin_model=MarginModelConfig( model_type="my_package.my_module:CustomMarginModel", config={ "risk_multiplier": 1.5, "use_leverage": False, "volatility_threshold": 0.02, }, ) ``` The model is applied to the simulated exchange during backtest execution. ## Trade ID derivation The simulated exchange (used by both backtest and sandbox execution) emits a deterministic `TradeId` for each generated fill. The ID is formatted as `T-{hash:016x}-{count:03d}`, where the 16-character hex is an FNV-1a hash of `(venue, raw_id, ts_init)` and the trailing counter distinguishes multiple fills at the same `ts_init` (e.g. several legs of a bar-driven fill). **Properties**: - Deterministic across runs: the same replayed data produces the same `TradeId` every time, so downstream dedup and golden-output comparisons stay stable. - Collision-safe across resets: `ts_init` is pinned in backtest data and monotonic in live/sandbox, so a `BacktestEngine.reset()` (or an in-memory `IdsGenerator` reset in a sandbox with persisted orders) cannot mint a `TradeId` that collides with one already in the cache. - Bounded length: the hash keeps the identifier under the 36-character `TradeId` cap regardless of venue name length. The `use_random_ids` venue flag still governs `VenueOrderId` and `PositionId` generation, but `TradeId` is always deterministic and is not affected by the flag. ## Related guides - [Strategies](strategies.md) - Develop strategies to backtest. - [Visualization](visualization.md) - Generate tearsheets from backtest results. - [Reports](reports.md) - Analyze backtest performance data. # Cache Source: https://nautilustrader.io/docs/latest/concepts/cache/ The `Cache` is a central in-memory database that stores and manages all trading-related data, from market data to order history to custom calculations. The Cache serves multiple purposes: 1. **Stores market data**: - Stores recent market history (e.g., order books, quotes, trades, bars). - Gives you access to both current and historical market data for your strategy. 2. **Tracks trading data**: - Maintains complete `Order` history and current execution state. - Tracks all `Position`s and `Account` information. - Stores `Instrument` definitions and `Currency` information. 3. **Stores custom data**: - You can store any user-defined objects or data in the `Cache` for later use. - Enables data sharing between different strategies. ## How caching works **Built-in types**: - The system automatically adds data to the `Cache` as it flows through. - In live contexts, the engine applies updates asynchronously, so you might see a brief delay between an event and its appearance in the `Cache`. - For quotes, trades, and bars the `DataEngine` writes to the `Cache` before publishing to subscribers, so the latest value is available in the cache by the time your handler runs. Order book deltas and depth snapshots are published directly without a cache write; book state is maintained separately through `BookUpdater` subscriptions: ```mermaid flowchart LR data[Data] engine[DataEngine] cache[Cache] callback["Strategy callback:
on_quote_tick(...)"] data --> engine --> cache --> callback ``` For the full step-by-step trace, see [Data flow: life of a quote tick](architecture.md#data-flow-life-of-a-quote-tick). ### Basic example Within a strategy, you can access the `Cache` through `self.cache`. Here’s a typical example: :::note Within a `Strategy` class, `self` refers to the strategy instance. ::: ```python def on_bar(self, bar: Bar) -> None: # Current bar is provided in the parameter 'bar' # Get historical bars from Cache last_bar = self.cache.bar(self.bar_type, index=0) # Last bar (practically the same as the 'bar' parameter) previous_bar = self.cache.bar(self.bar_type, index=1) # Previous bar third_last_bar = self.cache.bar(self.bar_type, index=2) # Third last bar # Get current position information if self.last_position_opened_id is not None: position = self.cache.position(self.last_position_opened_id) if position.is_open: # Check position details current_pnl = position.unrealized_pnl # Get all open orders for our instrument open_orders = self.cache.orders_open(instrument_id=self.instrument_id) ``` ## Configuration Use the `CacheConfig` class to configure the `Cache` behavior and capacity. You can provide this configuration either to a `BacktestEngine` or a `TradingNode`, depending on your [environment context](architecture.md#environment-contexts). Here's a basic example of configuring the `Cache`: ```python from nautilus_trader.config import CacheConfig, BacktestEngineConfig, TradingNodeConfig # For backtesting engine_config = BacktestEngineConfig( cache=CacheConfig( tick_capacity=10_000, # Store last 10,000 ticks per instrument bar_capacity=5_000, # Store last 5,000 bars per bar type ), ) # For live trading node_config = TradingNodeConfig( cache=CacheConfig( tick_capacity=10_000, bar_capacity=5_000, ), ) ``` :::tip By default, the `Cache` keeps the last 10,000 bars for each bar type and 10,000 trade ticks per instrument. These limits provide a good balance between memory usage and data availability. Increase them if your strategy needs more historical data. ::: ### Configuration options The `CacheConfig` type supports these parameters: ```rust use nautilus_common::{cache::CacheConfig, enums::SerializationEncoding}; let config = CacheConfig { encoding: SerializationEncoding::MsgPack, timestamps_as_iso8601: false, buffer_interval_ms: None, bulk_read_batch_size: None, use_trader_prefix: true, use_instance_id: false, flush_on_start: false, drop_instruments_on_reset: true, tick_capacity: 10_000, bar_capacity: 10_000, persist_account_events: true, save_market_data: false, }; ``` :::note Each bar type maintains its own separate capacity. For example, if you're using both 1-minute and 5-minute bars, each stores up to `bar_capacity` bars. When `bar_capacity` is reached, the `Cache` automatically removes the oldest data. ::: ### Database configuration For persistence between system restarts, you can configure a database backend. `CacheConfig` controls cache behavior only. Connection settings belong to the concrete cache database technology config, such as `RedisCacheConfig` or `PostgresCacheConfig`. When is it useful to use persistence? - **Long-running systems**: If you want your data to survive system restarts, upgrading, or unexpected failures, having a database configuration helps to pick up exactly where you left off. - **Historical insights**: When you need to preserve past trading data for detailed post-analysis or audits. - **Multi-node or distributed setups**: If multiple services or nodes need to access the same state, a persistent store helps ensure shared and consistent data. Rust-native callers build a concrete database config and use the `CacheDatabaseFactory` trait to construct the adapter passed into the system builder: ```rust use nautilus_common::{ cache::{CacheConfig, database::CacheDatabaseFactory}, enums::SerializationEncoding, }; use nautilus_infrastructure::redis::cache::RedisCacheConfig; let config = CacheConfig { encoding: SerializationEncoding::MsgPack, timestamps_as_iso8601: true, buffer_interval_ms: Some(100), ..Default::default() }; let database = RedisCacheConfig { host: Some("localhost".to_string()), port: Some(6379), connection_timeout: 2, response_timeout: 2, ..Default::default() }; let cache_database = database .create(trader_id, instance_id, config.clone()) .await?; ``` ## Using the cache ### Accessing market data The `Cache` provides a full interface for accessing order books, quotes, trades, and bars. All market data in the cache uses reverse indexing, so the most recent entry sits at index 0. #### Bar access ```python # Get a list of all cached bars for a bar type bars = self.cache.bars(bar_type) # Returns list[Bar] or an empty list if no bars found # Get the most recent bar latest_bar = self.cache.bar(bar_type) # Returns Bar or None if no such object exists # Get a specific historical bar by index (0 = most recent) second_last_bar = self.cache.bar(bar_type, index=1) # Returns Bar or None if no such object exists # Check if bars exist and get count bar_count = self.cache.bar_count(bar_type) # Returns number of bars in cache for the specified bar type has_bars = self.cache.has_bars(bar_type) # Returns bool indicating if any bars exist for the specified bar type ``` #### Quote ticks ```python # Get quotes quotes = self.cache.quote_ticks(instrument_id) # Returns list[QuoteTick] or an empty list if no quotes found latest_quote = self.cache.quote_tick(instrument_id) # Returns QuoteTick or None if no such object exists second_last_quote = self.cache.quote_tick(instrument_id, index=1) # Returns QuoteTick or None if no such object exists # Check quote availability quote_count = self.cache.quote_tick_count(instrument_id) # Returns the number of quotes in cache for this instrument has_quotes = self.cache.has_quote_ticks(instrument_id) # Returns bool indicating if any quotes exist for this instrument ``` #### Trade ticks ```python # Get trades trades = self.cache.trade_ticks(instrument_id) # Returns list[TradeTick] or an empty list if no trades found latest_trade = self.cache.trade_tick(instrument_id) # Returns TradeTick or None if no such object exists second_last_trade = self.cache.trade_tick(instrument_id, index=1) # Returns TradeTick or None if no such object exists # Check trade availability trade_count = self.cache.trade_tick_count(instrument_id) # Returns the number of trades in cache for this instrument has_trades = self.cache.has_trade_ticks(instrument_id) # Returns bool indicating if any trades exist ``` #### Order book ```python # Get current order book book = self.cache.order_book(instrument_id) # Returns OrderBook or None if no such object exists # Check if order book exists has_book = self.cache.has_order_book(instrument_id) # Returns bool indicating if an order book exists # Get count of order book updates update_count = self.cache.book_update_count(instrument_id) # Returns the number of updates received ``` #### Price access ```python from nautilus_trader.core.rust.model import PriceType # Get current price by type; Returns Price or None. price = self.cache.price( instrument_id=instrument_id, price_type=PriceType.MID, # Options: BID, ASK, MID, LAST ) ``` #### Bar types ```python from nautilus_trader.core.rust.model import PriceType, AggregationSource # Get all available bar types for an instrument; Returns list[BarType]. bar_types = self.cache.bar_types( instrument_id=instrument_id, price_type=PriceType.LAST, # Options: BID, ASK, MID, LAST aggregation_source=AggregationSource.EXTERNAL, ) ``` #### Simple example ```python class MarketDataStrategy(Strategy): def on_start(self): # Subscribe to 1-minute bars self.bar_type = BarType.from_str(f"{self.instrument_id}-1-MINUTE-LAST-EXTERNAL") # example of instrument_id = "EUR/USD.FXCM" self.subscribe_bars(self.bar_type) def on_bar(self, bar: Bar) -> None: bars = self.cache.bars(self.bar_type)[:3] if len(bars) < 3: # Wait until we have at least 3 bars return # Access last 3 bars for analysis current_bar = bars[0] # Most recent bar prev_bar = bars[1] # Second to last bar prev_prev_bar = bars[2] # Third to last bar # Get latest quote and trade latest_quote = self.cache.quote_tick(self.instrument_id) latest_trade = self.cache.trade_tick(self.instrument_id) if latest_quote is not None: current_spread = latest_quote.ask_price - latest_quote.bid_price self.log.info(f"Current spread: {current_spread}") ``` ### Trading objects The `Cache` provides access to all trading objects within the system, including: - Orders - Positions - Accounts - Instruments #### Orders You can access and query orders through multiple methods, with flexible filtering options by venue, strategy, instrument, and order side. ##### Basic order access ```python # Get a specific order by its client order ID order = self.cache.order(ClientOrderId("O-123")) # Get all orders in the system orders = self.cache.orders() # Get orders filtered by specific criteria orders_for_venue = self.cache.orders(venue=venue) # All orders for a specific venue orders_for_strategy = self.cache.orders(strategy_id=strategy_id) # All orders for a specific strategy orders_for_instrument = self.cache.orders(instrument_id=instrument_id) # All orders for an instrument ``` ##### Order state queries ```python # Get orders by their current state open_orders = self.cache.orders_open() # Orders currently active at the venue closed_orders = self.cache.orders_closed() # Orders that have completed their lifecycle emulated_orders = self.cache.orders_emulated() # Orders being simulated locally by the system inflight_orders = self.cache.orders_inflight() # Orders submitted (or modified) to venue, but not yet confirmed local_active_orders = self.cache.orders_active_local() # Orders still managed locally (initialized, emulated, or released) # Check specific order states exists = self.cache.order_exists(client_order_id) # Checks if an order with the given ID exists in the cache is_open = self.cache.is_order_open(client_order_id) # Checks if an order is currently open is_closed = self.cache.is_order_closed(client_order_id) # Checks if an order is closed is_emulated = self.cache.is_order_emulated(client_order_id) # Checks if an order is being simulated locally is_inflight = self.cache.is_order_inflight(client_order_id) # Checks if an order is submitted or modified, but not yet confirmed is_active_local = self.cache.is_order_active_local(client_order_id) # Checks if an order is still managed locally ``` ##### Order statistics ```python # Get counts of orders in different states open_count = self.cache.orders_open_count() # Number of open orders closed_count = self.cache.orders_closed_count() # Number of closed orders emulated_count = self.cache.orders_emulated_count() # Number of emulated orders inflight_count = self.cache.orders_inflight_count() # Number of inflight orders local_active_count = self.cache.orders_active_local_count() # Number of locally active orders (initialized, emulated, or released) total_count = self.cache.orders_total_count() # Total number of orders in the system # Get filtered order counts buy_orders_count = self.cache.orders_open_count(side=OrderSide.BUY) # Number of currently open BUY orders venue_orders_count = self.cache.orders_total_count(venue=venue) # Total number of orders for a given venue ``` #### Positions The `Cache` maintains a record of all positions and offers several ways to query them. ##### Position access ```python # Get a specific position by its ID position = self.cache.position(PositionId("P-123")) # Get positions by their state all_positions = self.cache.positions() # All positions in the system open_positions = self.cache.positions_open() # All currently open positions closed_positions = self.cache.positions_closed() # All closed positions # Get positions filtered by various criteria venue_positions = self.cache.positions(venue=venue) # Positions for a specific venue instrument_positions = self.cache.positions(instrument_id=instrument_id) # Positions for a specific instrument strategy_positions = self.cache.positions(strategy_id=strategy_id) # Positions for a specific strategy long_positions = self.cache.positions(side=PositionSide.LONG) # All long positions ``` ##### Position state queries ```python # Check position states exists = self.cache.position_exists(position_id) # Checks if a position with the given ID exists is_open = self.cache.is_position_open(position_id) # Checks if a position is open is_closed = self.cache.is_position_closed(position_id) # Checks if a position is closed # Get position and order relationships orders = self.cache.orders_for_position(position_id) # All orders related to a specific position position = self.cache.position_for_order(client_order_id) # Find the position associated with a specific order ``` ##### Position statistics ```python # Get position counts in different states open_count = self.cache.positions_open_count() # Number of currently open positions closed_count = self.cache.positions_closed_count() # Number of closed positions total_count = self.cache.positions_total_count() # Total number of positions in the system # Get filtered position counts long_positions_count = self.cache.positions_open_count(side=PositionSide.LONG) # Number of open long positions instrument_positions_count = self.cache.positions_total_count(instrument_id=instrument_id) # Number of positions for a given instrument ``` #### Accounts ```python # Access account information account = self.cache.account(account_id) # Retrieve account by ID account = self.cache.account_for_venue(venue) # Retrieve account for a specific venue account_id = self.cache.account_id(venue) # Retrieve account ID for a venue ``` #### Instruments and currencies ##### Instruments ```python # Get instrument information instrument = self.cache.instrument(instrument_id) # Retrieve a specific instrument by its ID all_instruments = self.cache.instruments() # Retrieve all instruments in the cache # Get filtered instruments venue_instruments = self.cache.instruments(venue=venue) # Instruments for a specific venue instruments_by_underlying = self.cache.instruments(underlying="ES") # Instruments by underlying # Get instrument identifiers instrument_ids = self.cache.instrument_ids() # Get all instrument IDs venue_instrument_ids = self.cache.instrument_ids(venue=venue) # Get instrument IDs for a specific venue ``` ### Purging cached data Long-running sessions accumulate closed orders, closed positions, account events, and unused instruments. The cache exposes targeted and bulk purge methods so strategies and the live trading engine can keep memory bounded without restarting the system. #### Targeted purges Use these to drop a single entity. Each refuses to purge while the entity is still active. - `cache.purge_order(client_order_id)`: removes the order and every order-keyed index entry. Skips open orders. - `cache.purge_position(position_id)`: removes the position, its snapshots, and position-keyed index entries. Skips open positions. - `cache.purge_instrument(instrument_id)`: removes the instrument and every per-instrument map (order book, quotes, trades, mark/index/funding prices, instrument status, greeks, and bars referencing the instrument). Skips while any associated order is non-terminal (anything that has not reached a closed state, including initialized, submitted, accepted, emulated, released, and inflight orders) or any associated position is non-closed. ```python class HousekeepingStrategy(Strategy): def on_start(self) -> None: # Drop instruments that are no longer in the watchlist. for instrument_id in self.cache.instrument_ids(venue=self.venue): if instrument_id not in self.watchlist: self.cache.purge_instrument(instrument_id) ``` :::warning `purge_instrument` is intended for actors and strategies with their own lifecycle logic for deciding when an instrument is no longer needed. Purging an instrument that another component still relies on causes missing instrument lookups and loses market-data history. Active subscriptions belong to the data engine, so unsubscribe before purging if you no longer want updates. ::: #### Bulk purges Use these to sweep older entries by age. They take the current timestamp and a buffer or lookback window in seconds. - `cache.purge_closed_orders(ts_now, buffer_secs)`: closed orders whose close timestamp is older than `buffer_secs`. - `cache.purge_closed_positions(ts_now, buffer_secs)`: closed positions whose close timestamp is older than `buffer_secs`. - `cache.purge_account_events(ts_now, lookback_secs)`: account state events older than `lookback_secs`. A value of `0` purges all events. #### Automatic purging in live trading `LiveExecEngineConfig` schedules the bulk purges on a timer. Set the interval to enable the loop and the buffer or lookback to control how recent entries are protected. The following defaults work well for most live sessions: ```python from nautilus_trader.config import LiveExecEngineConfig exec_engine = LiveExecEngineConfig( purge_closed_orders_interval_mins=15, purge_closed_orders_buffer_mins=60, purge_closed_positions_interval_mins=15, purge_closed_positions_buffer_mins=60, purge_account_events_interval_mins=15, purge_account_events_lookback_mins=60, ) ``` A 60-minute buffer keeps recent activity available for reconciliation while still trimming long-tail growth. Tune these down for HFT sessions and up if you need longer historical lookbacks for analytics. See [Configure live trading: memory management](../how_to/configure_live_trading.md) for the full parameter reference. :::note The instrument purge has no automatic loop because the right time to drop an instrument depends on strategy state, not age. Call `cache.purge_instrument` from the actor or strategy that owns the instrument's lifecycle. ::: --- ### Custom data The `Cache` can also store and retrieve custom data types in addition to built-in market data and trading objects. Use it to share any user-defined data between system components, primarily actors and strategies. #### Basic storage and retrieval ```python # Call this code inside Strategy methods (`self` refers to Strategy) # Store data self.cache.add(key="my_key", value=b"some binary data") # Retrieve data stored_data = self.cache.get("my_key") # Returns bytes or None ``` For more complex use cases, the `Cache` can store custom data objects that inherit from the `nautilus_trader.core.Data` base class. :::warning The `Cache` is not designed to be a full database replacement. For large datasets or complex querying needs, consider using a dedicated database system. ::: ## Best practices and common questions ### Cache vs. portfolio usage The `Cache` and `Portfolio` components serve different but complementary purposes in NautilusTrader: **Cache**: - Maintains the historical knowledge and current state of the trading system. - Updates immediately when local state changes (for example, initializing an order before submission). - Updates asynchronously as external events occur (for example, when an order fills). - Provides a complete history of trading activity and market data. - Keeps every event the strategy receives in the cache. **Portfolio**: - Aggregates position, exposure, and account information. - Provides current state without history. **Example**: ```python class MyStrategy(Strategy): def on_position_changed(self, event: PositionEvent) -> None: # Use Cache when you need historical perspective position_history = self.cache.position_snapshots(event.position_id) # Use Portfolio when you need current real-time state current_exposure = self.portfolio.net_exposure(event.instrument_id) ``` ### Cache vs. strategy variables Choosing between storing data in the `Cache` versus strategy variables depends on your specific needs: **Cache storage**: - Use for data that needs to be shared between strategies. - Best for data that needs to persist between system restarts. - Acts as a central database accessible to all components. - Ideal for state that needs to survive strategy resets. **Strategy variables**: - Use for strategy-specific calculations. - Better for temporary values and intermediate results. - Provides faster access and better encapsulation. - Best for data that only your strategy needs. **Example**: The following example shows how you might store data in the `Cache` so multiple strategies can access the same information. ```python import pickle class MyStrategy(Strategy): def on_start(self): # Prepare data you want to share with other strategies shared_data = { "last_reset": self.clock.timestamp_ns(), "trading_enabled": True, # Include any other fields that you want other strategies to read } # Store it in the cache with a descriptive key # This way, multiple strategies can call self.cache.get("shared_strategy_info") # to retrieve the same data self.cache.add("shared_strategy_info", pickle.dumps(shared_data)) ``` Another strategy can retrieve the cached data as follows: ```python import pickle class AnotherStrategy(Strategy): def on_start(self): # Load the shared data from the same key data_bytes = self.cache.get("shared_strategy_info") if data_bytes is not None: shared_data = pickle.loads(data_bytes) self.log.info(f"Shared data retrieved: {shared_data}") ``` ## Related guides - [Data](data.md) - Data types stored in the cache. - [Strategies](strategies.md) - Strategies access cache for market data and state. - [Reports](reports.md) - Generate reports from cached data. # Configuration Source: https://nautilustrader.io/docs/latest/concepts/configuration/ NautilusTrader uses typed configuration structs throughout the platform. Each component (data clients, execution clients, engines, strategies) has a dedicated config struct that controls its behavior. ## Design principles ### Defaults resolve at the config boundary Config structs carry concrete values for fields that always have a sensible default. Timeouts, retry counts, backoff delays, and heartbeat intervals are plain types like `u64` or `u32` with defaults baked in. Downstream code receives resolved values and does not repeat defaulting logic. ### Option means semantic absence, not "use default" `Option` fields appear only when `None` carries real meaning: a feature is off, a lookback window is unbounded, or a value is inherited from the environment at runtime. If a field always resolves to a concrete value, it is not wrapped in `Option`. This distinction makes config semantics visible in the type. A plain `u64` field always has a value. An `Option` field might be absent, and the code that consumes it will branch on that. ### Single source of truth for defaults Each config struct uses `bon::Builder` to define defaults in one place via `#[builder(default = value)]` annotations. The `Default` impl delegates to the builder (`Self::builder().build()`), so there is no second copy of default values that could drift out of sync. ### Config decoding fails on unknown fields Config decoding fails fast on unknown fields. Nautilus treats extra keys as bugs, not as harmless input. This catches misspellings, stale names after config renames, and copy-paste mistakes before a node or client starts with the wrong settings. ## Python configs Python config classes (msgspec structs) accept `None` for optional parameters. For plain `T` fields, `None` means "use the default." For `Option` fields, `None` preserves the field's optional meaning (disabled, unbounded, etc.). All Python config classes inherit from `NautilusConfig`, which sets `forbid_unknown_fields=True` on the underlying `msgspec.Struct`. Unknown keys now raise `msgspec.ValidationError` during decoding. ```python from nautilus_trader.adapters.bybit.config import BybitDataClientConfig # All defaults: 60s timeout, 3 retries, etc. config = BybitDataClientConfig() # Override just the timeout config = BybitDataClientConfig(http_timeout_secs=30) # Disable instrument status polling config = BybitDataClientConfig(instrument_status_poll_secs=None) ``` ## Rust configs All config structs derive [`bon::Builder`](https://bon-rs.com), which generates a type-safe builder with compile-time checks for required fields. Fields with `#[builder(default = value)]` can be omitted from the builder call and will use their declared default. Three equivalent ways to construct a config: Rust config structs that deserialize with Serde also set `#[serde(deny_unknown_fields)]`. Unknown keys now fail deserialization instead of being ignored. ```rust // Builder: only set what differs from defaults let config = BybitDataClientConfig::builder() .http_timeout_secs(30) .build(); // Struct literal with default spread let config = BybitDataClientConfig { http_timeout_secs: 30, ..Default::default() }; // Full defaults let config = BybitDataClientConfig::default(); ``` All three produce identical results for unspecified fields. ## Common config fields Most adapter configs share a common set of fields: | Field | Type | Default | Purpose | |------------------------------------|--------|---------|-------------------------------| | `http_timeout_secs` | `u64` | 60 | REST request timeout. | | `max_retries` | `u32` | 3 | Maximum retry attempts. | | `retry_delay_initial_ms` | `u64` | 1,000 | Initial backoff delay. | | `retry_delay_max_ms` | `u64` | 10,000 | Maximum backoff delay. | | `heartbeat_interval_secs` | `u64` | varies | WebSocket keepalive interval. | | `recv_window_ms` | `u64` | varies | Signed request expiry window. | | `update_instruments_interval_mins` | varies | varies | Periodic instrument refresh. | Adapter-specific fields (rate limits, polling intervals, margin modes) are documented in each adapter's integration guide. ## Engine configs Engine configs (`LiveExecEngineConfig`, `DataEngineConfig`, etc.) follow the same pattern. Fields like `reconciliation`, `inflight_check_interval_ms`, and `open_check_threshold_ms` are plain types with builder defaults. Genuinely optional features use `Option`: ```python from nautilus_trader.config import LiveExecEngineConfig config = LiveExecEngineConfig( reconciliation=True, open_check_interval_secs=30.0, # Enable open order polling open_check_lookback_mins=60, # Look back 60 minutes # position_check_interval_secs=None # Disabled by default ) ``` # Continuous Futures Source: https://nautilustrader.io/docs/latest/concepts/continuous_futures/ A continuous future is a derived series that splices consecutive futures contracts into one adjusted price stream. Each underlying contract expires; the continuous series stays active by rolling to the next contract at a transition point and shifting historical prices into the new contract's frame so the resulting series has no roll-induced jumps. Nautilus models a continuous future as a target `BarType` plus an explicit list of roll transitions supplied in request or subscribe params. The data engine walks per-contract segments, computes the cumulative price adjustment per segment, and feeds adjusted source data through the normal bar aggregation path. ## Adjustment modes `ContinuousFutureAdjustmentType` combines direction (backward or forward) with operation (spread or ratio): | Mode | Operation | Anchor segment | |-------------------|-----------------|----------------------| | `BACKWARD_SPREAD` | Additive | Most recent contract | | `FORWARD_SPREAD` | Additive | First contract | | `BACKWARD_RATIO` | Multiplicative | Most recent contract | | `FORWARD_RATIO` | Multiplicative | First contract | The cumulative adjustment at segment `k` of `N` transitions is: ```text BACKWARD_SPREAD: sum over i in [k, N) of (post_i - pre_i) FORWARD_SPREAD: sum over i in [0, k) of (pre_i - post_i) BACKWARD_RATIO: product over i in [k, N) of (post_i / pre_i) FORWARD_RATIO: product over i in [0, k) of (pre_i / post_i) ``` Spread modes accumulate additive offsets. Ratio modes accumulate multiplicative factors and require strictly positive prices. ## Inputs A continuous-future request or subscription is any `RequestBars` or `SubscribeBars` that carries a `continuous_future_transitions` entry in `params`: ```python params = { "continuous_future_transitions": [ { "transition_time_ns": 1773671460000000000, # when ESH26 rolls to ESM26 "pre_instrument_id": "ESH26.XCME", "post_instrument_id": "ESM26.XCME", "pre_price": "6001.00", # last ESH26 price pre-roll "post_price": "5995.50", # first ESM26 price post-roll }, # ... more transitions ... ], "continuous_future_adjustment_mode": ContinuousFutureAdjustmentType.BACKWARD_SPREAD, # Optional: cap the upper end of cumulative adjustment at the transition whose # post_instrument_id matches (the backward-mode anchor). # "last_post_instrument_id": "ESM26.XCME", # Optional: cap the lower end of cumulative adjustment at the transition whose # pre_instrument_id matches (the forward-mode anchor). # "first_pre_instrument_id": "ESM26.XCME", } ``` The `bar_type` on the request or command is the **target** continuous bar type, for example `"ES.XCME-1-MINUTE-LAST-INTERNAL@1-MINUTE-EXTERNAL"`. The root identifier (`ES.XCME`) is the continuous root, not a real contract. Each segment's raw source data comes from the real contract in the transitions list. The continuous target bar type must be **internally aggregated**. Externally aggregated bars are not supported as continuous targets, but they can serve as the per-segment source. ### Bounded chains The two optional bounds restrict the active portion of the transition table: - `last_post_instrument_id` caps the upper end at the first transition whose `post_instrument_id` matches. Backward modes use this as the anchor (cumulative adjustment for the anchor segment is zero); forward modes use it to cap how far later contracts accumulate. - `first_pre_instrument_id` caps the lower end at the first transition whose `pre_instrument_id` matches. Forward modes use this as the anchor; backward modes use it to cap how far earlier contracts accumulate. This lets callers pass a wide transition table while anchoring the adjusted series to a specific contract on either side. ## Validation The Rust request path (`crates/data/src/engine/requests.rs`) and the Cython request and subscription paths (`engine.pyx::_continuous_future_validate_transitions`) validate transition params before allocating any aggregator: - `continuous_future_adjustment_mode` must parse as a valid `ContinuousFutureAdjustmentType`. - `continuous_future_transitions` must be a list or tuple of dict rows. - Each row must include a non-negative integer `transition_time_ns`, and transition times must be strictly increasing. - Each `pre_instrument_id` and `post_instrument_id` must parse as a valid `InstrumentId` whose venue equals the target venue. - The chain must be continuous: row `i`'s `post_instrument_id` must equal row `i + 1`'s `pre_instrument_id`. - Each row must include finite `pre_price` and `post_price`. Ratio modes additionally require both prices to be positive. - If the caller supplies `last_post_instrument_id`, it must parse as an `InstrumentId`, match the target venue, and appear as a `post_instrument_id` in the transition list. The same applies to `first_pre_instrument_id`. On validation failure the Rust request returns an error before it allocates child segment state. The Cython request handler calls `_abort_request` to drop workflow state it had begun to set up; the Cython subscription path logs a specific error and returns. ## Target instrument auto-synthesis The continuous root (for example `ES.XCME`) is a synthetic id with no market data of its own, but downstream consumers (aggregators, cache lookups, serialization) still expect an `Instrument` in the cache. After validation, the Rust request path and the Cython request and subscription paths ensure the target instrument exists: - If the target id is already cached, the target setup is a no-op. Callers can pre-register a custom continuous instrument and the engine respects it. - Otherwise the target setup fetches the first segment's instrument from the cache and clones it, overriding only `id`, `raw_symbol`, and clearing `activation_ns` and `expiration_ns` to `0`. Every other field (currency, precision, increment, multiplier, lot size, underlying, fees, margins, exchange, tick scheme, info) is reused from the segment. - If the first segment is not yet in the cache or is not a `FuturesContract`, the setup logs a warning and returns. The caller must then register the continuous instrument manually. ## Architecture overview ```mermaid flowchart TD User([User/Strategy]) -->|"params['continuous_future_transitions']"| Entry{"Entry point"} Entry -->|RequestBars| ReqPath[Request path] Entry -->|SubscribeBars| SubPath[Subscription path] ReqPath --> OuterReq[Outer loop: segments] SubPath --> OuterSub[Outer loop: segments + time alerts] OuterReq -->|per segment| SubReq[Inner request for segment contract] OuterSub -->|per segment| LiveSub[Inner subscribe for segment contract] SubReq --> Agg[(Primary aggregator
BarBuilder.set_adjustment)] LiveSub --> Agg2[(Live aggregator
BarBuilder.set_adjustment)] Agg -->|Rust request path| ReqAgg[(Request-scoped aggregator chain)] Agg -->|Cython request path| Chain[Cython chain aggregators] Agg2 -->|adjusted bars| MsgBus[(msgbus: data.bars.*)] Chain -->|final bars| PipelineBus[(Cython msgbus: data.pipeline.bars.*)] ``` The design has two entry points, one outer loop shape (walk the segments), two ways to fetch per-segment data (historical sub-requests or live sub-subscriptions), and one adjustment mechanism (`BarBuilder.set_adjustment` at each segment boundary). ## Segments A **segment** is a contiguous time slice owned by one real contract. Transitions separate segments. Given `transitions[0..N)`: - Segment 0: `(-inf, transitions[0].time)` on `transitions[0].pre_instrument_id`. - Segment k, with k in `[1, N)`: `[transitions[k-1].time, transitions[k].time)` on `transitions[k].pre_instrument_id`. - Segment N: `[transitions[N-1].time, +inf)` on `transitions[N-1].post_instrument_id`. The request and subscription paths return the next segment starting at `cursor_ns`, clamped to `end_ns`. ## Request flow The request path mirrors `_handle_long_request` one level up: each iteration fires one inner request for one segment's worth of data, and the inner's completion callback advances the cursor. ```mermaid sequenceDiagram participant User participant Engine as DataEngine participant Agg as Primary aggregator participant Client as DataClient User->>Engine: request(RequestBars w/ transitions) Engine->>Agg: init aggregators, set cursor loop one iteration per segment Engine->>Agg: BarBuilder.set_adjustment(offset, mode) Engine->>Client: inner Request_ for segment contract Client-->>Engine: DataResponse Engine->>Agg: route child response through request-scoped aggregation Engine->>Engine: advance cursor end Engine->>User: terminal parent response ``` If the caller sets `time_range_generator` and `durations_seconds` in params, the inner request inherits them and itself becomes a long request that chunks the segment's time range into N further sub-sub-requests. The outer continuous-future loop ignores the inner chunking: each inner request still emits exactly one combined response back, which triggers the outer loop's next segment. ### Chain aggregators If the caller sets `bar_types = (bar_type_1, bar_type_2)` for multi-level internal aggregation, the setup creates all aggregators keyed by `parent.id`. The Rust request path routes segment source responses into the primary continuous target, then forwards emitted bars to matching request-scoped downstream aggregators. The Cython path wires pipeline topics between levels so the chain walks up automatically. Only the primary builder has `set_adjustment` called on it; higher levels re-aggregate already adjusted data in both paths. ## Subscription flow A small state machine drives each active subscription via a single pending time alert: ```mermaid stateDiagram-v2 [*] --> Active: subscribe(segment_i active, timer for transition_i) Active --> Active: roll(deactivate segment_i, activate segment_{i+1}, schedule next timer) Active --> [*]: unsubscribe(cancel timer, deactivate segment) ``` When a transition fires, the engine deactivates the current segment (unsubscribes the source), activates the next segment (resolves the new source, applies the new offset, subscribes), and re-arms the timer for the next transition. ## Source resolution For any continuous-future target `BarType`, the raw data feeding the primary aggregator lives on the **segment contract**, not the continuous id. The target's shape decides the source type: ```mermaid flowchart TD Target[target_bar_type] --> Check1{is_composite?} Check1 -->|yes| Ref[reference = target.composite] Check1 -->|no| RefNo[reference = target] Ref --> Check2{externally_aggregated?} RefNo --> Check2 Check2 -->|yes| Bars["source = bars (RequestBars / SubscribeBars)"] Check2 -->|no| Check3{price_type} Check3 -->|LAST| Trades["source = trades (TradeTicks)"] Check3 -->|MID/BID/ASK| Quotes["source = quotes (QuoteTicks)"] ``` ## BarBuilder adjustment The builder applies adjustment **at ingress** on every `update(price, ...)` and `update_bar(bar, ...)` call. The running OHLC state always sits in the adjusted (common) frame, so a mid-bar adjustment change only affects subsequent prices. No partial-bar buffering is needed. ```mermaid flowchart LR Tick[raw price] --> AdjCheck{adjustment_mode} AdjCheck -->|inactive| Raw[pass through] AdjCheck -->|spread| SpreadApply[price + adjustment_raw] AdjCheck -->|ratio| RatioApply[price * adjustment_ratio] Raw --> Update[update OHLC state] SpreadApply --> Update RatioApply --> Update Update --> Build[build on trigger] ``` The `BarBuilder` only cares about the ratio-vs-spread distinction to decide add-or-multiply. The engine collapses direction information into the sign and magnitude of the cumulative offset before calling `set_adjustment`. The `reset()` method clears per-bar OHLCV state for the next bar in the series but intentionally preserves the adjustment configuration: rolls happen far less often than bar resets, so the adjustment is treated as segment-scoped state. ## Mid-bar roll boundary If a roll lands inside an in-progress target bar, the builder keeps the current OHLC state and applies the new adjustment only to subsequent updates. The pre-boundary portion stays at the old offset; the post-boundary portion uses the new offset. This is the intentional policy: rewriting the running OHLC at every roll would require buffering raw input per segment, which adds cost without changing the adjusted result for the common case where the adjusted segment is built seamlessly across the boundary. ## Limitations - The feature requires supplied transition metadata. The engine does not discover rolls, choose contracts, or infer roll prices: that is the caller's responsibility. - Ratio adjustment goes through `float` in the hot path (`price_as_f64 * ratio` then `price_new`). For high-precision instruments the rounding can shift the resulting raw by 1 ULP versus the equivalent `Decimal` multiplication. Spread mode is exact because it works directly on `PriceRaw` (int64/int128). # Custom Data Source: https://nautilustrader.io/docs/latest/concepts/custom_data/ Nautilus Trader supports custom data authored in Python and Rust, and moves that data through the same runtime, persistence, and query pipeline used by the rest of the platform. This document explains how custom data is: - Registered at runtime. - Wrapped across the Python/Rust boundary. - Serialized to and from Arrow/Parquet. - Routed through actors and strategies. ## Goals The custom-data architecture satisfies the following requirements: - Let users define custom data in pure Python without writing Rust code. - Let Rust-defined custom data use native Rust JSON and Arrow handlers. - Preserve a single user-facing `CustomData` wrapper at the PyO3 boundary. - Support persistence in `ParquetDataCatalog` using dynamic type registration instead of hardcoded schemas. - Make custom data routable through the normal data-engine, actor, and strategy subscription flow. ## High-level model There are two supported authoring modes: | Mode | Example | Registration path | Encode/decode path | Wrapper backend | |-------------------|----------------------------------------------------|---------------------------------------------------------|---------------------------------|---------------------------| | Pure Python | `@customdataclass_pyo3` class | `register_custom_data_class(...)` | Python callback + Arrow C FFI | `PythonCustomDataWrapper` | | Same‑binary Rust | `#[custom_data]` or `#[custom_data(pyo3)]` type | `ensure_custom_data_registered::()` and native extractor | Native Rust | Native Rust payload | Both modes converge on the same outer PyO3 `CustomData` wrapper and the same `DataType` identity model. ## End-to-end flow ```mermaid sequenceDiagram participant U as User code participant P as Python layer participant R as Rust model/catalog participant G as Global DataRegistry participant S as Storage U->>P: define class/type U->>P: register_custom_data_class(...) or module init P->>R: install type registration R->>G: store JSON/Arrow/extractor handlers U->>P: CustomData(data_type, data) P->>R: write_custom_data([...]) R->>G: lookup encoder by type_name G-->>R: encoder R->>S: write RecordBatch to Parquet U->>P: query(type_name, ...) P->>R: query catalog R->>S: read RecordBatch + metadata R->>G: lookup decoder by type_name G-->>R: decoder R-->>P: CustomData wrappers P-->>U: typed data via .data ``` ## Core components ### `DataRegistry` `crates/model/src/data/registry.rs` is the central runtime registry module for custom data in the main process. Registration uses atomic `DashMap::entry()` so that concurrent `register_*` and `ensure_*` calls do not race. The module contains several `OnceLock`-initialized `DashMap` singletons: - JSON deserializers keyed by `type_name`. - Arrow schemas, encoders, and decoders keyed by `type_name`. - Python extractors that convert a Python object into `Arc`. - Rust extractor factories that produce Python extractors for same-binary types. Instead of hardcoding every type into the main binary, Nautilus resolves handlers at runtime using the `type_name` stored in `DataType` and Parquet metadata. ### `CustomData` The outer PyO3 `CustomData` wrapper is the common container that crosses the FFI boundary. Constructor signature: `CustomData(data_type, data)` where `DataType` comes first, then the inner payload. It contains: - A `DataType`. - An inner custom payload implementing `CustomDataTrait` (wrapped in `Arc`). Timestamps (`ts_event`, `ts_init`) are delegated to the inner `CustomDataTrait` implementation and exposed as properties on the wrapper. On the Python side, `CustomData` exposes value semantics: `__eq__` and `__repr__` are implemented (equality uses the Rust `PartialEq` logic). Instances are intentionally unhashable so that equality remains consistent with the inner payload comparison. This wrapper is shared across both custom-data modes. User code interacts with one API even though the underlying payload may be: - A Python-backed wrapper. - A same-binary Rust value. #### `CustomData` JSON envelope When serialized to JSON (e.g. for `to_json_bytes` / `from_json_bytes`, SQL cache, or Redis), `CustomData` uses a single canonical envelope so that deserialization does not depend on user payload field names: - `type`: The custom type name (from `CustomDataTrait::type_name`). - `data_type`: An object with `type_name`, `metadata`, and optional `identifier`. - `payload`: The inner payload only (the result of `CustomDataTrait::to_json` parsed as a value). Registered deserializers receive only this value in `from_json`, so user structs can use any field names (including `value`) without conflicting with wrapper metadata. This envelope is produced by Rust `CustomData` serialization and consumed by `DataRegistry` when deserializing custom data from JSON. ### `DataType` `DataType` identifies custom data for routing and persistence. Constructor: `DataType(type_name, metadata=None, identifier=None)`. It includes: - `type_name`. - Optional `metadata`. - Optional `identifier` (used only for catalog pathing, not for routing or equality). Equality, hashing, and topic routing are derived from `type_name` and `metadata` only. Two `DataType` values with the same type name and metadata but different identifiers compare equal and publish to the same message bus topic. The `identifier` affects only the storage path under `data/custom//`. Custom-data storage and queries use `DataType`, not just the bare Rust/Python class name. This allows the same logical type to be stored under different metadata or identifiers while still decoding through the same registered handler. ## Registration architecture Registration bridges the gap between Python objects and Rust trait objects. ```mermaid flowchart TD A[User-defined custom type] --> B{Mode} B --> C[Pure Python] B --> D[Same-binary Rust] C --> F[register_custom_data_class] D --> G[ensure_custom_data_registered and native extractor] F --> I[Python callbacks registered] G --> J[Native JSON and Arrow handlers registered] I --> L[Main-process DataRegistry] J --> L ``` ### Pure Python registration When Python code calls `register_custom_data_class(MyType)`: 1. The type is registered in the Python serialization layer for JSON and Arrow support. 2. Rust registers a Python extractor that wraps Python instances as `PythonCustomDataWrapper`. 3. Rust registers Arrow schema/encode/decode callbacks in `DataRegistry`. This path is flexible and user-friendly, but Arrow encoding and reconstruction rely on Python callbacks. ### Same-binary Rust registration For Rust types defined inside Nautilus: 1. `#[custom_data]` or `#[custom_data(pyo3)]` generates the necessary trait, JSON, and Arrow implementations. 2. `ensure_custom_data_registered::()` inserts native schema/encoder/decoder handlers into `DataRegistry`. 3. For PyO3-exposed types, a native extractor can convert Python instances back into the concrete Rust type rather than a Python fallback wrapper. This path stays fully native in Rust for encode/decode. ### Registration precedence `register_custom_data_class(...)` resolves types in the following order: 1. Same-binary native Rust registration. 2. Pure Python fallback registration. That ordering preserves the fastest available path for types already known natively by the main binary. ## Wrapper backends Internally, the outer `CustomData` wrapper can hold different payload implementations. ### `PythonCustomDataWrapper` Used for pure Python custom data. Responsibilities: - Stores a reference to the Python object. - Caches `ts_event`, `ts_init`, and `type_name`. - Implements `CustomDataTrait`. - Calls Python methods for JSON and Arrow-related operations under the GIL. This is the fallback path when the main process does not have a native Rust representation for the type. ### Native same-binary Rust payload For Rust types compiled into Nautilus, the inner payload is the concrete Rust type itself and can be downcast directly from `Arc`. No Python callback path is needed for serialization or decode. ## Persistence architecture ### Why dynamic Arrow registration is needed Built-in Nautilus data types have schemas and encoders known statically to the Rust binary. Custom data does not. The persistence layer therefore resolves custom data dynamically using the registered `type_name`. ### Catalog write flow `ParquetDataCatalog` expects custom writes to come in as `CustomData` values. The custom-data write path: 1. Extracts `type_name`, `metadata`, and `identifier` from `DataType`. 2. Looks up the Arrow encoder in `DataRegistry`. 3. Encodes the values to a `RecordBatch`. 4. Appends a `data_type` column containing the persisted `DataType`. 5. Attaches `type_name` and metadata to the Arrow schema. 6. Writes the batch to Parquet under the custom-data path. The path layout is: - `data/custom//` Identifiers are normalized before becoming path segments. ### Catalog read flow On query: 1. The catalog reads matching Parquet files. 2. Extracts `type_name` from schema metadata. 3. Asks `DataRegistry` for the registered decoder. 4. Decodes the `RecordBatch` into `Vec`. 5. Reconstructs `CustomData` with the original `DataType`. This makes custom-data query resolution symmetric with write-time registration. When converting a Feather stream to Parquet (e.g. after a backtest), the custom-data branch decodes batches and writes them via `write_custom_data_batch` so that custom data written through the Feather writer is correctly converted to Parquet. ## The Arrow C FFI bridge Pure Python custom data cannot provide native Rust Arrow encode logic directly. For those types, Nautilus uses the Arrow C FFI interface to pass `RecordBatch` data between Python and Rust without serialization overhead. ```mermaid sequenceDiagram participant R as Rust encoder participant P as Python custom class participant F as Arrow C FFI structs participant C as Parquet writer R->>P: encode_record_batch_py(items) P->>P: build pyarrow.RecordBatch P-->>F: _export_to_c (FFI_ArrowArray + FFI_ArrowSchema) F-->>R: reconstruct native RecordBatch R->>C: write Parquet ``` ### Pure Python encode path For pure Python classes: 1. Rust acquires the GIL. 2. Rust calls `encode_record_batch_py(...)` on the Python class. 3. Python converts objects to a `pyarrow.RecordBatch`. 4. Python exports the batch via `_export_to_c` into Arrow C FFI structs. 5. Rust reconstructs a native `RecordBatch` from the FFI structs and writes it. ### Pure Python decode path For the reverse direction: 1. Rust converts its `RecordBatch` into Arrow C FFI structs. 2. Python imports the batch via `RecordBatch._import_from_c`. 3. Python calls `decode_record_batch_py(metadata, batch)` on the class. 4. Rust wraps the returned Python objects in `PythonCustomDataWrapper`. ### Native paths The Arrow C FFI bridge is not used for same-binary Rust custom data. Those types use native Rust encode/decode handlers registered in the main process. ## Reconstruction on query When custom data is loaded back from the catalog, reconstruction depends on the backend: - Same-binary Rust types decode directly to native Rust values. - Pure Python types reconstruct through the registered Python class using `from_dict` or `from_json`. In all cases the caller receives the same outer `CustomData` wrapper at the PyO3 API boundary. ## Runtime integration Custom data is not only a persistence feature. It also participates in Nautilus runtime routing. Relevant integrations include: - `crates/data/src/engine/mod.rs` publishes `CustomData` through the message bus. - `crates/common/src/msgbus/switchboard.rs` derives custom topics from `DataType`. - `crates/common/src/actor/*` routes custom data into actor subscriptions. - `crates/trading/src/python/strategy.rs` exposes custom data to Python strategy `on_data`. - `crates/backtest/src/engine.rs` treats `Data::Custom` as data-engine-delivered input rather than exchange-routed data. A registered custom type can be persisted, queried, subscribed to, and consumed through the same runtime interfaces as other data families. ## SQL cache and database integration The SQL cache/database layer also supports `CustomData`. Current behavior: - PostgreSQL stores custom data in the `custom` table. - The stored record includes `data_type`, `metadata`, `identifier`, and full JSON payload. - Reads reconstruct `CustomData` using `CustomData::from_json_bytes(...)`. - Python SQL bindings expose `add_custom_data` and `load_custom_data`. - Redis cache stores custom data under keys `custom::` with full `CustomData` JSON as value. - Redis `add_custom_data` and `load_custom_data` filter by `DataType` (type_name, metadata, identifier) and return results sorted by `ts_init`; this is exposed via the PyO3 `RedisCacheDatabase` API. ## Cython custom data The Cython `@customdataclass` system is separate from this architecture. This document describes the PyO3 custom-data system: - PyO3 `CustomData`. - Dynamic runtime registration. - Arrow/Parquet persistence. - Native Rust execution paths. ## Practical implications This architecture gives Nautilus two important properties: 1. Python-first extensibility for users who only want to write Python. 2. Native Rust performance for built-in or compiled custom types. The result is one conceptual custom-data system with two backends, rather than separate feature silos for Python-only and Rust-only data types. # Data Source: https://nautilustrader.io/docs/latest/concepts/data/ Common built-in data types include: - `OrderBookDelta` (L1/L2/L3): Represents the most granular order book updates. - `OrderBookDeltas` (L1/L2/L3): Batches multiple order book deltas for more efficient processing. - `OrderBookDepth10`: Aggregated order book snapshot (up to 10 levels per bid and ask side). - `QuoteTick`: Represents the best bid and ask prices along with their sizes at the top-of-book. - `TradeTick`: A single trade/match event between counterparties. - `Bar`: OHLCV (Open, High, Low, Close, Volume) bar/candle, aggregated using a specified *aggregation method*. - `MarkPriceUpdate`: The current mark price for an instrument (typically used in derivatives trading). - `IndexPriceUpdate`: The index price for an instrument (underlying price used for mark price calculations). - `FundingRateUpdate`: The funding rate for perpetual contracts (periodic payments between long and short positions). - `InstrumentStatus`: An instrument-level status event. - `InstrumentClose`: The closing price of an instrument. See the API reference for the full set of built-in data classes and wrappers. NautilusTrader operates primarily on granular order book data for the highest realism in execution simulations. Backtests can also run on any supported market data type, depending on the desired simulation fidelity. When data flows over the message bus, topic-addressable data stays under the `data` root. Live streams use `data....`; the data pipeline path uses `data.pipeline....`. See [Message Bus](message_bus.md#topic-hierarchy) for the topic hierarchy. ## Order books A high-performance order book implemented in Rust is available to maintain order book state based on provided data. `OrderBook` instances are maintained per instrument for both backtesting and live trading, with the following book types available: - `L3_MBO`: **Market by order (MBO)** or L3 data, uses every order book event at every price level, keyed by order ID. - `L2_MBP`: **Market by price (MBP)** or L2 data, aggregates order book events by price level. - `L1_MBP`: **Market by price (MBP)** or L1 data, also known as best bid and offer (BBO), captures only top-level updates. :::note Top-of-book data, such as `QuoteTick`, `TradeTick` and `Bar`, can also be used for backtesting, with markets operating on `L1_MBP` book types. ::: ### Delta flags and event boundaries Each `OrderBookDelta` carries a `flags` field using `RecordFlag` bitmask values to signal event boundaries to the `DataEngine`: - `F_LAST`: Marks the final delta in a logical event group. When `buffer_deltas` is enabled, the `DataEngine` accumulates deltas and only publishes to subscribers when it encounters `F_LAST`. Every event group **must** end with a delta that has `F_LAST` set. - `F_SNAPSHOT`: Marks deltas that belong to a snapshot (as opposed to an incremental update). Snapshot sequences begin with a `Clear` action followed by `Add` deltas reconstructing the full book state. The last delta in a snapshot has both `F_SNAPSHOT | F_LAST` set. :::warning A missing `F_LAST` on the final delta in an event group causes buffered consumers to accumulate deltas indefinitely without publishing. This applies to incremental updates and snapshots alike, including empty book snapshots where only a `Clear` delta is emitted. ::: ## Instruments All market data belongs to an instrument. The instrument definition supplies the identity, precision, price and size increments, limits, currencies, and contract semantics that make the data meaningful. See [Instruments](instruments/) for the instrument taxonomy and per-type guides. ## Bars and aggregation ### Introduction to bars A *bar* (also known as a candle, candlestick or kline) is a data structure that represents price and volume information over a specific period, including: - Opening price - Highest price - Lowest price - Closing price - Traded volume (or ticks as a volume proxy) The system generates bars using an *aggregation method* that groups data by specific criteria. ### Purpose of data aggregation Data aggregation in NautilusTrader transforms granular market data into structured bars or candles for several reasons: - To provide data for technical indicators and strategy development. - Because time-aggregated data (like minute bars) are often sufficient for many strategies. - To reduce costs compared to high-frequency L1/L2/L3 market data. ### Aggregation methods The platform implements various aggregation methods: | Name | Description | Category | |:-------------------|:---------------------------------------------------------------------------|:-------------| | `TICK` | Aggregation of a number of ticks. | Threshold | | `TICK_IMBALANCE` | Aggregation of the buy/sell imbalance of ticks. | Threshold | | `TICK_RUNS` | Aggregation of sequential buy/sell runs of ticks. | Information | | `VOLUME` | Aggregation of traded volume. | Threshold | | `VOLUME_IMBALANCE` | Aggregation of the buy/sell imbalance of traded volume. | Threshold | | `VOLUME_RUNS` | Aggregation of sequential runs of buy/sell traded volume. | Information | | `VALUE` | Aggregation of the notional value of trades (also known as "Dollar bars"). | Threshold | | `VALUE_IMBALANCE` | Aggregation of the buy/sell imbalance of trading by notional value. | Threshold | | `VALUE_RUNS` | Aggregation of sequential buy/sell runs of trading by notional value. | Information | | `RENKO` | Aggregation based on fixed price movements (brick size in ticks). | Threshold | | `MILLISECOND` | Aggregation of time intervals with millisecond granularity. | Time | | `SECOND` | Aggregation of time intervals with second granularity. | Time | | `MINUTE` | Aggregation of time intervals with minute granularity. | Time | | `HOUR` | Aggregation of time intervals with hour granularity. | Time | | `DAY` | Aggregation of time intervals with day granularity. | Time | | `WEEK` | Aggregation of time intervals with week granularity. | Time | | `MONTH` | Aggregation of time intervals with month granularity. | Time | | `YEAR` | Aggregation of time intervals with year granularity. | Time | ### Information-driven bars Information-driven bars adapt their sampling frequency to market activity rather than using fixed intervals. They are based on the concept of *aggressor side* (whether the trade initiator was a buyer or seller) and come in two families: **imbalance** and **runs**. **Imbalance bars** close when the *net* buy/sell activity reaches a threshold. Each trade contributes a signed value: positive for buyer-initiated trades and negative for seller-initiated. The bar closes when the absolute imbalance reaches the configured step. This means that opposing trades cancel each other out, so imbalance bars tend to form more slowly in balanced markets and faster during directional moves. **Runs bars** close when *consecutive* activity from the same aggressor side reaches a threshold. Unlike imbalance bars, runs bars reset their counter when the aggressor side changes. This makes them sensitive to sustained one-sided pressure rather than net imbalance. Both families have three variants based on what is measured: | Variant | Imbalance | Runs | What is measured | |:--------|:-------------------|:--------------|:------------------------------------------| | Tick | `TICK_IMBALANCE` | `TICK_RUNS` | Number of trades (each trade counts as 1) | | Volume | `VOLUME_IMBALANCE` | `VOLUME_RUNS` | Traded volume (quantity) | | Value | `VALUE_IMBALANCE` | `VALUE_RUNS` | Notional value (price x quantity) | :::note Information-driven bars require `TradeTick` data because they need the `aggressor_side` field to classify each trade. They cannot be aggregated from `QuoteTick` data alone. ::: ### Types of aggregation NautilusTrader implements three distinct data aggregation methods: 1. **Trade-to-bar aggregation**: Creates bars from `TradeTick` objects (executed trades) - Use case: For strategies analyzing execution prices or when working directly with trade data. - Always uses the `LAST` price type in the bar specification. 2. **Quote-to-bar aggregation**: Creates bars from `QuoteTick` objects (bid/ask prices) - Use case: For strategies focusing on bid/ask spreads or market depth analysis. - Uses `BID`, `ASK`, or `MID` price types in the bar specification. 3. **Bar-to-bar aggregation**: Creates larger-timeframe `Bar` objects from smaller-timeframe `Bar` objects - Use case: For resampling existing smaller timeframe bars (1-minute) into larger timeframes (5-minute, hourly). - Always requires the `@` symbol in the specification. ### Bar types NautilusTrader defines a unique *bar type* (`BarType` class) based on the following components: - **Instrument ID** (`InstrumentId`): Specifies the particular instrument for the bar. - **Bar Specification** (`BarSpecification`): - `step`: Defines the interval or frequency of each bar. - `aggregation`: Specifies the method used for data aggregation (see the above table). - `price_type`: Indicates the price basis of the bar (e.g., bid, ask, mid, last). - **Aggregation Source** (`AggregationSource`): Indicates whether the bar was aggregated internally (within Nautilus) or externally (by a trading venue or data provider). :::note `BarSpecification` validates fixed-subunit time aggregations so bars align cleanly with their parent clock or calendar unit. `MILLISECOND` steps must divide 1000 and be less than 1000; `SECOND` and `MINUTE` steps must divide 60 and be less than 60; `HOUR` steps must divide 24 and be less than 24; and `MONTH` steps must divide 12 and be less than 12. Use the next larger aggregation when the step equals a parent unit, such as `1-HOUR` instead of `60-MINUTE`. `DAY`, `WEEK`, `YEAR`, threshold, information, and `RENKO` bars are not restricted by this fixed-subunit rule. A future version will allow advanced users to override this validation for arbitrary bar periods that do not align to clock or calendar boundaries. ::: Bar types can also be classified as either *standard* or *composite*: - **Standard**: Generated from granular market data, such as quote-ticks or trade-ticks. - **Composite**: Derived from a higher-granularity bar type through subsampling (like 5-MINUTE bars aggregate from 1-MINUTE bars). ### Aggregation sources Bar data aggregation can be either *internal* or *external*: - `INTERNAL`: The bar is aggregated inside the local Nautilus system boundary. - `EXTERNAL`: The bar is aggregated outside the local Nautilus system boundary (typically by a trading venue or data provider). For bar-to-bar aggregation, the target bar type is always `INTERNAL` (since you're doing the aggregation within NautilusTrader), but the source bars can be either `INTERNAL` or `EXTERNAL`, i.e., you can aggregate externally provided bars or already aggregated internal bars. ### Defining bar types with *string syntax* #### Standard bars You can define standard bar types from strings using the following convention: `{instrument_id}-{step}-{aggregation}-{price_type}-{INTERNAL | EXTERNAL}` For example, to define a `BarType` for AAPL trades (last price) on Nasdaq (XNAS) using a 5-minute interval aggregated from trades locally by Nautilus: ```python bar_type = BarType.from_str("AAPL.XNAS-5-MINUTE-LAST-INTERNAL") ``` #### Composite bars Composite bars are derived by aggregating higher-granularity bars into the desired bar type. To define a composite bar, use this convention: `{instrument_id}-{step}-{aggregation}-{price_type}-INTERNAL@{step}-{aggregation}-{INTERNAL | EXTERNAL}` **Notes**: - The derived bar type must use an `INTERNAL` aggregation source (since this is how the bar is aggregated). - The sampled bar type must have a higher granularity than the derived bar type. - The sampled instrument ID is inferred to match that of the derived bar type. - Composite bars can be aggregated *from* `INTERNAL` or `EXTERNAL` aggregation sources. For example, to define a `BarType` for AAPL trades (last price) on Nasdaq (XNAS) using a 5-minute interval aggregated locally by Nautilus, from 1-minute interval bars aggregated externally: ```python bar_type = BarType.from_str("AAPL.XNAS-5-MINUTE-LAST-INTERNAL@1-MINUTE-EXTERNAL") ``` ### Aggregation syntax examples The `BarType` string format encodes both the target bar type and, optionally, the source data type: ``` {instrument_id}-{step}-{aggregation}-{price_type}-{source}@{step}-{aggregation}-{source} ``` The part after the `@` symbol is optional and only used for bar-to-bar aggregation: - **Without `@`**: Aggregates from `TradeTick` objects (when price_type is `LAST`) or `QuoteTick` objects (when price_type is `BID`, `ASK`, or `MID`). - **With `@`**: Aggregates from existing `Bar` objects (specifying the source bar type). #### Trade-to-bar example ```python def on_start(self) -> None: # Define a bar type for aggregating from TradeTick objects # Uses price_type=LAST which indicates TradeTick data as source bar_type = BarType.from_str("6EH4.XCME-50-VOLUME-LAST-INTERNAL") start = self.clock.utc_now() - timedelta(days=30) # Request historical data (will receive bars in on_historical_data handler) self.request_bars(bar_type, start=start) # Subscribe to live data (will receive bars in on_bar handler) self.subscribe_bars(bar_type) ``` #### Quote-to-bar example ```python def on_start(self) -> None: # Create 1-minute bars from ASK prices (in QuoteTick objects) bar_type_ask = BarType.from_str("6EH4.XCME-1-MINUTE-ASK-INTERNAL") # Create 1-minute bars from BID prices (in QuoteTick objects) bar_type_bid = BarType.from_str("6EH4.XCME-1-MINUTE-BID-INTERNAL") # Create 1-minute bars from MID prices (middle between ASK and BID prices in QuoteTick objects) bar_type_mid = BarType.from_str("6EH4.XCME-1-MINUTE-MID-INTERNAL") start = self.clock.utc_now() - timedelta(days=30) # Request historical data and subscribe to live data self.request_bars(bar_type_ask, start=start) # Historical bars processed in on_historical_data self.subscribe_bars(bar_type_ask) # Live bars processed in on_bar ``` #### Bar-to-bar example ```python def on_start(self) -> None: # Create 5-minute bars from 1-minute bars (Bar objects) # Format: target_bar_type@source_bar_type # Note: price type (LAST) is only needed on the left target side, not on the source side bar_type = BarType.from_str("6EH4.XCME-5-MINUTE-LAST-INTERNAL@1-MINUTE-EXTERNAL") start = self.clock.utc_now() - timedelta(days=30) # Request historical data by providing the dependency-ordered aggregation chain self.request_aggregated_bars([bar_type], start=start) # Subscribe to live updates (processed in on_bar(...) handler) self.subscribe_bars(bar_type) ``` #### Advanced bar-to-bar example You can create complex aggregation chains where you aggregate from already aggregated bars: ```python # First create 1-minute bars from TradeTick objects (LAST indicates TradeTick source) primary_bar_type = BarType.from_str("6EH4.XCME-1-MINUTE-LAST-INTERNAL") # Then create 5-minute bars from 1-minute bars # Note the @1-MINUTE-INTERNAL part identifying the source bars intermediate_bar_type = BarType.from_str("6EH4.XCME-5-MINUTE-LAST-INTERNAL@1-MINUTE-INTERNAL") # Then create hourly bars from 5-minute bars # Note the @5-MINUTE-INTERNAL part identifying the source bars hourly_bar_type = BarType.from_str("6EH4.XCME-1-HOUR-LAST-INTERNAL@5-MINUTE-INTERNAL") ``` ### Working with bars: request vs. subscribe NautilusTrader provides two distinct operations for working with bars: - **`request_bars()`**: Fetches historical data for a standard `BarType`, processed by the `on_historical_data()` handler. - **`request_aggregated_bars()`**: Fetches historical data for a dependency-ordered list of bar types, building internal bars on the fly. - **`subscribe_bars()`**: Establishes a real-time data feed processed by the `on_bar()` handler. It expects the instrument for the `BarType` to already be loaded in the cache. The same cache precondition applies to quote, trade, order book, and other live subscriptions. These methods work together in a typical workflow: 1. First, `request_bars()` loads historical data to initialize indicators or state of strategy with past market behavior. 2. Then, `subscribe_bars()` ensures the strategy continues receiving new bars as they form in real-time. :::tip[Request and subscribe ordering] When `validate_data_sequence=True` (common with live adapters such as Interactive Brokers), calling `subscribe_bars()` immediately after `request_bars()` can cause a race condition: live bars arriving before the historical batch may cause the validator to discard older warmup bars. To avoid it, pass a `callback` to `request_bars()` and subscribe from inside it, as shown in the example below. ::: Example usage in `on_start()`: ```python def on_start(self) -> None: # Define bar type bar_type = BarType.from_str("6EH4.XCME-5-MINUTE-LAST-INTERNAL") start = self.clock.utc_now() - timedelta(days=30) # Register indicators before requesting history so they receive historical updates too self.register_indicator_for_bars(bar_type, self.my_indicator) # Request historical data to initialize indicators # These bars will be delivered to the on_historical_data(...) handler in strategy # Subscribe to real-time bars as a callback to the request so the live stream # only starts once history is loaded (see tip above) # New live bars will be delivered to the on_bar(...) handler in strategy self.request_bars( bar_type, start=start, callback=lambda _: self.subscribe_bars(bar_type), ) ``` Required handlers in your strategy to receive the data: ```python def on_historical_data(self, data): # Processes historical Data objects from request_bars() or request_aggregated_bars() # Note: indicators registered with register_indicator_for_bars # are updated automatically with historical data pass def on_bar(self, bar): # Processes individual bars in real-time from subscribe_bars() # Indicators registered with this bar type will update automatically and they will be updated before this handler is called pass ``` ### Historical data requests with aggregation When requesting historical bars for backtesting or initializing indicators, use `request_bars()` for standard bar types and `request_aggregated_bars()` for on-the-fly aggregation: ```python start = self.clock.utc_now() - timedelta(days=30) # Request raw 1-minute bars (aggregated from TradeTick objects as indicated by LAST price type) self.request_bars( BarType.from_str("6EH4.XCME-1-MINUTE-LAST-EXTERNAL"), start=start, ) # Request bars that are aggregated from historical trade ticks self.request_aggregated_bars( [BarType.from_str("6EH4.XCME-100-VOLUME-LAST-INTERNAL")], start=start, ) # Request 5-minute bars aggregated from 1-minute bars self.request_aggregated_bars( [BarType.from_str("6EH4.XCME-5-MINUTE-LAST-INTERNAL@1-MINUTE-EXTERNAL")], start=start, ) ``` ### Common pitfalls **Register indicators before requesting data**: Ensure indicators are registered before requesting historical data so they get updated properly. ```python start = self.clock.utc_now() - timedelta(days=30) # Correct order self.register_indicator_for_bars(bar_type, self.ema) self.request_bars(bar_type, start=start) # Incorrect order self.request_bars(bar_type, start=start) # Indicator won't receive historical data self.register_indicator_for_bars(bar_type, self.ema) ``` ### Performance considerations Bar aggregators track OHLC prices via the fixed-point `Price` type. Threshold comparisons for tick and volume aggregators, including their imbalance and runs variants, use integer arithmetic, while value-based aggregators (value, value imbalance, and value runs) currently use `f64` for notional value and signed accumulation (these are being migrated to fixed-point integer arithmetic). The choice of aggregation method has a modest impact on per-update overhead: - **Time bars** are the most efficient for high-throughput data. The aggregator accumulates OHLCV state per update; bar emission is driven by a timer rather than per-tick logic. - **Threshold bars** (tick, volume, value) add a lightweight counter or accumulator check per update. Volume and value bars may split a single large trade across multiple bars when it exceeds the remaining threshold. - **Information-driven bars** (imbalance, runs) require tracking aggressor side and signed accumulation per update. The overhead is slightly higher than threshold bars but still minimal. - **Renko bars** are price-driven and can emit multiple bars from a single large price move. Otherwise the per-update cost is comparable to threshold bars. - **Composite bars** (bar-to-bar) are the most efficient way to produce higher-timeframe bars when lower-timeframe bars are already available, as each input bar represents an already aggregated period rather than a single tick. ### Time bar configuration Time bar behavior is controlled through `DataEngineConfig`. The following options apply to all time-based aggregation (millisecond through year): | Option | Type | Default | Description | |:------------------------------------|:-------|:--------------|:------------------------------------------------------------------------------------------------------------------------------------------------| | `time_bars_interval_type` | `str` | `"left-open"` | `"left-open"`: start excluded, end included. `"right-open"`: start included, end excluded. | | `time_bars_timestamp_on_close` | `bool` | `True` | When `True`, `ts_event` is the bar close time. When `False`, `ts_event` is the bar open time. | | `time_bars_skip_first_non_full_bar` | `bool` | `False` | Skip emitting a bar when aggregation starts mid‑interval, avoiding partial bars on startup. | | `time_bars_build_with_no_updates` | `bool` | `True` | When `True`, bars are emitted even if no market updates arrived during the interval. | | `time_bars_origin_offset` | `dict` | `None` | Maps `BarAggregation` types to `pd.Timedelta` or `pd.DateOffset` values for shifting bar alignment (e.g., align to 09:30 market open). | | `time_bars_build_delay` | `int` | `0` | Delay in microseconds before building a bar. Useful in backtests to ensure data at bar boundary timestamps is processed before the timer fires. | ```python from nautilus_trader.data.config import DataEngineConfig config = DataEngineConfig( time_bars_timestamp_on_close=True, time_bars_build_with_no_updates=False, time_bars_skip_first_non_full_bar=True, ) ``` ## Timestamps The platform uses two fundamental timestamp fields that appear across many objects, including market data, orders, and events. These timestamps serve distinct purposes and help maintain precise timing information throughout the system: - `ts_event`: UNIX timestamp (nanoseconds) representing when an event actually occurred. - `ts_init`: UNIX timestamp (nanoseconds) representing when Nautilus created the internal object representing that event. ### Examples | **Event Type** | **`ts_event`** | **`ts_init`** | | -----------------| ------------------------------------------------------| --------------| | `TradeTick` | Time when trade occurred at the exchange. | Time when Nautilus received the trade data. | | `QuoteTick` | Time when quote occurred at the exchange. | Time when Nautilus received the quote data. | | `OrderBookDelta` | Time when order book update occurred at the exchange. | Time when Nautilus received the order book update. | | `Bar` | Time of the bar's closing (exact minute/hour). | Time when Nautilus generated (for internal bars) or received the bar data (for external bars). | | `DefiData` | Time the block or pool event occurred. | Time when Nautilus created the object from the chain data. | | `OrderFilled` | Time when order was filled at the exchange. | Time when Nautilus received and processed the fill confirmation. | | `OrderCanceled` | Time when cancellation was processed at the exchange. | Time when Nautilus received and processed the cancellation confirmation. | | `NewsEvent` | Time when the news was published. | Time when the event object was created (if internal event) or received (if external event) in Nautilus. | | Custom event | Time when event conditions actually occurred. | Time when the event object was created (if internal event) or received (if external event) in Nautilus. | :::note The `ts_init` field represents a more general concept than "time of reception" for events. It denotes the timestamp when an object, such as a data point or command, was initialized within Nautilus. This distinction is important because `ts_init` is not exclusive to "received events". It applies to any internal initialization process. For example, the `ts_init` field is also used for commands, where the concept of reception does not apply. This broader definition ensures consistent handling of initialization timestamps across various object types in the system. ::: ### Latency analysis The dual timestamp system enables latency analysis within the platform: - Latency can be calculated as `ts_init - ts_event`. - This difference represents total system latency, including network transmission time, processing overhead, and any queueing delays. - It's important to remember that the clocks producing these timestamps are likely not synchronized. ### Environment-specific behavior #### Backtesting environment - Data is ordered by `ts_init` using a stable sort. - DeFi data (`DefiData`) breaks `ts_init` ties by on-chain position (block number, transaction index, log index) so events from the same block replay in canonical chain order. - This behavior ensures deterministic processing order and simulates realistic system behavior, including latencies. #### Live trading environment - The system processes data as it arrives to minimize latency and enable real-time decisions. - For venue-sourced data, `ts_init` is typically when Nautilus creates the local object after receiving the update. - `ts_event` reflects the time the event occurred externally, enabling accurate comparisons between external event timing and system reception. - We can use the difference between `ts_init` and `ts_event` to detect network or processing delays. ### Other notes and considerations - For data from external sources, `ts_init` is usually the local receipt or normalization time, but clock skew means it is not guaranteed to be greater than or equal to `ts_event`. - For data created within Nautilus, `ts_init` and `ts_event` can be the same because the object is initialized at the same time the event happens. - Not every type with a `ts_init` field necessarily has a `ts_event` field. This reflects cases where: - The initialization of an object happens at the same time as the event itself. - The concept of an external event time does not apply. #### Persisted data The `ts_init` field preserves the original initialization timestamp. For venue data this is typically receipt time; for internally created data it is the creation time of that object. ## Data flow From the `DataEngine` onward, data follows the same pathway regardless of [environment context](architecture.md#environment-contexts) (backtest, sandbox, live). In live and sandbox modes a venue adapter creates a normalized data object and sends it through a channel; in backtests the engine feeds data directly. Either way the `DataEngine` stores it in the `Cache` (for cached types) and publishes it on the `MessageBus` to subscribed handlers. For a step-by-step trace with a sequence diagram, see [Data flow: life of a quote tick](architecture.md#data-flow-life-of-a-quote-tick). For users who need more flexibility, the platform also supports the creation of custom data types. For details on how to implement user-defined data types, see the [Custom Data](#custom-data) section below. ## Loading data NautilusTrader supports data loading and conversion for three main use cases: - Providing data for a `BacktestEngine` to run backtests. - Persisting the Nautilus-specific Parquet format for the data catalog via `ParquetDataCatalog.write_data(...)` to be later used with a `BacktestNode`. - For research purposes (to ensure data is consistent between research and backtesting). Regardless of the destination, the process remains the same: converting diverse external data formats into Nautilus data structures. To achieve this, two main components are necessary: - A type of DataLoader (normally specific per raw source/format) which can read the data and return a `pd.DataFrame` with the correct schema for the desired Nautilus object. - A type of DataWrangler (specific per data type) which takes this `pd.DataFrame` and returns a `list[Data]` of Nautilus objects. ### Data loaders Data loader components are typically specific for the raw source/format and per integration. For instance, Binance order book data is stored in its raw CSV file form with an entirely different format to [Databento Binary Encoding (DBN)](https://databento.com/docs/knowledge-base/new-users/dbn-encoding/getting-started-with-dbn) files. ### Data wranglers Data wranglers are implemented per specific Nautilus data type, and can be found in the `nautilus_trader.persistence.wranglers` module. Common v1 wranglers include: - `OrderBookDeltaDataWrangler` - `QuoteTickDataWrangler` - `TradeTickDataWrangler` - `BarDataWrangler` For Arrow v2 / PyO3 workflows, the v2 module also provides `OrderBookDepth10DataWranglerV2`. :::warning There are a number of **DataWrangler v2** components, which will take a `pd.DataFrame` typically with a different fixed width Nautilus Arrow v2 schema, and output PyO3 Nautilus objects which are only compatible with the new version of the Nautilus core, currently in development. **These PyO3 data objects are not compatible where v1 legacy Cython objects are expected (e.g., adding directly to a `BacktestEngine`).** ::: ### Fixed-point precision and raw values NautilusTrader uses fixed-point arithmetic for `Price` and `Quantity` types for precise financial calculations without floating-point errors. Understanding how raw values work is essential when creating data or working with catalogs. #### Raw value requirements When constructing `Price` or `Quantity` using `from_raw()`, the raw value **must** be a valid multiple of the scale factor for the given precision. Valid raw values should come from: - Accessing the `.raw` field of an existing value (e.g., `price.raw`). - Using the Nautilus fixed-point conversion functions. - Values from Nautilus-produced Arrow data. :::warning Raw values that are not valid multiples will cause a panic. The raw value must be divisible by `10^(FIXED_PRECISION - precision)` where `FIXED_PRECISION` is 9 (standard mode) or 16 (high-precision mode). ::: #### Automatic raw value correction Catalog data can contain raw values with floating-point precision errors. This happens when raw values are produced with `int(value * FIXED_SCALAR)` instead of precision-aware conversion: ```python int(value * FIXED_SCALAR) # Introduces floating-point errors round(value * 10**precision) * scale # Correct precision-aware conversion ``` For example, `int(0.67068 * 1e9)` produces `670680000000001` instead of the expected `670680000000000`. The Arrow decode path automatically corrects these values by rounding to the nearest valid multiple, so affected catalogs work without data migration. :::note This correction adds a small amount of overhead during data decoding. ::: ### Transformation pipeline **Process flow**: 1. Raw data (e.g., CSV) is input into the pipeline. 2. DataLoader processes the raw data and converts it into a `pd.DataFrame`. 3. DataWrangler further processes the `pd.DataFrame` to generate a list of Nautilus objects. 4. The Nautilus `list[Data]` is the output of the data loading process. The following diagram illustrates how raw data is transformed into Nautilus data structures: ```mermaid flowchart LR raw["Raw data (CSV)"] loader[DataLoader] wrangler[DataWrangler] output["Nautilus list[Data]"] raw --> loader loader -->|"pd.DataFrame"| wrangler wrangler --> output ``` Concretely, this would involve: - `BinanceOrderBookDeltaDataLoader.load(...)` which reads CSV files provided by Binance from disk, and returns a `pd.DataFrame`. - `OrderBookDeltaDataWrangler.process(...)` which takes the `pd.DataFrame` and returns `list[OrderBookDelta]`. The following example shows how to accomplish the above in Python: ```python from nautilus_trader import TEST_DATA_DIR from nautilus_trader.adapters.binance.loaders import BinanceOrderBookDeltaDataLoader from nautilus_trader.persistence.wranglers import OrderBookDeltaDataWrangler from nautilus_trader.test_kit.providers import TestInstrumentProvider # Load raw data data_path = TEST_DATA_DIR / "binance" / "btcusdt-depth-snap.csv" df = BinanceOrderBookDeltaDataLoader.load(data_path) # Set up a wrangler instrument = TestInstrumentProvider.btcusdt_binance() wrangler = OrderBookDeltaDataWrangler(instrument) # Process to a list `OrderBookDelta` Nautilus objects deltas = wrangler.process(df) ``` ## Data catalog The data catalog is a central store for Nautilus data, persisted in the [Parquet](https://parquet.apache.org) file format. It is the primary data management system for both backtesting and live trading scenarios, providing efficient storage, retrieval, and streaming capabilities for market data. ### Overview and architecture The NautilusTrader data catalog is built on a dual-backend architecture that combines the performance of Rust with the flexibility of Python: **Core components:** - **ParquetDataCatalog**: The main Python interface for data operations. - **Rust backend**: High-performance query engine for core data types (`OrderBookDelta`, `OrderBookDeltas`, `OrderBookDepth10`, `QuoteTick`, `TradeTick`, `Bar`, `MarkPriceUpdate`) and registered same-binary Rust custom data. - **PyArrow backend**: Flexible fallback for custom data types and advanced filtering. - **fsspec integration**: Support for local and cloud storage (S3, GCS, Azure, etc.). **Key benefits**: - **Performance**: Rust backend provides optimized query performance for core market data types. - **Flexibility**: PyArrow backend handles custom data types and complex filtering scenarios. - **Scalability**: Efficient compression and columnar storage reduce storage costs and improve I/O performance. - **Cloud native**: Built-in support for cloud storage providers through fsspec. - **No dependencies**: Self-contained solution requiring no external databases or services. **Storage format advantages:** - Superior compression ratio and read performance compared to CSV/JSON/HDF5. - Columnar storage enables efficient filtering and aggregation. - Schema evolution support for data model changes. - Cross-language compatibility (Python, Rust, Java, C++, etc.). The Arrow schemas used for the Parquet format are defined in two places: the Rust `model` and `persistence` crates for core market data types, and the Python `serialization/arrow/schema.py` module for additional types. ### Initializing The data catalog can be initialized from a `NAUTILUS_PATH` environment variable, or by explicitly passing in a path like object. :::note[NAUTILUS_PATH environment variable] The `NAUTILUS_PATH` environment variable should point to the **root** directory containing your Nautilus data. The catalog will automatically append `/catalog` to this path. For example: - If `NAUTILUS_PATH=/home/user/trading_data`. - Then the catalog will be located at `/home/user/trading_data/catalog`. This is a common pattern when using `ParquetDataCatalog.from_env()` - make sure your `NAUTILUS_PATH` points to the parent directory, not the catalog directory itself. ::: The following example shows how to initialize a data catalog where there is pre-existing data already written to disk at the given path. ```python from pathlib import Path from nautilus_trader.persistence.catalog import ParquetDataCatalog CATALOG_PATH = Path.cwd() / "catalog" # Create a new catalog instance catalog = ParquetDataCatalog(CATALOG_PATH) # Alternative: Environment-based initialization catalog = ParquetDataCatalog.from_env() # Uses NAUTILUS_PATH environment variable ``` ### Filesystem protocols and storage options The catalog supports multiple filesystem protocols through fsspec integration, working across local and cloud storage systems. #### Supported filesystem protocols **Local filesystem (`file`):** ```python catalog = ParquetDataCatalog( path="/path/to/catalog", fs_protocol="file", # Default protocol ) ``` **Amazon S3 (`s3`):** ```python catalog = ParquetDataCatalog( path="s3://my-bucket/nautilus-data/", fs_protocol="s3", fs_storage_options={ "key": "your-access-key-id", "secret": "your-secret-access-key", "endpoint_url": "https://s3.amazonaws.com", # Optional custom endpoint } ) ``` **Google Cloud Storage (`gcs`):** ```python catalog = ParquetDataCatalog( path="gcs://my-bucket/nautilus-data/", fs_protocol="gcs", fs_storage_options={ "project": "my-project-id", "token": "/path/to/service-account.json", # Or "cloud" for default credentials } ) ``` **Azure Blob Storage :** `abfs` protocol ```python catalog = ParquetDataCatalog( path="abfs://container@account.dfs.core.windows.net/nautilus-data/", fs_protocol="abfs", fs_storage_options={ "account_name": "your-storage-account", "account_key": "your-account-key", # Or use SAS token: "sas_token": "your-sas-token" } ) ``` `az` protocol ```python catalog = ParquetDataCatalog( path="az://container/nautilus-data/", fs_protocol="az", fs_storage_options={ "account_name": "your-storage-account", "account_key": "your-account-key", # Or use SAS token: "sas_token": "your-sas-token" } ) ``` #### URI-based initialization For convenience, you can use URI strings that automatically parse protocol and storage options: ```python # Local filesystem catalog = ParquetDataCatalog.from_uri("/path/to/catalog") # S3 bucket catalog = ParquetDataCatalog.from_uri("s3://my-bucket/nautilus-data/") # With storage options catalog = ParquetDataCatalog.from_uri( "s3://my-bucket/nautilus-data/", fs_storage_options={ "access_key_id": "your-key", "secret_access_key": "your-secret" } ) ``` ### Writing data Store data in the catalog using the `write_data()` method. All Nautilus built-in `Data` objects are supported, and any data which inherits from `Data` can be written. ```python # Write a list of data objects catalog.write_data(quote_ticks) # Write with custom timestamp range catalog.write_data( trade_ticks, start=1704067200000000000, # Optional start timestamp override (UNIX nanoseconds) end=1704153600000000000, # Optional end timestamp override (UNIX nanoseconds) ) # Skip disjoint check for overlapping data catalog.write_data(bars, skip_disjoint_check=True) ``` ### File naming and data organization The catalog automatically generates filenames based on the timestamp range of the data being written. Files are named using the pattern `{start_timestamp}_{end_timestamp}.parquet`, where each timestamp is an ISO 8601 value converted to a filename-safe form by replacing `:` and `.` with `-`. Data is organized in directories by data type and identifier (instrument ID, bar type, or custom identifier). Identifiers are made URI-safe by removing `/`: ``` catalog/ ├── data/ │ ├── quote_ticks/ │ │ └── EURUSD.SIM/ │ │ └── 2024-01-01T00-00-00-000000000Z_2024-01-01T23-59-59-999999999Z.parquet │ └── trade_ticks/ │ └── BTCUSD.BINANCE/ │ └── 2024-01-01T00-00-00-000000000Z_2024-01-01T23-59-59-999999999Z.parquet ``` **Rust backend data types (enhanced performance):** The following data types use optimized Rust implementations: - `OrderBookDelta`. - `OrderBookDeltas`. - `OrderBookDepth10`. - `QuoteTick`. - `TradeTick`. - `Bar`. - `MarkPriceUpdate`. :::warning By default, overlapping writes raise a `ValueError` to maintain data integrity. Use `skip_disjoint_check=True` in `write_data()` to bypass this check when needed. ::: ### Reading data Use the `query()` method to read data back from the catalog: ```python from nautilus_trader.model import QuoteTick, TradeTick # Query quote ticks for a specific instrument and time range quotes = catalog.query( data_cls=QuoteTick, identifiers=["EUR/USD.SIM"], start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z" ) # Query trade ticks for a specific instrument and time range trades = catalog.query( data_cls=TradeTick, identifiers=["BTC/USD.BINANCE"], start="2024-01-01", end="2024-01-02", ) ``` ### `BacktestDataConfig` - data specification for backtests The `BacktestDataConfig` class is the primary mechanism for specifying data requirements before a backtest starts. It defines what data should be loaded from the catalog and how it should be filtered and processed during the backtest execution. #### Core parameters **Required parameters:** - `catalog_path`: Path to the data catalog directory. - `data_cls`: The data type class (e.g., QuoteTick, TradeTick, OrderBookDelta, Bar). **Optional parameters:** - `catalog_fs_protocol`: Filesystem protocol ('file', 's3', 'gcs', etc.). - `catalog_fs_storage_options`: Storage-specific options (credentials, region, etc.). - `catalog_fs_rust_storage_options`: Storage-specific options for the Rust backend. - `instrument_id`: Specific instrument to load data for. - `instrument_ids`: List of instruments (alternative to single instrument_id). - `start_time`: Start time for data filtering (ISO string or UNIX nanoseconds). - `end_time`: End time for data filtering (ISO string or UNIX nanoseconds). - `filter_expr`: Additional PyArrow filter expressions. - `client_id`: Client ID for custom data types. - `metadata`: Additional metadata for data queries. - `bar_spec`: Bar specification for bar data (e.g., `"1-MINUTE-LAST"`). When combined with `instrument_id` or `instrument_ids`, this builds `...-EXTERNAL` bar identifiers. - `bar_types`: Explicit list of full bar types. Use this for `INTERNAL` bars or composite bars. - `optimize_file_loading`: Load directories instead of individual files when supported. #### Basic usage examples **Loading quote ticks:** ```python from nautilus_trader.config import BacktestDataConfig from nautilus_trader.model import QuoteTick, InstrumentId data_config = BacktestDataConfig( catalog_path="/path/to/catalog", data_cls=QuoteTick, instrument_id=InstrumentId.from_str("EUR/USD.SIM"), start_time="2024-01-01T00:00:00Z", end_time="2024-01-02T00:00:00Z", ) ``` **Loading multiple instruments:** ```python data_config = BacktestDataConfig( catalog_path="/path/to/catalog", data_cls=TradeTick, instrument_ids=["BTC/USD.BINANCE", "ETH/USD.BINANCE"], start_time="2024-01-01T00:00:00Z", end_time="2024-01-02T00:00:00Z", ) ``` **Loading Bar Data:** ```python data_config = BacktestDataConfig( catalog_path="/path/to/catalog", data_cls=Bar, instrument_id=InstrumentId.from_str("AAPL.NASDAQ"), bar_spec="5-MINUTE-LAST", # Loads AAPL.NASDAQ-5-MINUTE-LAST-EXTERNAL start_time="2024-01-01", end_time="2024-01-31", ) ``` #### Advanced configuration examples **Cloud Storage with Custom Filtering:** ```python data_config = BacktestDataConfig( catalog_path="s3://my-bucket/nautilus-data/", catalog_fs_protocol="s3", catalog_fs_storage_options={ "key": "your-access-key", "secret": "your-secret-key", "region": "us-east-1" }, data_cls=OrderBookDelta, instrument_id=InstrumentId.from_str("BTC/USD.COINBASE"), start_time="2024-01-01T09:30:00Z", end_time="2024-01-01T16:00:00Z", ) ``` **Custom Data with Client ID:** ```python data_config = BacktestDataConfig( catalog_path="/path/to/catalog", data_cls="my_package.data.NewsEventData", client_id="NewsClient", metadata={"source": "reuters", "category": "earnings"}, start_time="2024-01-01", end_time="2024-01-31", ) ``` #### Integration with BacktestRunConfig The `BacktestDataConfig` objects are integrated into the backtesting framework through `BacktestRunConfig`: ```python from nautilus_trader.config import BacktestRunConfig, BacktestVenueConfig # Define multiple data configurations data_configs = [ BacktestDataConfig( catalog_path="/path/to/catalog", data_cls=QuoteTick, instrument_id="EUR/USD.SIM", start_time="2024-01-01", end_time="2024-01-02", ), BacktestDataConfig( catalog_path="/path/to/catalog", data_cls=TradeTick, instrument_id="EUR/USD.SIM", start_time="2024-01-01", end_time="2024-01-02", ), ] # Create backtest run configuration run_config = BacktestRunConfig( venues=[BacktestVenueConfig(name="SIM", oms_type="HEDGING")], data=data_configs, # List of data configurations start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z", ) ``` #### Data loading process When a backtest runs, the `BacktestNode` processes each `BacktestDataConfig`: 1. **Catalog Loading**: Creates a `ParquetDataCatalog` instance from the config. 2. **Query Construction**: Builds query parameters from config attributes. 3. **Data Retrieval**: Executes catalog queries using the appropriate backend. 4. **Instrument Loading**: Loads instrument definitions if needed. 5. **Engine Integration**: Adds data to the backtest engine with proper sorting. The system automatically handles: - Instrument ID resolution and validation. - Data type validation and conversion. - Memory-efficient streaming for large datasets. - Error handling and logging. ### DataCatalogConfig - on-the-fly data loading The `DataCatalogConfig` class provides configuration for on-the-fly data loading scenarios, particularly useful for backtests where the number of possible instruments is vast, Unlike `BacktestDataConfig` which pre-specifies data for backtests, `DataCatalogConfig` enables flexible catalog access during runtime. Catalogs defined this way can also be used for requesting historical data. #### Core parameters **Required Parameters:** - `path`: Path to the data catalog directory. **Optional Parameters:** - `fs_protocol`: Filesystem protocol ('file', 's3', 'gcs', 'azure', etc.). - `fs_storage_options`: Protocol-specific storage options. - `fs_rust_storage_options`: Protocol-specific storage options for the Rust backend. - `name`: Optional name identifier for the catalog configuration. #### Basic usage examples **Local Catalog Configuration:** ```python from nautilus_trader.persistence.config import DataCatalogConfig catalog_config = DataCatalogConfig( path="/path/to/catalog", fs_protocol="file", name="local_market_data" ) # Convert to catalog instance catalog = catalog_config.as_catalog() ``` **Cloud storage configuration:** ```python catalog_config = DataCatalogConfig( path="s3://my-bucket/market-data/", fs_protocol="s3", fs_storage_options={ "key": "your-access-key", "secret": "your-secret-key", "region": "us-west-2", "endpoint_url": "https://s3.us-west-2.amazonaws.com" }, name="cloud_market_data" ) ``` #### Integration with live trading `DataCatalogConfig` is commonly used in live trading configurations for historical data access: ```python from nautilus_trader.config import TradingNodeConfig from nautilus_trader.persistence.config import DataCatalogConfig # Configure catalog for live system catalog_config = DataCatalogConfig( path="/data/nautilus/catalog", fs_protocol="file", name="historical_data" ) # Use in trading node configuration node_config = TradingNodeConfig( # ... other configurations catalogs=[catalog_config], # Enable historical data access ) ``` #### Streaming configuration For streaming data to catalogs during live trading or backtesting, use `StreamingConfig`: ```python from nautilus_trader.persistence.config import StreamingConfig, RotationMode import pandas as pd streaming_config = StreamingConfig( catalog_path="/path/to/streaming/catalog", fs_protocol="file", flush_interval_ms=1000, # Flush every second replace_existing=False, rotation_mode=RotationMode.INTERVAL, rotation_interval=pd.Timedelta(hours=1), max_file_size=1024 * 1024 * 100, # 100MB max file size ) ``` #### Use cases **Historical Data Analysis:** - Load historical data during live trading for strategy calculations. - Access reference data for instrument lookups. - Retrieve past performance metrics. **Dynamic data loading:** - Load data based on runtime conditions. - Implement custom data loading strategies. - Support multiple catalog sources. **Research and development:** - Interactive data exploration in Jupyter notebooks. - Ad-hoc analysis and backtesting. - Data quality validation and monitoring. ### Query system and dual backend architecture The catalog's query system uses a dual-backend architecture that selects the query engine based on data type and query parameters. #### Backend selection logic **Rust backend (high performance):** - **Supported Types**: OrderBookDelta, OrderBookDeltas, OrderBookDepth10, QuoteTick, TradeTick, Bar, MarkPriceUpdate. - **Conditions**: Used when `files` parameter is None (automatic file discovery). - **Benefits**: Optimized performance, memory efficiency, native Arrow integration. Registered same-binary Rust custom data types can also use this path. **PyArrow backend (flexible):** - **Supported Types**: All data types including custom data classes. - **Conditions**: Used for custom data types or when `files` parameter is specified. - **Benefits**: Advanced filtering, custom data support, complex query expressions. #### Query methods and parameters **Core query parameters:** ```python catalog.query( data_cls=QuoteTick, # Data type to query identifiers=["EUR/USD.SIM"], # Instrument identifiers start="2024-01-01T00:00:00Z", # Start time (various formats supported) end="2024-01-02T00:00:00Z", # End time files=None, # Leave unset for automatic file discovery ) ``` - `where=` passes a DataFusion SQL predicate to Rust-backed queries. - `filter_expr=` passes a parsed PyArrow dataset expression to PyArrow-backed queries. **Time format support:** - ISO 8601 strings: `"2024-01-01T00:00:00Z"`. - UNIX nanoseconds: `1704067200000000000` (or ISO format: `"2024-01-01T00:00:00Z"`). - Pandas Timestamps: `pd.Timestamp("2024-01-01", tz="UTC")`. - Python datetime objects (timezone-aware recommended). **Filtering notes:** - Use `where=` for Rust-backed built-in market data queries. - Use `filter_expr=` for PyArrow-backed queries, including custom data and queries forced onto the PyArrow path with `files=`. ### Catalog operations The catalog provides several operation functions for maintaining and organizing data files. These operations help optimize storage, improve query performance, and maintain data integrity. #### Reset file names Reset parquet file names to match their actual content timestamps. This ensures filename-based filtering works correctly. **Reset all files in catalog:** ```python # Reset all parquet files in the catalog catalog.reset_all_file_names() ``` **Reset specific data type:** ```python # Reset filenames for all quote tick files catalog.reset_data_file_names(QuoteTick) # Reset filenames for specific instrument's trade files catalog.reset_data_file_names(TradeTick, "BTC/USD.BINANCE") ``` #### Consolidate catalog Combine multiple small parquet files into larger files to improve query performance and reduce storage overhead. **Consolidate entire catalog:** ```python # Consolidate all files in the catalog catalog.consolidate_catalog() # Consolidate files within a specific time range catalog.consolidate_catalog( start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z", ensure_contiguous_files=True ) ``` **Consolidate specific data type:** ```python # Consolidate all quote tick files catalog.consolidate_data(QuoteTick) # Consolidate specific instrument's files catalog.consolidate_data( TradeTick, identifier="BTC/USD.BINANCE", start="2024-01-01", end="2024-01-31" ) ``` #### Consolidate catalog by period Split data files into fixed time periods for standardized file organization. **Consolidate entire catalog by period:** ```python import pandas as pd # Consolidate all files by 1-day periods catalog.consolidate_catalog_by_period( period=pd.Timedelta(days=1) ) # Consolidate by 1-hour periods within time range catalog.consolidate_catalog_by_period( period=pd.Timedelta(hours=1), start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z" ) ``` **Consolidate specific data by period:** ```python # Consolidate quote data by 4-hour periods catalog.consolidate_data_by_period( data_cls=QuoteTick, period=pd.Timedelta(hours=4) ) # Consolidate specific instrument by 30-minute periods catalog.consolidate_data_by_period( data_cls=TradeTick, identifier="EUR/USD.SIM", period=pd.Timedelta(minutes=30), start="2024-01-01", end="2024-01-31" ) ``` #### Delete data range Remove data within a specified time range for specific data types and instruments. This operation permanently deletes data and handles file intersections intelligently. **Delete entire catalog range:** ```python # Delete all data within a time range across the entire catalog catalog.delete_catalog_range( start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z" ) # Delete all data from the beginning up to a specific time catalog.delete_catalog_range(end="2024-01-01T00:00:00Z") ``` **Delete specific data type:** ```python # Delete all quote tick data for a specific instrument catalog.delete_data_range( data_cls=QuoteTick, identifier="BTC/USD.BINANCE" ) # Delete trade data within a specific time range catalog.delete_data_range( data_cls=TradeTick, identifier="EUR/USD.SIM", start="2024-01-01T00:00:00Z", end="2024-01-31T23:59:59Z" ) ``` :::warning Delete operations permanently remove data and cannot be undone. Files that partially overlap the deletion range are split to preserve data outside the range. ::: ### Feather streaming and conversion The catalog supports streaming data to temporary feather files during backtests, which can then be converted to permanent parquet format for efficient querying. **Example: option greeks streaming** ```python from option_trader.greeks import GreeksData from nautilus_trader.persistence.config import StreamingConfig # 1. Configure streaming for custom data streaming = StreamingConfig( catalog_path=catalog.path, include_types=[GreeksData], flush_interval_ms=1000, ) # 2. Run backtest with streaming enabled engine_config = BacktestEngineConfig(streaming=streaming) results = node.run() # 3. Convert streamed data to permanent catalog catalog.convert_stream_to_data( results[0].instance_id, GreeksData, ) # 4. Query converted data greeks_data = catalog.query( data_cls=GreeksData, start="2024-01-01", end="2024-01-31", where="delta > 0.5", ) ``` ### Catalog summary The NautilusTrader data catalog provides market data management: **Core features**: - **Dual Backend**: Rust performance + Python flexibility. - **Multi-Protocol**: Local, S3, GCS, Azure storage. - **Streaming**: Feather -> Parquet conversion pipeline. - **Operations**: Reset file names, consolidate data, period-based organization. **Key use cases**: - **Backtesting**: Pre-configured data loading via BacktestDataConfig. - **Live Trading**: On-demand data access via DataCatalogConfig. - **Maintenance**: File consolidation and organization operations. - **Research**: Interactive querying and analysis. ## Data migrations NautilusTrader defines an internal data format specified in the `nautilus_model` crate. These models are serialized into Arrow record batches and written to Parquet files. Nautilus backtesting is most efficient when using these Nautilus-format Parquet files. However, migrating the data model between [precision modes](../getting_started/installation.md#precision-mode) and schema changes can be challenging. This guide explains how to handle data migrations using our utility tools. ### Migration tools The `nautilus_persistence` crate provides two key utilities: #### `to_json` Converts Parquet files to JSON while preserving metadata: - Creates two files: - `.json`: Contains the deserialized data - `.metadata.json`: Contains schema metadata and row group configuration - Automatically detects data type from filename: - `OrderBookDelta` (contains "deltas" or "order_book_delta") - `QuoteTick` (contains "quotes" or "quote_tick") - `TradeTick` (contains "trades" or "trade_tick") - `Bar` (contains "bars") #### `to_parquet` Converts JSON back to Parquet format: - Reads both the data JSON and metadata JSON files. - Preserves row group sizes from original metadata. - Uses ZSTD compression. - Creates `.parquet`. ### Migration process The following migration examples both use trades data (you can also migrate the other data types in the same way). All commands should be run from the root of the `persistence` crate directory. #### Migrating from standard-precision (64-bit) to high-precision (128-bit) This example describes a scenario where you want to migrate from standard-precision schema to high-precision schema. :::note If you're migrating from a catalog that used the `Int64` and `UInt64` Arrow data types for prices and sizes, be sure to check out commit [e284162](https://github.com/nautechsystems/nautilus_trader/commit/e284162cf27a3222115aeb5d10d599c8cf09cf50) **before** compiling the code that writes the initial JSON. ::: **1. Convert from standard-precision Parquet to JSON**: ```bash cargo run --bin to_json trades.parquet ``` This will create `trades.json` and `trades.metadata.json` files. **2. Convert from JSON to high-precision Parquet**: Add the `--features high-precision` flag to write data as high-precision (128-bit) schema Parquet. ```bash cargo run --features high-precision --bin to_parquet trades.json ``` This will create a `trades.parquet` file with high-precision schema data. #### Migrating schema changes This example describes a scenario where you want to migrate from one schema version to another. **1. Convert from old schema Parquet to JSON**: Add the `--features high-precision` flag if the source data uses a high-precision (128-bit) schema. ```bash cargo run --bin to_json trades.parquet ``` This will create `trades.json` and `trades.metadata.json` files. **2. Switch to new schema version**: ```bash git checkout ``` **3. Convert from JSON back to new schema Parquet**: ```bash cargo run --features high-precision --bin to_parquet trades.json ``` This will create a `trades.parquet` file with the new schema. ### Best practices - Always test migrations with a small dataset first. - Maintain backups of original files. - Verify data integrity after migration. - Perform migrations in a staging environment before applying them to production data. ## Custom data Due to the modular nature of the Nautilus design, it is possible to set up systems with very flexible data streams, including custom user-defined data types. This guide covers some possible use cases for this functionality. It's possible to create custom data types within the Nautilus system. First you will need to define your data by subclassing from `Data`. :::info As `Data` holds no state, it is not strictly necessary to call `super().__init__()`. ::: ```python from nautilus_trader.core import Data class MyDataPoint(Data): """ This is an example of a user-defined data class, inheriting from the base class `Data`. The fields `label`, `x`, `y`, and `z` in this class are examples of arbitrary user data. """ def __init__( self, label: str, x: int, y: int, z: int, ts_event: int, ts_init: int, ) -> None: self.label = label self.x = x self.y = y self.z = z self._ts_event = ts_event self._ts_init = ts_init @property def ts_event(self) -> int: """ UNIX timestamp (nanoseconds) when the data event occurred. Returns ------- int """ return self._ts_event @property def ts_init(self) -> int: """ UNIX timestamp (nanoseconds) when the object was initialized. Returns ------- int """ return self._ts_init ``` The `Data` abstract base class acts as a contract within the system and requires two properties for all types of data: `ts_event` and `ts_init`. These represent the UNIX nanosecond timestamps for when the event occurred and when the object was initialized, respectively. The recommended approach to satisfy the contract is to assign `ts_event` and `ts_init` to backing fields, and then implement the `@property` for each as shown above (for completeness, the docstrings are copied from the `Data` base class). :::info These timestamps enable Nautilus to correctly order data streams for backtests using monotonically increasing `ts_init` UNIX nanoseconds. ::: We can now work with this data type for backtesting and live trading. For instance, we could now create an adapter which is able to parse and create objects of this type - and send them back to the `DataEngine` for consumption by subscribers. You can publish a custom data type within your actor/strategy using the message bus in the following way: ```python self.publish_data( DataType(MyDataPoint, metadata={"some_optional_category": 1}), MyDataPoint(...), ) ``` The `metadata` dictionary optionally adds more granular information that is used in the topic name to publish data with the message bus. Extra metadata information can also be passed to a `BacktestDataConfig` configuration object in order to enrich and describe custom data objects used in a backtesting context: ```python from nautilus_trader.config import BacktestDataConfig data_config = BacktestDataConfig( catalog_path=str(catalog.path), data_cls=MyDataPoint, metadata={"some_optional_category": 1}, ) ``` You can subscribe to custom data types within your actor/strategy in the following way: ```python self.subscribe_data( data_type=DataType(MyDataPoint, metadata={"some_optional_category": 1}), client_id=ClientId("MY_ADAPTER"), ) ``` The `client_id` provides an identifier to route the data subscription to a specific client. This will result in your actor/strategy passing these received `MyDataPoint` objects to your `on_data` method. You will need to check the type, as this method acts as a flexible handler for all custom data. ```python def on_data(self, data: Data) -> None: # First check the type of data if isinstance(data, MyDataPoint): # Do something with the data ``` ### Publishing and receiving signal data Here is an example of publishing and receiving signal data using the `MessageBus` from an actor or strategy. A signal is an automatically generated custom data identified by a name containing only one value of a basic type (str, float, int, bool or bytes). ```python self.publish_signal("signal_name", value, ts_event) self.subscribe_signal("signal_name") def on_signal(self, signal): print("Signal", signal) ``` ### Option greeks example This example demonstrates how to create a custom data type for option Greeks, specifically the delta. By following these steps, you can create custom data types, subscribe to them, publish them, and store them in the `Cache` or `ParquetDataCatalog` for efficient retrieval. ```python import msgspec from nautilus_trader.core import Data from nautilus_trader.core.datetime import unix_nanos_to_iso8601 from nautilus_trader.model import DataType from nautilus_trader.serialization.base import register_serializable_type from nautilus_trader.serialization.arrow.serializer import register_arrow import pyarrow as pa from nautilus_trader.model import InstrumentId from nautilus_trader.core.datetime import dt_to_unix_nanos, unix_nanos_to_dt, format_iso8601 class GreeksData(Data): def __init__( self, instrument_id: InstrumentId = InstrumentId.from_str("ES.GLBX"), ts_event: int = 0, ts_init: int = 0, delta: float = 0.0, ) -> None: self.instrument_id = instrument_id self._ts_event = ts_event self._ts_init = ts_init self.delta = delta def __repr__(self): return (f"GreeksData(ts_init={unix_nanos_to_iso8601(self._ts_init)}, instrument_id={self.instrument_id}, delta={self.delta:.2f})") @property def ts_event(self): return self._ts_event @property def ts_init(self): return self._ts_init def to_dict(self): return { "instrument_id": self.instrument_id.value, "ts_event": self._ts_event, "ts_init": self._ts_init, "delta": self.delta, } @classmethod def from_dict(cls, data: dict): return GreeksData(InstrumentId.from_str(data["instrument_id"]), data["ts_event"], data["ts_init"], data["delta"]) def to_bytes(self): return msgspec.msgpack.encode(self.to_dict()) @classmethod def from_bytes(cls, data: bytes): return cls.from_dict(msgspec.msgpack.decode(data)) def to_catalog(self): return pa.RecordBatch.from_pylist([self.to_dict()], schema=GreeksData.schema()) @classmethod def from_catalog(cls, table: pa.Table): return [GreeksData.from_dict(d) for d in table.to_pylist()] @classmethod def schema(cls): return pa.schema( { "instrument_id": pa.string(), "ts_event": pa.int64(), "ts_init": pa.int64(), "delta": pa.float64(), } ) ``` #### Publishing and receiving data Here is an example of publishing and receiving data using the `MessageBus` from an actor or strategy: ```python register_serializable_type(GreeksData, GreeksData.to_dict, GreeksData.from_dict) def publish_greeks(self, greeks_data: GreeksData): self.publish_data(DataType(GreeksData), greeks_data) def subscribe_to_greeks(self): self.subscribe_data(DataType(GreeksData)) def on_data(self, data): if isinstance(data, GreeksData): print("Data", data) ``` #### Writing and reading data using the cache Here is an example of writing and reading data using the `Cache` from an actor or strategy: ```python def greeks_key(instrument_id: InstrumentId): return f"{instrument_id}_GREEKS" def cache_greeks(self, greeks_data: GreeksData): self.cache.add(greeks_key(greeks_data.instrument_id), greeks_data.to_bytes()) def greeks_from_cache(self, instrument_id: InstrumentId): return GreeksData.from_bytes(self.cache.get(greeks_key(instrument_id))) ``` #### Writing and reading data using a catalog For streaming custom data to feather files or writing it to parquet files in a catalog (`register_arrow` needs to be used): ```python register_arrow(GreeksData, GreeksData.schema(), GreeksData.to_catalog, GreeksData.from_catalog) from nautilus_trader.persistence.catalog import ParquetDataCatalog catalog = ParquetDataCatalog('.') catalog.write_data([GreeksData()]) ``` ### Creating a custom data class automatically The `@customdataclass` decorator enables the creation of a custom data class with default implementations for all the features described above. Each method can also be overridden if needed. Here is an example of its usage: ```python from nautilus_trader.model.custom import customdataclass @customdataclass class GreeksTestData(Data): instrument_id: InstrumentId = InstrumentId.from_str("ES.GLBX") delta: float = 0.0 GreeksTestData( instrument_id=InstrumentId.from_str("CL.GLBX"), delta=1000.0, ts_event=1, ts_init=2, ) ``` #### Python-only custom data with the PyO3 catalog To use custom data with the Rust-backed catalog (`ParquetDataCatalog` from `nautilus_pyo3`), use the `@customdataclass_pyo3()` decorator instead of `@customdataclass`. This adds the methods the Rust catalog expects (JSON and Arrow IPC serialization). After defining your class, register it once. You can pass either the **type** (recommended) or a **sample instance**: ```python from nautilus_trader.core.nautilus_pyo3 import ParquetDataCatalog from nautilus_trader.core.nautilus_pyo3.model import CustomData from nautilus_trader.core.nautilus_pyo3.model import DataType from nautilus_trader.core.nautilus_pyo3.model import register_custom_data_class from nautilus_trader.model.custom import customdataclass_pyo3 @customdataclass_pyo3() class MarketTickPython: symbol: str = "" price: float = 0.0 volume: int = 0 # Register by type (no instance needed; call once, e.g. at startup) register_custom_data_class(MarketTickPython) catalog = ParquetDataCatalog("/path/to/catalog") data_type = DataType("MarketTickPython", metadata={"exchange": "NASDAQ"}) wrapped = [ CustomData( data_type, MarketTickPython(ts_event=1, ts_init=1, symbol="AAPL", price=150.5, volume=1000), ), ] catalog.write_custom_data(wrapped) result = catalog.query("MarketTickPython") ticks = [item.data for item in result] ``` See `nautilus_trader.model.custom.customdataclass_pyo3` for details. #### Custom data type stub For better IDE code suggestions, you can create a `.pyi` stub file with the proper constructor signature for your custom data types as well as type hints for attributes. This is particularly useful when the constructor is dynamically generated at runtime, as it allows the IDE to recognize and provide suggestions for the class's methods and attributes. For instance, if you have a custom data class defined in `greeks.py`, you can create a corresponding `greeks.pyi` file with the following constructor signature: ```python from nautilus_trader.core import Data from nautilus_trader.model import InstrumentId class GreeksData(Data): instrument_id: InstrumentId delta: float def __init__( self, ts_event: int = 0, ts_init: int = 0, instrument_id: InstrumentId = InstrumentId.from_str("ES.GLBX"), delta: float = 0.0, ) -> GreeksData: ... ``` ## Related guides - [Instruments](instruments/) - Financial instruments referenced by data. - [Options](options.md) - Option instruments, chain subscriptions, and strike filtering. - [Greeks](greeks.md) - Venue-provided and locally computed option Greeks. - [Cache](cache.md) - Data storage and retrieval. - [Adapters](adapters.md) - Data sources and connectivity. # DST Source: https://nautilustrader.io/docs/latest/concepts/dst/ **Deterministic simulation testing (DST)** runs NautilusTrader under a seed-controlled runtime so that timing-sensitive execution behavior is bitwise reproducible from a single integer. This guide explains what DST is, how NautilusTrader supports it, what guarantees the support provides, and where those guarantees stop. The goal is a published contract that external users and auditors can verify: the determinism NautilusTrader claims is backed by source-level evidence and enforced at commit time by a pre- commit hook that runs in continuous integration. ## Introduction ### What DST is DST is a testing technique for concurrent systems. A single seed fully determines an execution, including task scheduling, timer firings, and random values. Two runs with the same seed, binary, and configuration produce identical observable behavior. When a property fails, the seed is the reproduction: the same seed replays the failure every time. Scheduling decisions in an async runtime come from ambient process state: task wake order, timer resolution, thread scheduling, hash seeds. None of that is controlled by the test harness, which is why a race that surfaces once in CI is usually hard to reproduce on demand. DST replaces those ambient sources with a seeded pseudorandom sequence, so the interleaving is a function of the seed. FoundationDB applied the pattern to a production distributed database starting around 2009; in the Rust ecosystem, [madsim](https://github.com/madsim-rs/madsim) intercepts `tokio` primitives to provide a deterministic scheduler. The bugs DST targets are the ones that escape unit, integration, property, and acceptance testing: channel wakeup ordering, drain races at shutdown, startup sequencing, reconciliation ordering, recovery-path correctness. All involve interleavings that other test layers cannot exhaustively cover but a deterministic scheduler can explore systematically. ### What this guide covers NautilusTrader's DST support has two halves: - **The contract**: what the runtime guarantees under seed-controlled execution, and under which conditions. - **The enforcement**: the source-level seams that implement the contract and the pre-commit hook that keeps them in place. ## Goals - **Seed-reproducible execution** for the in-scope portion of the NautilusTrader runtime. - **Honest scope**. The contract lists what is covered and what is not. No silent fallbacks to real wall-clock time or unseeded RNG; conditions that weaken the guarantee are enumerated. - **Enforcement in source**. A pre-commit hook fails commits that add banned patterns to the DST path, so the contract stays true without relying on reviewer attention. - **Minimum necessary instrumentation**. The seams route time, task scheduling, and randomness through a deterministic source only where the contract requires it; everything else runs unchanged. ## Approach `madsim` determinizes only the `tokio` primitives that route through its aliased submodules (`time`, `task`, `runtime`, `signal`). Wall-clock reads, monotonic reads, RNG draws, hash iteration, and `select!` polling bypass `tokio` entirely and need their own seams. Layer 1 swaps the aliased submodules for `madsim`; Layer 2 supplies the seams. ### Layer 1: runtime swap Under the `simulation` Cargo feature on `nautilus-common`, four `tokio` submodules are routed through `madsim` when `RUSTFLAGS="--cfg madsim"` is set: - `time` (timers, intervals, monotonic `Instant`). - `task` (spawning and joining async tasks). - `runtime` (the runtime builder and handle). - `signal` (process signals such as `ctrl_c`; re-export is available, call-site adoption is partial as noted under [scope boundaries](#signal-handling)). These re-exports live in `nautilus_common::live::dst`. DST-path call sites for `time`, `task`, and `runtime` import from this module rather than directly from `tokio`, so toggling the feature switches the async runtime in one place for the primitives that are fully routed. Under normal builds, the re-exports resolve to real `tokio`. Under `simulation` + `cfg(madsim)`, they resolve to `madsim`'s deterministic counterparts. Everything else that `tokio` provides (`sync`, `io`, `select!` as a macro, `fs`, `net`) uses real `tokio` unconditionally. Transitive crates (`tokio-tungstenite`, `tokio-rustls`, `reqwest`) are unaffected. ### Layer 2: nondeterminism substitution Nondeterminism outside the async runtime is redirected through explicit seams: - **Wall-clock reads** go through `nautilus_core::time::duration_since_unix_epoch`. Under simulation this routes to `madsim::time::TimeHandle::try_current()`, preserving Unix-epoch semantics for order and fill timestamps. When called outside a madsim runtime (plain `#[rstest]` test bodies), it falls back to `SystemTime::now()`, which under `cfg(madsim)` is libc-intercepted to the same real syscall a normal build would use. Production paths under simulation always run inside a runtime, so they continue to receive virtual time. - **Monotonic reads** go through `nautilus_common::live::dst::time::Instant`. The type resolves to `tokio::time::Instant` on normal builds (for compatibility with `tokio::test(start_paused)` test helpers) and `madsim::time::Instant` under simulation. - **Network-local monotonic reads** go through `nautilus_network::dst::time`. The crate sits below `nautilus-common` in the dependency graph and exposes a local re-export module with the same semantics. - **Hash iteration order** in the reconciliation manager and the order matching engine uses `IndexMap` and `IndexSet` rather than `AHashMap` and `AHashSet`. `AHash` randomizes its hasher per process; insertion-order iteration is needed where order drives downstream event publication or the sequence in which a seeded `FillModel` RNG is consumed. - **`tokio::select!` polling order** uses the `biased;` modifier at every production site on the DST path. Unbiased `select!` polls branches in an order chosen by an unintercepted RNG. ## Determinism contract Under the conditions below, a run identified by `(seed, binary hash, configuration hash)` on the same platform produces bitwise-identical: 1. Scheduling order of async tasks. 2. Timer firings (virtual monotonic and virtual wall-clock). 3. RNG output from `madsim::rand`. 4. Channel delivery order on `tokio` primitives. ### Required conditions The contract holds only when all of the following are true: 1. The `simulation` Cargo feature is active and `RUSTFLAGS="--cfg madsim"` is set. Both are required. The feature activates the deterministic runtime; the cfg flag activates `madsim`'s libc-level `clock_gettime` and `getrandom` intercepts. One without the other silently falls back to real `tokio` and breaks determinism without an error. 2. Every `tokio::select!` call site on the DST path uses the `biased;` modifier. 3. Monotonic time reads route through the DST seam (either `nautilus_common::live::dst::time` or `nautilus_network::dst::time`), not `std::time::Instant::now` directly. 4. Wall-clock time reads route through `nautilus_core::time::duration_since_unix_epoch`. 5. Randomness routes through `madsim::rand`. `rand::thread_rng`, `rand::rng()`, `fastrand`, `getrandom`, and `OsRng` are not intercepted. 6. Iteration-order-sensitive collections use `IndexMap` or `IndexSet`, not `AHashMap` or `AHashSet`. 7. `tokio::task::LocalSet` construction is cfg-gated out under simulation. `madsim` does not provide `LocalSet`; `spawn_local` works without it. 8. `tokio::task::spawn_blocking` call sites are cfg-gated or removed. A blocking call escapes the deterministic scheduler. ## Static enforcement Static enforcement has two layers: - Clippy policy in `clippy.toml` and `[workspace.lints.clippy]` blocks APIs that are invalid across the workspace DST contract: direct `getrandom::{fill,u32,u64}` calls and `tokio::task::LocalSet`. - A pre-commit hook named `check-dst-conventions` enforces scoped, path-aware, and cfg-aware structural checks that Clippy cannot express cleanly. The hook lives at `.pre-commit-hooks/check_dst_conventions.sh`, runs as part of the standard pre-commit suite, and runs in continuous integration. It covers the 16 in-scope workspace crates and fails the commit when it detects any of: - Raw `std::time::Instant::now()`, `SystemTime::now()`, or `chrono::Utc::now()` reads, including bare forms when the enclosing file imports the type from `std::time`, or from `chrono` for `Utc`. - Raw RNG usage (`rand::thread_rng`, `rand::rng()`, `fastrand::`, `getrandom::`, `OsRng`) or `Uuid::new_v4()` without cfg gating. - `tokio::select!` blocks missing `biased;` within the first three lines. - `std::thread::spawn`, `std::thread::Builder::new`, or `tokio::task::spawn_blocking` calls that lack a preceding `#[cfg(test)]`, `#[cfg(not(madsim))]`, or `#[cfg(not(all(feature = "simulation", madsim)))]` attribute. - `AHashMap` or `AHashSet` in iteration-order-sensitive files on the DST path. The full set of files is under audit; enforcement currently covers `crates/live/src/manager.rs` and `crates/execution/src/matching_engine/engine.rs`, and expands as further files are reviewed. - Direct `tokio::net::TcpStream::connect` / `tokio::net::TcpListener::bind` reaches that bypass `nautilus_network::net`. The seam re-exports `tokio::net` types under normal builds and swaps to `turmoil::net` under the `turmoil` feature, so all TCP entry points share a single cfg-gated swap point. The hook supports two exception forms: - An inline `// dst-ok` marker on a specific line, typically accompanied by a short reason (for example, log-only wall-clock timing that does not affect state). - A small file-level allowlist in the hook script itself for sites classified as leave-alone in the codebase audit (log timing in the cache module, progress reporting in the DeFi module). Test files, files under `tests/`, `python/`, and `ffi/` directories, and lines inside an inline `#[cfg(test)]` module are excluded because they are not part of the DST path. ### In-scope crates The hook applies to the 16 workspace crates in the transitive closure of `nautilus-live`: - `analysis`, `common`, `core`, `cryptography`, `data`, `execution`, `indicators`, `live`, `model`, `network`, `persistence`, `portfolio`, `risk`, `serialization`, `system`, `trading`. Adapter crates and infrastructure crates (Redis, Postgres) are out of scope. Their DST suitability requires a separate audit before they enter the DST path. ## Network seed soaks The `nautilus-network` Turmoil tests use two layers: - Fixed-seed tests run in the nightly test suite. These cover connect, reconnect, partition, close during reconnect, close during backoff, and repeated server-drop scenarios with reproducible seeds. - An ignored reconnect soak sweeps Turmoil seeds until stopped, or until `NAUTILUS_TURMOIL_SOAK_COUNT` seeds have run. Each seed runs the Tungstenite WebSocket backend first and the Sockudo backend second when `transport-sockudo` is enabled, so both backends see the same schedule search path. Run the continuous soak with: ```bash scripts/soak-network-turmoil.sh ``` Run a bounded soak with: ```bash env NAUTILUS_TURMOIL_SOAK_COUNT=100 scripts/soak-network-turmoil.sh ``` The soak uses deterministic seed sweep, random node order, randomized link latency, repeated server-side drops, reconnect state cycling, and exact application-message order checks. It does not enable Turmoil `fail_rate`: for TCP, that breaks links without a retransmit model, which would overstate the client delivery contract for an order-preservation test. These Turmoil tests run against the simulated network and are not gated to Linux. Several real localhost socket and WebSocket unit tests use `target_os = "linux"` for CI stability, so macOS local runs do not exercise that host TCP coverage. Use macOS for the Turmoil seed sweep and use Linux CI, or a Linux workstation, before treating the full network test set as covered. ## Implementation notes Concrete changes the DST audit produced in this repository. Use this as the starting point when investigating whether a code path is on the DST path and how it routes today. ### Iteration-order seams Production sites where `AHashMap` / `AHashSet` flipped to `IndexMap` / `IndexSet` because the iteration order is observable on the DST path: - **Matching engine** (`crates/execution/src/matching_engine/engine.rs`): nine fields (`execution_bar_types`, `execution_bar_deltas`, `account_ids`, `cached_filled_qty`, `bid_consumption`, `ask_consumption`, `queue_ahead`, `queue_excess`, `queue_pending`). Iterated removes use `.shift_remove()`. Closes [#3914](https://github.com/nautechsystems/nautilus_trader/issues/3914). - **Reconciliation manager** (`crates/live/src/manager.rs`): hook-enforced, plus `ReconciliationResult.orders` and `ReconciliationResult.fills`. - **Account trait** (`crates/model/src/accounts/`): `balances`, `balances_total`, `balances_free`, `balances_locked`, `starting_balances` returns. Storage fields on `BaseAccount` and `MarginAccount` are `IndexMap`. - **Position events** (`crates/model/src/position.rs`): `Position::commissions` flipped to `IndexMap` (consumed via `.values()` in `events/position/snapshot.rs`). - **Portfolio aggregation** (`crates/portfolio/src/portfolio.rs`): `unrealized_pnls`, `realized_pnls`, `net_positions` storage; `accumulate_mark_values` builds `IndexMap`. - **Data engine** (`crates/data/src/engine/`): `book_snapshot_counts`, `bar_aggregators`, `BookSnapshotInfos`. Iterated removes use `.shift_remove()`. - **Execution engine** (`crates/execution/src/engine/`): `ExecutionEngine.clients`, plus the `client_ids` / `venues` accumulators in `get_clients_for_orders()`. - **Trading algorithm** (`crates/trading/src/algorithm/core.rs`): `strategy_event_handlers` (drives ordered `msgbus::unsubscribe_*` fan-out). - **Analyzer** (`crates/analysis/src/analyzer.rs`): `account_balances`, `account_balances_starting`. - **Cache API** (`crates/common/src/cache/mod.rs`): `get_orders_for_ids` and `get_positions_for_ids` sort their `Vec` returns by `client_order_id` / `position_id` before returning. Storage stays on `AHashSet` (set semantics). Remaining `AHashMap` / `AHashSet` sites in the in-scope crates are lookup-only, behind concurrent shared-ownership wrappers (`Arc`, `AtomicMap`), or feed into commutative aggregation. Any new in-scope site that drives observable iteration order is a regression that the per-area audit guards against. ### Time seams `Instant::now` / `SystemTime::now` call sites that remain on the DST path are either inside `#[cfg(test)]`, file-allowlisted in the hook, or carry an inline `// dst-ok` marker with a reason: - `crates/common/src/testing.rs:81,108` `wait_until` / `wait_until_async` - `crates/execution/src/engine/mod.rs:822,847` init log timing - `crates/common/src/cache/mod.rs:569,904,3895` log and audit timing (file-allowlisted) - `crates/model/src/defi/reporting.rs:59,123` progress logging (file-allowlisted) - `crates/core/src/time.rs` seam definition site (file-allowlisted) `chrono::Utc::now` is hook-banned in the in-scope crates. The remaining call sites are the logging bridge and writer (scoped out under "Logging runs on real OS threads"). The `crates/core/src/datetime.rs::is_within_last_24_hours` helper used to reach `chrono::Utc::now` from non-logging paths; it now routes through `nautilus_core::time::nanos_since_unix_epoch()` and compares in `u64` nanos directly. ### Randomness seams Production RNG sites on the DST path: - `crates/core/src/uuid.rs::UUID4::new()` routes through `madsim::rand::thread_rng()` when called inside a madsim runtime under simulation, falling back to `rand::rng()` outside one (and on normal builds). Production paths under simulation always run inside a runtime, so they consume seeded bytes; plain `#[rstest]` tests under `cfg(madsim)` use the host RNG. Reachable from order and event factories in `nautilus-common` and `nautilus-risk`. - `crates/execution/src/models/fill.rs::default_std_rng()` routes the same way. Called from `ProbabilisticFillState::new()` when no seed is provided. With a seed, `StdRng::seed_from_u64` is deterministic by construction. - `crates/execution/src/matching_engine/ids_generator.rs:167,179` uses `nautilus_core::UUID4::new()` for the `use_random_ids` path. The default ID scheme (`{venue}-{raw_id}-{count}`) is deterministic without it. Allowed-with-marker: `crates/network/src/backoff.rs:105` for reconnect jitter, `// dst-ok` (transport layer). ### Tokio submodule split `madsim` aliases `time`, `task`, `runtime`, and `signal`. Other tokio submodules (`sync`, `io`, `select!`, `fs`, `net`) stay on real tokio under simulation. Extending the swap further would require rebuilding `tokio-tungstenite`, `tokio-rustls`, and `reqwest` against shimmed `tokio::net::TcpStream`, which the audit ruled out as too invasive. In-scope sites that touch real `tokio::net` / `tokio::io` directly: - `crates/network/src/net.rs:37` re-exports `tokio::net::{TcpListener, TcpStream}` - `crates/network/src/socket/client.rs:46,356` `tokio::io::{AsyncReadExt, AsyncWriteExt}` - `crates/network/src/tls.rs:22` `tokio::io::{AsyncRead, AsyncWrite}` - `crates/network/src/websocket/types.rs:26,29` aliases `MaybeTlsStream` These run on real sockets even under simulation. Channel delivery order on `tokio::sync` stays deterministic because the sender and receiver tasks are scheduled by the madsim executor even though the channel implementation is real. ### Raw thread escape rules Rule 4 of the hook bans raw thread spawning outside three escape cases: - `#[cfg(test)]` test modules. - `#[cfg(not(madsim))]` or `#[cfg(not(all(feature = "simulation", madsim)))]` production sites (e.g. the logging writer thread). - An inline `// dst-ok` marker. `tokio::task::LocalSet` and `tokio::task::spawn_blocking` are not supported under `madsim`. The codebase audit found no production sites for either inside the in-scope crates; new sites must carry a cfg gate or `// dst-ok` marker. ### Logging tests under simulation The logging writer thread is cfg-gated out under simulation; under `cfg(madsim)` log events are dropped. Tests that init the file-logging writer would either hang or assert against an empty log file, so the affected submodules are gated out at the module boundary: - `crates/common/src/logging/logger.rs::tests::serial_tests` (eight tests). - `crates/common/src/logging/macros.rs::tests` (two tests). `logger.rs::tests::sim_tests::test_init_under_madsim_skips_writer_thread_and_forces_bypass` runs under simulation and pins the gated behaviour. ## Scope boundaries The contract is deliberately narrow. The following weakenings are explicit, not oversights. ### Python is not in DST scope DST runs under a native Rust test harness. No Python interpreter starts during a DST run. The PyO3 bindings under `crates/*/src/python/`, the `ffi/` directories, and the Python packages under `nautilus_trader/` are excluded from the contract as a policy, not as a weakness. Any code reachable only from Python call paths is out of scope; any Rust path reachable from the native DST harness must satisfy the contract even if the same type is also exported to Python. The `check-dst-conventions` hook encodes this policy by skipping `/python/` and `/ffi/` paths in the in-scope crates. Clock, RNG, and threading call sites behind those paths do not apply to the contract. The primary objective of DST is reliability of the Rust engine itself: the order lifecycle, reconciliation, matching, risk, and execution state machines. Deterministic replay of user strategies is a later, secondary goal that becomes available once strategies are authored in Rust or run through a Rust-native test harness. In the meantime, a Python strategy that calls `time.time()`, issues arbitrary network requests, or relies on thread scheduling can vary its command stream between runs; the Rust core will process the varying stream deterministically, but end-to-end replay from a Python entry point is not guaranteed. ### Platform-scoped `madsim`'s libc overrides for `clock_gettime` and `getrandom` are platform-specific. Cross-platform bitwise reproducibility is not claimed. A seed that reproduces a failure on Linux x86_64 may not reproduce on macOS aarch64. ### Non-aliased dependencies escape silently Any dependency that reaches the OS through a non-aliased path (direct `libc` calls, `std::net` bypass, crates using `fastrand` or `OsRng`) escapes the simulator without raising an error. The in-scope crates have been audited; adapter crates and infrastructure crates require their own audits before entering the DST path. ### Transport-layer I/O is not simulated `tokio-tungstenite`, `tokio-rustls`, `reqwest`, `redis`, and `sqlx` use real `tokio` internally. Under simulation, WebSocket and HTTP I/O run on real networking. This is intentional: the initial target is order lifecycle determinism, not transport fault injection. Transport-layer determinism would require per-crate `madsim` shims that do not exist at this time. Test modules that drive real localhost sockets (`crates/network/src/socket/client.rs::tests`, `::rust_tests`; `crates/network/src/websocket/client.rs::tests`, `::rust_tests`; `crates/network/tests/websocket_proxy.rs`) are cfg-gated out under `all(feature = "simulation", madsim)` because their production code paths reach `dst::time::*` (madsim time primitives), which panic when called from a `#[tokio::test]` runtime. The retry test modules (`crates/network/src/retry.rs::tests`, `::proptest_tests`) run under simulation: each test attribute is `cfg_attr`-swapped between `#[tokio::test(start_paused = true)]` and `#[madsim::test]`, time reads and sleeps route through `crate::dst::time`, and explicit virtual-time advances go through a `cfg`-gated `advance_clock` helper so the same body covers both runtimes. ### Signal handling `nautilus_common::live::dst::signal` exposes a routed `ctrl_c` re-export. The `crates/live/src/node.rs` run loop routes through it, so node shutdown driven by `ctrl_c` is injectable from test code under `cfg(madsim)` via `madsim::runtime::Handle::send_ctrl_c`. Adapter-bin entry points still call `tokio::signal::ctrl_c` directly and remain scoped out. ### Logging runs on real OS threads The logging subsystem spawns a writer thread via `std::thread::Builder` and uses `std::sync::mpsc`. Under simulation, the thread is not spawned and log events are dropped. Log output is outside the determinism contract: the writer only writes, never reads or mutates simulation state. ### Adapters Adapter crates are out of scope for the initial DST contract. Each adapter has its own set of `chrono::Utc::now`, `SystemTime::now`, `Uuid::new_v4`, and transport-layer call sites. An adapter that enters the DST path must be audited for direct clock, RNG, and transport usage before its behavior can be covered by the contract. ### Process-global lazy state consumes RNG bytes on first call The contract holds within a single run-of-the-runtime. A few process-global lazy initialisations consume RNG bytes on first call, which matters for harnesses that run two seeded executions inside one process and compare traces. - The `Ustr::from()` interner allocates and seeds its internal map on first use. - `ahash::RandomState` seeds itself via `getrandom` (which `madsim` hooks under `cfg(madsim)`) on first instance creation. Single-runtime tests (one body invocation per seed, fresh process) are unaffected: the consumption is part of the seeded execution and reproduces deterministically. Same-seed equality frameworks that invoke the body twice in one process to diff the traces will see drift between runs. Run 1 pays the lazy-init cost and consumes RNG bytes; run 2 inherits the warm state and starts at a different offset in the RNG sequence. Workaround: pre-warm process-global state outside the runtime before the comparison, for example by calling `Ustr::from("")` and constructing one `ahash::RandomState` once at process start. Both runs then start with the warmed state and consume the RNG sequence identically. ### Adapter factories no longer expose `Rc>` Commit `f0ea66da15` ("Standardize adapter cache access via `CacheView`") replaced the mutable `Rc>` parameter on `DataClientFactory::create` and `ExecutionClientFactory::create` with a `CacheView`. `CacheView` exposes `borrow()` for read access but does not expose the inner `Rc` handle. This blocks DST-style harness factories that need to construct an `OrderMatchingEngine` inline inside the factory, because `OrderMatchingEngine::new` still takes `Rc>` and there is no public accessor to recover that handle from a `CacheView`. Workarounds: - Pin the harness consumer to a commit before `f0ea66da15`. - Refactor the harness so it builds the `OrderMatchingEngine` outside the factory (where the kernel `Cache` handle is still reachable) and passes the constructed engine into the client. A longer-term escape hatch would be either an accessor on `CacheView` for the inner handle or an alternative `OrderMatchingEngine` constructor that accepts a `CacheView`. Neither is in the tree today. ## Relationship to other testing layers DST complements existing testing; it does not replace any of it. | Layer | Covers | DST relationship | |-------------------------|------------------------------------------------------|-------------------------------------------------| | Unit tests | Pure logic, calculations, parsers, transformers. | Unchanged. | | Integration tests | Component interaction, I/O boundaries. | Unchanged. DST runs alongside, not in place of. | | Property‑based tests | Invariants over input domains (parsers, roundtrips). | Unchanged. | | Acceptance tests | End‑to‑end backtest and live scenarios. | Unchanged. | | Deterministic sim (DST) | Async timing, scheduling, recovery correctness. | Adds seed‑replayable exploration. | DST's unique value is in the intersection of async concurrency and state-machine correctness. Bugs such as "a message at shutdown is dropped under a specific wakeup ordering" or "a reconciliation event is lost when iteration order reverses" are the target class. For anything else, the pre-existing test layers are the right tool. ## Status As of the current state of this repository: - Layer 1 (runtime swap) is implemented. `nautilus_common::live::dst` exposes routed re-exports for `time`, `task`, `runtime`, and `signal`. Production call sites for `time`, `task`, and `runtime` route through the seam; signal call-site adoption is partial (see "Signal handling" under "Scope boundaries"). - Layer 2 (nondeterminism substitution) is implemented across the 16 in-scope crates. Seams exist for wall-clock time, monotonic time, randomness, and iteration order. The audit closures and remaining allowed call sites are enumerated under "Implementation notes". - Static enforcement via `check-dst-conventions` is active in pre-commit and CI. The hook covers the load-bearing conditions; the `// dst-ok` marker convention permits per-line exceptions when justified. - Build-and-test smoke gate under `cfg(madsim)` runs via the `dst` workflow (`.github/workflows/dst.yml`, invokes `make cargo-test-sim`). It compiles the in-scope crates with `--features simulation` and runs every test that is sim-compatible today. Crates that consume `nautilus-model` types (`nautilus-common`, `nautilus-execution`) also run a second leg with `--features "simulation,high-precision"` so the seam-routed code paths are exercised under both fixed-point widths (`QuantityRaw` / `PriceRaw` as `u64` vs `u128`). - All of `nautilus-common`. This leg compiles with `nautilus-core/simulation` propagated, so the explicit `wall_clock_now` cfg branch is selected for every test in the suite. Plain `#[rstest]` tests run outside a madsim runtime and route through the seam's `SystemTime::now()` fallback (the same path madsim's libc shim takes outside a runtime). The `live::dst::tests::test_dst_wall_clock_advances_with_virtual_time` test in this leg uses `#[madsim::test]` and asserts that `nanos_since_unix_epoch` advances with `madsim::time::sleep`, so virtual wall-clock behavior is validated end-to-end on the common leg. - All of `nautilus-network` (transport-bound test modules are gated out at the source). Includes the seam pinning tests for sleep / timeout virtual time and the rate-limiter, plus the retry suites that exercise backoff timing under virtual time. - All of `nautilus-execution`. The matching engine, fill model, and execution-engine state machines run under the deterministic scheduler with the seeded RNG. - The cross-crate seam pinning tests in `nautilus-core` (`wall_clock_now` virtual time). Each leg runs with its own crate's `--features simulation` and uses `#[madsim::test]` where applicable, so the explicit cfg branches and virtual time are both validated. Together this catches drift in the cfg-gated DST seams and exercises the in-scope state machines under the deterministic scheduler; it does not yet exercise determinism end-to-end. - End-to-end runtime verification (same-seed diff over an in-scope code path) is out of scope for this repository. The structural conditions (Rule 1 to Rule 6) are enforced; the claim that a seed reproduces identical observable behavior across runs is plausible from the seam design but is not yet verified by a regression gate. ## Further reading - `.pre-commit-hooks/check_dst_conventions.sh` defines the six enforcement rules in full and documents the `// dst-ok` marker convention. - External references: the [FoundationDB testing philosophy](https://apple.github.io/foundationdb/testing.html), the [TigerBeetle simulation testing blog posts](https://tigerbeetle.com/blog/), and the [madsim repository](https://github.com/madsim-rs/madsim) for the deterministic runtime. # Event Sourcing Source: https://nautilustrader.io/docs/latest/concepts/event_sourcing/ Event sourcing gives NautilusTrader a durable, ordered record of the messages that change engine state. The event store records those messages at the system boundary, then readers, replay tools, and verifiers use the same log to reconstruct what happened and to rebuild state. **The core philosophy**: - The event store is the durable authority for state-affecting history. - The cache is a write-through projection, not the source of truth. - Cache replay rebuilds state by applying captured history to cache-owned state. - Market data stays in the data catalog; the event store records the messages that affect state. - External I/O becomes replayable only when Nautilus captures it as commands, raw reports, or other state-affecting inputs. :::note Event-store capture, replay, verification, recovery, and retention planning have targeted test coverage, but the API surface is still evolving. Treat the concepts here as the design contract for current development, and use the crate README for current API details. ::: ## Why event sourcing The cache answers "what is true now." The event store answers "how did Nautilus get here." It gives readers, replay tools, and verifiers a run-scoped history that does not require strategy logic, venue queries, or the live cache to explain past state. The event store provides Nautilus with a durable basis to: - Prove whether a sealed run is clean before replay or archive. - Inspect the exact command, report, and event sequence behind an order or component intent. - Rebuild cache state from captured history, including a snapshot anchor plus the run tail. - Trace an intent through the engine-side messages that followed from it. - Seal stale run files before the next run starts after a process exit or writer halt. ## Terms - Run: one kernel session for one instance, binary, and config. - Entry: one captured message plus replay metadata. - `seq`: the per-run sequence assigned by the writer and used as replay order. - High-watermark: the largest `seq` durably acknowledged by the backend. - Snapshot anchor: the high-watermark recorded with a cache snapshot. - Headers: correlation and causation metadata propagated with captured messages. ## What the store records The event store records state-affecting message bus traffic for one trading instance and one run. A run starts when the kernel starts and ends when the process stops cleanly or crashes. **Captured entries include**: - Execution commands such as submit, modify, and cancel. - Data subscription commands that define the actor or strategy observation window. - Fired time events and generated order, position, and account events. - Raw venue execution reports before reconciliation synthesizes derived events. - Reconciliation outputs produced from those raw reports. - Request and response messages, or their audit-relevant metadata, that cross the bus and affect state. - Run lifecycle entries such as `RunStarted` and `RunEnded`. The store does not replace the data catalog. Market-data observations remain in the Feather streaming catalog. The event store records the command stream, raw reports, generated events, and metadata needed to replay how the engine reacted to that world. ## Boundaries The event store is intentionally narrow: - It does not replace the data catalog. - It does not provide analytics or OLAP queries. - It does not aggregate multiple trader instances into a consensus log. - It does not yet define redaction, encryption-at-rest, or tamper evidence. ## Capture flow Capture happens at the message bus dispatch boundary, before downstream handlers observe the message. That placement matters because any handler that can mutate state must see only messages that the event store has already accepted. ```mermaid flowchart LR Producer["Engine, adapter, strategy, or component"] --> Bus["MessageBus publish/send"] Bus --> Tap["Capture tap"] Tap --> Adapter["BusCaptureAdapter"] Adapter --> Writer["EventStoreWriter"] Writer --> Backend["redb run file"] Bus --> Handlers["Downstream handlers"] Backend --> Reader["Reader, replay, verifier"] ``` **The operational steps are**: - The producer publishes or sends a state-affecting message. - The bus capture tap builds an event-store entry before downstream handlers run. - The writer assigns the next `seq`, writes a batch, and advances the high-watermark after the backend acknowledges durability. - Handlers run after the captured entry has reached the writer boundary. - Readers scan sealed or running backends without exposing append operations. The writer uses a bounded channel. If the writer stalls past its configured threshold, Nautilus halts instead of dropping entries or allowing unaudited state changes. Some messages legitimately cross more than one tap-visible boundary: the execution engine sends an order event to the portfolio endpoint and publishes the same event on its strategy topic, and trading commands hop from strategy to risk to execution. The capture adapter dedups on the registered message identity (event id, command id), so each logical message becomes exactly one entry and replay never applies the same event twice. ## Lifecycle options `EventStoreConfig` remains the serializable run policy. Process-local construction policy lives in `EventStoreLifecycleOptions`, which advanced callers pass through `EventStoreLifecycle::boot_with_options(...)`. By default, the lifecycle opens `RedbBackend` and installs the default encoder registry. Callers can use lifecycle options to: - Supply a custom encoder registry before the bus tap starts capture. - Supply a backend opener that returns any `EventStore` implementation for the new run. The backend opener is the simulation-safe path for memory capture. A DST harness or focused test can open `MemoryBackend` through the normal lifecycle, keep the same bus tap and writer semantics, and read the captured entries in-process after seal. Under `cfg(madsim)`, the writer commits each submit synchronously, so the captured `seq` order is deterministic. With a `MemoryBackend` opener, capture needs no `redb` run file. ## Entry model Each event-store entry is one captured message plus metadata: - `seq`: the per-run replay-order authority. - `ts_init`: the domain timestamp on the captured message. - `ts_publish`: the bus-accepted time when that ordering detail matters. - `topic`: the bus topic or logical endpoint. - `payload_type`: the encoded message type. - `payload`: the encoded message bytes. - `headers`: correlation and causation metadata. - `entry_hash`: the canonical hash over the entry content. `seq` orders replay. Timestamps help explain the run, but they do not override `seq`. The current secondary indices support lookup by `client_order_id` and `venue_order_id`. A `correlation_id` index can be added when a concrete inspection caller needs that lookup pattern; until then, correlation scans can walk the captured stream. ## Correlation model Nautilus records three identity levels so readers can answer scope, lineage, and message identity questions. - `correlation_id`: the logical workflow or chain. A component `intent_id` is recorded in this field at the dispatch boundary. - `causation_id`: the direct parent message that caused this message. - `command_id`, `event_id`, or `report_id`: the identity of this specific message. ```mermaid flowchart TD Intent["Component intent_id"] --> Correlation["correlation_id"] Command["SubmitOrder command_id"] --> Event["OrderAccepted event_id"] Event --> Fill["OrderFilled event_id"] Correlation --> Command Correlation --> Event Correlation --> Fill Command -. "causation_id" .-> Event Event -. "causation_id" .-> Fill ``` This lets operators ask two common questions: - "Show everything in this workflow": filter or scan by `correlation_id`. - "Show why this event happened": walk `causation_id` back to the direct parent. ## Run files and manifests The default backend is `redb`. It stores one file per run under: ```text //.redb ``` Each run file contains: - Entries keyed by `seq`. - Secondary indices for order identifiers. - A manifest written at run start and sealed at run end. - An optional snapshot anchor for cache restore. The manifest records the run identity and reproducibility inputs: - Run identity: - `run_id` - `parent_run_id` - `instance_id` - Build identity: - `binary_hash` - `crate_versions` - `feature_flags` - Adapter versions - Configuration identity: - `config_hash` - Registered components - Optional seed - Lifecycle state: - `start_ts_init` - `end_ts_init` - `high_watermark` - Status Run status is one of `Running`, `Ended`, `CrashedRecovered`, or `Quarantined`. ## Run lifecycle ```mermaid flowchart TD Start["RunStarted entry"] --> Running["Running manifest"] Running --> Capture["Capture state-affecting entries"] Capture --> Anchor["Record optional snapshot anchors"] Anchor --> Capture Capture --> RunEnded["RunEnded entry"] RunEnded --> Ended["Ended manifest"] ``` Operationally: - `RunStarted` is the first entry of a fresh run. A repeated `open()` in the same process seals the current session before it starts a new run. - While the manifest is `Running`, the bus tap records state-affecting entries and cache snapshots can record anchors against the durable high-watermark. - A clean shutdown, kernel drop, or reset/rerun seal appends `RunEnded` and seals the manifest as `Ended`. - A fail-stopped (halted) session skips the in-process seal; the recovery sweep on the next boot owns it. The halt signal is scoped to the run that fired it: a later `open()` re-arms a fresh signal, so one halt does not poison subsequent runs in the same process. ## Recovery sealing A predecessor is an older run file for the same instance whose manifest still says `Running`. This means the previous process did not finish the normal lifecycle, or the writer halted before the manifest seal completed. ```mermaid flowchart TD Predecessor["Running predecessor"] --> Scan["Scan durable tail"] Scan --> Empty["No durable entries"] Empty --> Recovered["Seal as CrashedRecovered"] Scan --> TailEnded["Tail contains RunEnded"] TailEnded --> Ended["Seal as Ended"] Scan --> CleanTail["Clean tail without RunEnded"] CleanTail --> Recovered Scan --> BadTail["Hash, gap, or structural failure"] BadTail --> Quarantined["Seal as Quarantined"] Recovered --> Parent["Eligible parent_run_id"] Ended --> NoParent["No parent link"] Quarantined --> NoParent ``` Boot recovery scans each `Running` predecessor and chooses a final manifest status from the durable tail. A clean tail without `RunEnded` seals as `CrashedRecovered`, a tail ending in `RunEnded` seals as `Ended`, and a hash mismatch, gap, or structural corruption seals as `Quarantined`. The sweep never leaves the trader unbootable because one run file is damaged. A hard-killed process (SIGKILL, OOM kill, power loss) leaves a file that redb refuses to open read-only; the listing falls back to a writable open, which performs redb's repair pass before recovery proceeds. A file that still cannot be opened, or that lacks a manifest, is skipped with a logged error and retried on the next boot, so recovery and retention continue over the healthy runs. Only `CrashedRecovered` predecessors become `parent_run_id`. A configured `replay_from_run_id` overrides a recovered parent after validation. The read-only verifier is separate: it can inspect a sealed run without mutating it and reports `quarantine=not-performed`. ## Replay modes Replay follows one ordering rule: apply event-store entries in `seq` order. `ts_init` and `ts_publish` explain when messages happened, but `seq` is the durable replay order. The Rust replay-input API keeps planning separate from execution: - Event-store-only replay inputs return entries only. - Catalog-joined replay inputs add caller-selected catalog slices for context analysis. Catalog planners take explicit `CatalogSliceSelector` values and a read-only `ReplayCatalog`. Planning resolves catalog time bounds from the event-store scan unless the selector supplies explicit bounds, reports missing catalog slices, and preserves `seq` as the entry ordering authority. Loading returns `ReplayInputs`: event-store entries in `seq` order plus catalog records grouped under their selected slice. Rust callers can enable the off-by-default `persistence` feature and wrap a `ParquetDataCatalog` with `nautilus_event_store::ParquetReplayCatalog` to plan selected catalog files and filename-derived intervals. The bridge can load `quotes`, `trades`, and `bars` into typed `CatalogReplayRecord` values. :::note The persistence bridge is read-only: it uses catalog discovery and query APIs but **does not write to the catalog**. Unsupported catalog classes fail loading until replay adds a typed payload contract for that class. ::: ## Data marker sidecar :::note The marker sidecar shipped in `crates/event_store/src/markers/`. It is opt-in via `EventStoreConfig.data_markers` (`crates/system/src/event_store.rs`) and stays off by default. ::: Exact data delivery order is not inferred from catalog timestamps. The marker sidecar records data observed at the message-bus dispatch boundary, beside the event-store run, without writing full market-data payloads into `EventStoreEntry` rows. The sidecar supports one audit claim: when marker capture is enabled, Nautilus observed data delivery in `marker_seq` order at the bus boundary for that run, and each marker carries enough identity to join back to candidate catalog rows. It cannot prove that catalog timestamps alone define bus order, reconstruct a data point when the catalog row is absent or changed, prove venue send order before Nautilus observed the message, or say anything about runs where marker capture was disabled. Markers do not consume event-store `seq` values and do not create gaps in the entry table. Each marker has its own monotonically increasing `marker_seq` plus `event_seq_before`, the largest event-store `seq` assigned before the marker was observed. A sealed-run analyzer can derive the next event-store entry after a marker from `event_seq_before + 1`; markers that share the same `event_seq_before` are ordered by `marker_seq`. Event-store `seq` remains the replay-order authority for state-affecting entries. The sidecar has two marker kinds: - **Cursor snapshots** (`DataCursorSnapshot`): the default capture mode. Each snapshot records `marker_seq`, `event_seq_before`, `ts_init`, and the `StreamCursor` entries that advanced since the previous snapshot. A `StreamCursor` carries the stream `slot`, the highest `ts_init` seen in that slot (`ts_init_hi`), and the record `count`. A `StreamDictEntry` maps each `slot` to its `data_cls` (`BookDeltas`, `BookDepth10`, `Quote`, `Trade`, `Bar`) and instrument `identifier`. - **High-fidelity markers** (`HiFiMarker`): opt-in per instrument via `DataMarkerConfig.high_fidelity`. Each records `marker_seq`, `event_seq_before`, `slot`, `ts_event`, `ts_init`, `same_ts_ordinal`, and a 32-byte `record_fingerprint` over the canonical typed row fields. `same_ts_ordinal` and `record_fingerprint` disambiguate duplicate same-timestamp data without storing prices, quantities, sizes, or MessagePack payloads. If two catalog rows are byte-identical for the same key and timestamp, the sidecar can prove that Nautilus observed two deliveries in a specific marker order; it cannot name a unique physical catalog row after catalog compaction rewrites row order. The stable contract is the marker schema, opt-in capture and reader primitives, gap-free marker verification, and catalog join rules. Analysis tools can build on that contract to select windows, interpret venue-specific data, rank or cluster markers, present reports, and package run bundles. With marker capture disabled, no data marker writer is installed. Cache replay and live restart do not read this sidecar: snapshot-tail replay still applies event-store entries in `seq` order, and live restart still boots from cache-owned state plus the event-store parent link. These APIs **do not**: - Open live venue clients - Run strategies or actors - Re-run reconciliation - Delete files - Replay the clock registration/cancel lifecycle Kernel-managed replay uses `EventStoreConfig::replay_from_run_id`. When set, the kernel restores cache state from the sealed run, records that run as the parent of the fresh child run, and skips live engines, clients, startup, and venue reconciliation. The cache replay loader is state-only. It restores the cache-owned snapshot, scans the event-store tail in `seq` order, decodes supported cache-affecting payloads, and applies them directly to `Cache`. Supported payloads include: - Synthesized account, order, and position events - Captured order lists - Complete data responses for instruments, quotes, trades, funding rates, and bars It **does not**: - Publish replayed entries to the live message bus - Run strategy or actor code - Query venues - Run reconciliation - Derive identifiers again - Re-arm clocks Fired `TimeEvent`s and raw venue reports are inspection records on this path; replay applies the synthesized order, position, and account events captured later in the run. ## Snapshot-anchored recovery Cache snapshots are owned by the cache. The event store stores only the snapshot anchor: the high-watermark at snapshot time plus a content-addressed reference to the snapshot blob. ```mermaid sequenceDiagram participant Cache participant Store as Event store participant Replay Cache->>Store: Record snapshot anchor at high-watermark N Replay->>Store: Read manifest and latest anchor Replay->>Cache: Load snapshot blob from anchor Replay->>Store: Scan entries with seq > N Replay->>Replay: Apply tail in seq order ``` Recovery cases are ordered by how far the message progressed: - Before enqueue: the message never reached the writer, so producer retry policy applies. - After enqueue, before commit: the in-flight batch is not durable, so the high-watermark does not advance. - After commit, before snapshot anchor: recovery loads the prior snapshot and replays the tail. - After snapshot anchor: recovery loads the latest snapshot and replays entries after the anchor. :::info Live restart still uses snapshot-plus-reconcile. Event-store recovery becomes the live restart path only after capture coverage and replay rules cover every state-affecting path. ::: Replay correctness depends on four checks: - Entries are addressed by immutable `seq` values. - Writes reject out-of-order commits. - Readers detect gaps inside the high-watermark. - Snapshot replay plans reject anchors that point past the durable high-watermark. ## Retention planning Retention uses whole run files as the reclaim unit. The event store exposes a non-destructive planner that lists sealed run manifests, inspects their latest snapshot-anchor status, and returns candidate run files for a later supervisor or operator process to reclaim. The planner supports three modes: - `Full`: keep every sealed run and return no reclaim candidates. - `Bounded { keep_last }`: keep the newest sealed runs and also keep at least one known-good restore point. - `SnapshotAnchored`: reclaim only sealed runs older than the newest known-good restore point. A known-good restore point is a sealed, non-`Quarantined` run with a valid snapshot anchor whose high-watermark does not exceed the run's durable high-watermark (the last entry actually on disk, not the manifest's recorded value, so a tail-trimmed run cannot pose as a restore point). `Running` runs are never listed as sealed runs or selected as reclaim candidates. Missing, corrupt, or invalid snapshot anchors do not count as restore points, so the planner returns no candidates when it cannot prove that at least one usable restore point remains. ## Verification coverage The event-store test suite pins the load-bearing correctness guarantees for the current alpha surface: - The default encoder registry covers the audited state-affecting capture surface. - Fired `TimeEvent`s hit the installed event-store tap through `TimeEventHandler::run`. - The writer halts under bounded backpressure instead of dropping accepted entries. - Entry hash verification detects byte-level payload corruption. - Process-isolated verification reports truncated or zero-tailed run files as corrupt. - Cache replay reconstructs the same observed account, order, and position state as a live cache for generated captured event streams. - The same order event dispatched across multiple bus boundaries is captured once. - Snapshot anchors that fail to decode or point past the durable high-watermark surface as verifier findings instead of verifying clean. - Catalog-joined replay input planning covers selected slices, missing slices, time bounds, and event-store `seq` ordering. - Crash recovery seals `Running` predecessors as `Ended`, `CrashedRecovered`, or `Quarantined` based on the durable tail, and only `CrashedRecovered` runs become parents. - Boot recovery repairs hard-crashed run files and skips unreadable ones instead of failing the sweep. ## Integrity and verification Every entry carries a canonical hash over its full content. Readers and verifiers recompute the hash and report mismatches. The verifier also checks manifest/high-watermark status, validates secondary indices against the entry table, and reports snapshot anchors that fail to decode or point past the durable high-watermark, so a run whose restore path is broken cannot verify clean. Run verification is process-isolated. This matters because some corrupted `redb` files can panic on open or first read, and release builds use `panic = "abort"`. The verifier runs the scan in a worker subprocess so a bad file aborts the worker, not the caller. Verify a sealed run file: ```bash cargo run -p nautilus-event-store --bin verify -- /path/to/run.redb ``` Clean output looks like: ```text clean run_id=1700000000-cafe0001 status=Ended high_watermark=3 entries_scanned=3 ``` Corrupt output includes `quarantine=not-performed`: ```text corrupt run_id=1700000000-cafe0001 status=Ended high_watermark=3 entries_scanned=3 findings=1 quarantine=not-performed - hash mismatch at seq 2 ``` Exit codes: - `0`: the run is clean. - `1`: the run has corrupt findings, or the worker aborted or timed out. - `2`: the verifier could not open or run against the requested file. :::note The verifier reports corruption but does not mutate run files. Quarantine is an operator or supervisor policy. ::: ## Operational use today Current alpha use is focused on local inspection and verification of run files. Verify a run after copying or restoring it: ```bash cargo run -p nautilus-event-store --bin verify -- ./event_store/trader-001/1700000000-cafe0001.redb ``` Increase the verifier timeout for a large sealed run: ```bash env NAUTILUS_EVENT_STORE_VERIFY_TIMEOUT_SECS=120 \ cargo run -p nautilus-event-store --bin verify -- ./event_store/trader-001/1700000000-cafe0001.redb ``` Read a sealed run from Rust: ```rust use nautilus_event_store::{EventStoreReader, RedbBackend, ScanDirection}; fn inspect_run() -> Result<(), Box> { let backend = RedbBackend::open_sealed_file("./event_store/trader-001/1700000000-cafe0001.redb")?; let reader = EventStoreReader::new(backend); let high_watermark = reader.high_watermark()?; for entry in reader.scan_range(1, high_watermark, ScanDirection::Forward) { let entry = entry?; println!("{} {}", entry.seq, entry.topic); } Ok(()) } ``` The verifier is read-only inspection. It reports corruption without changing the run file, so quarantine decisions remain outside this command path. ## Relationship to DST The event store and deterministic simulation testing (DST) solve different parts of replay. - The event store supplies the captured input history. - DST controls scheduling, time, seeded randomness, and other in-scope nondeterminism. - Together they let a run reproduce engine behavior inside the deterministic simulation scope when identified by: - `seed` - `binary_hash` - `config_hash` - `schema_version` - `log` Under `cfg(madsim)`, the writer commits synchronously instead of spawning its writer thread. When a simulation harness supplies a `MemoryBackend` opener through lifecycle options, capture stays in-process and does not require `redb` files. Redb remains the default durable backend outside that advanced options path. Adapter network I/O remains outside bit-identical replay unless Nautilus captures the relevant raw inputs and routes them through deterministic interfaces. # Events Source: https://nautilustrader.io/docs/latest/concepts/events/ Nautilus is event-driven: every state change in the system is represented by an event object that flows through the `MessageBus` to strategy and actor handlers. This guide covers the event types, how they are dispatched, and how order fills produce position events. ## Event categories | Category | Examples | Origin | |----------|-------------------------------------------------|---------------------------------| | Order | `OrderAccepted`, `OrderFilled`, `OrderCanceled` | `ExecutionEngine` (from venue) | | Position | `PositionOpened`, `PositionChanged` | `ExecutionEngine` (from fills) | | Account | `AccountState` | `ExecutionClient` / `Portfolio` | | Time | `TimeEvent` | `Clock` (timers and alerts) | ## Handler dispatch When an event reaches a strategy, the system calls handlers in a fixed priority order. The first matching handler runs, then the next level, so you can handle events at whatever granularity you need. ### Order events 1. Specific handler (e.g. `on_order_filled`) 2. `on_order_event` (receives all order events) 3. `on_event` (receives everything) ### Position events 1. Specific handler (e.g. `on_position_opened`) 2. `on_position_event` (receives all position events) 3. `on_event` (receives everything) ### Time events Timers and alerts produce `TimeEvent` objects. Pass a `callback` when calling `set_timer` or `set_time_alert` to direct events to your own method. If you omit the callback, the event is delivered to `on_event` instead. ## Order events Each order event corresponds to a state transition in the [order state machine](orders/index.md#order-state-flow). The `ExecutionEngine` applies the event to the order, updates the `Cache`, and publishes it on the `MessageBus`. The table below shows the primary transitions; partially filled and triggered orders support additional transitions documented in the full [order state flow](orders/index.md#order-state-flow). | Event | Primary transition | Handler | |------------------------|-------------------------------------|----------------------------| | `OrderInitialized` | (created locally) | `on_order_initialized` | | `OrderDenied` | Initialized -> Denied | `on_order_denied` | | `OrderEmulated` | Initialized -> Emulated | `on_order_emulated` | | `OrderReleased` | Emulated -> Released | `on_order_released` | | `OrderSubmitted` | Initialized/Released -> Submitted | `on_order_submitted` | | `OrderAccepted` | Submitted -> Accepted | `on_order_accepted` | | `OrderRejected` | Submitted -> Rejected | `on_order_rejected` | | `OrderTriggered` | Accepted -> Triggered | `on_order_triggered` | | `OrderPendingUpdate` | Accepted -> PendingUpdate | `on_order_pending_update` | | `OrderPendingCancel` | Accepted -> PendingCancel | `on_order_pending_cancel` | | `OrderUpdated` | PendingUpdate -> Accepted | `on_order_updated` | | `OrderModifyRejected` | PendingUpdate -> Accepted | `on_order_modify_rejected` | | `OrderCancelRejected` | PendingCancel -> Accepted | `on_order_cancel_rejected` | | `OrderCanceled` | PendingCancel/Accepted -> Canceled | `on_order_canceled` | | `OrderExpired` | Accepted -> Expired | `on_order_expired` | | `OrderFilled` | Accepted -> Filled/PartiallyFilled | `on_order_filled` | ### Common order event fields All order events share these fields: | Field | Description | |--------------------|------------------------------------------| | `trader_id` | Trader instance identifier. | | `strategy_id` | Strategy that submitted the order. | | `instrument_id` | Instrument for the order. | | `client_order_id` | Client‑assigned order identifier. | | `venue_order_id` | Venue‑assigned order identifier. | | `account_id` | Account the order belongs to. | | `reconciliation` | Whether generated during reconciliation. | | `event_id` | Unique event identifier. | | `ts_event` | Timestamp when the event occurred. | | `ts_init` | Timestamp when the event was created. | Individual events add type-specific fields (e.g. `OrderFilled` adds `last_qty`, `last_px`, `trade_id`, `commission`). See the API reference for the full field list per event type. :::tip Override `on_order_event` to handle all order events in one place. The specific handlers fire first, so you can mix both approaches. ::: ## Position events Position events are a direct consequence of fill events. The `ExecutionEngine` processes each `OrderFilled`, updates or creates a position, and emits the corresponding position event. | Event | When it fires | Handler | |---------------------|-------------------------------------------|-----------------------| | `PositionOpened` | First fill creates a new position. | `on_position_opened` | | `PositionChanged` | Subsequent fill changes quantity or side. | `on_position_changed` | | `PositionClosed` | Fill reduces quantity to zero. | `on_position_closed` | ### From fill to position: the causal chain The following diagram shows how a single `OrderFilled` event produces a position event. This is the key link between order management and position tracking. ```mermaid sequenceDiagram participant Venue as Venue participant EE as ExecutionEngine participant Cache as Cache participant Strategy as Strategy Venue-->>EE: OrderFilled EE->>EE: apply fill to order EE->>Cache: update order state EE->>EE: determine position ID alt No existing position EE->>Cache: add new Position EE->>Strategy: PositionOpened else Position open, not closed by fill EE->>Cache: update Position EE->>Strategy: PositionChanged else Fill closes the position EE->>Cache: update Position EE->>Strategy: PositionClosed end ``` **Step by step:** 1. **Fill arrives.** The `ExecutionEngine` receives an `OrderFilled` event from the venue adapter. 2. **Order state updates.** The engine applies the fill to the order object and writes the updated order to the `Cache`. 3. **Position ID resolved.** The engine determines which position this fill belongs to, based on OMS type and strategy configuration. 4. **Position created or updated.** Three outcomes: - **No position exists** for this ID: the engine creates a `Position` from the fill, adds it to the `Cache`, and emits `PositionOpened`. - **Position exists and remains open** after the fill: the engine applies the fill to the position, updates the `Cache`, and emits `PositionChanged`. - **Position exists and closes** (quantity reaches zero): the engine applies the fill, updates the `Cache`, and emits `PositionClosed`. 5. **Flip case.** When a fill reverses the position (e.g. long 10 filled sell 15), the engine splits the fill into two parts: one that closes the original position (`PositionClosed`) and one that opens the new position (`PositionOpened`). ### Position event fields | Field | Opened | Changed | Closed | Description | |----------------------|--------|---------|--------|-----------------------------------| | `trader_id` | ✓ | ✓ | ✓ | Trader instance identifier. | | `strategy_id` | ✓ | ✓ | ✓ | Strategy that owns the position. | | `instrument_id` | ✓ | ✓ | ✓ | Instrument for the position. | | `position_id` | ✓ | ✓ | ✓ | Unique position identifier. | | `account_id` | ✓ | ✓ | ✓ | Account the position belongs to. | | `opening_order_id` | ✓ | ✓ | ✓ | Order that opened the position. | | `closing_order_id` | - | - | ✓ | Order that closed the position. | | `entry` | ✓ | ✓ | ✓ | Side of the opening fill. | | `side` | ✓ | ✓ | ✓ | Current position side. | | `signed_qty` | ✓ | ✓ | ✓ | Signed quantity (negative=short). | | `quantity` | ✓ | ✓ | ✓ | Unsigned position quantity. | | `peak_qty` | - | ✓ | ✓ | Largest quantity held. | | `last_qty` | ✓ | ✓ | ✓ | Quantity of the last fill. | | `last_px` | ✓ | ✓ | ✓ | Price of the last fill. | | `currency` | ✓ | ✓ | ✓ | Settlement currency. | | `avg_px_open` | ✓ | ✓ | ✓ | Average entry price. | | `avg_px_close` | - | ✓ | ✓ | Average exit price. | | `realized_return` | - | ✓ | ✓ | Realized return as a ratio. | | `realized_pnl` | - | ✓ | ✓ | Realized profit and loss. | | `unrealized_pnl` | - | ✓ | ✓ | Unrealized profit and loss. | | `duration_ns` | - | - | ✓ | Time held in nanoseconds. | | `ts_opened` | - | ✓ | ✓ | Timestamp when position opened. | | `ts_closed` | - | - | ✓ | Timestamp when position closed. | | `event_id` | ✓ | ✓ | ✓ | Unique event identifier. | | `ts_event` | ✓ | ✓ | ✓ | Timestamp of the triggering fill. | | `ts_init` | ✓ | ✓ | ✓ | Timestamp when event was created. | ### Tracing orders to positions The `Cache` provides methods to navigate between orders and positions: ```python # From a position, find all orders that contributed fills orders = self.cache.orders_for_position(position.id) # From an order, find the position it belongs to position = self.cache.position_for_order(order.client_order_id) # The opening order is stored directly on the position opening_order_id = position.opening_order_id ``` ## Account events `AccountState` events represent balance and margin snapshots. They fire when: - The venue reports an account update (via the execution client). - The `Portfolio` recalculates account state after a position update (for margin accounts with `calculate_account_state` enabled). Account state contains balances, margins, account type, and base currency. The `Portfolio` subscribes to these events internally to maintain exposure and balance tracking. ## Event subscriptions Beyond strategy handlers, actors can subscribe to specific event streams for instruments they do not trade. These subscriptions use the `MessageBus` directly and do not involve the `DataEngine`. | Method | Handler | Receives | |------------------------------|-----------------------|--------------------------------| | `subscribe_order_fills()` | `on_order_filled()` | All fills for an instrument. | | `subscribe_order_cancels()` | `on_order_canceled()` | All cancels for an instrument. | These are useful for monitoring actors that track execution quality or fill rates across strategies without participating in order management. For details and examples, see [Order fill subscriptions](actors.md#order-fill-subscriptions) and [Order cancel subscriptions](actors.md#order-cancel-subscriptions). ## Related guides - [Orders](orders/) - Order types and state machine. - [Positions](positions.md) - Position lifecycle and PnL. - [Execution](execution.md) - Execution flow and risk checks. - [Strategies](strategies.md) - Handler implementations in strategies. - [Architecture](architecture.md) - Data and execution flow patterns. # Execution Source: https://nautilustrader.io/docs/latest/concepts/execution/ NautilusTrader can handle trade execution and order management for multiple strategies and venues simultaneously (per instance). Several interacting components are involved in execution, making it important to understand the possible flows of execution messages (commands and events). The main execution-related components include: - `Strategy` - `ExecAlgorithm` (execution algorithms) - `OrderEmulator` - `RiskEngine` - `ExecutionEngine` or `LiveExecutionEngine` - `ExecutionClient` or `LiveExecutionClient` ## Execution flow The `Strategy` base class inherits from `Actor` and contains all common data methods. It also provides methods for managing orders and trade execution: - `submit_order(...)` - `submit_order_list(...)` - `modify_order(...)` - `cancel_order(...)` - `cancel_orders(...)` - `cancel_all_orders(...)` - `close_position(...)` - `close_all_positions(...)` - `query_account(...)` - `query_order(...)` These methods create the necessary execution commands and send them on the message bus to the relevant components (point-to-point). They also publish events such as `OrderInitialized`. There is not a single linear path for every command: - `submit_order(...)` routes to `OrderEmulator` for emulated orders, to an `ExecAlgorithm` when `exec_algorithm_id` is set, and to the `RiskEngine` otherwise. - `submit_order_list(...)` follows the same branching behavior based on emulation and `exec_algorithm_id`. - `modify_order(...)` routes to the `OrderEmulator` for emulated orders and to the `RiskEngine` otherwise. - Cancel and query commands can route directly to the `OrderEmulator`, `ExecAlgorithm`, or `ExecutionEngine`, depending on the command and order state. For new order submission, the typical flow looks like this: `Strategy` -> `OrderEmulator` or `ExecAlgorithm` or `RiskEngine` From there, the downstream flow is typically: `OrderEmulator` -> `ExecAlgorithm` or `ExecutionEngine` `ExecAlgorithm` -> `RiskEngine` -> `ExecutionEngine` -> `ExecutionClient` This diagram illustrates message flow (commands and events) across the Nautilus execution components. ```mermaid flowchart LR strategy[Strategy] emulator[OrderEmulator] algo[ExecAlgorithm] risk[RiskEngine] engine[ExecutionEngine] client[ExecutionClient] strategy --> emulator strategy --> algo strategy --> risk strategy --> engine emulator -. OrderReleased .-> risk emulator --> algo emulator --> engine algo --> risk risk <--> engine engine <--> client ``` ## Order denied reasons A local denial (`OrderDenied`) carries a standardized `CATEGORY_CONDITION` reason code followed by `key=value` context, for example `QUANTITY_EXCEEDS_MAXIMUM: effective_quantity=15, max_quantity=10`. The table covers local denials emitted by the risk and execution engines. These codes are the source of truth for locally-denied orders; venue rejections (`OrderRejected`) pass through the venue's own text unchanged. | Code | Description | | ------------------------------------- | --------------------------------------------------------------------------------- | | `CLIENT_VENUE_MISMATCH` | The execution client does not handle the order venue. | | `CUM_MARGIN_EXCEEDS_FREE_BALANCE` | The cumulative initial margin exceeds the account free balance. | | `CUM_NOTIONAL_EXCEEDS_FREE_BALANCE` | The cumulative order notional exceeds the account free balance. | | `EXPIRE_TIME_IN_PAST` | The order's expire time is in the past. | | `INSTRUMENT_NOT_FOUND` | The instrument was not found in the cache. | | `INVALID_CLIENT_ORDER_ID` | The client order ID is invalid for the venue. | | `INVALID_MAX_NOTIONAL_PER_ORDER` | The configured maximum notional per order is invalid. | | `INVALID_ORDER_SIDE` | The order side is invalid for this operation. | | `INVALID_POSITION_ID` | The supplied position ID is invalid for the order submission. | | `MARGIN_EXCEEDS_FREE_BALANCE` | The order initial margin exceeds the account free balance. | | `MISSING_EXPIRE_TIME` | A GTD order is missing its expire time. | | `MISSING_TRAILING_OFFSET` | The order is missing a required trailing offset. | | `MISSING_TRAILING_OFFSET_TYPE` | The order is missing a required trailing offset type. | | `MISSING_TRIGGER_TYPE` | The order is missing a required trigger type. | | `NOTIONAL_BELOW_MINIMUM` | The order notional is below the instrument minimum. | | `NOTIONAL_EXCEEDS_FREE_BALANCE` | The order notional exceeds the account free balance. | | `NOTIONAL_EXCEEDS_MAXIMUM` | The order notional exceeds the instrument maximum. | | `NOTIONAL_EXCEEDS_MAX_PER_ORDER` | The order notional exceeds the configured maximum per order. | | `NO_EXECUTION_CLIENT` | No execution client was found for the routed command. | | `ORDER_LIST_DENIED` | The order was denied because its order list failed risk checks. | | `ORDER_LIST_INCOMPLETE` | The order list is missing orders in the cache. | | `POSITION_NOT_FOUND` | The position for a reduce‑only order was not found. | | `QUANTITY_BELOW_MINIMUM` | The effective order quantity is below the instrument minimum. | | `QUANTITY_CONVERSION_FAILED` | The order quantity could not be converted for risk checks. | | `QUANTITY_EXCEEDS_MAXIMUM` | The effective order quantity exceeds the instrument maximum. | | `RATE_LIMIT_EXCEEDED` | The order submission rate limit was exceeded. | | `REDUCE_ONLY_WOULD_INCREASE_POSITION` | A reduce‑only order would increase the position. | | `STREAM_RECONCILING` | A post‑reconnect stream reconciliation is in progress; retry once it completes. | | `SUBMIT_FAILED` | Submitting the order to the execution client failed. | | `TRADING_HALTED` | Trading is halted; new orders are denied. | | `TRADING_STATE_REDUCING` | Trading is reducing; the order would increase exposure. | | `TRAILING_STOP_CALC_FAILED` | The trailing stop trigger price could not be calculated. | | `UNSUPPORTED_ORDER_LIST` | The venue does not support the requested order list. | | `UNSUPPORTED_ORDER_TYPE` | The order type is not supported by the venue. | | `UNSUPPORTED_TIME_IN_FORCE` | The order's time in force is not supported. | | `UNSUPPORTED_TP_SL` | The venue does not support the requested take‑profit/stop‑loss parameters. | | `UNSUPPORTED_TRAILING_OFFSET_TYPE` | The order's trailing offset type is not supported. | | `VALIDATION_FAILED` | The order failed adapter validation before submission. | ## Order Management System (OMS) An order management system (OMS) type refers to the method used for assigning orders to positions and tracking those positions for an instrument. OMS types apply to both strategies and venues (simulated and real). Even if a venue doesn't explicitly state the method in use, an OMS type is always in effect. The OMS type for a component can be specified using the `OmsType` enum. The `OmsType` enum has three variants: - `UNSPECIFIED`: The OMS type defaults based on where it is applied (details below) - `NETTING`: Positions are combined into a single position per instrument ID - `HEDGING`: Multiple positions per instrument ID are supported (both long and short) The table below describes different configuration combinations and their applicable scenarios. When the strategy and venue OMS types differ, the `ExecutionEngine` handles this by overriding or assigning `position_id` values for received `OrderFilled` events. A "virtual position" refers to a position ID that exists within the Nautilus system but not on the venue in reality. | Strategy OMS | Venue OMS | Description | |:-------------|:----------|:------------------------------------------------------------------------------------------------------------------------------------------------------------| | `NETTING` | `NETTING` | The strategy uses the venue's native OMS type, with a single position ID per instrument ID. | | `HEDGING` | `HEDGING` | The strategy uses the venue's native OMS type, with multiple position IDs per instrument ID (both `LONG` and `SHORT`). | | `NETTING` | `HEDGING` | The strategy **overrides** the venue's native OMS type. The venue tracks multiple positions per instrument ID, but Nautilus maintains a single position ID. | | `HEDGING` | `NETTING` | The strategy **overrides** the venue's native OMS type. The venue tracks a single position per instrument ID, but Nautilus maintains multiple position IDs. | :::note Configuring OMS types separately for strategies and venues increases platform complexity but allows for a wide range of trading styles and preferences (see below). ::: OMS config examples: - Most cryptocurrency exchanges use a `NETTING` OMS type, representing a single position per market. It may be desirable for a trader to track multiple "virtual" positions for a strategy. - Some FX ECNs or brokers use a `HEDGING` OMS type, tracking multiple positions both `LONG` and `SHORT`. The trader may only care about the NET position per currency pair. :::info Nautilus does not yet support venue-side hedging modes such as Binance `BOTH` vs. `LONG/SHORT` where the venue nets per direction. It is advised to keep Binance account configurations as `BOTH` so that a single position is netted. ::: ### OMS configuration If a strategy OMS type is not explicitly set using the `oms_type` configuration option, it will default to `UNSPECIFIED`. This means the `ExecutionEngine` will not override any venue `position_id`s, and the OMS type will follow the venue's OMS type. :::tip When configuring a backtest, you can specify the `oms_type` for the venue. For accuracy, match this with the OMS type used by the venue. ::: ### Custom position IDs and NETTING Custom position IDs are only valid under `HEDGING` OMS. Under `NETTING` there is by definition a single position per (instrument, strategy), and the engine assigns it a deterministic ID of the form `{instrument_id}-{strategy_id}`. The `ExecutionEngine` enforces this at submit time. If the effective OMS resolves to `NETTING` and `submit_order` (or `submit_order_list`) is called with a `position_id` that does not match `{instrument_id}-{strategy_id}`, the order is denied with an `OrderDenied` event explaining the mismatch. This rule still permits the common closing idiom: `Strategy.close_position(position)` forwards `position.id`, which under `NETTING` is exactly the deterministic ID, so it is accepted. To label or partition positions with arbitrary IDs, configure the strategy with `oms_type=HEDGING`. For `submit_order_list`, the engine additionally denies any mixed-instrument list when a `position_id` is supplied, regardless of OMS. A position belongs to a single instrument, so the combination is rejected with an explicit `OrderDenied` reason. See [Order lists](orders/advanced.md#order-lists) for the broader set of mixed-instrument caveats. ## Risk engine The `RiskEngine` is a component of every Nautilus system, including backtest, sandbox, and live environments. It sits on the submit and modify path, and it also receives order events such as `OrderReleased` from the `OrderEmulator`. Cancel and query commands route directly to other execution components and do not pass through the `RiskEngine`. Unless specifically bypassed in the `RiskEngineConfig`, the engine validates: - Price and trigger-price precision for the instrument. - Positive prices, unless the instrument allows negative prices (options, futures spreads, option spreads, and spot commodities). - Quantity precision and base-quantity min/max bounds. - GTD orders have not already expired. - `reduce_only` orders do not increase the referenced position. - Engine-level `max_notional_per_order` limits and instrument `max_notional` limits. - Cash-account balance impact for non-margin accounts. - Submit and modify rate limits. - Trading-state restrictions (`ACTIVE`, `HALTED`, `REDUCING`). If a submit-time risk check fails, the system generates an `OrderDenied` event with a standardized [reason code](#order-denied-reasons). If a modify-time risk check fails, it generates an `OrderModifyRejected` event. ### Trading state Additionally, the current trading state of a Nautilus system affects order flow. The `TradingState` enum has three variants: - `ACTIVE`: Submit and modify commands operate normally. - `HALTED`: New submit and modify commands are denied. Cancels still pass through. - `REDUCING`: Cancels are allowed, and only submit or modify commands that do not increase exposure are accepted. See the [`RiskEngineConfig` API Reference](/docs/python-api-latest/config.html#nautilus_trader.risk.config.RiskEngineConfig) for further details. ## Execution algorithms The platform supports custom execution algorithm components and provides built-in algorithms such as TWAP (Time-Weighted Average Price). ### TWAP (Time-Weighted Average Price) The TWAP algorithm spreads execution evenly over a specified time horizon. It receives a primary order representing the total size and direction, then spawns smaller child orders executed at regular intervals. This reduces the market impact of the full order size by spreading trade volume over time. The algorithm will immediately submit the first order, with the final order submitted being the primary order at the end of the horizon period. Using the TWAP algorithm as an example (found in `nautilus_trader/examples/algorithms/twap.py`), this example demonstrates how to initialize and register a TWAP execution algorithm directly with a `BacktestEngine` (assuming an engine is already initialized): ```python from nautilus_trader.examples.algorithms.twap import TWAPExecAlgorithm # `engine` is an initialized BacktestEngine instance exec_algorithm = TWAPExecAlgorithm() engine.add_exec_algorithm(exec_algorithm) ``` For this particular algorithm, two parameters must be specified: - `horizon_secs` - `interval_secs` The `horizon_secs` parameter determines the time period over which the algorithm will execute, while the `interval_secs` parameter sets the time between individual order executions. These parameters determine how a primary order is split into a series of spawned orders. ```python from decimal import Decimal from nautilus_trader.model.data import BarType from nautilus_trader.test_kit.providers import TestInstrumentProvider from nautilus_trader.examples.strategies.ema_cross_twap import EMACrossTWAP, EMACrossTWAPConfig # Configure your strategy config = EMACrossTWAPConfig( instrument_id=TestInstrumentProvider.ethusdt_binance().id, bar_type=BarType.from_str("ETHUSDT.BINANCE-250-TICK-LAST-INTERNAL"), trade_size=Decimal("0.05"), fast_ema_period=10, slow_ema_period=20, twap_horizon_secs=10.0, # execution algorithm parameter (total horizon in seconds) twap_interval_secs=2.5, # execution algorithm parameter (seconds between orders) ) # Instantiate your strategy strategy = EMACrossTWAP(config=config) ``` Alternatively, you can specify these parameters dynamically per order, determining them based on actual market conditions. In this case, the strategy configuration parameters could be provided to an execution model which determines the horizon and interval. :::info There is no limit to the number of execution algorithm parameters you can create. The parameters must be a dictionary with string keys and primitive values (values that can be serialized over the wire, such as ints, floats, and strings). ::: ### Writing execution algorithms To build a custom execution algorithm, define a class that inherits from `ExecAlgorithm`. An execution algorithm is a type of `Actor`, so it's capable of the following: - Request and subscribe to data. - Access the `Cache`. - Set time alerts and/or timers using a `Clock`. Additionally it can: - Access the central `Portfolio`. - Spawn secondary orders from a received primary (original) order. Once an execution algorithm is registered, and the system is running, it will receive orders off the messages bus which are addressed to its `ExecAlgorithmId` via the `exec_algorithm_id` order parameter. The order may also carry the `exec_algorithm_params` being a `dict[str, Any]`. :::warning Because of the flexibility of the `exec_algorithm_params` dictionary, it's important to thoroughly validate all of the key value pairs for correct operation of the algorithm (for starters that the dictionary is not `None` and all necessary parameters actually exist). ::: Received orders will arrive via the following `on_order(...)` method. These received orders are known as "primary" (original) orders when being handled by an execution algorithm. ```python from nautilus_trader.model.orders.base import Order def on_order(self, order: Order) -> None: # Handle the order here ``` When the algorithm is ready to spawn a secondary order, it can use one of the following methods: - `spawn_market(...)` (spawns a `MARKET` order) - `spawn_market_to_limit(...)` (spawns a `MARKET_TO_LIMIT` order) - `spawn_limit(...)` (spawns a `LIMIT` order) :::note Additional order types will be implemented in future versions, as the need arises. ::: Each of these methods takes the primary (original) `Order` as the first argument. By default, the primary order quantity is reduced by the spawned `quantity`. This can be disabled by passing `reduce_primary=False`. :::warning When `reduce_primary=True`, the spawned quantity must not exceed the primary order's `leaves_qty` (remaining unfilled quantity). ::: :::note If a spawned order is denied or rejected before acceptance, the deducted quantity is automatically restored to the primary order. Once accepted by the venue, the reduction is considered committed. ::: An execution algorithm can keep spawning secondary orders, submit the remaining primary order, or do both depending on its design. The built-in TWAP example submits the remaining primary order on the final interval. ### Spawned orders All secondary orders spawned from an execution algorithm will carry a `exec_spawn_id` which is the `ClientOrderId` of the primary (original) order, and whose `client_order_id` derives from this original identifier with the following convention: - `exec_spawn_id` (primary order `client_order_id` value) - `spawn_sequence` (the sequence number for the spawned order) ``` {exec_spawn_id}-E{spawn_sequence} ``` e.g. `O-20230404-001-000-E1` (for the first spawned order) :::note The "primary" and "secondary" / "spawn" terminology was specifically chosen to avoid conflict or confusion with the "parent" and "child" contingent orders terminology (an execution algorithm may also deal with contingent orders). ::: ### Managing execution algorithm orders The `Cache` provides several methods to aid in managing (keeping track of) the activity of an execution algorithm. Calling the below method will return all execution algorithm orders for the given query filters. ```python def orders_for_exec_algorithm( self, exec_algorithm_id: ExecAlgorithmId, venue: Venue | None = None, instrument_id: InstrumentId | None = None, strategy_id: StrategyId | None = None, side: OrderSide = OrderSide.NO_ORDER_SIDE, account_id: AccountId | None = None, ) -> list[Order]: ``` As well as more specifically querying the orders for a certain execution series/spawn. Calling the below method will return all orders for the given `exec_spawn_id` (if found). ```python def orders_for_exec_spawn(self, exec_spawn_id: ClientOrderId) -> list[Order]: ``` :::note This also includes the primary (original) order. ::: ## Own order books Own order books are L3 order books that track only your own (user) orders organized by price level, maintained separately from the venue's public order books. ### Purpose Own order books serve several purposes: - Monitor the state of your orders within the venue's public book in real-time. - Validate order placement by checking available liquidity at price levels before submission. - Help prevent self-trading by identifying price levels where your own orders already exist. - Support advanced order management strategies that depend on queue position. - Enable reconciliation between internal state and venue state during live trading. ### Lifecycle Own order books are maintained per instrument and automatically updated as orders transition through their lifecycle. Orders are added when submitted or accepted, updated when modified, and removed when filled, canceled, rejected, or expired. Only orders with prices can be represented in own order books. Market orders and other order types without explicit prices are excluded since they cannot be positioned at specific price levels. ### Safe cancellation queries When querying own order books for orders to cancel, use a `status` filter that **excludes** `PENDING_CANCEL` to avoid processing orders already being cancelled. :::warning Including `PENDING_CANCEL` in status filters can cause: - Duplicate cancel attempts on the same order. - Inflated open order counts (orders in `PENDING_CANCEL` remain "open" until confirmed canceled). - Order state explosion when multiple strategies attempt to cancel the same orders. ::: The optional `accepted_buffer_ns` many methods expose is a time-based guard that only returns orders whose `ts_accepted` is at least that many nanoseconds in the past. When `accepted_buffer_ns > 0`, you must also provide `ts_now`. Orders that have not yet been accepted by the venue still have `ts_accepted = 0`, so they are included once the buffer window elapses. To exclude those inflight orders you must pair the buffer with an explicit status filter (for example, restrict to `ACCEPTED` / `PARTIALLY_FILLED`). ### Auditing During live trading, own order books can be periodically audited against the cache's open and inflight order indexes to ensure consistency. The audit verifies that closed orders are removed and that inflight orders (submitted but not yet accepted) remain tracked during venue latency windows. The audit interval can be configured using the `own_books_audit_interval_secs` parameter in live trading configurations. ## Overfills An overfill occurs when the cumulative fill quantity for an order exceeds the original order quantity. For example, an order for 100 units that receives fills totaling 110 units has an overfill of 10 units. ### How overfills occur Overfills can result from two fundamentally different causes: - Duplicate fill events (a network/messaging issue). - Genuine overfills at the matching engine (a real execution outcome). **Genuine overfills at the matching engine** In some cases, the matching engine actually executes more quantity than the order requested. This is a real execution outcome, not a duplicate event: - **Matching engine race conditions**: In fast markets with high concurrency, an order may match against multiple counterparties nearly simultaneously before being fully removed from the book. - **Minimum lot size constraints**: If an order's remaining quantity falls below the venue's minimum tradeable lot, some matching engines fill the minimum lot anyway rather than leaving an untradeable remainder. - **DEX/AMM mechanics**: Decentralized exchanges using automated market makers may have execution mechanics where actual fill quantities differ slightly from requested due to price impact calculations. - **Multi-fill atomicity**: Some venues do not guarantee atomic fill quantities across partial executions, allowing aggregate fills to exceed the original order quantity. **Duplicate fill events** Separate from genuine overfills, the same fill event may be delivered multiple times: - WebSocket reconnection replaying previously received events. - The venue's internal retry or delivery guarantee mechanisms. - API timing issues in the venue's execution reporting. The system handles duplicate events via `trade_id` deduplication (see below), but duplicates with different `trade_id` values require overfill handling. **Race conditions with reconciliation** During live trading, the system maintains state through two parallel channels: - Real-time fill events arriving via WebSocket. - Periodic reconciliation polling the venue for fill history and position status. If the same fill arrives through both channels with different identifiers before deduplication can occur, both may be applied to the order. This is particularly likely during: - System startup when reconciliation runs while WebSocket connections are establishing. - Network instability causing reconnections mid-fill. - High-frequency trading where fills arrive faster than reconciliation cycles. The likelihood of reconciliation race conditions increases when: - **Thresholds are reduced**: The `open_check_threshold_ms` and `inflight_check_threshold_ms` settings (both default to 5,000 ms) define how long the engine waits before acting on discrepancies. Reducing these below the round-trip latency to your venue increases the chance of processing a fill via reconciliation before the real-time event arrives (or vice versa). - **Reconciliation frequency is increased**: Setting `open_check_interval_secs` or `position_check_interval_secs` to aggressive values (e.g., 1-2 seconds) increases how often the system polls the venue, creating more opportunities for race conditions with real-time events. - **Startup delay is reduced**: The `reconciliation_startup_delay_secs` setting (default 10 seconds) provides time for WebSocket connections to stabilize before continuous reconciliation begins. Reducing this increases the chance of duplicate fills during the startup window. See [Continuous reconciliation](../how_to/configure_live_trading.md#continuous-reconciliation) for configuration details. ### System behavior The `ExecutionEngine` checks for potential overfills before applying each fill event by comparing the order's current `filled_qty` plus the incoming `last_qty` against the original `quantity`. The `allow_overfills` configuration option (default: `False`) controls how overfills are handled: | `allow_overfills` | Behavior | |-------------------|--------------------------------------------------------------------------| | `False` | Logs and rejects the fill, preserving the order's current state. | | `True` | Logs a warning, applies the fill, and tracks the excess in `overfill_qty`. | When overfills are allowed, the order's `overfill_qty` field tracks the excess quantity. The order transitions to `FILLED` status and `leaves_qty` is clamped to zero. ### Duplicate fill detection The `Order` model enforces that each `trade_id` can only be applied once. Inside `Order.apply()`, a hard check raises an error if the incoming fill's `trade_id` already exists on the order. This is the invariant that prevents double-counting executions. **Core engine path (backtest and real-time event processing)** In the core `ExecutionEngine` (used for backtests and processing real-time fill events), before calling `apply()`, the engine checks `Order.is_duplicate_fill()` which compares: - `trade_id` - `order_side` - `last_px` - `last_qty` If all fields match an existing fill exactly, the event is skipped gracefully with a warning log. This avoids raising an error for benign exact replays (e.g., from WebSocket reconnection). If the `trade_id` matches but other fields differ ("noisy replay"), the 4-field check passes but `Order.apply()` will raise an error due to the duplicate `trade_id`. The engine catches this error, logs the exception with full context, and drops the fill - it does not crash. **Live reconciliation sanitizer** During live reconciliation, `LiveExecutionEngine` pre-filters on `trade_id` alone *before* generating fill events. This check runs before the 4-field check described above. If a fill report arrives with a `trade_id` that already exists on the order, it is skipped regardless of whether the price or quantity differs. When the data does differ, a warning is logged to alert operators to potential venue data quality issues. This pre-filtering ensures that "noisy duplicates" from venue replays or reconciliation races are filtered out before they can trigger model integrity errors. If a venue legitimately needs to correct fill data, it should use proper execution report semantics rather than resending with the same `trade_id`. Reconciliation-generated `trade_id` values are deterministic hashes of the reconciliation fill inputs, so a restart that replays reconciliation produces the same `trade_id` and is deduped by this sanitizer rather than being treated as a new fill. ### Configuration For live trading, enable overfill tolerance in the `LiveExecEngineConfig`: ```python from nautilus_trader.live.config import LiveExecEngineConfig config = LiveExecEngineConfig( allow_overfills=True, # Log warning instead of rejecting ) ``` :::tip Enable `allow_overfills=True` when trading on venues known to emit duplicate fills or when position reconciliation races with exchange fill events are expected. Monitor the logs for overfill warnings to identify patterns that may require venue-specific handling. ::: :::warning When `allow_overfills=False` (the default), rejected fills may cause position discrepancies between the system and the venue. Use the [reconciliation](live.md#execution-reconciliation) features to detect and resolve such discrepancies. ::: ## Reconciliation reports The execution engine consumes four reconciliation report variants emitted by adapters in live trading. Each variant has a different role and a different fallback when the matching order is not yet in the local cache. | Variant | Use case | Order missing from cache | |------------------------|--------------------------------------------------------------|---------------------------------------| | `OrderStatusReport` | Standalone order state update. | External order created from the report; if status is `PartiallyFilled`/`Filled`, an inferred fill is synthesised from `avg_px`/`filled_qty`. | | `FillReport` | Standalone execution. | External order is created from the fill (`OrderType::Market`, quantity `last_qty`); the real fill is then applied so its `trade_id` and `commission` are preserved. | | `OrderWithFills` | Order status update bundled with the fills that produced it. | External order created without an inferred fill; the supplied fills are applied first; any residual gap between `report.filled_qty` and the sum of supplied `last_qty`s is closed with an inferred fill. | | `PositionStatusReport` | Position snapshot from the venue. | Logged; positions are derived from fills, not bootstrapped here. | ### When to use each variant Adapters choose the variant based on what the venue's wire format actually delivers for a given event: - Use `OrderStatusReport` for ordinary order lifecycle updates (Accepted, PartiallyFilled, Canceled, Expired) where fill detail arrives separately on a different stream. - Use `FillReport` for venues that only surface a fill for venue-initiated closures and never open a user-level order (the canonical example is Hyperliquid liquidations: the user receives a `userFills` entry with `liquidation` metadata but no entry on the orders stream). - Use `OrderWithFills` when a single venue event maps to both a status update and one or more fills, and the adapter has both available at the same point in time. Bundling lets the engine apply real fill metadata (`trade_id`, `commission`) and only synthesise an inferred fill for the residual quantity. Binance Futures uses this for exchange-generated ADL, liquidation, and settlement orders via `dispatch_exchange_generated_fill`. ### External order creation When a report references an order that is not in the cache (a venue-initiated ADL / liquidation / settlement, an order placed by a different process, or an order that has not yet been observed locally), the engine creates an *external order* and routes ownership to: - The strategy that has claimed the instrument via `register_external_order_claims`, or - The `EXTERNAL` strategy as a default fallback. The external order's `client_order_id` is taken from the report when present, otherwise derived from the `venue_order_id`. The order is added to the cache, the venue order ID index is registered, and the engine emits the appropriate lifecycle events (`OrderAccepted`, `OrderFilled`, `OrderCanceled`, `OrderExpired`) so positions update through the normal event pipeline. This means a Hyperliquid liquidation that arrives as a single `FillReport` and a Binance ADL that arrives as a bundled `OrderWithFills` both update the local position without any strategy-side handling. ## Related guides - [Events](events.md) - Order and position event types and dispatch. - [Orders](orders/) - Order types and management. - [Positions](positions.md) - Position tracking from executions. - [Strategies](strategies.md) - Order submission from strategies. # Greeks Source: https://nautilustrader.io/docs/latest/concepts/greeks/ Nautilus provides two paths for working with option Greeks (sensitivities of option prices to changes in market variables): 1. **Venue-provided Greeks (Rust/PyO3)**: real-time Greeks streamed from venues like Deribit, Bybit, and OKX via the `OptionGreeks` data type and the option chain aggregation system. 2. **Local Greeks calculator (Cython/Python and Rust/PyO3)**: the `GreeksCalculator` class computes Black-Scholes Greeks from cached market data, with support for portfolio aggregation, shock scenarios, and beta weighting. Either path works independently or together. Venue-provided Greeks arrive through the data subscription system and require no local computation. The local calculator covers venues that do not stream Greeks, backtesting, and custom adjustments (shocks, beta weighting, percent Greeks). ## Venue-provided Greeks (Rust/PyO3) ### OptionGreeks The `OptionGreeks` type represents venue-provided sensitivities for a single option contract. It is a Rust-native type exposed to Python via PyO3. | Field | Type | Description | |--------------------|--------------------|-----------------------------------------------------| | `instrument_id` | `InstrumentId` | The option contract these Greeks apply to. | | `convention` | `GreeksConvention` | Numeraire convention for the Greeks. | | `delta` | `float` | Rate of change of option price per unit underlying. | | `gamma` | `float` | Rate of change of delta per unit underlying. | | `vega` | `float` | Sensitivity to a 1% change in implied volatility. | | `theta` | `float` | Daily time decay (dV/dt / 365.25). | | `rho` | `float` | Sensitivity to a change in interest rate. | | `mark_iv` | `float` or None | Mark implied volatility. | | `bid_iv` | `float` or None | Bid implied volatility. | | `ask_iv` | `float` or None | Ask implied volatility. | | `underlying_price` | `float` or None | Underlying price at time of calculation. | | `open_interest` | `float` or None | Open interest for the contract. | | `ts_event` | `int` | UNIX timestamp (nanoseconds) of the event. | | `ts_init` | `int` | UNIX timestamp (nanoseconds) when initialized. | Subscribe from an actor or strategy: ```python self.subscribe_option_greeks(instrument_id, client_id=ClientId("DERIBIT")) ``` Handle updates: ```python def on_option_greeks(self, greeks: OptionGreeks) -> None: self.log.info(f"delta={greeks.delta:.4f} gamma={greeks.gamma:.6f}") ``` See the [Options](options.md) guide for the full subscription API including option chain aggregation, strike range filtering, and snapshot modes. ### Persistence and replay `OptionGreeks` is a native member of the `Data` enum, so it persists to the data catalog and replays in backtests as built-in market data (not custom data). Writing and querying use the standard catalog API: ```python catalog.write_data(greeks) # greeks: list[OptionGreeks] greeks = catalog.query(data_cls=OptionGreeks) ``` During replay, persisted Greeks reach a subscribed actor or strategy through the same `on_option_greeks` handler used for live data. They also feed option-chain aggregation: when a strategy subscribes to an `OptionChainSlice`, the backtest data engine joins replayed `OptionGreeks` with replayed `QuoteTick` BBO updates for each option instrument. The `underlying_price` field seeds ATM, and `delta` supports delta-based strike selection through `StrikeRange.delta(target, tolerance)`. ### Core schema versus custom data The native `OptionGreeks` fields are the canonical core schema: the five standard Greeks (`delta`, `gamma`, `vega`, `theta`, `rho`) plus implied volatility, underlying price, open interest, and convention. These field names are stable. There is no single complete Greeks shape, so venue- or model-specific values such as `vanna`, `volga`, `charm`, calibration inputs, or surface metadata belong in [custom data](custom_data.md) rather than the native type. Optional venue fields are nullable. Fields required to interpret the values, such as `convention`, are non-nullable and carry defaults. ### Underlying Rust types The core Rust implementation lives in `crates/model/src/data/greeks.rs`: - `OptionGreekValues`: a plain struct with `delta`, `gamma`, `vega`, `theta`, `rho` fields. Implements `Add` and `Mul` for aggregation. - `OptionGreeks` (in `crates/model/src/data/option_chain.rs`): wraps `OptionGreekValues` with `instrument_id`, `convention`, implied volatility fields, and timestamps. Implements `Deref` so you can access Greeks fields directly. - `HasGreeks` trait: provides a `greeks()` method returning `OptionGreekValues`. Implemented by both `OptionGreekValues` and `OptionGreeks`. ### Black-Scholes functions (Rust/PyO3) Low-level pricing functions exposed to Python from `crates/model/src/data/greeks.rs`: ```python from nautilus_trader.model import ( black_scholes_greeks, imply_vol, imply_vol_and_greeks, refine_vol_and_greeks, ) # Compute Greeks given known volatility result = black_scholes_greeks(s=100.0, r=0.05, b=0.0, vol=0.20, is_call=True, k=100.0, t=0.25) # result.delta, result.gamma, result.vega, result.theta, result.price, result.vol # Imply volatility from market price, then compute Greeks result = imply_vol_and_greeks(s=100.0, r=0.05, b=0.0, is_call=True, k=100.0, t=0.25, price=5.0) # Refine volatility from a starting vol estimate (faster convergence) result = refine_vol_and_greeks(s=100.0, r=0.05, b=0.0, is_call=True, k=100.0, t=0.25, target_price=5.0, initial_vol=0.18) ``` The `BlackScholesGreeksResult` returned by these functions contains: `price`, `vol`, `delta`, `gamma`, `vega`, `theta`, and `itm_prob`. **Conventions:** - Vega is scaled by 0.01 (sensitivity to a 1 percentage point vol change). - Theta is scaled by 1/365.25 (daily decay). - American-style options are priced as European for Greeks computation. ## Local Greeks calculators ### GreeksCalculator The legacy Cython `GreeksCalculator` class in `nautilus_trader/model/greeks.pyx` computes Black-Scholes Greeks from cached market data. A PyO3 calculator is also exposed from `nautilus_trader.common.GreeksCalculator` for the v2 runtime surface. Both calculators use the cache and clock and are accessible from actors or strategies. ```python from nautilus_trader.model.greeks import GreeksCalculator # legacy Cython # v2 PyO3: from nautilus_trader.common import GreeksCalculator # Typically created in on_start() calculator = GreeksCalculator(cache=self.cache, clock=self.clock) ``` #### Instrument Greeks Compute Greeks for a single instrument (option or underlying) with quantity of 1: ```python greeks = calculator.instrument_greeks( instrument_id=option_id, flat_interest_rate=0.0425, # used if no yield curve in cache ) # Both surfaces return GreeksData or None while market data is warming up. ``` Both calculators: 1. Look up the instrument and its underlying in the cache. 2. Retrieve current prices (MID preferred, LAST as fallback). 3. Look up yield curves from the cache (falls back to `flat_interest_rate`). 4. Imply volatility from the market price using `imply_vol_and_greeks`. 5. Return a `GreeksData` object with all computed values. Missing prices return `None`, which lets strategies treat warm-up as a normal no-op path. The v2 PyO3 surface still raises a Python exception for setup errors such as a missing instrument definition. For non-option instruments (futures, equities), the calculator returns a `GreeksData` with `delta=1` (or beta-weighted delta) and no gamma/vega/theta. **Shock scenarios**: apply hypothetical changes to spot, volatility, or time: ```python greeks = calculator.instrument_greeks( instrument_id=option_id, spot_shock=10.0, # +10 points on underlying vol_shock=0.02, # +2% absolute vol increase time_to_expiry_shock=1/365, # roll forward one day ) ``` **Volatility update**: refine implied vol from a cached starting point for faster convergence: ```python greeks = calculator.instrument_greeks( instrument_id=option_id, update_vol=True, # use cached vol as starting point cache_greeks=True, # store result for next iteration ) ``` **Beta-weighted Greeks**: express delta and gamma in terms of an index: ```python greeks = calculator.instrument_greeks( instrument_id=option_id, index_instrument_id=InstrumentId.from_str("SPX.CBOE"), beta_weights={underlying_id: 1.15}, percent_greeks=True, ) ``` **Time-weighted vega**: normalize vega across different expirations: ```python greeks = calculator.instrument_greeks( instrument_id=option_id, vega_time_weight_base=30, # normalize to 30-day vega ) ``` #### Portfolio Greeks Aggregate Greeks across all open positions matching filter criteria: ```python portfolio = calculator.portfolio_greeks( underlyings=["AAPL", "MSFT"], venue=Venue("CBOE"), strategy_id=StrategyId("DELTA_HEDGE-001"), flat_interest_rate=0.0425, index_instrument_id=InstrumentId.from_str("SPX.CBOE"), beta_weights=beta_dict, percent_greeks=True, ) # Returns PortfolioGreeks: pnl, price, delta, gamma, vega, theta ``` Filters: - `underlyings`: list of symbol prefixes (e.g., `["AAPL"]` matches AAPL stock and all AAPL options). - `venue`: restrict to a single venue. - `instrument_id`: restrict to a single instrument. - `strategy_id`: restrict to a single strategy. - `side`: filter by position side (LONG, SHORT). - `greeks_filter`: callable that accepts `PortfolioGreeks` per position; return `True` to include. ### GreeksData On the legacy Python surface, `GreeksData` is a Python custom data class (`@customdataclass`) that carries the full context of a single instrument's Greeks computation. It extends `Data` and supports Arrow serialization, cache storage, and catalog persistence. The v2/PyO3 surface exposes the same core fields from Rust. | Field | Type | Description | |---------------------|-----------------|--------------------------------------------------------| | `instrument_id` | `InstrumentId` | The instrument. | | `is_call` | `bool` | True for call, False for put. | | `strike` | `float` | Strike price. | | `expiry` | `int` | Expiry date as YYYYMMDD integer. | | `expiry_in_days` | `int` | Days to expiry. | | `expiry_in_years` | `float` | Years to expiry (days / 365.25). | | `multiplier` | `float` | Contract multiplier. | | `quantity` | `float` | Position quantity (always 1 from `instrument_greeks`). | | `underlying_price` | `float` | Underlying price used in calculation. | | `interest_rate` | `float` | Interest rate used. | | `cost_of_carry` | `float` | Cost of carry (r - dividend yield; 0 for futures). | | `vol` | `float` | Implied volatility. | | `pnl` | `float` | PnL relative to position entry (if position provided). | | `price` | `float` | Model price. | | `delta` | `float` | Delta. | | `gamma` | `float` | Gamma. | | `vega` | `float` | Vega (dV / 1% vol change). | | `theta` | `float` | Theta (daily decay). | | `itm_prob` | `float` | In‑the‑money probability. | `GreeksData` scales to portfolio level via its `to_portfolio_greeks()` method, which multiplies all values by the contract `multiplier`. The `*` operator applies position quantity: ```python position_greeks = signed_qty * instrument_greeks # returns PortfolioGreeks ``` ### PortfolioGreeks `PortfolioGreeks` is the aggregated result from `portfolio_greeks()`. It supports addition (`+`) for combining positions and scalar multiplication (`*`) for scaling: | Field | Type | Description | |---------|---------|------------------------| | `pnl` | `float` | Aggregate PnL. | | `price` | `float` | Aggregate model value. | | `delta` | `float` | Portfolio delta. | | `gamma` | `float` | Portfolio gamma. | | `vega` | `float` | Portfolio vega. | | `theta` | `float` | Portfolio theta. | ### YieldCurveData `YieldCurveData` stores an interest rate or dividend yield curve. The `GreeksCalculator` looks up curves from the cache by currency code (for interest rates) or by underlying instrument ID (for dividend yields). ```python from nautilus_trader.model.greeks_data import YieldCurveData import numpy as np curve = YieldCurveData( ts_event=0, ts_init=0, curve_name="USD", tenors=np.array([0.25, 0.5, 1.0, 2.0]), interest_rates=np.array([0.04, 0.042, 0.045, 0.048]), ) # Callable: interpolates rate for a given tenor rate = curve(0.75) # quadratic interpolation ``` ## Choosing between the two paths | Criterion | Venue‑provided (`OptionGreeks`) | Local calculator (`GreeksCalculator`) | |------------------------------|----------------------------------------|------------------------------------------| | Computation | Done by the venue | Local Black‑Scholes | | Latency | Arrives with market data | Computed on demand | | Venues | Deribit, Bybit, OKX | Any venue with option instruments | | Shock scenarios | Not supported | Spot, vol, and time shocks | | Portfolio aggregation | Manual (iterate `OptionChainSlice`) | Built‑in via `portfolio_greeks()` | | Beta weighting | Not supported | Built‑in | | Backtest support | Via recorded `OptionGreeks` data | From cached prices at any point in time | | Greeks available | delta, gamma, vega, theta, rho, IV, OI | delta, gamma, vega, theta, itm_prob, vol | | Data type | `OptionGreeks` (Rust/PyO3) | `GreeksData` / `PortfolioGreeks` | ## Greek definitions For reference, the Greeks that Nautilus computes: | Greek | Symbol | Definition | |------------|--------|-------------------------------------------------------------------------------| | Delta | `d` | First derivative of option price with respect to underlying price (dV/dS). | | Gamma | `g` | Second derivative of option price with respect to underlying price (d2V/dS2). | | Vega | `v` | Sensitivity to a 1 percentage point change in implied volatility (dV/dVol). | | Theta | `t` | Daily time decay: change in option price per calendar day (dV/dt / 365.25). | | Rho | `r` | Sensitivity to a change in the risk‑free interest rate (dV/dr). | | ITM prob | - | Probability that the option finishes in the money: P(ϕS_T > ϕK), where ϕ = 1 for calls and ϕ = -1 for puts. | ## Examples Complete working examples are available in the repository: - `examples/live/bybit/bybit_option_greeks.py`: subscribe to Bybit venue-provided Greeks. - `examples/live/deribit/deribit_option_greeks.py`: subscribe to Deribit venue-provided Greeks. - `examples/live/okx/okx_option_greeks.py`: subscribe to OKX venue-provided Greeks. ## Related guides - [Options](options.md) - Option instruments, chain subscriptions, and strike filtering. - [Data](data.md) - Built-in data types, custom data, and the subscription model. - [Actors](actors.md) - Subscription and handler reference. - [Strategies](strategies.md) - Strategy implementation and handler methods. # Concepts Source: https://nautilustrader.io/docs/latest/concepts/ These guides explain the core components, architecture, and design of NautilusTrader. } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> :::note If there are discrepancies between these guides and the API reference, the API reference is correct. ::: # Live Trading Source: https://nautilustrader.io/docs/latest/concepts/live/ NautilusTrader deploys backtested strategies to live markets with no code changes. The same actors, strategies, and execution algorithms run against both the backtest engine and a live trading node. :::warning **Live trading involves real financial risk. Before deploying to production, understand system configuration, node operations, execution reconciliation, and the differences between backtesting and live trading.** ::: ## Configuration For how config structs handle defaults, `T` vs `Option` semantics, and builder patterns, see the [Configuration](configuration.md) concept guide. For step-by-step setup of `TradingNodeConfig`, execution engine options, strategy configuration, and multi-venue wiring, see the [Configure a live trading node](../how_to/configure_live_trading.md) how-to guide. ## Live execution policies ### Order command outcome policy Live command outcomes are one of: - Confirmed by the venue. - Definitively rejected by the venue or refused by the venue API. - Denied locally by Nautilus before a submit leaves the system boundary. - Logged as unresolved while Nautilus checks the venue for the final state. Rejection events only appear for definitive command outcomes, not for ambiguous failures: - `OrderRejected`. - `OrderModifyRejected`. - `OrderCancelRejected`. Before an order enters `SUBMITTED`, Nautilus can deny a submit locally. These failures appear as `OrderDenied`, no `OrderSubmitted` event is emitted. After a submit reaches the venue API, Nautilus emits `OrderRejected` when the order response proves that the venue did not accept the order. This includes structured venue submit rejects and API refusals where venue semantics guarantee non‑acceptance, such as HTTP 400, 401, 403, and 429 status responses. A status code is definitive only when venue‑specific semantics prove non‑acceptance. | Command type | Rejection event | When it appears | |--------------------------------------|-----------------------|-------------------------------------------------------------------| | Submit and submit order list | `OrderRejected` | A venue or API response proves an order was not accepted. | | Modify | `OrderModifyRejected` | The venue returns a command‑specific modify reject. | | Cancel, cancel‑all, and batch‑cancel | `OrderCancelRejected` | The venue returns a command‑specific or per‑order cancel reject. | A successful response can still contain per‑order failure fields, and those fields are definitive command outcomes. A whole‑request failure without per‑order results remains unresolved unless the target command is proven refused. Other local validation failures are reported differently: - Cancel, modify, cancel‑all, and batch‑cancel commands that fail local checks log warnings and do not produce rejection events. :::note[Ambiguous outcomes] These failures leave the venue outcome unknown: - Transport errors, WebSocket send failures, request timeouts, and disconnects. - Canceled local tasks, missing acknowledgements, and server errors. - Parse failures after a request may have reached the venue. - Whole‑batch failures without per‑order venue results. - In‑flight retry exhaustion for `PENDING_UPDATE` and `PENDING_CANCEL`. - Rate limits, except create‑order API refusals treated as definitive submit rejections. When the outcome is unknown, Nautilus logs the failure, keeps the order in its current in‑flight state, and waits for WebSocket updates, open‑order polling, in‑flight checks, or startup reconciliation to resolve the state. ::: :::note[Terminology] An **in‑flight order** is one awaiting venue acknowledgement: - `SUBMITTED` - initial submission, awaiting accept/reject. - `PENDING_UPDATE` - modification requested, awaiting confirmation. - `PENDING_CANCEL` - cancellation requested, awaiting confirmation. These orders are monitored by WebSocket updates, open‑order polling, in‑flight checks, and startup reconciliation. ::: For never‑acknowledged submits, the `LiveExecutionEngine` in‑flight check queries the venue. If the order stays unconfirmed beyond `inflight_check_retries`, the engine resolves it to `REJECTED`. Pending cancel and update outcomes remain unresolved until venue reconciliation confirms their final state. See the Runtime checks table below. ## Execution reconciliation Execution reconciliation aligns the venue's actual order and position state with the system's internal state built from events. Only the `LiveExecutionEngine` performs reconciliation, since backtesting controls both sides. Two scenarios: - **Cached state exists**: report data generates missing events to align the state. - **No cached state**: all orders and positions at the venue are generated from scratch. :::tip Persist all execution events to the cache database. This reduces reliance on venue history and allows full recovery even with short lookback windows. ::: ### Reconciliation configuration Unless `reconciliation` is set to false, the execution engine reconciles state for each venue at startup. The `reconciliation_lookback_mins` parameter controls how far back the engine requests history. :::tip Leave `reconciliation_lookback_mins` unset. This lets the engine request the maximum execution history the venue provides. ::: :::warning Executions before the lookback window still generate alignment events, but with some information loss that a longer window would avoid. Some venues also filter or drop older execution data. Persisting all events to the cache database prevents both issues. ::: Each strategy can claim venue-sourced external orders and materialized reconciliation activity for an instrument ID via the `external_order_claims` config parameter. This lets a strategy resume managing open orders and positions when no cached state exists. Unclaimed external orders use strategy ID `EXTERNAL` with tag `VENUE`. Unclaimed orders generated during position reconciliation use strategy ID `EXTERNAL` with tag `RECONCILIATION`. Claimed orders and fills use the claiming strategy ID and have no external/reconciliation tag, so the strategy can continue managing the recovered state. :::tip To detect unclaimed external orders in your strategy, check `order.strategy_id.value == "EXTERNAL"`. These orders participate in portfolio calculations and position tracking like any other order. ::: For all live trading options, see the `LiveExecEngineConfig` [API Reference](/docs/python-api-latest/config.html#nautilus_trader.live.config.LiveExecEngineConfig). ### Reconciliation procedure All adapter execution clients follow the same reconciliation procedure, calling three methods to produce an execution mass status: - `generate_order_status_reports` - `generate_fill_reports` - `generate_position_status_reports` ```mermaid flowchart TD Start[Startup Reconciliation] --> Fetch[Fetch venue reports
orders, fills, positions] Fetch --> Dedup[Deduplicate reports
log warnings for duplicates] Dedup --> Orders[Order Reconciliation
align order states, generate missing events] Orders --> Fills[Fill Reconciliation
verify fills, generate missing OrderFilled events] Fills --> Pos[Position Reconciliation
compare net positions per instrument] Pos --> Match{Positions
match venue?} Match -->|Yes| Done[Reconciliation complete
system ready for trading] Match -->|No| Gen[Generate missing orders
strategy: EXTERNAL, tag: RECONCILIATION] Gen --> Done ``` The system reconciles its state against these reports, which represent external reality: - **Duplicate check**: - Deduplicates order reports within the batch and logs warnings. - Logs duplicate trade IDs as warnings for investigation. - **Order reconciliation**: - Generates and applies events to move orders from cached state to current state. - Infers `OrderFilled` events for missing trade reports. - Generates external order events for unrecognized client order IDs or reports missing a client order ID. - Verifies fill report data consistency with tolerance-based price and commission comparisons. - **Position reconciliation**: - Matches the net position per account and instrument against venue position reports using instrument precision. - Generates external order events when order reconciliation leaves a position that differs from the venue. - When `generate_missing_orders` is enabled (default: True), generates orders with strategy ID `EXTERNAL` and tag `RECONCILIATION` to align discrepancies. - Logs a warning when NETTING ownership is split across multiple strategies for the same account and instrument, since venue position reports are account-level net positions. - Falls through a price hierarchy when generating reconciliation orders: 1. **Calculated reconciliation price** (preferred): targets the correct average position. 2. **Market mid-price**: uses the current bid-ask midpoint. 3. **Current position average**: uses the existing position's average price. 4. **MARKET order** (last resort): used only when no price data exists (no positions, no market data). - Uses LIMIT orders when a price can be determined (cases 1-3) to preserve PnL accuracy. - Skips zero quantity differences after precision rounding. - **Partial window adjustment**: - When `reconciliation_lookback_mins` is set, the window may miss opening fills. - The system adjusts fills using lifecycle analysis to reconstruct positions accurately: - Detects zero-crossings (position qty crosses through FLAT) to identify separate lifecycles. - Adds synthetic opening fills when the earliest lifecycle is incomplete. - Filters out closed lifecycles when the current lifecycle matches the venue position. - Replaces a mismatched current lifecycle with a synthetic fill reflecting the venue position. - Synthetic fills use calculated reconciliation prices to target correct average positions. - See [Partial window adjustment scenarios](#partial-window-adjustment-scenarios) for details. - **Exception handling**: - Individual adapter failures do not abort the entire reconciliation process. - Fill reports arriving before order status reports are deferred until order state is available. If reconciliation fails, the system logs an error and does not start. ### Common reconciliation scenarios The tables below cover startup reconciliation (mass status) and runtime checks (in‑flight order checks, open‑order polls, own‑books audits). #### Startup reconciliation | Scenario | Description | System behavior | |----------------------------------------|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------| | **Order state discrepancy** | Local state differs from venue (e.g., local `SUBMITTED`, venue `REJECTED`). | Updates local order to match venue state, emits missing events. | | **Missed fills** | Venue filled an order but the engine missed the event. | Generates missing `OrderFilled` events. | | **Multiple fills** | Order has partial fills, some missed by the engine. | Reconstructs complete fill history from venue reports. | | **External orders** | Orders exist on venue but not in local cache. | Creates unclaimed orders with strategy ID `EXTERNAL` and tag `VENUE`. | | **Partially filled then canceled** | Order partially filled then canceled by venue. | Updates state to `CANCELED`, preserves fill history. | | **Different fill data** | Venue reports different fill price/commission than cached. | Preserves cached data, logs discrepancies. | | **Filtered orders** | Orders marked for filtering via config. | Skips based on `filtered_client_order_ids` or instrument filters. | | **Duplicate order reports** | Multiple orders share the same identifier. | Deduplicates with warning logged. | | **Position quantity mismatch (long)** | Internal long position differs from venue (e.g., 100 vs 150). | Generates BUY LIMIT with calculated price when `generate_missing_orders=True`. | | **Position quantity mismatch (short)** | Internal short position differs from venue (e.g., -100 vs -150). | Generates SELL LIMIT with calculated price when `generate_missing_orders=True`. | | **Position reduction** | Venue position smaller than internal (e.g., internal 150 long, venue 100 long). | Generates opposite‑side LIMIT order with calculated price. | | **Position side flip** | Internal position opposite of venue (e.g., internal 100 long, venue 50 short). | Generates LIMIT order to close internal and open external position. | | **Internal reconciliation orders** | Orders generated to align position discrepancies. | Uses a claim when configured; otherwise `EXTERNAL` + `RECONCILIATION`. | #### Runtime checks Continuous reconciliation starts after startup reconciliation completes. It: - Monitors in‑flight orders for delays exceeding a configured threshold. - Reconciles open orders with the venue at configured intervals. - Checks position status with the venue at configured intervals. - Audits internal *own* order books against the venue's public books. The loop waits for startup reconciliation to finish before starting periodic checks. The `reconciliation_startup_delay_secs` parameter adds a further delay *after* startup reconciliation completes, giving the system time to stabilize. | Scenario | Description | System behavior | |-------------------------------------|------------------------------------------------------------|------------------------------------------------------| | **Explicit submit API refusal** | API refuses create‑order before acceptance. | Emits `OrderRejected`. | | **Ambiguous submit failure** | Submit fails without confirmed venue refusal. | Logs failure and waits for reconciliation. | | **In‑flight submit timeout** | `SUBMITTED` remains unconfirmed beyond retry exhaustion. | Resolves to `REJECTED`. | | **In‑flight cancel/update timeout** | `PENDING_CANCEL` or `PENDING_UPDATE` exceeds the retries. | Logs warning and remains unresolved. | | **Open orders check discrepancy** | Periodic poll detects a venue state change. | Confirms status and applies transitions. | | **Position check discrepancy** | Periodic poll detects a position mismatch. | Generates reconciliation events when eligible. | | **Own books audit mismatch** | Own order books diverge from venue public books. | Audits and logs inconsistencies. | **In‑flight order timeout resolution** (venue does not respond after max retries): | Current status | Resolved to | Rationale | |------------------|--------------|---------------------------------------| | `SUBMITTED` | `REJECTED` | No acceptance received from venue. | | `PENDING_UPDATE` | *Unresolved* | Modification outcome remains unknown. | | `PENDING_CANCEL` | *Unresolved* | Cancellation outcome remains unknown. | **Order consistency checks** (when cache state differs from venue state): The *Not found* rows apply only in full‑history mode (`open_check_open_only=False`); open‑only mode is the default. | Cache status | Venue status | Resolution | Rationale | |--------------------|--------------|--------------|---------------------------------------------------------------------| | `SUBMITTED` | *Not found* | `REJECTED` | Order never confirmed by venue (e.g., lost during network error). | | `ACCEPTED` | *Not found* | `REJECTED` | Order doesn't exist at venue, likely was never successfully placed. | | `ACCEPTED` | `CANCELED` | `CANCELED` | Venue canceled the order (user action or venue‑initiated). | | `ACCEPTED` | `EXPIRED` | `EXPIRED` | Order reached GTD expiration at venue. | | `ACCEPTED` | `REJECTED` | `REJECTED` | Venue rejected after initial acceptance (rare but possible). | | `PENDING_UPDATE` | *Not found* | *Unresolved* | Modification outcome remains unknown. | | `PENDING_CANCEL` | *Not found* | *Unresolved* | Cancellation outcome remains unknown. | | `PARTIALLY_FILLED` | `CANCELED` | `CANCELED` | Order canceled at venue with fills preserved. | | `PARTIALLY_FILLED` | *Not found* | `CANCELED` | Order doesn't exist but had fills (reconciles fill history). | :::note **Runtime reconciliation caveats:** - **Open‑only mode**: venue "open orders" endpoints exclude closed orders by design, making it impossible to distinguish missing orders from recently closed ones. Pending cancel/update orders remain unresolved when a missing‑order check cannot prove the final venue state. - **Recent order protection**: the engine skips reconciliation for orders whose last event falls within the `open_check_threshold_ms` window. This prevents false positives from race conditions where the venue is still processing. - **Targeted query safeguard**: before applying a terminal "not found" resolution, the engine issues a single‑order query to the venue. This catches false negatives from bulk query limitations or timing delays. - **Position report failures**: if a venue position query fails, the engine skips cached positions for that venue during the cycle instead of treating missing reports as flat. - **`FILLED` orders** that are "not found" at the venue are silently ignored. Venues commonly drop completed orders from their query results. ::: **Retry coordination.** The in‑flight loop and open‑order loop share a single retry counter (`_recon_check_retries`), bounded by `inflight_check_retries` and `open_check_missing_retries` respectively. The stricter limit wins for states eligible for terminal resolution and avoids duplicate venue queries for the same order state. When the open‑order loop exhausts retries, the engine issues one targeted `GenerateOrderStatusReport` probe before applying a terminal state or leaving an ambiguous pending cancel/update unresolved. If the venue returns the order, reconciliation proceeds and the retry counter resets. Position checks use separate retry counters per instrument and account. A successful position match clears the counter, while repeated unresolved discrepancies stop active reconciliation for that pair until the discrepancy clears. **Single‑order query throttling.** The engine caps single‑order queries per cycle via `max_single_order_queries_per_cycle`. Remaining orders are deferred to the next cycle. `single_order_query_delay_ms` spaces out consecutive queries to avoid rate limits. This handles bulk query failures across hundreds of orders without overwhelming the venue API. ### Common reconciliation issues - **Missing trade reports**: Some venues filter out older trades. Increase `reconciliation_lookback_mins` or cache all events locally. - **Position mismatches**: External orders that predate the lookback window cause position drift. Flatten the account before restarting to reset state. - **Split NETTING ownership**: Multiple strategies can hold cached positions for the same account and instrument, but venues report a single account-level net position. Prefer one claiming strategy per NETTING account/instrument pair when resuming external state. - **Duplicate order IDs**: Deduplicated with warnings logged. Frequent duplicates may indicate venue data integrity issues. - **Precision differences**: Small decimal differences are handled using instrument precision. Large discrepancies may indicate missing orders. - **Out-of-order reports**: Fill reports arriving before order status reports are deferred until order state is available. :::tip For persistent issues, drop cached state or flatten accounts before restarting. ::: ### Reconciliation invariants The reconciliation system maintains four invariants: 1. **Position quantity**: the final quantity matches the venue within instrument precision. 2. **Average entry price**: the position's average entry price matches the venue's reported price within tolerance (default 0.01%). 3. **PnL integrity**: all generated fills, including synthetic fills, use calculated prices that preserve correct unrealized PnL. 4. **ID determinism**: synthetic `trade_id` and `venue_order_id` values emitted during reconciliation are deterministic functions of the logical event. The same logical fill or position-adjustment order produces the same ID across restarts, so replayed reconciliation events dedupe against earlier runs instead of being treated as new. These hold even when: - The reconciliation window misses complete fill history. - Fills are missing from venue reports. - Position lifecycles span beyond the lookback window. - Multiple zero-crossings have occurred. ### Partial window adjustment scenarios When `reconciliation_lookback_mins` limits the window, the system analyzes position lifecycles from fills and adjusts to reconstruct positions accurately. | Scenario | Description | System behavior | |--------------------------------------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------| | **Complete lifecycle** | All fills from opening to current state are captured. | No adjustment. | | **Incomplete single lifecycle** | Window misses opening fills, no zero‑crossings. | Adds synthetic opening fill with calculated price. | | **Multiple lifecycles, current matches** | Zero‑crossings detected, current lifecycle matches venue. | Filters out old lifecycles, returns current only. | | **Multiple lifecycles, current mismatch** | Zero‑crossings detected, current lifecycle differs from venue. | Replaces current lifecycle with a single synthetic fill. | | **Flat position** | Venue reports FLAT regardless of fill history. | No adjustment. | | **No fills** | Window contains no fill reports. | No adjustment, empty result. | **Key concepts:** - **Zero-crossing**: position quantity crosses through zero (FLAT), marking a lifecycle boundary. - **Lifecycle**: a sequence of fills between zero-crossings representing one open-close cycle. - **Synthetic fill**: a calculated fill report representing missing activity, priced to achieve the correct average position. - **Tolerance**: position matching uses configurable price tolerance (default 0.0001 = 0.01%) to absorb minor calculation differences. ## Shutdown on error Set `LiveNodeConfig.shutdown_on_error=True` so that a Rust error log requests a live node shutdown. The Rust logger records the first `log::error!` emitted after the kernel starts, including error logs from other threads, and the kernel publishes a `ShutdownSystem` command when the live event loop next checks for shutdown. The shutdown request follows the normal live node stop path. The node stops the trader, awaits the post-stop delay, disconnects clients, and stops the engines. It does not abort the process. ```python from nautilus_trader.live import LiveNodeConfig config = LiveNodeConfig(shutdown_on_error=True) ``` Error logs suppressed by component filters or logging bypass mode still request shutdown. The trigger is cleared and re-armed when a new kernel run starts, so a process can restart a node without reinitializing the logging system. The per-engine `graceful_shutdown_on_error` option has been removed; configure shutdown-on-error at the node/kernel level instead. Shutdown-on-error observes Rust `log` records, not Python `logging.error(...)` calls. ## Related guides - [Configure a live trading node](../how_to/configure_live_trading.md) - Node and engine configuration. - [Adapters](adapters.md) - Venue connectivity. - [Execution](execution.md) - Order execution in live environments. - [Backtesting](backtesting.md) - Testing strategies before deployment. # Logging Source: https://nautilustrader.io/docs/latest/concepts/logging/ The platform provides logging for both backtesting and live trading using a high-performance logging subsystem implemented in Rust with a standardized facade from the `log` crate. The core logger operates in a separate thread and uses a multi-producer single-consumer (MPSC) channel to receive log messages. This design ensures that the main thread remains performant, avoiding potential bottlenecks caused by log string formatting or file I/O operations. Logging output is configurable and supports: - **stdout/stderr writer** for console output - **file writer** for persistent storage of logs :::info Infrastructure such as [Vector](https://github.com/vectordotdev/vector) can be integrated to collect and aggregate events within your system. ::: ## Architecture The logging subsystem captures events from multiple sources and routes them through an MPSC channel to a dedicated logging thread: ```mermaid flowchart TB subgraph Sources["Log Sources"] PY["Python Logger"] NAUT["Nautilus Rust Components"] LOG["External Rust Libraries
(using log crate)
rustls, etc."] end subgraph Filtering["Filtering"] LF["log_level / log_level_file
(LoggingConfig)"] end subgraph Logger["Nautilus Logger"] NL["Logger
(implements log::Log)"] end subgraph Channel["MPSC Channel"] TX["Sender (tx)"] RX["Receiver (rx)"] end subgraph Thread["Logging Thread"] LT["Log Writer"] end subgraph Output["Output"] STDOUT["stdout/stderr"] FILE["Log Files"] end PY --> NL NAUT --> NL LOG --> LF --> NL NL --> TX --> RX --> LT LT --> STDOUT LT --> FILE subgraph Tracing["Tracing Subscriber (optional)"] TRACE["External Rust Libraries
(using tracing crate)
hyper_util, h2, tokio, etc."] EF["RUST_LOG
(EnvFilter)"] FMT["fmt::Layer"] end TRACE --> EF --> FMT --> STDOUT ``` - **Python and Nautilus components**: Log directly through the Nautilus Logger. - **External `log` crate users**: Filtered by `log_level`/`log_level_file` in `LoggingConfig`. - **External `tracing` crate users**: When enabled, output goes directly to stdout (separate from Nautilus logging), filtered by the `RUST_LOG` environment variable. - **Logging thread**: All Nautilus log events are sent through an MPSC channel to a dedicated thread, ensuring the main thread isn't blocked by I/O operations. ## Configuration Logging can be configured by importing the `LoggingConfig` object. By default, log events with an 'INFO' `LogLevel` and higher are written to stdout/stderr. Log level (`LogLevel`) values include the following (matching standard log level conventions). The following log levels are supported: - `OFF` - Disable logging. - `TRACE` - Most verbose; only emitted by Rust components (cannot be generated from Python). - `DEBUG` - Detailed diagnostic information. - `INFO` - General operational messages. - `WARNING` - Potential issues that don't prevent operation. - `ERROR` - Errors that may affect functionality. :::tip You can set `TRACE` as a filter level to capture trace logs from Rust components, even though Python code cannot emit them directly. ::: See the `LoggingConfig` [API Reference](/docs/python-api-latest/config.html#nautilus_trader.common.config.LoggingConfig) for further details. Logging can be configured in the following ways: - Minimum `LogLevel` for stdout/stderr. - Minimum `LogLevel` for log files. - Maximum size before rotating a log file. - Maximum number of backup log files to maintain when rotating. - Automatic log file naming with date or timestamp components, or custom log file name. - Directory for writing log files. - Plain text or JSON log file formatting. - Filtering of individual components by log level. - ANSI colors in log lines. - Bypass logging entirely. - Print Rust config to stdout at initialization. - Optionally initialize logging via the PyO3 bridge (`use_pyo3`) to capture log events emitted by Rust components. - Truncate existing log file on startup if it already exists (`clear_log_file`) ### Standard output logging Log messages are written to the console via stdout/stderr writers. The minimum log level can be configured using the `log_level` parameter. ### File logging Log files are written to the current working directory by default. The naming convention and rotation behavior are configurable and follow specific patterns based on your settings. You can specify a custom log directory using `log_directory` and/or a custom file basename using `log_file_name`. **Log file formats:** - `None` (default) - Plain text format with `.log` extension. - `"json"` - JSON format with `.json` extension, useful for log aggregation tools. For detailed information about log file naming conventions and rotation behavior, see the [Log file rotation](#log-file-rotation) and [Log file naming convention](#log-file-naming-convention) sections below. #### Log file rotation Rotation behavior depends on both the presence of a size limit and whether a custom file name is provided: - **Size-based rotation**: - Enabled by specifying the `log_file_max_size` parameter (e.g., `100_000_000` for 100 MB). - When writing a log entry would make the current file exceed this size, the file is closed and a new one is created. - **Date-based rotation (default naming only)**: - Applies when no `log_file_max_size` is specified and no custom `log_file_name` is provided. - At each UTC date change (midnight), the current log file is closed and a new one is started, creating one file per UTC day. - **No rotation**: - When a custom `log_file_name` is provided without a `log_file_max_size`, logs continue to append to the same file. - Note: Size-based rotation takes precedence - if both a custom name and size limit are provided, rotation still occurs. - **Backup file management**: - Controlled by the `log_file_max_backup_count` parameter (default: 5), limiting the total number of rotated files kept. - When this limit is exceeded, the oldest backup files are automatically removed. #### Log file naming convention The default naming convention ensures log files are uniquely identifiable and timestamped. The format depends on whether file rotation is enabled: **With file rotation enabled**: - **Format**: `{trader_id}_{%Y-%m-%d_%H%M%S:%3f}_{instance_id}.{log|json}` - **Example**: `TESTER-001_2025-04-09_210721:521_d7dc12c8-7008-4042-8ac4-017c3db0fc38.log` - **Components**: - `{trader_id}`: The trader identifier (e.g., `TESTER-001`). - `{%Y-%m-%d_%H%M%S:%3f}`: Full ISO 8601-compliant datetime with millisecond resolution. - `{instance_id}`: A unique instance identifier. - `{log|json}`: File suffix based on format setting. **Without size-based rotation (default naming)**: - **Format**: `{trader_id}_{%Y-%m-%d}_{instance_id}.{log|json}` - **Example**: `TESTER-001_2025-04-09_d7dc12c8-7008-4042-8ac4-017c3db0fc38.log` - **Components**: - `{trader_id}`: The trader identifier. - `{%Y-%m-%d}`: Date only (YYYY-MM-DD). - `{instance_id}`: A unique instance identifier. - `{log|json}`: File suffix based on format setting. - **Note**: With default naming and no size limit, logs rotate daily at UTC midnight. **Custom naming**: If `log_file_name` is set (e.g., `my_custom_log`): - With rotation disabled: The file will be named exactly as provided (e.g., `my_custom_log.log`). - With rotation enabled: The file will include the custom name and timestamp (e.g., `my_custom_log_2025-04-09_210721:521.log`). ### Component log filtering The `log_component_levels` parameter can be used to set log levels for each component individually. The input value should be a dictionary of component ID strings to log level strings: `dict[str, str]`. Below is an example of a trading node logging configuration that includes some of the options mentioned above: ```python from nautilus_trader.config import LoggingConfig from nautilus_trader.config import TradingNodeConfig config_node = TradingNodeConfig( trader_id="TESTER-001", logging=LoggingConfig( log_level="INFO", log_level_file="DEBUG", log_file_format="json", log_component_levels={ "Portfolio": "INFO" }, ), ... # Omitted ) ``` For backtesting, the `BacktestEngineConfig` class can be used instead of `TradingNodeConfig`, as the same options are available. ### Environment variable configuration The `NAUTILUS_LOG` environment variable provides an alternative way to configure logging using a semicolon-separated spec string. This is useful for Rust-only binaries or when you want to override logging settings without modifying code. ```bash export NAUTILUS_LOG="stdout=Info;fileout=Debug;RiskEngine=Error;is_colored" ``` **Supported keys:** | Key | Type | Description | |-----------------------|-----------|--------------------------------------------------| | `stdout` | Log level | Maximum level for stdout output. | | `fileout` | Log level | Maximum level for file output. | | `is_colored` | Flag | Enable ANSI colors (default: true). | | `print_config` | Flag | Print config to stdout at startup. | | `log_components_only` | Flag | Only log components with explicit filters. | | `` | Log level | Component‑specific level (exact match). | | `` | Log level | Module‑specific level (prefix match, Rust only). | Flags are enabled by their presence in the spec string (no value needed). Log levels are case-insensitive: `Off`, `Trace`, `Debug`, `Info`, `Warning` (or `Warn`), `Error`. :::note For Rust-only binaries, setting `NAUTILUS_LOG` enables lazy initialization of the logging subsystem on first use, without requiring explicit `init_logging()` calls. ::: ### Components-only logging When focusing on a subset of noisy systems, enable `log_components_only` to log messages only from components explicitly listed in `log_component_levels`. All other components are suppressed regardless of the global `log_level` or file level. Example (Python configuration): ```python logging = LoggingConfig( log_level="INFO", log_component_levels={ "RiskEngine": "DEBUG", "Portfolio": "INFO", }, log_components_only=True, ) ``` If configuring via the environment using the Rust spec string, include `log_components_only` alongside component filters, for example: ```bash export NAUTILUS_LOG="stdout=Info;log_components_only;RiskEngine=Debug;Portfolio=Info" ``` ### Module path filtering (Rust only) When using the `NAUTILUS_LOG` environment variable, you can filter by Rust module paths in addition to component names. Keys containing `::` are treated as module path filters with prefix matching, while keys without `::` are component filters with exact matching. ```bash # Filter all adapters to Warn, but allow Debug for OKX specifically export NAUTILUS_LOG="stdout=Info;nautilus_okx=Warn;nautilus_okx::websocket=Debug" ``` The longest matching prefix takes precedence. In the example above, `nautilus_okx::websocket::handler` would use the `Debug` level (longer prefix), while `nautilus_okx::data` would use `Warn`. :::tip Rust log macros automatically capture the module path when no explicit component is provided. This enables module-level filtering to work with standard logging calls. ::: :::note Module path filtering is only available via the `NAUTILUS_LOG` environment variable. The Python `log_component_levels` configuration uses component name matching only. ::: :::warning If `log_components_only=True` (or `log_components_only` is present in the spec string) and `log_component_levels` is empty, no log messages will be emitted to stdout/stderr or files. Add at least one component filter or disable components-only logging. ::: ### Log colors ANSI color codes improve log readability in terminals. In environments that do not support ANSI color rendering (such as some cloud environments or text editors), these color codes may not be appropriate as they can appear as raw text. To accommodate for such scenarios, the `LoggingConfig.log_colors` option can be set to `false`. Disabling `log_colors` will prevent the addition of ANSI color codes to the log messages, which avoids raw escape codes in environments without color support. ## Using a logger directly It's possible to use `Logger` objects directly, and these can be initialized anywhere (very similar to the Python built-in `logging` API). If you ***aren't*** using an object which already initializes a `NautilusKernel` (and logging) such as `BacktestEngine` or `TradingNode`, then you can activate logging in the following way: ```python from nautilus_trader.common.component import init_logging from nautilus_trader.common.component import Logger log_guard = init_logging() logger = Logger("MyLogger") ``` See the [`init_logging` API Reference](/docs/python-api-latest/common.html) for further details. :::warning Only one logging subsystem can be initialized per process with an `init_logging` call. Multiple `LogGuard` instances (up to 255) can exist concurrently, and the logging thread will remain active until all guards are dropped. ::: ## LogGuard: managing log lifecycle The `LogGuard` ensures that the logging subsystem remains active and operational throughout the lifecycle of a process. It prevents premature shutdown of the logging subsystem when running multiple engines in the same process. ### Reference counting implementation The logging system uses reference counting to track active `LogGuard` instances: - **Counter increments**: When a new `LogGuard` is created, an atomic counter is incremented. - **Counter decrements**: When a `LogGuard` is dropped, the counter is decremented. - **Logging thread termination**: When the counter reaches zero (last `LogGuard` dropped), the logging thread is properly joined to ensure all pending log messages are written before the process terminates. - **Maximum guards**: The system supports up to 255 concurrent `LogGuard` instances. Attempting to create more raises a `RuntimeError`. This mechanism ensures that: 1. `LogGuard` keeps the logging thread alive and flushes on drop; abrupt termination (crashes, kill signals) can still lose buffered logs. 2. The logging thread remains active as long as any `LogGuard` exists. 3. On graceful shutdown, all buffered logs are properly flushed to their destinations. ### Why use LogGuard? Without a `LogGuard`, any attempt to run sequential engines in the same process may result in errors such as: ``` Error sending log event: [INFO] ... ``` This occurs because the logging subsystem's underlying channel and Rust `Logger` are closed when the first engine is disposed. As a result, subsequent engines lose access to the logging subsystem, leading to these errors. By using a `LogGuard`, you can ensure consistent logging behavior across multiple backtests or engine runs in the same process. The `LogGuard` retains the resources of the logging subsystem and ensures that logs continue to function correctly, even as engines are disposed and initialized. :::note Using `LogGuard` is required to maintain consistent logging behavior throughout a process with multiple engines. ::: ## Running multiple engines The following example demonstrates how to use a `LogGuard` when running multiple engines sequentially in the same process: ```python log_guard = None # Initialize LogGuard reference for i in range(number_of_backtests): engine = setup_engine(...) # Assign reference to LogGuard if log_guard is None: log_guard = engine.get_log_guard() # Add actors and execute the engine actors = setup_actors(...) engine.add_actors(actors) engine.run() engine.dispose() # Dispose safely ``` ### Steps - **Initialize LogGuard once**: The `LogGuard` is obtained from the first engine (`engine.get_log_guard()`) and is retained throughout the process. This ensures that the logging subsystem remains active. - **Dispose engines safely**: Each engine is safely disposed of after its backtest completes. The `LogGuard` remains valid after `engine.dispose()` - only the engine is cleaned up, not the logging subsystem. - **Reuse LogGuard**: The same `LogGuard` instance is reused for subsequent engines, preventing the logging subsystem from shutting down prematurely. ### Considerations - **Multiple LogGuards per process**: The system supports up to 255 concurrent `LogGuard` instances per process. Each guard increments a reference counter when created and decrements it when dropped. - **Thread safety**: The logging subsystem, including `LogGuard`, is thread-safe, ensuring consistent behavior even in multi-threaded environments. - **Automatic cleanup**: When the last `LogGuard` is dropped (reference count reaches zero), the logging thread is properly joined to ensure all pending logs are written before the process terminates. ## Tracing subscriber for external Rust libraries External Rust crates that use the `tracing` crate can have their log output displayed by enabling the tracing subscriber. This is useful for debugging external dependencies or when integrating custom Rust components (such as feature extractors or adapters) compiled as separate PyO3 extensions. ### Enabling the subscriber Enable the tracing subscriber by setting `use_tracing=True` in `LoggingConfig`: ```python from nautilus_trader.config import LoggingConfig from nautilus_trader.config import TradingNodeConfig config_node = TradingNodeConfig( trader_id="TESTER-001", logging=LoggingConfig( log_level="INFO", use_tracing=True, ), ... # Omitted ) ``` Alternatively, call `init_tracing()` directly: ```python from nautilus_trader.core import nautilus_pyo3 nautilus_pyo3.init_tracing() ``` ### Filtering with RUST_LOG The `RUST_LOG` environment variable controls which tracing events are displayed: ```bash # Show debug logs from your crate, warn and above from hyper RUST_LOG=my_feature_extractor=debug,hyper=warn python my_script.py ``` If `RUST_LOG` is not set, the default filter level is `warn`. ### How it works The tracing subscriber uses a `tracing-subscriber` fmt layer with a custom formatter to output directly to stdout. This is separate from the Nautilus logging infrastructure - tracing output uses a Nautilus-aligned format with nanosecond timestamps. Example tracing output: ``` 2026-01-24T05:51:42.809619000Z [DEBUG] hyper_util::client::legacy::connect::http: connecting to 104.18.5.240:443 2026-01-24T05:51:42.810543000Z [DEBUG] hyper_util::client::legacy::pool: pooling idle connection for ("https", api.example.com) ``` **Differences from Nautilus logging:** - Tracing output goes directly to stdout, not through the Nautilus logging thread. - Tracing events are not written to Nautilus log files. - Filtering is controlled exclusively by `RUST_LOG`, independent of `LoggingConfig`. For external libraries that use the `log` crate (such as `rustls`), their events go through the Nautilus logger and are filtered by `log_level`/`log_level_file` in `LoggingConfig`. :::tip `RUST_LOG` only affects crates using `tracing`. For crates using `log`, configure verbosity via `LoggingConfig` or the `NAUTILUS_LOG` environment variable (e.g., `NAUTILUS_LOG=stdout=Debug`). ::: :::note The tracing subscriber can only be initialized once per process. When using `use_tracing=True` in `LoggingConfig`, subsequent kernel creations safely skip re-initialization. Direct calls to `init_tracing()` when already initialized will raise an error. ::: ## Platform-specific considerations ### Windows shutdown behavior On Windows, non-deterministic garbage collection during interpreter shutdown can occasionally prevent the logging thread from joining properly. When the last `LogGuard` is dropped, the logging subsystem signals the background thread to close and joins it to ensure all pending messages are written. If Python's garbage collector delays dropping the guard until after interpreter shutdown has begun, this join may not complete, resulting in truncated logs. This issue is tracked in GitHub [issue #3027](https://github.com/nautechsystems/nautilus_trader/issues/3027). A more deterministic shutdown mechanism is under consideration. ## Related guides - [Architecture](architecture.md) - System architecture including logging infrastructure. # Message Bus Source: https://nautilustrader.io/docs/latest/concepts/message_bus/ The `MessageBus` enables communication between system components through message passing. This design creates a loosely coupled architecture where components interact without direct dependencies. The *messaging patterns* include: - Point-to-Point - Publish/Subscribe - Request/Response Messages exchanged via the `MessageBus` fall into three categories: - Data - Events - Commands ## Topic hierarchy Nautilus keeps market data topics under the `data` root. Live data publications use the direct `data....` topics, for example `data.book.deltas.XCME.ESZ24`. When requested, replayed, or workflow-generated data flows over the message bus as topic-addressable data, the `DataEngine` publishes it under `data.pipeline....`. Long requests, grouped requests, and aggregation chains can split, transform, and fan data back in before the parent request completes. These messages are still data messages, but they do not claim the same live ordering and timing semantics as normal real-time publications. For example, book deltas on the pipeline path use `data.pipeline.book.deltas.XCME.ESZ24`. Correlated request responses are delivered through response handlers keyed by correlation ID. The `data.response` topic is a capture channel for response publications, not the pipeline data path. ## Message integrity Once a message is created, its fields must not be mutated. This includes container fields such as `params` maps. Components can read a message and derive local state from it, but they must not rewrite the original. Immutable messages keep every consumer seeing the same input, preserve what was true at emission time, and remove a class of shared-state races. Replay, debugging, and audit all depend on messages remaining stable after dispatch. Three ownership rules follow from this: - Caller-supplied request options stay on the message. - Response metadata returned to the caller stays on the response. - Component workflow state (bounded date ranges, grouping state, replay cursors, counters, processing flags) stays in component-owned context keyed by message or request ID. When a component needs a derived message, it creates a new one with the required values instead of rewriting the original. ## Data and signal publishing While the `MessageBus` is a lower-level component that users typically interact with indirectly, `Actor` and `Strategy` classes provide convenient methods built on top of it: ```python def publish_data(self, data_type: DataType, data: Data) -> None: def publish_signal(self, name: str, value, ts_event: int = 0) -> None: ``` These methods allow you to publish custom data and signals efficiently without needing to work directly with the `MessageBus` interface. ## Direct access For advanced users or specialized use cases, direct access to the message bus is available within `Actor` and `Strategy` classes through the `self.msgbus` reference, which provides the full message bus interface. To publish a custom message directly, you can specify a topic as a `str` and any Python `object` as the message payload, for example: ```python self.msgbus.publish("MyTopic", "MyMessage") ``` ## Messaging styles NautilusTrader is an **event-driven** framework where components communicate by sending and receiving messages. Understanding the different messaging styles helps when building trading systems. This guide explains the three primary messaging patterns available in NautilusTrader: | **Messaging Style** | **Purpose** | **Best For** | |:---------------------------------------------|:--------------------------------------------|:------------------------------------------------------| | **MessageBus - Publish/Subscribe to topics** | Low‑level, direct access to the message bus | Custom events, system‑level communication | | **Actor‑Based - Publish/Subscribe Data** | Structured trading data exchange | Trading metrics, indicators, data needing persistence | | **Actor‑Based - Publish/Subscribe Signal** | Lightweight notifications | Simple alerts, flags, status updates | Each approach serves different purposes. This section helps you decide which pattern to use. ### MessageBus publish/subscribe to topics #### Concept The `MessageBus` is the central hub for all messages in NautilusTrader. It enables a **publish/subscribe** pattern where components can publish events to **named topics**, and other components can subscribe to receive those messages. This decouples components, allowing them to interact indirectly via the message bus. #### Key benefits and use cases The message bus approach is ideal when you need: - **Cross-component communication** within the system. - **Flexibility** to define any topic and send any type of payload (any Python object). - **Decoupling** between publishers and subscribers who don't need to know about each other. - **Global Reach** where messages can be received by multiple subscribers. - Working with events that don't fit within the predefined `Actor` model. - Advanced scenarios requiring full control over messaging. #### Considerations - You must track topic names manually (typos could result in missed messages). - You must define handlers manually. #### Quick overview code ```python from nautilus_trader.core.message import Event # Define a custom event class Each10thBarEvent(Event): TOPIC = "each_10th_bar" # Topic name def __init__(self, bar): self.bar = bar # Subscribe in a component (in Strategy) self.msgbus.subscribe(Each10thBarEvent.TOPIC, self.on_each_10th_bar) # Publish an event (in Strategy) event = Each10thBarEvent(bar) self.msgbus.publish(Each10thBarEvent.TOPIC, event) # Handler (in Strategy) def on_each_10th_bar(self, event: Each10thBarEvent): self.log.info(f"Received 10th bar: {event.bar}") ``` #### Full example [MessageBus Example](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/example_09_messaging_with_msgbus) ### Actor-based publish/subscribe data #### Concept This approach provides a way to exchange trading specific data between `Actor`s in the system. (note: each `Strategy` inherits from `Actor`). It inherits from `Data`, which ensures proper timestamping and ordering of events - crucial for correct backtest processing. #### Key benefits and use cases The Data publish/subscribe approach works well when you need: - **Exchange of structured trading data** like market data, indicators, custom metrics, or option greeks. - **Proper event ordering** via built-in timestamps (`ts_event`, `ts_init`) crucial for backtest accuracy. - **Data persistence and serialization** through the `@customdataclass` decorator, integrating with NautilusTrader's data catalog system. - **Standardized trading data exchange** between system components. #### Considerations - Requires defining a class that inherits from `Data` or uses `@customdataclass`. #### Inheriting from `Data` vs. using `@customdataclass` **Inheriting from `Data` class:** - Defines abstract properties `ts_event` and `ts_init` that must be implemented by the subclass. These ensure proper data ordering in backtests based on timestamps. **The `@customdataclass` decorator:** - Adds `ts_event` and `ts_init` attributes if they are not already present. - Provides serialization functions: `to_dict()`, `from_dict()`, `to_bytes()`, `to_arrow()`, etc. - Enables data persistence and external communication. #### Quick overview code ```python from nautilus_trader.core.data import Data from nautilus_trader.model.custom import customdataclass @customdataclass class GreeksData(Data): delta: float gamma: float # Publish data (in Actor / Strategy) data = GreeksData(delta=0.75, gamma=0.1, ts_event=1_630_000_000_000_000_000, ts_init=1_630_000_000_000_000_000) self.publish_data(GreeksData, data) # Subscribe to receiving data (in Actor / Strategy) self.subscribe_data(GreeksData) # Handler (this is static callback function with fixed name) def on_data(self, data: Data): if isinstance(data, GreeksData): self.log.info(f"Delta: {data.delta}, Gamma: {data.gamma}") ``` #### Full example [Actor-Based Data Example](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/example_10_messaging_with_actor_data) ### Actor-based publish/subscribe signal #### Concept **Signals** are a lightweight way to publish and subscribe to simple notifications within the actor framework. This is the simplest messaging approach, requiring no custom class definitions. #### Key benefits and use cases The Signal messaging approach works well when you need: - **Simple, lightweight notifications/alerts** like "RiskThresholdExceeded" or "TrendUp". - **Quick, on-the-fly messaging** without defining custom classes. - **Broadcasting alerts or flags** as primitive data (`int`, `float`, or `str`). - **Easy API integration** with straightforward methods (`publish_signal`, `subscribe_signal`). - **Multiple subscriber communication** where all subscribers receive signals when published. - **Minimal setup overhead** with no class definitions required. #### Considerations - Each signal can contain only **single value** of type: `int`, `float`, and `str`. That means no support for complex data structures or other Python types. - In the `on_signal` handler, you can only differentiate between signals using `signal.value`, as the signal name is not accessible in the handler. #### Quick overview code ```python # Define signal constants for better organization (optional but recommended) import types from nautilus_trader.core.datetime import unix_nanos_to_dt from nautilus_trader.common.enums import LogColor signals = types.SimpleNamespace() signals.NEW_HIGHEST_PRICE = "NewHighestPriceReached" signals.NEW_LOWEST_PRICE = "NewLowestPriceReached" # Subscribe to signals (in Actor/Strategy) self.subscribe_signal(signals.NEW_HIGHEST_PRICE) self.subscribe_signal(signals.NEW_LOWEST_PRICE) # Publish a signal (in Actor/Strategy) self.publish_signal( name=signals.NEW_HIGHEST_PRICE, value=signals.NEW_HIGHEST_PRICE, # value can be the same as name for simplicity ts_event=bar.ts_event, # timestamp from triggering event ) # Handler (this is static callback function with fixed name) def on_signal(self, signal): # IMPORTANT: We match against signal.value, not signal.name match signal.value: case signals.NEW_HIGHEST_PRICE: self.log.info( f"New highest price was reached. | " f"Signal value: {signal.value} | " f"Signal time: {unix_nanos_to_dt(signal.ts_event)}", color=LogColor.GREEN ) case signals.NEW_LOWEST_PRICE: self.log.info( f"New lowest price was reached. | " f"Signal value: {signal.value} | " f"Signal time: {unix_nanos_to_dt(signal.ts_event)}", color=LogColor.RED ) ``` #### Full example [Actor-Based Signal Example](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/example_11_messaging_with_actor_signals) ### Summary and decision guide Here's a quick reference to help you decide which messaging style to use: #### Decision guide: Which style to choose? | **Use Case** | **Recommended Approach** | **Setup required** | |:--------------------------------------------|:--------------------------------------------------------------------------------|:-------------------| | Custom events or system‑level communication | `MessageBus` + Pub/Sub to topic | Topic + Handler management | | Structured trading data | `Actor` + Pub/Sub Data + optional `@customdataclass` if serialization is needed | New class definition inheriting from `Data` (handler `on_data` is predefined) | | Simple alerts/notifications | `Actor` + Pub/Sub Signal | Signal name only | ## External egress and ingress The `MessageBus` can write serialized messages to external streams. This section describes the external egress and ingress sides of the external bus. Rust-native live nodes use injected `MessageBusExternalEgress` and `MessageBusExternalIngress` surfaces, so the core node does not depend on Redis, a broker, shared-memory implementation, or socket protocol. :::info Redis is currently supported as one external backing for serializable messages. The minimum supported Redis version is 6.2, required for [streams](https://redis.io/docs/latest/develop/data-types/streams/) functionality. ::: When external egress is configured, outgoing publish messages are first dispatched to in-process subscribers, then serialized into the existing `BusMessage` wire record: - `topic`: the exact message bus topic used by the internal publish call, for example `data.quotes.BINANCE.BTCUSDT` or `events.order.S-001`. - `type`: the canonical payload type name, for example `QuoteTick` or `OrderEventAny`. - `encoding`: the payload encoding selected from the message bus encoding policy. - `payload`: serialized bytes encoded with the selected encoding. External egress receives that record as `publish(BusMessage)`. This outbound call must not block the node's bus thread. Bounded egress implementations drop on a full queue instead of applying back-pressure to the trading loop. Closing the message bus closes the configured egress. Inbound external streams are exposed through the separate Rust `MessageBusExternalIngress` trait. Ingress yields the same `BusMessage { topic, payload_type, encoding, payload }` shape. `republish_external_message` decodes supported inbound messages and republishes them internally without forwarding the message back out. The inbound payload type must first be registered for streaming on the receiving message bus; unregistered types are skipped without decoding. For Redis, messages are transmitted via a Multiple-Producer Single-Consumer (MPSC) channel to a separate Rust task. That task writes the message to Redis streams. Offloading I/O to a separate thread keeps the main thread unblocked. With MessagePack or JSON, Rust-native external egress forwards serializable typed publications. This includes instruments, quotes, trades, bars, book deltas, depth-10 snapshots, mark/index/funding updates, option greeks, account state, portfolio snapshots, order events, position events, and custom data. With the `defi` feature this also includes DeFi blocks, pools, liquidity updates, fee collects, and flash events. Full order book snapshots, greeks data, option chain slices, and DeFi pool swaps are not forwarded because those types do not implement Serde serialization. With SBE or Cap'n Proto, Rust-native external egress forwards the built-in market data payloads with schema codecs: quotes, trades, bars, book deltas, depth-10 snapshots, mark price updates, index price updates, funding rate updates, and option greeks. Other payload types are dropped with a debug log when those schema encodings are selected. ### Serialization Nautilus supports serialization for: - All Nautilus built-in types (serialized as dictionaries `dict[str, Any]` containing serializable primitives). - Python primitive types (`str`, `int`, `float`, `bool`, `bytes`). You can add serialization support for custom types by registering them through the `serialization` subpackage. ```python def register_serializable_type( cls, to_dict: Callable[[Any], dict[str, Any]], from_dict: Callable[[dict[str, Any]], Any], ): ... ``` - `cls`: The type to register. - `to_dict`: The delegate to instantiate a dict of primitive types from the object. - `from_dict`: The delegate to instantiate the object from a dict of primitive types. ## Configuration The message bus external backing technology uses a behavior config plus a technology-owned backing config. `MessageBusConfig` controls message bus behavior. `RedisMessageBusConfig` owns Redis connection settings and implements `MessageBusBackingFactory`. ```rust use nautilus_common::{ enums::SerializationEncoding, msgbus::{backing::MessageBusBackingFactory, config::MessageBusConfig}, }; use nautilus_infrastructure::redis::msgbus::RedisMessageBusConfig; let config = MessageBusConfig { encoding: SerializationEncoding::Json, encoding_market_data: Some(SerializationEncoding::Sbe), timestamps_as_iso8601: true, buffer_interval_ms: Some(100), autotrim_mins: Some(30), use_trader_prefix: true, use_trader_id: true, use_instance_id: false, streams_prefix: "streams".to_string(), types_filter: Some(vec!["QuoteTick".to_string(), "TradeTick".to_string()]), ..Default::default() }; let backing = RedisMessageBusConfig::default(); let message_bus_backing = backing.create(trader_id, instance_id, config.clone())?; ``` ### Backing config A `RedisMessageBusConfig` is required when using the built-in Redis backing. For a default Redis setup on the local loopback you can pass `RedisMessageBusConfig::default()`. Redis selection is explicit in the Rust type. The config does not use a user-facing selector such as `type = "redis"` or `backing_type = "redis"`. Rust-native callers that inject `MessageBusExternalEgress` pass concrete connection details when they construct that egress surface. The core message bus does not require a `RedisMessageBusConfig` for injected egress. The Rust live runtime accepts `external_streams` in `MessageBusConfig`, and consumes inbound `BusMessage`s when callers inject a `MessageBusExternalIngress` with `LiveNodeBuilder::with_external_ingress`. The config names the external stream keys; the injected ingress is the concrete runtime source. Rust-native factory wiring from config to a backing remains the caller's responsibility. ### Encoding Rust-native external message bus egress supports these encoding names: - JSON (`json`) - MessagePack (`msgpack`) - Cap'n Proto (`capnp`, with the Rust `capnp` feature) - SBE (`sbe`, with the Rust `sbe` feature) Use the `encoding` config option to control the message writing encoding. Use `encoding_market_data` to override the encoding for market data payloads backed by the external bus binary codecs. Use `encoding_builtin` to override account state, portfolio snapshot, order event, and position event payloads. Custom and unmapped payload types always use `encoding`. `MessageBusConfig::validate` requires the default `encoding` to support custom payloads, so it must be JSON or MessagePack. Category overrides must be supported by every published payload type in that category. SBE and Cap'n Proto can currently be used only for `encoding_market_data`, and only when the matching Rust feature is enabled. `encoding_builtin = "sbe"` and `encoding_builtin = "capnp"` fail validation until those schema codecs cover the built-in event category. The legacy Python/Cython Redis serializer and the Redis cache payload path support MessagePack and JSON. SBE and Cap'n Proto are schema payload encodings for Rust-native external message bus egress, not Redis cache encodings. :::tip The `json` encoding is used by default for human readability and interoperability. Use `msgpack` when payload size and serialization performance are a primary concern. ::: ### Timestamp formatting By default timestamps are formatted as UNIX epoch nanosecond integers. Alternatively you can configure ISO 8601 string formatting by setting the `timestamps_as_iso8601` to `true`. ### Message stream keys Message stream keys are essential for identifying individual trader nodes and organizing messages within streams. They can be tailored to meet your specific requirements and use cases. In the context of message bus streams, a trader key is typically structured as follows: ``` trader:{trader_id}:{instance_id}:{streams_prefix} ``` These options control Redis stream keys. They do not rewrite the `topic` passed to an injected `MessageBusExternalEgress`; that topic remains the internal message bus publish topic. When `stream_per_topic` is `True`, Redis egress appends the topic to the stream key. When it is `False`, Redis stores all messages on the base stream key and keeps the topic as a message field. The following options are available for configuring message stream keys: #### Trader prefix If the key should begin with the `trader` string. #### Trader ID If the key should include the trader ID for the node. #### Instance ID Each trader node is assigned a unique 'instance ID,' which is a UUIDv4. This instance ID helps distinguish individual traders when messages are distributed across multiple streams. You can include the instance ID in the trader key by setting the `use_instance_id` configuration option to `True`. This is particularly useful when you need to track and identify traders across various streams in a multi-node trading system. #### Streams prefix The `streams_prefix` string enables you to group all streams for a single trader instance or organize messages for multiple instances. Configure this by passing a string to the `streams_prefix` configuration option, ensuring other prefixes are set to false. #### Stream per topic Indicates whether the producer will write a separate stream for each topic. This is particularly useful for Redis backings, which do not support wildcard topics when listening to streams. If set to False, all messages will be written to the same stream. :::info Redis does not support wildcard stream topics. For better compatibility with Redis, it is recommended to set this option to False. ::: ### Types filtering When messages are published on the message bus, they are serialized and written to a stream if a backing for the message bus is configured and enabled. To prevent flooding the stream with data like high-frequency quotes, you may filter out certain types of messages from external publication. To enable this filtering mechanism, pass a list of `type` objects to the `types_filter` parameter in the message bus configuration, specifying which types of messages should be excluded from external publication. ```python from nautilus_trader.config import MessageBusConfig from nautilus_trader.model.data import QuoteTick from nautilus_trader.model.data import TradeTick # Create a MessageBusConfig instance with types filtering message_bus = MessageBusConfig( types_filter=[QuoteTick, TradeTick] ) ``` ### Stream auto-trimming The `autotrim_mins` configuration parameter allows you to specify the lookback window in minutes for automatic stream trimming in your message streams. Automatic stream trimming helps manage the size of your message streams by removing older messages, ensuring that the streams remain manageable in terms of storage and performance. :::info The current Redis implementation will maintain the `autotrim_mins` as a maximum width (plus roughly a minute, as streams are trimmed no more than once per minute). Rather than a maximum lookback window based on the current wall clock time. ::: ## External streams The message bus within a `TradingNode` (node) is referred to as the "internal message bus". A producer node is one which publishes messages onto an external stream (see [external egress and ingress](#external-egress-and-ingress)). The consumer node listens to external streams to receive and publish deserialized message payloads on its internal message bus. ```mermaid flowchart TB producer[Producer Node] stream[Stream] consumer1[Consumer Node 1] consumer2[Consumer Node 2] producer --> stream stream --> consumer1 stream --> consumer2 ``` :::tip Set the `LiveDataEngineConfig.external_clients` with the list of `client_id`s intended to represent the external streaming clients. The `DataEngine` will filter out subscription commands for these clients, ensuring that the external streaming provides the necessary data for any subscriptions to these clients. When the Rust `DataEngine` skips an external-client subscription, it registers the corresponding streaming payload type for inbound republishing on the message bus. ::: ### Example configuration The following example details a streaming setup where a producer node publishes Binance data externally, and a downstream consumer node publishes these data messages onto its internal message bus. #### Producer node We configure the `MessageBus` of the producer node to publish to a `"binance"` stream. The settings `use_trader_id`, `use_trader_prefix`, and `use_instance_id` are all set to `false` to ensure a simple and predictable stream key that the consumer nodes can register for. ```rust let message_bus = MessageBusConfig { use_trader_id: false, use_trader_prefix: false, use_instance_id: false, streams_prefix: "binance".to_string(), // <--- stream_per_topic: false, autotrim_mins: Some(30), ..Default::default() }; let backing = RedisMessageBusConfig { connection_timeout: 2, response_timeout: 2, ..Default::default() }; ``` #### Consumer node We configure the `MessageBus` of the consumer node to receive messages from the same `"binance"` stream. The node listens to the external stream keys when a `MessageBusExternalIngress` is injected into the `LiveNodeBuilder`, then publishes these messages onto its internal message bus. We declare the client ID `"BINANCE_EXT"` as an external client so the `DataEngine` does not attempt to send data commands to this client ID. ```rust let data_engine = LiveDataEngineConfig { external_clients: Some(vec![ClientId::from("BINANCE_EXT")]), ..Default::default() }; let message_bus = MessageBusConfig { external_streams: Some(vec!["binance".to_string()]), // <--- ..Default::default() }; let backing = RedisMessageBusConfig { connection_timeout: 2, response_timeout: 2, ..Default::default() }; ``` ## Related guides - [Actors](actors.md) - Actors use the message bus for event handling. - [Architecture](architecture.md) - Message bus role in system architecture. # Options Source: https://nautilustrader.io/docs/latest/concepts/options/ Nautilus provides first-class support for options trading across traditional and crypto markets. This includes option-specific instrument types, venue-provided Greeks streaming, option chain aggregation, and a local Black-Scholes Greeks calculator for risk management. ## Option instrument types The platform defines several option instrument types: | Instrument | Description | |----------------------|------------------------------------------------------------------------------| | `OptionContract` | Exchange‑traded option on an underlying with strike and expiry. | | `OptionSpread` | Exchange‑defined multi‑leg option strategy as one line. | | `CryptoOption` | Crypto option with crypto quote/settlement; inverse or quanto style. | | `CryptoOptionSpread` | Crypto option spread with inverse, settlement currency, and fractional size. | | `BinaryOption` | Fixed‑payout option that settles to 0 or 1. | Greeks-relevant metadata varies by instrument type: - `OptionContract`, `CryptoOption`: full Greeks inputs including `strike_price`, `option_kind` (CALL/PUT), `expiration_utc`, `underlying`, `multiplier`. - `OptionSpread`, `CryptoOptionSpread`: a combination of up to 4 option legs, each weighted by a ratio. Has `underlying`, `expiration_utc`, and `strategy_type` (vertical, calendar, straddle, etc.). Per-leg `strike_price` and `option_kind` live on each leg's `OptionContract`/`CryptoOption`, not on the spread itself. Greeks are computed per leg and aggregated. Spreads are commonly used for orders (the exchange executes as a single order), while the individual legs appear as positions. `CryptoOptionSpread` additionally carries `is_inverse` and `settlement_currency` for venues like Deribit. - `BinaryOption`: has `expiration_utc` and `outcome`/`description`, but no `strike_price`, `option_kind`, or `underlying`. ## Subscribing to Greeks Venues like Deribit, Bybit, and OKX publish real-time Greeks alongside their options markets. Nautilus provides two subscription levels: - **Per-instrument Greeks**: subscribe to individual option contracts. - **Option chain slices**: subscribe to an aggregated view of an entire option series. ### Per-instrument Greeks Subscribe to venue-provided Greeks for a single option contract from an actor or strategy: ```python from nautilus_trader.model.identifiers import ClientId client_id = ClientId("DERIBIT") self.subscribe_option_greeks(instrument_id, client_id=client_id) ``` Handle incoming updates by implementing the `on_option_greeks` handler: ```python def on_option_greeks(self, greeks) -> None: self.log.info( f"{greeks.instrument_id}: " f"delta={greeks.delta:.4f} gamma={greeks.gamma:.6f} " f"vega={greeks.vega:.4f} theta={greeks.theta:.4f} " f"mark_iv={greeks.mark_iv} underlying={greeks.underlying_price}" ) ``` To stop receiving updates: ```python self.unsubscribe_option_greeks(instrument_id, client_id=client_id) ``` ### Option chain subscriptions An option chain subscription aggregates quotes and Greeks across all strikes in an option series into `OptionChainSlice` snapshots. The `DataEngine` creates one Rust `OptionChainManager` per series and owns the lifecycle: creating the manager, routing incoming data, running snapshot timers, and draining wire subscription changes. ```python from nautilus_trader.core import nautilus_pyo3 series_id = nautilus_pyo3.OptionSeriesId(...) # identifies the series (venue, underlying, expiry) # Subscribe to 5 strikes above and below ATM, snapshot every 1000ms strike_range = nautilus_pyo3.StrikeRange.atm_relative(strikes_above=5, strikes_below=5) self.subscribe_option_chain( series_id, strike_range=strike_range, snapshot_interval_ms=1000, ) ``` Handle snapshots by implementing the `on_option_chain` handler: ```python def on_option_chain(self, chain) -> None: for strike in chain.strikes(): call = chain.get_call(strike) put = chain.get_put(strike) if call and call.greeks: self.log.info(f"Call {strike}: delta={call.greeks.delta:.4f}") ``` ### Strike range filtering `StrikeRange` controls which strikes are active in a chain subscription: | Variant | Description | Example | |---------------|-----------------------------------------------------|------------------------------------------------| | `Fixed` | Subscribe to an explicit set of strikes. | `nautilus_pyo3.StrikeRange.fixed([...])` | | `AtmRelative` | N strikes above and N below the current ATM strike. | `nautilus_pyo3.StrikeRange.atm_relative(5, 5)` | | `AtmPercent` | All strikes within a percentage band around ATM. | `nautilus_pyo3.StrikeRange.atm_percent(0.10)` | | `Delta` | Strikes whose call or put delta is near a target. | `nautilus_pyo3.StrikeRange.delta(0.25, 0.05)` | For ATM-based variants, subscriptions are deferred until the ATM price is determined. ATM is derived from the forward price embedded in venue-provided `OptionGreeks` updates (the `underlying_price` field). It can also be seeded from an initial forward price fetched via HTTP, allowing instant bootstrap before live WebSocket ticks arrive. As ATM shifts, the active strike set rebalances automatically. `Delta` resolves from venue-provided Greeks: a strike is active when its call or put delta magnitude (calls positive, puts negative, compared by absolute value) falls within `tolerance` of `target`. A typical out-of-the-money target such as `0.25` selects a strike on each side of ATM. Before the ATM/forward price is known, `Delta` is deferred like other ATM-based ranges. After ATM is known, when no active strike's Greeks match the band (including before any Greeks arrive), `Delta` falls back to an ATM-relative window of five strikes either side of ATM. Before switching from the fallback window to selected delta strikes, the aggregator waits until every fallback leg has Greeks so partial early updates do not drop neighbouring strikes. ### Snapshot vs. raw mode The `snapshot_interval_ms` parameter controls publishing behavior: - **Snapshot mode** (`snapshot_interval_ms=1000`): Quotes and Greeks accumulate in a buffer and publish as an `OptionChainSlice` on a timer. Suitable for periodic portfolio rebalancing or UI display. - **Raw mode** (`snapshot_interval_ms=None`): Each quote or Greeks update publishes a slice immediately. Suitable for latency-sensitive strategies that react to individual updates. ## Backtesting option chains Option-chain backtests use the same `OptionChainManager` and `OptionChainAggregator` path as live subscriptions. The prerequisite is a Nautilus Parquet catalog that already contains the option instruments and the per-instrument data needed for the chain: - `QuoteTick` records for each option contract, carrying the replayed best bid and offer. - `OptionGreeks` records for each option contract, carrying delta, implied volatility, convention, and the `underlying_price` used to seed ATM. - `CryptoOption` or `OptionContract` instruments for the same instrument IDs. Tardis replays satisfy this contract when option book snapshots or quotes are written as `QuoteTick` and `option_summary` messages are written as `OptionGreeks`. The backtest does not download or request missing catalog data during the run. Configure a `BacktestNode` run with both data streams for the option instruments in the series: ```python data = [ BacktestDataConfig( data_type="QuoteTick", catalog_path="/path/to/catalog", instrument_ids=option_instrument_ids, ), BacktestDataConfig( data_type="OptionGreeks", catalog_path="/path/to/catalog", instrument_ids=option_instrument_ids, ), ] ``` Then subscribe from the strategy: ```python strike_range = StrikeRange.delta(0.25, 0.05) self.subscribe_option_chain( series_id, strike_range=strike_range, snapshot_interval_ms=1000, ) ``` Use `snapshot_interval_ms=None` for raw mode. Raw mode publishes a slice after each quote or Greeks update that changes the active chain. Use an integer interval for thinned snapshots. Thinned mode accumulates the latest BBO and Greeks per instrument and publishes the chain on the timer cadence, reducing event volume for large chains. Each `OptionChainSlice` joins the latest BBO and Greeks by instrument, then groups the result by strike and option kind. A quote can arrive before Greeks, and Greeks can arrive before a quote; the aggregator keeps latest state and attaches both when available. The `underlying_price` in `OptionGreeks` drives ATM detection. Selection can happen either in the subscription range or inside the strategy: - Moneyness: use `StrikeRange.atm_relative(...)` or `StrikeRange.atm_percent(...)`. - Delta: use `StrikeRange.delta(target, tolerance)`, or inspect `entry.greeks.delta` in `on_option_chain`. - Strike: use `StrikeRange.fixed([...])`, or read `chain.get_call(strike)` and `chain.get_put(strike)`. Matching is quote-driven for options. Market orders and marketable limits fill as takers against the opposing replayed BBO. Passive limit orders rest on the simulated book and can fill as makers when later BBO updates trade through the limit price. The model does not simulate L2 queue position for options. Structural option fee models are configured on the simulated venue, not inferred from the venue name: ```python from decimal import Decimal from nautilus_trader.execution import CappedOptionFeeModel from nautilus_trader.execution import TieredNotionalOptionFeeModel deribit_like = CappedOptionFeeModel( maker_rate=Decimal("0.0003"), taker_rate=Decimal("0.0003"), ) okx_like = TieredNotionalOptionFeeModel( maker_rate=Decimal("0.0002"), taker_rate=Decimal("0.0005"), ) ``` Pass one of these objects as `fee_model` on `BacktestVenueConfig`. The Rust surface uses `FeeModelAny::CappedOption(CappedOptionFeeModel::new(...))` and `FeeModelAny::TieredNotionalOption(TieredNotionalOptionFeeModel::new(...))`. See `examples/backtest/tardis_option_chain.py` and the Rust `tardis-option-chain` example in `crates/backtest/examples/`. ## Option chain architecture The option chain system is event-driven and built around per-series isolation. The `DataEngine` creates one Rust `OptionChainManager` per subscribed option series. The manager wraps `OptionChainAggregator` and `AtmTracker`, registers message bus handlers, publishes snapshots, and queues wire subscription changes for the engine to drain. A separate PyO3 `OptionChainManager` exposes the same aggregation core to Python. ```mermaid flowchart TD subgraph DataEngine DE[DataEngine] TMR[SnapshotTimer] end subgraph "OptionChainManager (per series)" MGR[OptionChainManager] AGG[OptionChainAggregator] ATM[AtmTracker] end DC[DataClient] -- QuoteTick --> DE DC -- OptionGreeks --> DE DE -- "handle_quote()" --> MGR DE -- "handle_greeks()" --> MGR MGR --> AGG MGR --> ATM ATM -- "forward price" --> AGG TMR -- "timer tick" --> DE DE -- "publish_slice()" --> MGR MGR -- "OptionChainSlice" --> DE DE -- publish --> MB((MessageBus)) MB -- "on_option_chain" --> S[Actor / Strategy] DE -- "sub/unsub" --> DC ``` ### Component responsibilities #### DataEngine Holds one `OptionChainManager` per active `OptionSeriesId`. On `SubscribeOptionChain`, it resolves instruments from the cache, requests forward prices for ATM-based ranges, creates the manager, subscribes active instruments to the data client, and sets up the snapshot timer. On each timer tick, the manager checks for rebalances, publishes a snapshot, and queues any wire subscription changes for the engine to drain. On `UnsubscribeOptionChain` or when all instruments expire, it tears down the manager, cancels the timer, and unsubscribes wire-level feeds. #### OptionChainManager A per-series Rust manager around `OptionChainAggregator` and `AtmTracker`. The `DataEngine` feeds it market data through `handle_quote()` and `handle_greeks()`. In snapshot mode, timer callbacks call `publish_slice()`. In raw mode, each active quote or Greeks update calls `publish_slice()` immediately. The Python-facing manager has `handle_*` methods that return whether the first ATM price bootstrapped the active instrument set; the Rust manager performs that bootstrap internally. #### OptionChainAggregator Accumulates quotes and Greeks into call/put buffers using keep-latest semantics. Instruments that did not update since the last snapshot are still included. Greeks that arrive before any quote for an instrument are held in a `pending_greeks` buffer and attached when the first quote arrives. On each `snapshot()` call, the aggregator produces an immutable `OptionChainSlice`. #### AtmTracker Derives the ATM price reactively from the `underlying_price` field in incoming `OptionGreeks` events (the venue-provided forward price for that expiry). It can be pre-seeded from an HTTP forward price response for instant bootstrap without waiting for WebSocket ticks. ### Bootstrap and rebalancing For ATM-based strike ranges (`AtmRelative`, `AtmPercent`), the active instrument set cannot be determined until the ATM price is known. There are two bootstrap paths: **Instant bootstrap (forward price available):** 1. `DataEngine` receives `SubscribeOptionChain`, resolves all instruments for the series from the cache, and requests forward prices from the data client. 2. When the forward price response arrives, the engine creates the manager with the ATM price pre-seeded. The manager computes the active strike set during construction. 3. The engine subscribes the active instruments immediately. **Deferred bootstrap (no forward price):** 1. Same as above, but no matching forward price is found in the response. 2. The engine creates the manager with no initial ATM price. The active set is empty and no wire subscriptions are made for the chain. 3. Bootstrap depends on relevant Greeks data already flowing from other subscriptions (e.g., per-instrument `subscribe_option_greeks` calls). When the engine feeds an `OptionGreeks` event with `underlying_price` through `handle_greeks()`, the manager bootstraps the active instrument set, registers message bus handlers, and queues the new wire subscriptions for the engine to drain. Once bootstrapped, the aggregator monitors ATM drift. On each snapshot timer tick, the engine calls `check_rebalance()` which returns any instruments to add or remove. A hysteresis threshold and cooldown period prevent thrashing near strike boundaries. ## OptionGreeks data type `OptionGreeks` carries venue-provided sensitivities and implied volatility for a single option contract: | Field | Type | Description | |--------------------|--------------------|-----------------------------------------------------| | `instrument_id` | `InstrumentId` | The option contract these Greeks apply to. | | `convention` | `GreeksConvention` | Numeraire convention for the Greeks. | | `delta` | `float` | Rate of change of option price per unit underlying. | | `gamma` | `float` | Rate of change of delta per unit underlying. | | `vega` | `float` | Sensitivity to a 1% change in implied volatility. | | `theta` | `float` | Daily time decay (dV/dt / 365.25). | | `rho` | `float` | Sensitivity to a change in interest rate. | | `mark_iv` | `float` or None | Mark implied volatility. | | `bid_iv` | `float` or None | Bid implied volatility. | | `ask_iv` | `float` or None | Ask implied volatility. | | `underlying_price` | `float` or None | Underlying price at time of calculation. | | `open_interest` | `float` or None | Open interest for the contract. | | `ts_event` | `int` | UNIX timestamp (nanoseconds) of the event. | | `ts_init` | `int` | UNIX timestamp (nanoseconds) when initialized. | ## OptionChainSlice data type `OptionChainSlice` is a point-in-time snapshot of an entire option series. Properties: | Property | Type | Description | |--------------|----------------------|-------------------------------------| | `series_id` | `OptionSeriesId` | The option series identifier. | | `atm_strike` | `Price` or None | Current ATM strike (if determined). | | `ts_event` | `int` | UNIX timestamp (nanoseconds). | | `ts_init` | `int` | UNIX timestamp (nanoseconds). | Call and put data are accessed through methods, not as direct properties. Each `OptionStrikeData` returned by these methods contains a `quote` (`QuoteTick`) and an optional `greeks` (`OptionGreeks`) for that strike. Methods: - `strikes()`: all unique strike prices in the chain. - `strike_count()`, `call_count()`, `put_count()`: counts. - `get_call(strike)`, `get_put(strike)`: full `OptionStrikeData`. - `get_call_greeks(strike)`, `get_put_greeks(strike)`: Greeks only. - `get_call_quote(strike)`, `get_put_quote(strike)`: quote only. - `is_empty()`: true if the chain has no data. ## Adapter support The following adapters currently support option Greeks subscriptions: | Adapter | Per‑instrument Greeks | Option chains | |---------|:---------------------:|:-------------:| | Deribit | ✓ | ✓ | | Bybit | ✓ | ✓ | | OKX | ✓ | - | ## See also - [Greeks](greeks.md) - Local Greeks calculation and portfolio risk management. - [Data](data.md) - Built-in data types and the subscription model. - [Actors](actors.md) - Subscription and handler reference table. # Order Book Source: https://nautilustrader.io/docs/latest/concepts/order_book/ NautilusTrader provides a high-performance order book implemented in Rust, capable of maintaining full book state from L1 through L3 data. The `OrderBook` is the primary component for tracking public market depth, while the `OwnOrderBook` tracks your own orders separately, enabling filtered views that show true available liquidity. :::note This guide documents the Rust API. These types are also available from Python via PyO3 bindings (`nautilus_pyo3.OrderBook`, `nautilus_pyo3.OwnOrderBook`). The v1 legacy Cython `OrderBook` (`nautilus_trader.model.book.OrderBook`) returned by `cache.order_book()` has a similar but not identical interface. Refer to the API reference for differences. ::: ## Book types `OrderBook` instances are maintained per instrument for both backtesting and live trading: - `L3_MBO`: **Market by order** data. Tracks every order at every price level, keyed by order ID. - `L2_MBP`: **Market by price** data. Aggregates orders by price level (one entry per price). - `L1_MBP`: **Top-of-book** data, also known as best bid and offer (BBO). Captures only the best prices. :::note Top-of-book data such as `QuoteTick`, `TradeTick` and `Bar` can also maintain `L1_MBP` books. ::: ## Subscribing to book data Strategies and actors subscribe to order book updates through the following methods. Subscriptions and handlers are part of the Python strategy/actor layer: ```python # L3/L2 incremental deltas self.subscribe_order_book_deltas(instrument_id) # Aggregated depth snapshots (up to 10 levels) self.subscribe_order_book_depth(instrument_id) # Full book snapshots at a timed interval self.subscribe_order_book_at_interval(instrument_id, interval_ms=1000) ``` Each subscription type delivers data to the corresponding handler: ```python def on_order_book_deltas(self, deltas: OrderBookDeltas) -> None: ... def on_order_book_depth(self, depth: OrderBookDepth10) -> None: ... def on_order_book(self, order_book: OrderBook) -> None: ... ``` ## Accessing the book The `OrderBook` exposes top-of-book accessors: ```rust let best_bid: Option = book.best_bid_price(); let best_ask: Option = book.best_ask_price(); let spread: Option = book.spread(); let midpoint: Option = book.midpoint(); ``` ## Analysis methods The `OrderBook` supports market depth analysis and execution simulation: ```rust // Average fill price for a given quantity let avg_px = book.get_avg_px_for_quantity(quantity, OrderSide::Buy); // Average price and quantity for a target exposure (notional) let (price, qty, exposure) = book.get_avg_px_qty_for_exposure(target_exposure, OrderSide::Buy); // Cumulative quantity available at or better than a price let qty = book.get_quantity_for_price(price, OrderSide::Buy); // Quantity at a specific price level only let qty = book.get_quantity_at_level(price, OrderSide::Buy, 2); // Simulate fills against the book let fills: Vec<(Price, Quantity)> = book.simulate_fills(&order); // All crossed levels regardless of order quantity let levels = book.get_all_crossed_levels(OrderSide::Buy, price, 2); ``` ## Integrity checks The `book_check_integrity` function validates that the book state is consistent with its type: - **L1_MBP**: No more than one level per side. - **L2_MBP**: No more than one order per price level. - **L3_MBO**: No structural constraints (any number of orders at any level). - **All types**: Best bid must not exceed best ask (crossed book). Locked markets (bid == ask) are considered valid. These checks run internally during delta application. The instrument ID of incoming deltas is also validated against the book's instrument ID, returning `BookIntegrityError::InstrumentMismatch` on mismatch. ## Pretty printing Both `OrderBook` and `OwnOrderBook` provide a `pprint` method that renders the book as a human-readable table: ```rust book.pprint(5, None); book.pprint(5, Some(Decimal::new(1, 2))); // group_size = 0.01 ``` The `group_size` parameter buckets price levels into coarser groups for instruments with fine tick sizes. The output is a formatted table with bids on the left, prices in the center, and asks on the right. ## Own order book The `OwnOrderBook` tracks your own working orders separately from the public book. Market making and other quoting strategies use it to estimate available liquidity at each price level after subtracting their own orders. Execution engines maintain own books when `manage_own_order_books` is enabled. The cache updates an existing own book as order events change state. Eligible orders have a price and do not use `IOC` or `FOK` time in force. Terminal events may still clean up an existing own book entry, even when the order would not otherwise be eligible for tracking. ### Order lifecycle The `OwnOrderBook` tracks orders through their lifecycle. Orders are added when submitted or materialized from reconciliation, updated as state changes arrive, and removed when they close. Updates include accepted, pending update, pending cancel, partially filled, filled, canceled, expired, rejected, and denied states as supported by the order model. Each `OwnBookOrder` carries: - `client_order_id`: Client order ID used to reconcile the own book with cache state. - `venue_order_id`: Venue order ID when one has been assigned. - `side`, `price`, and `size`: Order side and the remaining own-book price level. - `order_type` and `time_in_force`: Order type metadata used by filters and diagnostics. - `status`: Current order status, such as `SUBMITTED`, `ACCEPTED`, or `PENDING_CANCEL`. - `ts_last`: Timestamp of the latest order event applied to this own-book order. - `ts_accepted`: Timestamp when the order was accepted by the venue. - `ts_submitted`: Timestamp when the order was submitted. - `ts_init`: Timestamp when the order was initialized. These fields let filtered views include or exclude own orders by status and acceptance time (see [Status and time filtering](#status-and-time-filtering)). ### Auditing The `audit_open_orders` method reconciles an own book against a set of valid client order IDs. Any own-book order not in the provided set is removed and logged as an audit error. `Cache::audit_own_order_books` builds this set from open and in-flight orders so submitted orders are not removed during normal venue latency windows. Live systems can run this audit periodically through the own-books audit interval. ### Querying ```rust // Check if a specific order is tracked let in_book = own_book.is_order_in_book(&client_order_id); // Get all tracked order IDs per side let bid_ids = own_book.bid_client_order_ids(); let ask_ids = own_book.ask_client_order_ids(); // Aggregated quantities per price level let bid_qty = own_book.bid_quantity(None, None, None, None, None); let ask_qty = own_book.ask_quantity(None, None, None, None, None); // Pretty print own_book.pprint(5, None); ``` ### Filtered views Subtract your own orders from the public book to see net available liquidity: ```rust // Filtered maps of price -> quantity (own orders subtracted) let net_bids = book.bids_filtered_as_map(Some(10), Some(&own_book), None, None, None); let net_asks = book.asks_filtered_as_map(Some(10), Some(&own_book), None, None, None); // Full filtered OrderBook with all analysis methods available let filtered = book.filtered_view(Some(&own_book), Some(10), None, None, None); let avg_px = filtered.get_avg_px_for_quantity(quantity, OrderSide::Buy); ``` The `filtered_view` method returns a new `OrderBook` with your own sizes subtracted, giving access to the full set of analysis methods (`spread`, `midpoint`, `get_avg_px_for_quantity`, etc.) on the net book. ### Status and time filtering Filtered views support optional status and time-based filtering for own orders: ```rust let status = Some(AHashSet::from([OrderStatus::Accepted])); // Only subtract ACCEPTED orders (ignore SUBMITTED, PENDING_CANCEL, etc.) let filtered = book.filtered_view(Some(&own_book), None, status, None, None); ``` The `accepted_buffer_ns` parameter provides a grace period: when set, only orders where `ts_accepted + buffer <= now` are included. This excludes recently accepted orders that may not yet appear in the public book feed. The buffer applies to the `ts_accepted` field regardless of order status. Combine with a status filter to also exclude non-accepted orders. ```rust // Only subtract orders accepted at least 500ms ago let filtered = book.filtered_view( Some(&own_book), None, None, Some(500_000_000), Some(clock.timestamp_ns()), ); ``` ## Binary markets For binary/prediction markets (e.g., Polymarket), instruments have two complementary sides (YES and NO) where prices sum to 1.0. A bid on the NO side at 0.40 is economically equivalent to an ask on the YES side at 0.60. The `OwnOrderBook::combined_with_opposite` method handles this transformation, merging your orders from both sides into a single view: ```rust let yes_own = own_yes_book .cloned() .unwrap_or_else(|| OwnOrderBook::new(yes_instrument_id)); let no_own = own_no_book .cloned() .unwrap_or_else(|| OwnOrderBook::new(no_instrument_id)); // Merge NO-side orders with parity price transform (1 - price) let combined = yes_own.combined_with_opposite(&no_own).unwrap(); // Filter the public YES book using the combined own book let filtered = book.filtered_view(Some(&combined), None, None, None, None); ``` The transformation works as follows: - NO asks at price P become bids at price 1 - P in the combined book. - NO bids at price P become asks at price 1 - P in the combined book. This gives a complete picture of your own liquidity across both sides of the market. # Overview Source: https://nautilustrader.io/docs/latest/concepts/overview/ ## Introduction NautilusTrader is an open-source, production-grade, Rust-native engine for multi-asset, multi-venue trading systems. The system spans research, deterministic simulation, and live execution within a single event-driven architecture, with Python serving as the control plane for strategy logic, configuration, and orchestration. This separation provides the performance and safety of a compiled trading engine with the flexibility of Python for system composition and strategy development. Trading systems can also be written entirely in Rust for mission-critical workloads. The same execution semantics and deterministic time model operate in both research and live systems. Strategies deploy from research to production with no code changes, providing research-to-live parity and reducing the divergence that typically introduces deployment risk. NautilusTrader is asset-class-agnostic. Any venue with a REST API or WebSocket feed can be integrated through modular adapters. Current integrations span crypto exchanges (CEX and DEX), traditional markets (FX, equities, futures, options), and betting exchanges. ## Features - **Fast**: Rust core with asynchronous networking using [tokio](https://crates.io/crates/tokio). - **Reliable**: Type- and thread-safety backed by Rust, with optional Redis-backed state persistence. - **Portable**: Runs on Linux, macOS, and Windows. Deploy using Docker. - **Flexible**: Modular adapters integrate any REST API or WebSocket feed. - **Advanced**: Time in force `IOC`, `FOK`, `GTC`, `GTD`, `DAY`, `AT_THE_OPEN`, `AT_THE_CLOSE`, advanced order types and conditional triggers. Execution instructions `post-only`, `reduce-only`, and icebergs. Contingency orders including `OCO`, `OUO`, `OTO`. - **Customizable**: User-defined components, or assemble entire systems from scratch using the [cache](cache.md) and [message bus](message_bus.md). - **Backtesting**: Multiple venues, instruments, and strategies simultaneously using historical quote tick, trade tick, bar, order book, and custom data with nanosecond resolution. - **Live**: Identical strategy implementations between research and live deployment. - **Multi-venue**: Run market-making and cross-venue strategies across multiple venues simultaneously. - **AI training**: Engine fast enough to train AI trading agents (RL/ES). ## Why NautilusTrader? Trading strategy research typically happens in Python using vectorized approaches, while production trading systems are built separately using event-driven architectures in compiled languages. NautilusTrader removes this separation. A Rust-native core provides a deterministic event-driven runtime for both research and live execution, while Python serves as the control plane. The same architecture, execution semantics, and time model operate across both environments, allowing strategies to move from research to production without reimplementation. Python bindings are provided via [PyO3](https://pyo3.rs), with an ongoing migration from Cython. No Rust toolchain is required at install time. ## Use cases There are three main use cases for this software package: - Backtest trading systems on historical data (`backtest`). - Simulate trading systems with real-time data and virtual execution (`sandbox`). - Deploy trading systems live on real or paper accounts (`live`). The codebase provides a framework for building the software layer of systems that achieve the above. The default `backtest` and `live` system implementations live in their respectively named subpackages. A `sandbox` environment can be built using the sandbox adapter. :::note - All examples will use these default system implementations. - We consider trading strategies to be subcomponents of end-to-end trading systems, these systems include the application and infrastructure layers. ::: ## Distributed The platform integrates into larger distributed systems. Nearly all configuration and domain objects serialize using JSON, MessagePack, or Apache Arrow (Feather) for communication over the network. ## Common core The common system core is used by all node [environment contexts](architecture.md#environment-contexts) (`backtest`, `sandbox`, and `live`). User-defined `Actor`, `Strategy` and `ExecAlgorithm` components are managed consistently across these environment contexts. ## Backtesting Feed data to a `BacktestEngine` either directly or through a higher-level `BacktestNode` and `ParquetDataCatalog`, then run the data through the system with nanosecond resolution. ## Live trading A `TradingNode` ingests data and events from multiple data and execution clients, supporting both demo/paper trading accounts and real accounts. Running asynchronously on a single [event loop](https://docs.python.org/3/library/asyncio-eventloop.html) provides high performance, with the option to use the [uvloop](https://github.com/MagicStack/uvloop) implementation (available for Linux and macOS) for additional throughput. ## Domain model The platform features a trading domain model that includes various value types such as `Price` and `Quantity`, as well as more complex entities such as `Order` and `Position` objects, which are used to aggregate multiple events to determine state. ## Timestamps All timestamps use nanosecond precision in UTC. Timestamp strings follow ISO 8601 (RFC 3339) format with either 9 digits (nanoseconds) or 3 digits (milliseconds) of decimal precision, (but mostly nanoseconds) always maintaining all digits including trailing zeros. These can be seen in log messages, and debug/display outputs for objects. A timestamp string consists of: - Full date component always present: `YYYY-MM-DD`. - `T` separator between date and time components. - Always nanosecond precision (9 decimal places) or millisecond precision (3 decimal places) for certain cases such as GTD expiry times. - Always UTC timezone designated by `Z` suffix. Example: `2024-01-05T15:30:45.123456789Z` For the complete specification, refer to [RFC 3339: Date and Time on the Internet](https://datatracker.ietf.org/doc/html/rfc3339). ## UUIDs The platform uses Universally Unique Identifiers (UUID) version 4 (RFC 4122) for unique identifiers. Our high-performance implementation uses the `uuid` crate for correctness validation when parsing from strings, ensuring input UUIDs comply with the specification. A valid UUID v4 consists of: - 32 hexadecimal digits displayed in 5 groups. - Groups separated by hyphens: `8-4-4-4-12` format. - Version 4 designation (indicated by the third group starting with "4"). - RFC 4122 variant designation (indicated by the fourth group starting with "8", "9", "a", or "b"). Example: `2d89666b-1a1e-4a75-b193-4eb3b454c757` For the complete specification, refer to [RFC 4122: A Universally Unique Identifier (UUID) URN Namespace](https://datatracker.ietf.org/doc/html/rfc4122). ## Data types The following market data types can be requested historically, and also subscribed to as live streams when available from a venue / data provider, and implemented in an integrations adapter. - `OrderBookDelta` (L1/L2/L3) - `OrderBookDeltas` (container type) - `OrderBookDepth10` (fixed depth of 10 levels per side) - `QuoteTick` - `TradeTick` - `Bar` - `Instrument` - `InstrumentStatus` - `InstrumentClose` The following `PriceType` options can be used for bar aggregations: - `BID` - `ASK` - `MID` - `LAST` ## Bar aggregations The following `BarAggregation` methods are available: - `MILLISECOND` - `SECOND` - `MINUTE` - `HOUR` - `DAY` - `WEEK` - `MONTH` - `YEAR` - `TICK` - `VOLUME` - `VALUE` (a.k.a Dollar bars) - `RENKO` (price-based bricks) - `TICK_IMBALANCE` - `TICK_RUNS` - `VOLUME_IMBALANCE` - `VOLUME_RUNS` - `VALUE_IMBALANCE` - `VALUE_RUNS` All listed aggregations are implemented for internal aggregation. Information-driven aggregations require `TradeTick` data. The price types and bar aggregations can be combined with step sizes >= 1 in any way through a `BarSpecification`. This allows alternative bars to be aggregated for live trading. ## Account types The following account types are available for both live and backtest environments: - `Cash` single-currency (base currency) - `Cash` multi-currency - `Margin` single-currency (base currency) - `Margin` multi-currency - `Betting` single-currency ## Order types The following order types are available (when possible on a venue): - `MARKET` - `LIMIT` - `STOP_MARKET` - `STOP_LIMIT` - `MARKET_TO_LIMIT` - `MARKET_IF_TOUCHED` - `LIMIT_IF_TOUCHED` - `TRAILING_STOP_MARKET` - `TRAILING_STOP_LIMIT` ## Value types The following value types are backed by either 128-bit or 64-bit raw integer values, depending on the [precision mode](../getting_started/installation.md#precision-mode) used during compilation. - `Price` - `Quantity` - `Money` ### High-precision mode (128-bit) When the `high-precision` feature flag is **enabled** (default), values use the specification: | Type | Raw backing | Max precision | Min value | Max value | |:-------------|:------------|:--------------|:--------------------|:-------------------| | `Price` | `i128` | 16 | -17,014,118,346,046 | 17,014,118,346,046 | | `Money` | `i128` | 16 | -17,014,118,346,046 | 17,014,118,346,046 | | `Quantity` | `u128` | 16 | 0 | 34,028,236,692,093 | ### Standard-precision mode (64-bit) When the `high-precision` feature flag is **disabled**, values use the specification: | Type | Raw backing | Max precision | Min value | Max value | |:-------------|:------------|:--------------|:--------------------|:-------------------| | `Price` | `i64` | 9 | -9,223,372,036 | 9,223,372,036 | | `Money` | `i64` | 9 | -9,223,372,036 | 9,223,372,036 | | `Quantity` | `u64` | 9 | 0 | 18,446,744,073 | # Plugins Source: https://nautilustrader.io/docs/latest/concepts/plugins/ The `nautilus-plugin` crate defines the plug-in artifact contract for NautilusTrader. It lets an independently compiled Rust `cdylib` identify itself with versioned build metadata and a manifest, and provides the C-ABI boundary primitives used at that boundary. It covers artifact identity and the boundary types only; it does not load, register, or run plug-ins. :::warning The plug-in ABI is early alpha and the contract is unstable. Pin plug-in builds to the matching `nautilus-plugin` version. ::: A plug-in is a Rust `cdylib` that exports a single `nautilus_plugin_init` entry symbol. The `nautilus_plugin!` macro generates that symbol and the static manifest that carries the build identity: ```rust nautilus_plugin::nautilus_plugin! { name: "example-plugin", vendor: "Nautech", version: env!("CARGO_PKG_VERSION"), } ``` Set `crate-type = ["cdylib"]` in the artifact's `Cargo.toml` and depend on the matching `nautilus-plugin` version. See [the crate docs](https://docs.rs/nautilus-plugin) for the boundary and manifest types. # Portfolio Source: https://nautilustrader.io/docs/latest/concepts/portfolio/ The Portfolio is the central hub for managing and tracking all positions across active strategies for the trading node or backtest. It consolidates position data from multiple instruments, providing a unified view of your holdings, risk exposure, and overall performance. ## Currency conversion The Portfolio supports automatic currency conversion for PnL and exposure calculations, allowing you to view results in your preferred currency. This is particularly useful when trading across multiple instruments with different settlement currencies or managing multiple accounts with different base currencies. ### Supported conversions Currency conversion is available for the following portfolio queries: - `realized_pnl()` / `realized_pnls()` - Convert realized PnL to target currency. - `unrealized_pnl()` / `unrealized_pnls()` - Convert unrealized PnL to target currency. - `total_pnl()` / `total_pnls()` - Convert total PnL to target currency. - `net_exposure()` / `net_exposures()` - Convert net exposure to target currency. All methods accept an optional `target_currency` parameter to specify the desired output currency. ### Single account behavior When querying a single account without specifying `target_currency`, the Portfolio automatically converts values to that account's base currency: ```python # Returns exposure in the account's base currency (e.g., USD) exposure = portfolio.net_exposures(venue=BINANCE, account_id=account_id) ``` ### Multi-account behavior When querying multiple accounts simultaneously, behavior depends on whether you query all instruments (`net_exposures()`) or a single instrument (`net_exposure()`): **For `net_exposures()` (all instruments):** - **Same base currency**: Automatically converts to the common base currency. - **Different base currencies**: Returns a dict with multiple currencies, each converted to its account's base currency. Provide `target_currency` for single-currency results. **For `net_exposure()` (single instrument across accounts):** - **Different base currencies**: Returns `None` unless you provide `target_currency`. ```python # Scenario 1: Multiple accounts, all with USD base currency exposures = portfolio.net_exposures(venue=BINANCE) # Returns {USD: Money(...)} # Scenario 2: Multiple accounts with different base currencies (USD and EUR) exposures = portfolio.net_exposures(venue=BINANCE) # Returns {USD: Money(...), EUR: Money(...)} # Force single currency across accounts exposures = portfolio.net_exposures(venue=BINANCE, target_currency=USD) # Returns {USD: Money(...)} ``` ### Conversion failures When `target_currency` is provided and currency conversion fails, behavior depends on the method type: - **Single-value methods** (`realized_pnl`, `unrealized_pnl`, `total_pnl`, `net_exposure`): Return `None` and log an error to prevent incorrect values. - **Dict-returning methods** (`realized_pnls`, `unrealized_pnls`, `total_pnls`, `net_exposures`): Omit instruments that fail conversion but return results for successful conversions. :::warning Exchange rate data must be available when using `target_currency` for cross-currency aggregation. ::: ### Conversion price types When converting exposures to a target currency, the Portfolio uses different price types depending on the position composition: - **All long positions**: Uses `BID` prices (conservative for long exposure). - **All short positions**: Uses `ASK` prices (conservative for short exposure). - **Mixed positions**: Uses `MID` prices (neutral when both long and short exist). This ensures conversions reflect realistic market conditions where you would liquidate long positions at bid and cover short positions at ask. For mixed positions, mid-pricing provides a neutral valuation. If `use_mark_xrates` is enabled in the portfolio configuration, `MARK` prices replace `MID` prices for mixed positions and general conversions. ## Equity and mark-to-market The Portfolio exposes three pull-style queries for continuous portfolio valuation. Each returns per-currency results keyed by the relevant account base currency or native settlement currency. | Method | Returns | |------------------------------------------|------------------------------------------------------------------| | `mark_values(venue, account_id)` | Signed MTM totals for open positions. | | `equity(venue, account_id)` | Total equity combining balance and position valuation. | | `missing_price_instruments(venue)` | Instruments currently flagged as unpriceable. | Longs contribute positive notional, shorts contribute negative notional. Flat positions are skipped. ### Equity formula Equity combines the account balance with open-position valuation, using a different second term depending on account type: - **Cash and betting accounts**: `balances_total + Σ mark_value(open positions)`. - **Margin accounts**: `balances_total + Σ unrealized_pnl(open positions)`. The cash and betting path uses `mark_values()` internally. The margin path uses the same cached unrealized PnL pipeline that powers `unrealized_pnls()`. ### Price fallback Valuation asks `Cache` for a price in this order, stopping at the first match: 1. Mark price, if `use_mark_prices=true` in `PortfolioConfig` and a mark price is cached. 2. Side-appropriate quote: `BID` for longs, `ASK` for shorts. 3. Last trade price. 4. Most recent cached bar close (populated when `bar_updates=true`). If none of the four yield a price, the position goes into the missing-price tracker and is skipped in the sum. ### Base currency conversion When `convert_to_account_base_currency=true` (the default) and the account has a `base_currency` set, settlement-currency values are converted to the base currency using `MID` xrates from `Cache.get_xrate()`. With `use_mark_xrates=true`, the cached mark xrate from `Cache.get_mark_xrate()` is used first and falls back to `MID` if unavailable. The output dictionary then has a single key matching the base currency. When `convert_to_account_base_currency=false`, or the account has no `base_currency`, results are keyed by each position's native settlement currency and no xrate conversion is applied. If xrate data is unavailable for a required conversion, that position is treated as unpriceable and flagged via the missing-price tracker rather than silently valued at a 1.0 rate. ### Missing-price tracking The tracker is a per-venue set of instrument IDs that could not be priced on the last `mark_values()` or `equity()` call. It has two observable behaviors: - A warning log fires once per instrument on the transition from priced to unpriced, not on every subsequent call. The next transition back to priced clears the entry so a future drop re-warns. - When a venue goes flat (no open positions), its tracker entry is cleared so stale instruments do not remain flagged. Call `missing_price_instruments(venue)` to inspect the current set. :::tip If `equity()` understates what you expect, check `missing_price_instruments(venue)` before investigating the math. An empty quote, trade, and bar feed for one instrument is the most common cause of silent gaps. ::: ### Venue and account scope `mark_values` and `equity` accept an optional `account_id` to scope the aggregation to a single account. With `account_id=None`, results aggregate across every account on the venue. The missing-price tracker is venue-scoped. `missing_price_instruments` takes a venue only, and an account-filtered `mark_values(venue, account_id)` call does not clear the venue entry so flags raised by other accounts on the same venue survive. ## Portfolio statistics There are a variety of built-in portfolio statistics in `crates/analysis/src/statistics` which analyse a trading portfolio's performance for both backtests and live trading. The statistics are generally categorized as follows. - PnLs based statistics (per currency) - Returns based statistics - Positions based statistics - Orders based statistics Backtest statistics are exposed after a run through `engine.get_result()`. ## Custom statistics Custom metrics for post-run analysis can be computed from reports, snapshots, or position data and added to the dictionaries passed to visualization APIs such as `create_tearsheet_from_stats()`. For example, calculate a win rate from realized PnLs: ```python import pandas as pd def calculate_win_rate(realized_pnls: pd.Series) -> float: if realized_pnls.empty: return 0.0 winners = realized_pnls[realized_pnls > 0.0] return len(winners) / len(realized_pnls) ``` Then include the metric in offline tearsheet inputs: ```python stats_general = { "Win Rate": calculate_win_rate(realized_pnls), } ``` :::tip Your metric should handle degenerate inputs such as empty series or insufficient data. Return `None` for unknown or incalculable values, or a reasonable default like `0.0` when semantically appropriate. ::: ## Returns: position vs portfolio The analyzer tracks two distinct return series: - **Position returns** (`analyzer.position_returns()`) measure realized return per position as a side-aware price return relative to the average open price. This reflects the instrument's price movement between entry and exit, independent of account size or leverage. - **Portfolio returns** (`analyzer.portfolio_returns()`) measure daily percentage change in total account balance. A $900 gain on a $100,000 account reports roughly 0.9% for that day. When the analyzer has account state history spanning at least two distinct calendar days, it computes portfolio returns automatically and uses them as the primary series for statistics, tearsheets, and the monthly returns heatmap. Multiple snapshots on the same day count as one day, so intra-day trading alone does not produce portfolio returns. When portfolio returns are unavailable, it falls back to position returns. The convenience accessor `analyzer.returns()` resolves this preference: portfolio returns when present, position returns otherwise. ### Multi-currency accounts Portfolio returns require a single-currency balance history. When the account carries balances in multiple currencies, the analyzer cannot produce a single return series and falls back to position returns silently. Statistics and tearsheet charts use whichever series `returns()` resolves to. If you need portfolio-level returns for a multi-currency account, compute them externally by converting balances to a common currency before calculating percentage changes. ### Per-venue calculation In the backtest engine, the analyzer runs per venue (`engine.pyx`). Each venue's account produces its own portfolio return series. The tearsheet aggregates across all cached accounts to produce a combined return series for multi-venue backtests. ## Backtest analysis Following a backtest run, the engine passes realized PnLs, returns, positions, and orders data to each registered statistic. Any output is then displayed in the tear sheet under the `Portfolio Performance` heading, grouped as: - Realized PnL statistics (per currency) - Returns statistics (for the entire portfolio) - General statistics derived from position and order data (for the entire portfolio) ## Related guides - [Positions](positions.md) - Position tracking within portfolios. - [Reports](reports.md) - Generate portfolio analysis reports. - [Visualization](visualization.md) - Visualize portfolio performance. # Positions Source: https://nautilustrader.io/docs/latest/concepts/positions/ This guide explains how positions work in NautilusTrader, including their lifecycle, aggregation from order fills, profit and loss calculations, and the important concept of position snapshotting for netting OMS configurations. ## Overview A position represents an open exposure to a particular instrument in the market. Positions are fundamental to tracking trading performance and risk, as they aggregate all fills for a particular instrument and continuously calculate metrics like unrealized PnL, average entry price, and total exposure. The system automatically creates positions when orders fill and tracks them from open to close. The platform supports both netting and hedging position management styles through its OMS (Order Management System) configuration. ## Position lifecycle ### Creation The system opens a position on the first fill: - **NETTING OMS**: Opens on first fill for an instrument (one position per instrument). - **HEDGING OMS**: Opens on first fill for a new `position_id` (multiple positions per instrument). A position tracks: - Opening order and fill details. - Entry side (`LONG` or `SHORT`). - Initial quantity and average price. - Timestamps for initialization and opening. :::tip You can access positions through the Cache using `self.cache.position(position_id)` or `self.cache.positions(instrument_id=instrument_id)` from within your actors/strategies. ::: ### Updates As additional fills occur, the position: - Aggregates quantities from buy and sell fills. - Recalculates average entry and exit prices. - Updates peak quantity (maximum exposure reached). - Tracks all associated order IDs and trade IDs. - Accumulates commissions by currency. ### Closure A position closes when the net quantity becomes zero (`FLAT`). At closure: - The closing order ID is recorded. - Duration is calculated from open to close. - Final realized PnL is computed. - In `NETTING` OMS, when the position later reopens, the engine snapshots the closed state to preserve historical PnL (see [Position snapshotting](#position-snapshotting)). ## Order fill aggregation Positions aggregate order fills to maintain an accurate view of market exposure. The aggregation process handles both sides of trading activity: ### Buy fills When a BUY order fills: - Increases long exposure or reduces short exposure. - Updates average entry price for opening trades. - Updates average exit price for closing trades. - Calculates realized PnL for any closed portion. ### Sell fills When a SELL order fills: - Increases short exposure or reduces long exposure. - Updates average entry price for opening trades. - Updates average exit price for closing trades. - Calculates realized PnL for any closed portion. ### Net position calculation The position maintains a `signed_qty` field representing the net exposure: - Positive values indicate `LONG` positions. - Negative values indicate `SHORT` positions. - Zero indicates a `FLAT` (closed) position. ```python # Example: Position aggregation # Initial BUY 100 units at $50 signed_qty = +100 # LONG position # Subsequent SELL 150 units at $55 signed_qty = -50 # Now SHORT position # Final BUY 50 units at $52 signed_qty = 0 # Position FLAT (closed) ``` ## Position adjustments Position adjustments track quantity or PnL changes that occur outside of normal order fills, ensuring the position quantity accurately reflects the true net asset position. The system generates `PositionAdjusted` events for these scenarios. ### Base currency commissions When trading spot currency pairs (e.g., BTC/USDT) or FX spot, commissions paid in the base currency directly affect the net quantity received or delivered: - **Opening fills**: Commission is deducted from the traded quantity. A buy of 1.0 BTC with 0.001 BTC commission results in a net long position of 0.999 BTC. - **Closing fills**: Commission is applied to `signed_qty` because it affects actual inventory. Selling a 0.999 BTC LONG position with 0.000999 BTC commission leaves you SHORT 0.000999 BTC, not FLAT, because you gave up 0.999999 BTC total. - **Flips**: Commission affects the final position size on both sides of the flip. :::note Base currency commissions only apply to spot currency pairs and FX spot instruments where the commission currency matches `instrument.base_currency`. For other instruments, commissions are tracked separately and do not affect position quantity. ::: ### Funding payments Funding adjustments track periodic payments for perpetual futures without affecting position quantity. These are logged with `quantity_change = None` and can include PnL impacts. ### Adjustment tracking All adjustments are preserved in the position event history: - `position.adjustments` returns the list of all `PositionAdjusted` events. - Each adjustment includes type (`COMMISSION` or `FUNDING`), quantity change, and timestamps. - The adjustment history is cleared when positions close and reopen. When events are purged, commission adjustments tied to the removed fills are regenerated while non-commission adjustments (for example funding) are preserved. ## OMS types and position management NautilusTrader supports two primary OMS types that fundamentally affect how positions are tracked and managed. An `OmsType.UNSPECIFIED` option also exists, which defaults to the component's context. For full details, see the [Execution guide](execution.md#order-management-system-oms). ### `NETTING` In `NETTING` mode, all fills for an instrument are aggregated into a single position: - One position per instrument ID. - All fills contribute to the same position. - Position flips from `LONG` to `SHORT` (or vice versa) as net quantity changes. - Historical snapshots preserve closed position states. ### `HEDGING` In `HEDGING` mode, multiple positions can exist for the same instrument: - Multiple simultaneous `LONG` and `SHORT` positions. - Each position has a unique position ID. - Positions are tracked independently. - No automatic netting across positions. - Closed positions remain in cache history but do not reopen; new fills create new positions. :::warning When using `HEDGING` mode, be aware of increased margin requirements as each position consumes margin independently. Some venues may not support true hedging mode and will net positions automatically. ::: ### Strategy vs venue OMS The platform allows different OMS configurations for strategies and venues: | Strategy OMS | Venue OMS | Behavior | |--------------|-----------|-------------------------------------------------------------| | `NETTING` | `NETTING` | Single position per instrument at both strategy and venue. | | `HEDGING` | `HEDGING` | Multiple positions supported at both levels. | | `NETTING` | `HEDGING` | Venue tracks multiple, Nautilus maintains single position. | | `HEDGING` | `NETTING` | Venue tracks single, Nautilus maintains virtual positions. | :::tip For most trading scenarios, keeping strategy and venue OMS types aligned simplifies position management. Override configurations are primarily useful for prop trading desks or when interfacing with legacy systems. See the [Live guide](live.md) for venue-specific OMS configuration. ::: ## Position snapshotting Position snapshotting is an important feature for `NETTING` OMS configurations that preserves the state of closed positions for accurate PnL tracking and reporting. ### Why snapshotting matters In a `NETTING` system, when a position closes (becomes `FLAT`) and then reopens with a new trade, the position object is reset to track the new exposure. Without snapshotting, the historical realized PnL from the previous position cycle would be lost. ### How it works When a `NETTING` position closes and then receives a new fill for the same instrument, the execution engine snapshots the closed position state before resetting it, preserving: - Final quantities and prices. - Realized PnL. - All fill events. - Commission totals. This snapshot is stored in the cache indexed by position ID. The position then resets for the new cycle while previous snapshots remain accessible. The Portfolio aggregates PnL across all snapshots for accurate totals. :::note This historical snapshot mechanism differs from optional position state snapshots (`snapshot_positions`), which periodically record open-position state for telemetry. See the [Live guide](live.md) for `snapshot_positions` and `snapshot_positions_interval_secs` settings. ::: ### Example scenario ```python # NETTING OMS Example # Cycle 1: Open LONG position BUY 100 units at $50 # Position opens SELL 100 units at $55 # Position closes, PnL = $500 # Snapshot taken preserving $500 realized PnL # Cycle 2: Open SHORT position SELL 50 units at $54 # Position reopens (SHORT) BUY 50 units at $52 # Position closes, PnL = $100 # Snapshot taken preserving $100 realized PnL # Total realized PnL = $500 + $100 = $600 (from snapshots) ``` Without snapshotting, only the most recent cycle's PnL would be available, leading to incorrect reporting and analysis. ## PnL calculations NautilusTrader provides PnL calculations that account for instrument specifications and market conventions. ### Realized PnL Calculated when positions are partially or fully closed: ```python # For standard instruments realized_pnl = (exit_price - entry_price) * closed_quantity * multiplier # For inverse instruments (side-aware) # LONG: realized_pnl = closed_quantity * multiplier * (1/entry_price - 1/exit_price) # SHORT: realized_pnl = closed_quantity * multiplier * (1/exit_price - 1/entry_price) ``` The engine automatically applies the correct formula based on position side. ### Unrealized PnL Calculated using current market prices for open positions. The `price` parameter accepts any reference price (bid, ask, mid, last, or mark): ```python position.unrealized_pnl(last_price) # Using last traded price position.unrealized_pnl(bid_price) # Conservative for LONG positions position.unrealized_pnl(ask_price) # Conservative for SHORT positions ``` Returns `Money(0, settlement_currency)` for `FLAT` positions regardless of the price provided. ### Total PnL Combines realized and unrealized components: ```python total_pnl = position.total_pnl(current_price) # Returns realized_pnl + unrealized_pnl ``` ### Currency considerations - PnL is calculated in the instrument's settlement currency. - For Forex, this is typically the quote currency. - For inverse contracts, PnL may be in the base currency. - Portfolio aggregates realized PnL per instrument in settlement currency. - Multi-currency totals require conversion outside the Position class. ## Commissions and costs Positions track all trading costs: - Commissions are accumulated by currency. - Each fill's commission is added to the running total. - Multiple commission currencies are supported. - Realized PnL includes commissions only when denominated in the settlement currency. - Other commissions are tracked separately and may require conversion. ```python commissions = position.commissions() # Returns list[Money] with aggregated commission totals per currency notional = position.notional_value(current_price) # Returns Money in quote currency (standard) or base currency (inverse) ``` **Limitations:** - Panics if inverse instrument has no `base_currency` set. - Does not handle quanto contracts (returns quote currency instead of settlement currency). - For quanto instruments, use `instrument.calculate_notional_value()` instead. ## Position properties and state ### Identifiers - `id`: Unique position identifier. - `instrument_id`: The traded instrument. - `account_id`: Account where position is held. - `trader_id`: The trader who owns the position. - `strategy_id`: The strategy managing the position. - `opening_order_id`: Client order ID that opened the position. - `closing_order_id`: Client order ID that closed the position. ### Position state - `side`: Current position side (`LONG`, `SHORT`, or `FLAT`). - `entry`: Direction of the currently open position (`Buy` for `LONG`, `Sell` for `SHORT`). Updates when position flips direction. - `quantity`: Current absolute position size. - `signed_qty`: Signed position size (positive for `LONG`, negative for `SHORT`). - `peak_qty`: Maximum quantity reached during position lifetime. - `is_open`: Whether position is currently open. - `is_closed`: Whether position is closed (`FLAT`). - `is_long`: Whether position side is `LONG`. - `is_short`: Whether position side is `SHORT`. ### Pricing and valuation - `avg_px_open`: Average entry price. - `avg_px_close`: Average exit price when closing. - `realized_pnl`: Realized profit/loss. - `realized_return`: Realized return as decimal (e.g., 0.05 for 5%). - `quote_currency`: Quote currency of the instrument. - `base_currency`: Base currency if applicable. - `settlement_currency`: Currency for PnL settlement. ### Instrument specifications - `multiplier`: Contract multiplier. - `price_precision`: Decimal precision for prices. - `size_precision`: Decimal precision for quantities. - `is_inverse`: Whether instrument is inverse. ### Timestamps - `ts_init`: When position was initialized. - `ts_opened`: When position was opened. - `ts_last`: Last update timestamp. - `ts_closed`: When position was closed. - `duration_ns`: Duration from open to close in nanoseconds. ### Associated data - `symbol`: The instrument's ticker symbol. - `venue`: The trading venue. - `client_order_ids`: All client order IDs associated with position. - `venue_order_ids`: All venue order IDs associated with position. - `trade_ids`: All trade/fill IDs from venue. - `events`: All order fill events applied to position. - `event_count`: Total number of fill events applied. - `last_event`: Most recent fill event. - `last_trade_id`: Most recent trade ID. :::info For complete type information and detailed property documentation, see the Position [API Reference](/docs/python-api-latest/model/position.html#nautilus_trader.model.position.Position). ::: ## Events and tracking Positions maintain a complete history of events: - All order fill events are stored chronologically. - Associated client order IDs are tracked. - Trade IDs from the venue are preserved. - Event count indicates total fills applied. This historical data enables: - Detailed position analysis. - Trade reconciliation. - Performance attribution. - Audit trails. :::tip Use `position.events` to access the full history of fills for reconciliation. The `position.trade_ids` property helps match against broker statements. See the [Execution guide](execution.md) for reconciliation best practices. ::: ## Numerical precision Position calculations use 64-bit floating-point (`f64`) arithmetic for PnL and average price computations. While fixed-point types (`Price`, `Quantity`, `Money`) preserve exact precision at configured decimal places, internal calculations convert to `f64` for performance and overflow safety. ### Design rationale The platform uses `f64` for position calculations to balance performance and accuracy: - Floating-point operations are significantly faster than arbitrary-precision arithmetic. - Raw integer multiplication can overflow even with 128-bit integers. - Each calculation starts from precise fixed-point values, avoiding cumulative error. - IEEE-754 double precision provides ~15 decimal digits of accuracy. ### Validated precision characteristics Testing confirms `f64` arithmetic maintains accuracy for typical trading scenarios: - Standard amounts: No precision loss for amounts ≥ 0.01 in standard currencies. - High-precision instruments: 9-decimal crypto prices preserved within 1e-6 tolerance. - Sequential fills: 100 fills show no drift (commission accuracy to 1e-10). - Extreme prices: Handles range from 0.00001 to 99,999.99999 without overflow. - Round-trip trades: Opening and closing at same price produces exact PnL (commissions only). For implementation details, see `test_position_pnl_precision_*` tests in `crates/model/src/position.rs`. :::note For regulatory compliance or audit trails requiring exact decimal arithmetic, consider using `Decimal` types from external libraries. Very small amounts below `f64` epsilon (~1e-15) may round to zero. This does not affect realistic trading scenarios with standard currency precisions (typically 2-9 decimals). ::: ## Integration with other components Positions interact with several key components: - **Portfolio**: Aggregates positions across instruments and strategies. - **ExecutionEngine**: Creates and updates positions from fills. - **Cache**: Stores position state and snapshots. - **RiskEngine**: Monitors position limits and exposure. :::note Positions are not created for spread instruments. While contingent orders can still trigger for spreads, they operate without position linkage. The engine handles spread instruments separately from regular positions. ::: ## Summary Positions are central to tracking trading activity and performance. Understanding how positions aggregate fills, calculate PnL, and handle different OMS configurations matters when building trading strategies. Position snapshotting provides accurate historical tracking in `NETTING` mode, and the event history supports detailed analysis and reconciliation. ## Related guides - [Events](events.md) - How fills produce position events. - [Orders](orders/) - Orders that create and modify positions. - [Execution](execution.md) - Fill handling that updates positions. - [Portfolio](portfolio.md) - Portfolio-level position aggregation. # Reports Source: https://nautilustrader.io/docs/latest/concepts/reports/ This guide explains the portfolio analysis and reporting capabilities provided by the `ReportProvider` class, and how these reports are used for PnL accounting and backtest post-run analysis. ## Overview The `ReportProvider` class in NautilusTrader generates structured analytical reports from trading data, transforming raw orders, fills, positions, and account states into pandas DataFrames for analysis and visualization. These reports help you evaluate strategy performance, analyze execution quality, and verify PnL accounting. Reports can be generated using two approaches: - **Trader helper methods** (recommended): Convenient methods like `trader.generate_orders_report()`. - **ReportProvider directly**: For more control over data selection and filtering. Reports provide consistent analytics across both backtesting and live trading environments, enabling reliable performance evaluation and strategy comparison. ## Available reports The `ReportProvider` class offers several static methods to generate reports from trading data. Each report returns a pandas DataFrame with specific columns and indexing for easy analysis. ### Orders report Generates a full view of all orders: ```python # Using Trader helper method (recommended) orders_report = trader.generate_orders_report() # Or using ReportProvider directly from nautilus_trader.analysis import ReportProvider orders = cache.orders() orders_report = ReportProvider.generate_orders_report(orders) ``` **Returns `pd.DataFrame`. Key columns include:** | Column | Description | |--------------------|---------------------------------------------------------| | `client_order_id` | Index - unique order identifier. | | `instrument_id` | Trading instrument. | | `strategy_id` | Strategy that created the order. | | `trader_id` | Trader identifier. | | `account_id` | Account identifier (if assigned). | | `venue_order_id` | Venue‑assigned order ID (if accepted). | | `side` | BUY or SELL. | | `type` | MARKET, LIMIT, etc. | | `status` | Current order status. | | `quantity` | Original order quantity (string). | | `filled_qty` | Amount filled (string). | | `price` | Limit price (order‑type dependent). | | `avg_px` | Average fill price (if filled). | | `time_in_force` | Time‑in‑force instruction. | | `ts_init` | Order initialization timestamp (Unix nanoseconds). | | `ts_last` | Last update timestamp (Unix nanoseconds). | Additional columns vary by order type (e.g., `trigger_price` for stop orders, `expire_time` for GTD orders). See `Order.to_dict()` for the complete field list. ### Order fills report Provides a summary of filled orders (one row per order): ```python # Using Trader helper method (recommended) fills_report = trader.generate_order_fills_report() # Or using ReportProvider directly orders = cache.orders() fills_report = ReportProvider.generate_order_fills_report(orders) ``` This report includes only orders with `filled_qty > 0` and contains the same columns as the orders report, but filtered to executed orders only. Note that `ts_init` and `ts_last` are converted to datetime objects in this report for easier analysis. ### Fills report Details individual fill events (one row per fill): ```python # Using Trader helper method (recommended) fills_report = trader.generate_fills_report() # Or using ReportProvider directly orders = cache.orders() fills_report = ReportProvider.generate_fills_report(orders) ``` **Returns `pd.DataFrame`. Key columns include:** | Column | Description | |--------------------|------------------------------------------| | `client_order_id` | Index - order identifier. | | `trade_id` | Unique trade/fill identifier. | | `venue_order_id` | Venue‑assigned order ID. | | `instrument_id` | Trading instrument. | | `strategy_id` | Strategy that created the order. | | `account_id` | Account identifier. | | `position_id` | Associated position ID (if applicable). | | `order_side` | BUY or SELL. | | `order_type` | Order type (MARKET, LIMIT, etc.). | | `last_px` | Fill execution price (string). | | `last_qty` | Fill execution quantity (string). | | `currency` | Currency of the fill. | | `liquidity_side` | MAKER or TAKER. | | `commission` | Commission amount and currency. | | `ts_event` | Fill timestamp (datetime). | | `ts_init` | Initialization timestamp (datetime). | See `OrderFilled.to_dict()` for the complete field list. ### Positions report Position analysis including snapshots: ```python # Using Trader helper method (recommended) # Automatically includes snapshots for NETTING OMS positions_report = trader.generate_positions_report() # Or using ReportProvider directly positions = cache.positions() snapshots = cache.position_snapshots() # For NETTING OMS positions_report = ReportProvider.generate_positions_report( positions=positions, snapshots=snapshots ) ``` **Returns `pd.DataFrame`. Key columns include:** | Column | Description | |--------------------|------------------------------------------| | `position_id` | Index - unique position identifier. | | `instrument_id` | Trading instrument. | | `strategy_id` | Strategy that managed the position. | | `trader_id` | Trader identifier. | | `account_id` | Account identifier. | | `opening_order_id` | Order ID that opened the position. | | `closing_order_id` | Order ID that closed the position. | | `entry` | Entry side (BUY or SELL). | | `side` | Position side (LONG, SHORT, or FLAT). | | `quantity` | Current position size. | | `peak_qty` | Maximum size reached. | | `avg_px_open` | Average entry price. | | `avg_px_close` | Average exit price (if closed). | | `commissions` | List of commissions paid. | | `realized_pnl` | Realized profit/loss. | | `realized_return` | Return percentage. | | `ts_init` | Position initialization timestamp. | | `ts_opened` | Opening timestamp (datetime). | | `ts_last` | Last update timestamp. | | `ts_closed` | Closing timestamp (datetime or NA). | | `duration_ns` | Position duration in nanoseconds. | | `is_snapshot` | Whether this is a historical snapshot. | ### Account report Tracks account balance and margin changes over time: ```python # Using Trader helper method (recommended) # Requires venue parameter from nautilus_trader.model.identifiers import Venue venue = Venue("BINANCE") account_report = trader.generate_account_report(venue) # Or using ReportProvider directly account = cache.account(account_id) account_report = ReportProvider.generate_account_report(account) ``` **Returns `pd.DataFrame`. Columns include:** | Column | Description | |-----------------|--------------------------------------------| | `ts_event` | Index - timestamp of account state change. | | `account_id` | Account identifier. | | `account_type` | Type of account (e.g., SPOT, MARGIN). | | `base_currency` | Base currency for the account. | | `total` | Total balance amount (string). | | `free` | Available balance (string). | | `locked` | Balance locked in orders (string). | | `currency` | Currency of the balance. | | `reported` | Whether balance was reported by venue. | | `margins` | Margin information (list, if applicable). | | `info` | Additional venue‑specific information. | Each row represents a balance entry; accounts with multiple currencies produce multiple rows per account state event. ## PnL accounting considerations Accurate PnL accounting requires careful consideration of several factors: ### Position-based PnL - **Realized PnL**: Calculated when positions are partially or fully closed. - **Unrealized PnL**: Marked-to-market using current prices. - **Commission impact**: Only included when in settlement currency. :::warning PnL calculations depend on the OMS type. In `NETTING` OMS, position snapshots preserve historical PnL when positions reopen. Always include snapshots in reports for accurate total PnL calculation. In `HEDGING` OMS, snapshots are not used since each position has a unique ID and is never reopened. ::: ### Multi-currency accounting When dealing with multiple currencies: - Each position tracks PnL in its settlement currency. - Portfolio aggregation requires currency conversion. - Commission currencies may differ from settlement currency. ```python # Accessing PnL across positions for position in positions: realized = position.realized_pnl # In settlement currency unrealized = position.unrealized_pnl(last_price) # Handle multi-currency aggregation (illustrative) # Note: Currency conversion requires user-provided exchange rates if position.settlement_currency != base_currency: # Apply conversion rate from your data source # rate = get_exchange_rate(position.settlement_currency, base_currency) # realized_converted = realized.as_double() * rate pass ``` ### Snapshot considerations For `NETTING` OMS: ```python from nautilus_trader.model.objects import Money # Include snapshots for complete PnL (per currency) pnl_by_currency = {} # Add PnL from current positions for position in cache.positions(instrument_id=instrument_id): if position.realized_pnl: currency = position.realized_pnl.currency if currency not in pnl_by_currency: pnl_by_currency[currency] = 0.0 pnl_by_currency[currency] += position.realized_pnl.as_double() # Add PnL from historical snapshots for snapshot in cache.position_snapshots(instrument_id=instrument_id): if snapshot.realized_pnl: currency = snapshot.realized_pnl.currency if currency not in pnl_by_currency: pnl_by_currency[currency] = 0.0 pnl_by_currency[currency] += snapshot.realized_pnl.as_double() # Create Money objects for each currency total_pnls = [Money(amount, currency) for currency, amount in pnl_by_currency.items()] ``` ## Backtest post-run analysis After a backtest completes, analysis is available through result statistics and generated reports. ### Accessing backtest results ```python # After backtest run engine.run(start=start_time, end=end_time) # Access result statistics result = engine.get_result() # Generate reports from the backtest engine fills_report = engine.generate_fills_report() venue = engine.list_venues()[0] account_report = engine.generate_account_report(venue=venue) # Or access data directly for custom analysis orders = engine.cache.orders() positions = engine.cache.positions() snapshots = engine.cache.position_snapshots() ``` ### Portfolio statistics The backtest result provides performance metrics: ```python # Access backtest result statistics result = engine.get_result() # Get different categories of statistics stats_pnls = result.stats_pnls stats_returns = result.stats_returns stats_general = result.stats_general ``` :::info For detailed information about available statistics, see the [Portfolio guide](portfolio.md#portfolio-statistics). The Portfolio guide covers: - Built-in statistics categories (PnLs, returns, positions, orders based). - Portfolio report context. ::: ### Visualization NautilusTrader provides interactive tearsheets and plots via Plotly: ```python from nautilus_trader.analysis import create_tearsheet # After backtest run engine.run() # Generate interactive HTML tearsheet create_tearsheet(engine, output_path="tearsheet.html") ``` This creates an interactive HTML report with: - Equity curve - Drawdown analysis - Monthly returns heatmap - Performance statistics table - Returns distribution For more control, generate individual plots: ```python import pandas as pd from nautilus_trader.analysis import create_equity_curve returns = pd.Series( [0.01, -0.005, 0.002], index=pd.date_range("2024-01-01", periods=3, tz="UTC"), ) fig = create_equity_curve(returns, title="My Strategy Equity") fig.show() # Display in browser fig.write_image("equity.png") # Export to PNG (requires kaleido) ``` Install visualization dependencies: ```bash uv pip install "nautilus_trader[visualization]" ``` ## Report generation patterns ### Live trading During live trading, generate reports periodically: ```python import pandas as pd class ReportingActor(Actor): def on_start(self): # Schedule periodic reporting self.clock.set_timer( name="generate_reports", interval=pd.Timedelta(minutes=30), callback=self.generate_reports ) def generate_reports(self, event): # Generate and log reports positions_report = self.trader.generate_positions_report() # Save or transmit report positions_report.to_csv(f"positions_{event.ts_event}.csv") ``` ### Performance analysis For backtest analysis: ```python import pandas as pd # Run the backtest engine.run(start=start_time, end=end_time) # Collect results positions_closed = engine.cache.positions_closed() result = engine.get_result() stats_pnls = result.stats_pnls stats_returns = result.stats_returns stats_general = result.stats_general # Create summary dictionary results = { "total_positions": len(positions_closed), "pnl_total": stats_pnls.get("USD", {}).get("PnL (total)"), "sharpe_ratio": stats_returns.get("Sharpe Ratio (252 days)"), "profit_factor": stats_general.get("Profit Factor"), "win_rate": stats_general.get("Win Rate"), } # Display results results_df = pd.DataFrame([results]) print(results_df.T) # Transpose for vertical display ``` :::info Reports are generated from in-memory data structures. For large-scale analysis or long-running systems, consider persisting reports to a database for efficient querying. See the [Cache guide](cache.md) for persistence options. ::: ## Integration with other components The `ReportProvider` works with several system components: - **Cache**: Source of all trading data (orders, positions, accounts) for reports. - **Portfolio**: Uses reports for performance analysis and metrics calculation. - **BacktestEngine**: Uses reports for post-run analysis and visualization. - **Position snapshots**: Required for accurate PnL reporting in `NETTING` OMS. ## Summary The `ReportProvider` generates reports from orders, fills, positions, and account states as structured DataFrames for analysis and visualization. For accurate total PnL in `NETTING` OMS, include position snapshots when generating reports. ## Related guides - [Visualization](visualization.md) - Interactive tearsheets and charts from backtest results. - [Portfolio](portfolio.md) - Portfolio statistics and performance metrics. - [Backtesting](backtesting.md) - Running backtests that generate reports. - [Cache](cache.md) - Cache system that stores data for reports. # Rust Source: https://nautilustrader.io/docs/latest/concepts/rust/ Nautilus has a complete Rust implementation under the `crates/` directory. You can write actors, strategies, run backtests, and trade live without Python. The domain model is shared across all paths, and the v2 PyO3 path runs Python strategies on the Rust engine directly. :::warning The Rust API is under active development. Method signatures and trait requirements may change between releases. ::: ## System implementations Nautilus has three implementations. Understanding where each stands helps you choose the right one for your use case. - **v1 legacy**: Cython/Python classes under `nautilus_trader/`. Fully featured with the broadest component coverage. - **v2 Rust**: Pure Rust under `crates/`. Runs without Python. - **v2 PyO3**: Python user-components (actors, strategies) running on the Rust core via PyO3 bindings. Combines Python convenience with Rust engine performance. ### Capability matrix | Component | v1 legacy (Cython) | v2 Rust | v2 PyO3 (Python on Rust) | |-----------------------|--------------------|----------------|--------------------------| | Strategy | ✓ | ✓ | ✓ | | Actor | ✓ | ✓ | ✓ | | DataEngine | ✓ | ✓ | ✓ | | ExecutionEngine | ✓ | ✓ | ✓ | | RiskEngine | ✓ | ✓ | ✓ | | BacktestEngine | ✓ | ✓ | ✓ | | BacktestNode | ✓ | ✓ | ✓ | | LiveNode | ✓ | ✓ | ✓ | | OrderEmulator | ✓ | ✓ | ✓ | | Matching engine | ✓ | ✓ | ✓ | | Portfolio | ✓ | ✓ | ✓ | | Accounts | ✓ | ✓ | ✓ | | Cache | ✓ | ✓ | ✓ | | MessageBus | ✓ | ✓ | ✓ | | Data catalog | ✓ | ✓ | ✓ | | Indicators | ✓ | ✓ | ✓ | | Exec algorithms | TWAP | TWAP | TWAP | | Controller | ✓ | - | - | | Tearsheets | ✓ | - | ✓ | | Config serialization | ✓ | - | - | ### Adapters | Adapter | v1 legacy (Cython) | v2 Rust | v2 PyO3 | |---------------------|--------------------|---------|---------| | Architect AX | ✓ | ✓ | ✓ | | Betfair | ✓ | ✓ | ✓ | | Binance | ✓ | ✓ | ✓ | | BitMEX | ✓ | ✓ | ✓ | | Bybit | ✓ | ✓ | ✓ | | Databento | ✓ | ✓ | ✓ | | Deribit | ✓ | ✓ | ✓ | | dYdX | ✓ | ✓ | ✓ | | Hyperliquid | ✓ | ✓ | ✓ | | Interactive Brokers | ✓ | - | - | | Kraken | ✓ | ✓ | ✓ | | OKX | ✓ | ✓ | ✓ | | Polymarket | ✓ | ✓ | ✓ | | Sandbox | ✓ | ✓ | ✓ | | Tardis | ✓ | ✓ | ✓ | ### Choosing a path - **v1 legacy** is the most complete today. Use it if you need the Controller, Interactive Brokers, or config serialization. - **v2 Rust** gives native performance without a Python runtime. All core trading functionality is available. Use it for latency-sensitive deployments or teams that prefer a compiled language. - **v2 PyO3**: Python user-components (actors, strategies) run on the Rust core engine with Rust performance for data processing and execution, while keeping the Python authoring experience. ## Project setup The Nautilus crates are published to [crates.io](https://crates.io/crates/nautilus-backtest). Add them to your `Cargo.toml`: ```toml [dependencies] nautilus-backtest = "0.59" nautilus-common = "0.59" nautilus-execution = "0.59" nautilus-model = { version = "0.59", features = ["stubs"] } nautilus-trading = { version = "0.59", features = ["examples"] } anyhow = "1" log = "0.4" ``` For live trading, add the live crate and the adapter for your venue: ```toml [dependencies] nautilus-live = "0.59" nautilus-okx = "0.59" ``` To track the latest development branch, point all Nautilus dependencies at the same git source to avoid type mismatches between crates.io and git versions: ```toml [dependencies] nautilus-backtest = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop" } nautilus-common = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop" } nautilus-execution = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop" } nautilus-model = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["stubs"] } nautilus-trading = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["examples"] } ``` The minimum supported Rust version (MSRV) is **1.96.0**. ### Feature flags | Flag | Crate | Effect | |------------------|---------------------|---------------------------------------------------------------| | `high-precision` | `nautilus-model` | 16-digit fixed precision (default is 9). Required for crypto. | | `stubs` | `nautilus-model` | Test instrument stubs (`audusd_sim`, etc.). | | `examples` | `nautilus-trading` | Example strategies (`EmaCross`, `GridMarketMaker`). | | `streaming` | `nautilus-backtest` | Catalog‑based data streaming via `BacktestNode`. | | `defi` | `nautilus-model` | DeFi data types. Implies `high-precision`. | :::tip Standard 9-digit precision handles most traditional finance instruments. Enable `high-precision` for crypto venues where prices can have many decimal places (e.g. `0.00000001`). ::: ## Actors An actor receives market data, custom data/signals, and system events but does not manage orders. Implement the `DataActor` trait and use `nautilus_actor!` to wire your `DataActorCore` field into the runtime contract. Your type implements or derives `Debug`; the macro supplies the native runtime wiring. User code normally uses the `DataActor` facade methods for subscriptions, cache access, and clock access. ### Handler methods Override any handler on the `DataActor` trait to receive the corresponding data or event. All handlers have default no-op implementations, so you only override what you need. | Handler | Receives | |------------------------|---------------------------| | `on_start` | Actor started. | | `on_stop` | Actor stopped. | | `on_quote` | `QuoteTick` | | `on_trade` | `TradeTick` | | `on_bar` | `Bar` | | `on_book_deltas` | `OrderBookDeltas` | | `on_book` | `OrderBook` (at interval) | | `on_instrument` | `InstrumentAny` | | `on_mark_price` | `MarkPriceUpdate` | | `on_index_price` | `IndexPriceUpdate` | | `on_funding_rate` | `FundingRateUpdate` | | `on_option_greeks` | `OptionGreeks` | | `on_option_chain` | `OptionChainSlice` | | `on_instrument_status` | `InstrumentStatus` | | `on_order_filled` | `OrderFilled` | | `on_order_canceled` | `OrderCanceled` | | `on_time_event` | `TimeEvent` | For a step-by-step walkthrough, see the [Write an Actor (Rust)](../how_to/write_rust_actor.md) how-to guide. For a complete example, see [`BookImbalanceActor`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/actors/imbalance). ## Strategies A strategy extends an actor with order management. Implement `DataActor` for data handling and use `nautilus_strategy!` to wire your `StrategyCore` field into the strategy runtime contract. `StrategyCore` stores the runtime strategy state; normal strategy logic reaches it through facade methods on `self`. Runtime registration requires the native wiring generated by the macro, but normal strategy logic uses `Strategy` methods and the facade methods on `self`. ### Order management The `Strategy` trait provides order methods through the facade: | Method | Action | |-----------------------|-------------------------------------------| | `submit_order` | Submit a new order to the venue. | | `submit_order_list` | Submit a list of contingent orders. | | `modify_order` | Modify price, quantity, or trigger price. | | `cancel_order` | Cancel a specific order. | | `cancel_orders` | Cancel a filtered set of orders. | | `cancel_all_orders` | Cancel all orders for an instrument. | | `close_position` | Close a position with a market order. | | `close_all_positions` | Close all open positions. | The `OrderApi` (accessed via `self.order()`) builds orders and order lists: - `generate_client_order_id` - `generate_order_list_id` - `market` - `limit` - `stop_market` - `stop_limit` - `market_to_limit` - `market_if_touched` - `limit_if_touched` - `trailing_stop_market` - `trailing_stop_limit` - `bracket` - `create_list` ### Core wiring macros Rust actors, strategies, and execution algorithms keep their runtime core as a struct field. The macros tell the traits where that field lives. | Macro | Core field | Generates | |------------------------------------------------|--------------------------|---------------------------------| | `nautilus_actor!(Type)` | `DataActorCore` | Runtime wiring. | | `nautilus_strategy!(Type)` | `StrategyCore` | Runtime wiring and `Strategy`. | | `nautilus_execution_algorithm!(Type, { ... })` | `ExecutionAlgorithmCore` | Runtime wiring and algorithm. | The macros expect a field named `core`; pass a field name as the second argument when needed. They do not make the actor, strategy, or `StrategyCore` deref to runtime internals. The execution algorithm macro takes an `on_order()` implementation block because that method defines the algorithm's required order handling. Normal code uses facade methods such as: - `actor_id()` - `trader_id()` - `is_registered()` - `config()` - `strategy_id()` - `clock()` - `cache()` - `order()` - `portfolio()` ### Native traits Use facade methods by default: - `actor_id()` - `trader_id()` - `is_registered()` - `config()` - `strategy_id()` - `clock()` - `cache()` - `order()` - `portfolio()` `DataActorNative`, `StrategyNative`, and `ExecutionAlgorithmNative` are for native-only access below that facade. This section documents engine, runtime, and explicit latency-sensitive native Rust code, not the portable authoring path. | Authoring path | Native traits? | Normal API | |---------------------------|------------------|-------------------------------------| | Native Rust binary | Only when needed | `Strategy` and `DataActor` facades. | | Rust launched from Python | Only when needed | Same as native Rust. | | Python‑authored component | No | Facades only. | Native traits expose borrowed core state, `Rc>`, and runtime references. Use them when native Rust code intentionally accepts those borrow rules for an explicit latency-sensitive path. Engine, runtime, registration, PyO3, and testkit code can import `DataActorNative`, `StrategyNative`, or `ExecutionAlgorithmNative` when they need actor-core, strategy-core, or execution-algorithm-core access. Do not use them in ordinary portable actor, strategy, or execution algorithm logic or Python-authored components, because those types do not cross the Python boundary. `ExecutionAlgorithmCore` owns a `DataActorCore`, but it does not deref to one. Normal execution algorithm logic should use `id()`, `actor_id()`, `trader_id()`, `clock()`, and `cache()`. Reach for `ExecutionAlgorithmNative` only when the code needs native execution-algorithm state. Choose the smallest native handle and keep each borrow scoped. Use `order()` for normal strategy order construction. Reach for `order_factory()` only when native code needs the raw mutable factory borrow. #### `DataActorNative` methods | Native method | Return shape | Use when | |---------------|--------------------------|---------------------------------| | `core()` | `&DataActorCore` | Read actor internals. | | `core_mut()` | `&mut DataActorCore` | Mutate actor internals. | | `clock_mut()` | `RefMut<'_, dyn Clock>` | Need a mutable clock borrow. | | `clock_rc()` | `Rc>` | Store or pass the shared clock. | | `cache_ref()` | `Ref<'_, Cache>` | Need short live‑cache reads. | | `cache_rc()` | `Rc>` | Mutate, store, or pass cache. | #### `StrategyNative` methods | Native method | Return shape | Use when | |-----------------------|------------------------------|-----------------------------------| | `strategy_core()` | `&StrategyCore` | Read strategy internals. | | `strategy_core_mut()` | `&mut StrategyCore` | Mutate strategy internals. | | `order_factory()` | `RefMut<'_, OrderFactory>` | Need raw mutable factory borrow. | | `order_factory_rc()` | `Rc>` | Store or pass the factory. | | `portfolio_rc()` | `Rc>` | Store or pass the portfolio. | #### `ExecutionAlgorithmNative` methods | Native method | Return shape | Use when | |-----------------------------|--------------------------------|---------------------------------------| | `exec_algorithm_core()` | `&ExecutionAlgorithmCore` | Read execution algorithm internals. | | `exec_algorithm_core_mut()` | `&mut ExecutionAlgorithmCore` | Mutate execution algorithm internals. | For a step-by-step walkthrough, see the [Write a Strategy (Rust)](../how_to/write_rust_strategy.md) how-to guide. For complete examples, see [`EmaCross`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/ema_cross) and [`GridMarketMaker`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/grid_mm). ### Running Rust components Rust strategies and actors can run through two paths. The examples below use strategies, but the same pattern applies to bundled actors via `add_actor` (pure Rust) and `add_builtin_actor` (from Python). #### Pure Rust Write your strategy and `main` function in Rust, then build a standalone binary with `cargo build`. This path requires no Python runtime. ```rust let strategy = GridMarketMaker::new(config); node.add_strategy(strategy)?; node.run().await?; ``` See [Run Live Trading (Rust)](../how_to/run_rust_live_trading.md) for a full walkthrough. #### Built-in examples from Python Pass a type name and config to `add_builtin_strategy` to register a built-in example strategy from Python. This path exists to single-source the bundled example strategy code across Rust and Python docs, examples, and tests. It is not a first-class extension path for adding native strategies. For custom native components, use pure Rust. ```python from nautilus_trader.core.nautilus_pyo3.trading import GridMarketMakerConfig config = GridMarketMakerConfig( instrument_id=InstrumentId.from_str("BTC-USDT-SWAP.OKX"), max_position=Quantity.from_str("10.0"), trade_size=Quantity.from_str("0.1"), num_levels=5, grid_step_bps=15, ) node.add_builtin_strategy("GridMarketMaker", config) ``` Built-in strategy configs: | Config | Strategy | |--------------------------------|--------------------------| | `CompositeMarketMakerConfig` | `CompositeMarketMaker` | | `DeltaNeutralVolConfig` | `DeltaNeutralVol` | | `EmaCrossConfig` | `EmaCross` | | `ExecTesterConfig` | `ExecTester` | | `GridMarketMakerConfig` | `GridMarketMaker` | | `HurstVpinDirectionalConfig` | `HurstVpinDirectional` | `add_builtin_actor` follows the same bundled-only rule for actors used by examples and tests. Built-in actor configs (via `add_builtin_actor`): | Config | Actor | |----------------------------|-----------------------| | `BookImbalanceActorConfig` | `BookImbalanceActor` | | `DataTesterConfig` | `DataTester` | ## Backtesting For annotated walkthroughs of both APIs, see the [Run a Backtest (Rust)](../how_to/run_rust_backtest.md) how-to guide. ### `BacktestEngine` (low-level API) Construct the engine, add venues and instruments, load data, register strategies, and run. See the full working example: ```bash cargo run -p nautilus-backtest --features examples --example engine-ema-cross ``` Source: [`crates/backtest/examples/engine_ema_cross.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/backtest/examples/engine_ema_cross.rs) ### `BacktestNode` (high-level API) Loads data from a `ParquetDataCatalog` and supports streaming in configurable chunk sizes. Requires the `streaming` feature on `nautilus-backtest`. See the full working example: ```bash cargo run -p nautilus-backtest --features examples,streaming --example node-ema-cross ``` Source: [`crates/backtest/examples/node_ema_cross.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/backtest/examples/node_ema_cross.rs) ## Live trading For an annotated walkthrough, see the [Run Live Trading (Rust)](../how_to/run_rust_live_trading.md) how-to guide. The `LiveNode` connects to real venues through adapter clients. The builder pattern configures data and execution clients, then `run()` starts the async event loop. Each adapter provides its own factory and config types. | Adapter | Example | |----------------|----------------------------------------------------------| | Architect AX | `crates/adapters/architect_ax/examples/` | | Betfair | `crates/adapters/betfair/examples/` | | Binance | `crates/adapters/binance/examples/` | | BitMEX | `crates/adapters/bitmex/examples/` | | Blockchain | `crates/adapters/blockchain/examples/` | | Bybit | `crates/adapters/bybit/examples/` | | Databento | `crates/adapters/databento/examples/` | | Deribit | `crates/adapters/deribit/examples/` | | dYdX | `crates/adapters/dydx/examples/` | | Hyperliquid | `crates/adapters/hyperliquid/examples/` | | Kraken | `crates/adapters/kraken/examples/` | | OKX | `crates/adapters/okx/examples/` | | Polymarket | `crates/adapters/polymarket/examples/` | | Sandbox | `crates/adapters/sandbox/examples/` | | Tardis | `crates/adapters/tardis/examples/` | Most adapters include `node_data_tester.rs` and `node_exec_tester.rs` examples. These test data requests, streaming, and order execution against live venues. ## Related guides - [Write an Actor (Rust)](../how_to/write_rust_actor.md) - Step-by-step actor walkthrough. - [Write a Strategy (Rust)](../how_to/write_rust_strategy.md) - Step-by-step strategy walkthrough. - [Run a Backtest (Rust)](../how_to/run_rust_backtest.md) - BacktestEngine and BacktestNode usage. - [Run Live Trading (Rust)](../how_to/run_rust_live_trading.md) - LiveNode setup and venue connection. - [Architecture](architecture.md) - System design and data/execution flow. - [Actors](actors.md) - Actor concepts (applies to both Python and Rust). - [Strategies](strategies.md) - Strategy concepts and handler reference. - [Events](events.md) - Event types and handler dispatch. - [Backtesting](backtesting.md) - Backtest concepts and matching engine behavior. # Strategies Source: https://nautilustrader.io/docs/latest/concepts/strategies/ A strategy inherits the `Strategy` class and implements the methods its logic requires. **Capabilities**: - All `Actor` capabilities. - Order management. **Relationship with actors**: The `Strategy` class inherits from `Actor`, which means strategies have access to all actor functionality plus order management capabilities. :::tip We recommend reviewing the [Actors](actors.md) guide before diving into strategy development. ::: Strategies can be added to Nautilus systems in any [environment contexts](architecture.md#environment-contexts) and will start sending commands and receiving events based on their logic as soon as the system starts. With these building blocks of data ingest, event handling, and order management (discussed below), you can build any type of strategy including directional, momentum, re-balancing, pairs, market making, etc. See the [`Strategy` API Reference](/docs/python-api-latest/trading.html) for all available methods. There are two main parts of a Nautilus trading strategy: - The strategy implementation itself, defined by inheriting the `Strategy` class. - The *optional* strategy configuration, defined by inheriting the `StrategyConfig` class. :::tip Once a strategy is defined, the same source code can be used for backtesting and live trading. ::: The main capabilities of a strategy include: - Historical data requests. - Live data feed subscriptions. - Setting time alerts or timers. - Cache access. - Portfolio access. - Creating and managing orders and positions. :::info Rust implementation Rust strategy authors implement the `DataActor` callbacks they need and use `nautilus_strategy!` to generate the `Strategy` implementation, then call facade methods such as `clock()`, `cache()`, `order()`, and `portfolio()` on `self`. `DataActorNative` is native-only access to runtime wiring and actor-core state; `StrategyNative` exposes borrowed strategy state such as order factory, order manager, and portfolio access. Import them only for same-binary performance paths or internal runtime wiring. ::: ## Strategy implementation A trading strategy inherits from `Strategy`, so you must define a constructor. At minimum, initialize the base class: ```python from nautilus_trader.trading.strategy import Strategy class MyStrategy(Strategy): def __init__(self) -> None: super().__init__() # <-- the superclass must be called to initialize the strategy ``` From here, you can implement handlers as necessary to perform actions based on state transitions and events. :::warning Do not call components such as `clock` and `logger` in the `__init__` constructor (which is prior to registration). This is because the systems clock and logging subsystem have not yet been initialized. ::: ### Handlers Handlers are methods on the `Strategy` class that perform actions based on events or state changes. These methods use the `on_*` prefix. Implement any or all of them as your strategy requires. Multiple handlers exist for similar event types to give you control over granularity. Respond to a specific event with a dedicated handler, or use a generic handler for a range of related events (using typical switch statement logic). The system calls handlers in sequence from most specific to most general. #### Stateful actions Lifecycle state changes trigger these handlers. Recommendations: - Use the `on_start` method to initialize your strategy (e.g., fetch instruments, subscribe to data). - Use the `on_stop` method for cleanup tasks (e.g., cancel open orders, close open positions, unsubscribe from data). ```python def on_start(self) -> None: def on_stop(self) -> None: def on_resume(self) -> None: def on_reset(self) -> None: def on_dispose(self) -> None: def on_degrade(self) -> None: def on_fault(self) -> None: def on_save(self) -> dict[str, bytes]: # Returns user-defined dictionary of state to be saved def on_load(self, state: dict[str, bytes]) -> None: ``` #### Data handling These handlers receive data updates, including built-in market data and custom user-defined data. ```python from nautilus_trader.core import Data from nautilus_trader.model import OrderBook from nautilus_trader.model import Bar from nautilus_trader.model import QuoteTick from nautilus_trader.model import TradeTick from nautilus_trader.model import OrderBookDeltas from nautilus_trader.model import InstrumentClose from nautilus_trader.model import InstrumentStatus from nautilus_trader.model import OptionChainSlice from nautilus_trader.model import OptionGreeks from nautilus_trader.model.instruments import Instrument def on_order_book_deltas(self, deltas: OrderBookDeltas) -> None: def on_order_book(self, order_book: OrderBook) -> None: def on_quote_tick(self, tick: QuoteTick) -> None: def on_trade_tick(self, tick: TradeTick) -> None: def on_bar(self, bar: Bar) -> None: def on_instrument(self, instrument: Instrument) -> None: def on_instrument_status(self, data: InstrumentStatus) -> None: def on_instrument_close(self, data: InstrumentClose) -> None: def on_option_greeks(self, greeks: OptionGreeks) -> None: def on_option_chain(self, chain: OptionChainSlice) -> None: def on_historical_data(self, data: Data) -> None: def on_data(self, data: Data) -> None: # Custom data passed to this handler def on_signal(self, signal: Data) -> None: # Custom signals passed to this handler ``` #### Order management These handlers receive events related to orders. `OrderEvent` type messages are passed to handlers in the following sequence: 1. Specific handler (e.g., `on_order_accepted`, `on_order_rejected`, etc.) 2. `on_order_event(...)` 3. `on_event(...)` ```python from nautilus_trader.model.events import OrderAccepted from nautilus_trader.model.events import OrderCanceled from nautilus_trader.model.events import OrderCancelRejected from nautilus_trader.model.events import OrderDenied from nautilus_trader.model.events import OrderEmulated from nautilus_trader.model.events import OrderEvent from nautilus_trader.model.events import OrderExpired from nautilus_trader.model.events import OrderFilled from nautilus_trader.model.events import OrderInitialized from nautilus_trader.model.events import OrderModifyRejected from nautilus_trader.model.events import OrderPendingCancel from nautilus_trader.model.events import OrderPendingUpdate from nautilus_trader.model.events import OrderRejected from nautilus_trader.model.events import OrderReleased from nautilus_trader.model.events import OrderSubmitted from nautilus_trader.model.events import OrderTriggered from nautilus_trader.model.events import OrderUpdated def on_order_initialized(self, event: OrderInitialized) -> None: def on_order_denied(self, event: OrderDenied) -> None: def on_order_emulated(self, event: OrderEmulated) -> None: def on_order_released(self, event: OrderReleased) -> None: def on_order_submitted(self, event: OrderSubmitted) -> None: def on_order_rejected(self, event: OrderRejected) -> None: def on_order_accepted(self, event: OrderAccepted) -> None: def on_order_canceled(self, event: OrderCanceled) -> None: def on_order_expired(self, event: OrderExpired) -> None: def on_order_triggered(self, event: OrderTriggered) -> None: def on_order_pending_update(self, event: OrderPendingUpdate) -> None: def on_order_pending_cancel(self, event: OrderPendingCancel) -> None: def on_order_modify_rejected(self, event: OrderModifyRejected) -> None: def on_order_cancel_rejected(self, event: OrderCancelRejected) -> None: def on_order_updated(self, event: OrderUpdated) -> None: def on_order_filled(self, event: OrderFilled) -> None: def on_order_event(self, event: OrderEvent) -> None: # All order event messages are eventually passed to this handler ``` #### Position management These handlers receive events related to positions. `PositionEvent` type messages are passed to handlers in the following sequence: 1. Specific handler (e.g., `on_position_opened`, `on_position_changed`, etc.) 2. `on_position_event(...)` 3. `on_event(...)` ```python from nautilus_trader.model.events import PositionChanged from nautilus_trader.model.events import PositionClosed from nautilus_trader.model.events import PositionEvent from nautilus_trader.model.events import PositionOpened def on_position_opened(self, event: PositionOpened) -> None: def on_position_changed(self, event: PositionChanged) -> None: def on_position_closed(self, event: PositionClosed) -> None: def on_position_event(self, event: PositionEvent) -> None: # All position event messages are eventually passed to this handler ``` #### Generic event handling This handler will eventually receive all event messages which arrive at the strategy, including those for which no other specific handler exists. ```python from nautilus_trader.core.message import Event def on_event(self, event: Event) -> None: ``` #### Handler example The following example shows a typical `on_start` handler method implementation (taken from the example EMA cross strategy). Here we can see the following: - Indicators being registered to receive bar updates. - Historical data being requested (to hydrate the indicators). - Live data being subscribed to. The cache check matters in live trading. Direct subscriptions assume the instrument was loaded by the instrument provider config or by an earlier instrument request. ```python def on_start(self) -> None: """ Actions to be performed on strategy start. """ self.instrument = self.cache.instrument(self.instrument_id) if self.instrument is None: self.log.error(f"Could not find instrument for {self.instrument_id}") self.stop() # Transitions strategy to STOPPED state return # Register the indicators for updating self.register_indicator_for_bars(self.bar_type, self.fast_ema) self.register_indicator_for_bars(self.bar_type, self.slow_ema) # Get historical data and subscribe to live data self.request_bars( self.bar_type, callback=lambda _: self.subscribe_bars(self.bar_type), ) self.subscribe_quote_ticks(self.instrument_id) ``` Live bars are subscribed via the `request_bars()` `callback` so the stream starts only once history has loaded; see [Working with bars: request vs. subscribe](data.md#working-with-bars-request-vs-subscribe) for why this matters under `validate_data_sequence=True`. ### Clock and timers Strategies have access to a `Clock` which provides a number of methods for creating different timestamps, as well as setting time alerts or timers to trigger `TimeEvent`s. See the [`Clock` API Reference](/docs/python-api-latest/common.html) for all available methods. #### Current timestamps While there are multiple ways to obtain current timestamps, here are two commonly used methods as examples: To get the current UTC timestamp as a tz-aware `pd.Timestamp`: ```python import pandas as pd now: pd.Timestamp = self.clock.utc_now() ``` To get the current UTC timestamp as nanoseconds since the UNIX epoch: ```python unix_nanos: int = self.clock.timestamp_ns() ``` #### Time alerts Time alerts can be set which will result in a `TimeEvent` being dispatched to the `on_event` handler at the specified alert time. In a live context, this might be slightly delayed by a few microseconds. This example sets a time alert to trigger one minute from the current time: ```python import pandas as pd # Fire a TimeEvent one minute from now self.clock.set_time_alert( name="MyTimeAlert1", alert_time=self.clock.utc_now() + pd.Timedelta(minutes=1), ) ``` #### Timers Continuous timers can be set up which will generate a `TimeEvent` at regular intervals until the timer expires or is canceled. This example sets a timer to fire once per minute, starting immediately: ```python import pandas as pd # Fire a TimeEvent every minute self.clock.set_timer( name="MyTimer1", interval=pd.Timedelta(minutes=1), ) ``` ### Cache access The trader's central `Cache` stores data and execution objects (orders, positions, etc). Many methods are available with filtering. Here are some basic use cases. #### Fetching data The following example fetches data from the cache (assuming some instrument ID attribute is assigned). These methods return `None` if the requested data is not available. ```python last_quote = self.cache.quote_tick(self.instrument_id) last_trade = self.cache.trade_tick(self.instrument_id) last_bar = self.cache.bar(bar_type) ``` #### Fetching execution objects The following example shows how individual order and position objects can be fetched from the cache: ```python order = self.cache.order(client_order_id) position = self.cache.position(position_id) ``` See the [`Cache` API Reference](/docs/python-api-latest/cache.html) for all available methods. ### Portfolio access The trader's central `Portfolio` provides account and positional information. The following shows a general outline of available methods. #### Account and positional information ```python import decimal from nautilus_trader.accounting.accounts.base import Account from nautilus_trader.model import Venue from nautilus_trader.model import Currency from nautilus_trader.model import Money from nautilus_trader.model import InstrumentId def account(self, venue: Venue) -> Account def balances_locked(self, venue: Venue) -> dict[Currency, Money] def margins_init(self, venue: Venue) -> dict[Currency, Money] def margins_maint(self, venue: Venue) -> dict[Currency, Money] def unrealized_pnls(self, venue: Venue) -> dict[Currency, Money] def realized_pnls(self, venue: Venue) -> dict[Currency, Money] def net_exposures(self, venue: Venue) -> dict[Currency, Money] def unrealized_pnl(self, instrument_id: InstrumentId) -> Money def realized_pnl(self, instrument_id: InstrumentId) -> Money def net_exposure(self, instrument_id: InstrumentId) -> Money def net_position(self, instrument_id: InstrumentId) -> decimal.Decimal def is_net_long(self, instrument_id: InstrumentId) -> bool def is_net_short(self, instrument_id: InstrumentId) -> bool def is_flat(self, instrument_id: InstrumentId) -> bool def is_completely_flat(self) -> bool ``` See the [`Portfolio` API Reference](/docs/python-api-latest/portfolio.html) for all available methods. #### Reports and analysis The `Portfolio` also exposes a `PortfolioAnalyzer`, which accepts a flexible amount of data (to accommodate different lookback windows). The analyzer tracks and generates performance metrics and statistics. See the [`PortfolioAnalyzer` API Reference](/docs/python-api-latest/analysis.html) and [Portfolio statistics](portfolio.md#portfolio-statistics) guide. ### Trading commands The following trading commands are available for order management. See also the [Execution](../concepts/execution.md) guide for the full flow through the system. #### Submitting orders An `OrderFactory` is provided on the base class for every `Strategy` as a convenience, reducing the amount of boilerplate required to create different `Order` objects (although these objects can still be initialized directly with the `Order.__init__(...)` constructor if the trader prefers). The component a `SubmitOrder` or `SubmitOrderList` command will flow to for execution depends on the following: - If an `emulation_trigger` is specified, the command will *firstly* be sent to the `OrderEmulator`. - If an `exec_algorithm_id` is specified (with no `emulation_trigger`), the command will *firstly* be sent to the relevant `ExecAlgorithm`. - Otherwise, the command will *firstly* be sent to the `RiskEngine`. This example submits a `LIMIT` BUY order for emulation (see [Emulated Orders](orders/emulated.md)): ```python from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TriggerType from nautilus_trader.model.orders import LimitOrder def buy(self) -> None: """ Users simple buy method (example). """ order: LimitOrder = self.order_factory.limit( instrument_id=self.instrument_id, order_side=OrderSide.BUY, quantity=self.instrument.make_qty(self.trade_size), price=self.instrument.make_price(5000.00), emulation_trigger=TriggerType.LAST_PRICE, ) self.submit_order(order) ``` :::info You can specify both order emulation and an execution algorithm. In this case, the order is first sent to the `OrderEmulator`, and upon release is then routed to the `ExecAlgorithm`. ::: This example submits a `MARKET` BUY order to a TWAP execution algorithm: ```python from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model import ExecAlgorithmId def buy(self) -> None: """ Users simple buy method (example). """ order: MarketOrder = self.order_factory.market( instrument_id=self.instrument_id, order_side=OrderSide.BUY, quantity=self.instrument.make_qty(self.trade_size), time_in_force=TimeInForce.FOK, exec_algorithm_id=ExecAlgorithmId("TWAP"), exec_algorithm_params={"horizon_secs": 20, "interval_secs": 2.5}, ) self.submit_order(order) ``` #### Canceling orders Orders can be canceled individually, as a batch, or all orders for an instrument (with an optional side filter). If the order is already *closed* or already pending cancel, then a warning will be logged. If the order is currently *open* then the status will become `PENDING_CANCEL`. The component a `CancelOrder`, `CancelAllOrders` or `BatchCancelOrders` command will flow to for execution depends on the following: - If the order is currently emulated, the command will *firstly* be sent to the `OrderEmulator`. - If an `exec_algorithm_id` is specified (with no `emulation_trigger`), and the order is still active within the local system, the command will *firstly* be sent to the relevant `ExecAlgorithm`. - Otherwise, the order will *firstly* be sent to the `ExecutionEngine`. :::info Any managed GTD timer will also be canceled after the command has left the strategy. ::: The following shows how to cancel an individual order: ```python self.cancel_order(order) ``` The following shows how to cancel a batch of orders: ```python from nautilus_trader.model.orders import Order my_order_list: list[Order] = [order1, order2, order3] self.cancel_orders(my_order_list) ``` The following shows how to cancel all orders: ```python self.cancel_all_orders() ``` #### Modifying orders Orders can be modified individually when emulated, or *open* on a venue (if supported). If the order is already *closed* or already pending cancel, then a warning will be logged. If the order is currently *open* then the status will become `PENDING_UPDATE`. :::warning At least one value must differ from the original order for the command to be valid. ::: The component a `ModifyOrder` command will flow to for execution depends on the following: - If the order is currently emulated, the command will *firstly* be sent to the `OrderEmulator`. - Otherwise, the order will *firstly* be sent to the `RiskEngine`. :::info Once an order is under the control of an execution algorithm, it cannot be directly modified by a strategy (only canceled). ::: The following shows how to modify the size of `LIMIT` BUY order currently *open* on a venue: ```python from nautilus_trader.model import Quantity new_quantity: Quantity = Quantity.from_int(5) self.modify_order(order, new_quantity) ``` :::info The price and trigger price can also be modified (when emulated or supported by a venue). ::: #### Market exit The `market_exit()` method provides a graceful way to exit all positions and cancel all orders for a strategy. The strategy remains running after the exit completes, allowing you to re-enter positions later if desired. ```python self.market_exit() ``` The market exit process: 1. Cancels all open and in-flight orders for the strategy. 2. Closes all open positions with market orders. 3. Periodically checks (at `market_exit_interval_ms`) until all orders resolve and positions close. 4. Calls `post_market_exit()` once flat, or after `market_exit_max_attempts` is reached. Two hooks are available for custom logic: - `on_market_exit()` - Called when the exit process begins. - `post_market_exit()` - Called when the exit process completes. ```python class MyStrategy(Strategy): def on_market_exit(self) -> None: self.log.info("Beginning market exit...") def post_market_exit(self) -> None: self.log.info("Market exit complete") ``` During a market exit, non-reduce-only orders are automatically denied. For order lists, if any order in the list is non-reduce-only, the entire list is denied to preserve list semantics (e.g., bracket orders with interdependencies). To check if an exit is in progress (e.g., to skip order submission logic), use `is_exiting()`: ```python def on_quote_tick(self, tick: QuoteTick) -> None: if self.is_exiting(): return # Skip order logic during exit # ... normal order logic ``` To automatically perform a market exit when the strategy is stopped, set `manage_stop=True`: ```python config = StrategyConfig(manage_stop=True) ``` With this option, calling `stop()` will first perform a market exit, then stop the strategy once flat. Configuration options in `StrategyConfig`: - `manage_stop` (default: False) - If True, `stop()` performs a market exit before stopping. - `market_exit_interval_ms` (default: 100) - Interval between exit completion checks. - `market_exit_max_attempts` (default: 100) - Maximum checks before completing the exit. - `market_exit_time_in_force` (default: None/GTC) - Time in force for closing market orders. - `market_exit_reduce_only` (default: True) - If closing market orders should be reduce only. ## Strategy configuration A separate configuration class gives full flexibility over where and how a strategy is instantiated. Configurations serialize over the wire, enabling distributed backtesting and remote live trading. This is opt-in. You can skip configuration and pass parameters directly to your strategy constructor. If you want distributed backtests or remote live trading, define a configuration. Here is an example configuration: ```python from decimal import Decimal from nautilus_trader.config import StrategyConfig from nautilus_trader.model import Bar, BarType from nautilus_trader.model import InstrumentId from nautilus_trader.trading.strategy import Strategy # Configuration definition class MyStrategyConfig(StrategyConfig): instrument_id: InstrumentId # example value: "ETHUSDT-PERP.BINANCE" bar_type: BarType # example value: "ETHUSDT-PERP.BINANCE-15-MINUTE[LAST]-EXTERNAL" fast_ema_period: int = 10 slow_ema_period: int = 20 trade_size: Decimal order_id_tag: str # Strategy definition class MyStrategy(Strategy): def __init__(self, config: MyStrategyConfig) -> None: # Always initialize the parent Strategy class # After this, configuration is stored and available via `self.config` super().__init__(config) # Custom state variables self.time_started = None self.count_of_processed_bars: int = 0 def on_start(self) -> None: self.time_started = self.clock.utc_now() # Remember time, when strategy started self.subscribe_bars(self.config.bar_type) # See how configuration data are exposed via `self.config` def on_bar(self, bar: Bar): self.count_of_processed_bars += 1 # Update count of processed bars # Instantiate configuration with specific values. By setting: # - InstrumentId - we parameterize the instrument the strategy will trade. # - BarType - we parameterize bar-data, that strategy will trade. config = MyStrategyConfig( instrument_id=InstrumentId.from_str("ETHUSDT-PERP.BINANCE"), bar_type=BarType.from_str("ETHUSDT-PERP.BINANCE-15-MINUTE[LAST]-EXTERNAL"), trade_size=Decimal(1), order_id_tag="001", ) # Pass configuration to our trading strategy. strategy = MyStrategy(config=config) ``` Access configuration values through `self.config`. This provides clear separation between: - Configuration data (accessed via `self.config`): - Contains initial settings, that define how the strategy works. - Example: `self.config.trade_size`, `self.config.instrument_id` - Strategy state variables (as direct attributes): - Track any custom state of the strategy. - Example: `self.time_started`, `self.count_of_processed_bars` This separation makes code easier to understand and maintain. :::note Even though it often makes sense to define a strategy which will trade a single instrument. The number of instruments a single strategy can work with is only limited by machine resources. ::: ### Managed GTD expiry It's possible for the strategy to manage expiry for orders with a time in force of GTD (*Good 'till Date*). This may be desirable if the exchange/broker does not support this time in force option, or for any reason you prefer the strategy to manage this. To use this option, pass `manage_gtd_expiry=True` to your `StrategyConfig`. When an order is submitted with a time in force of GTD, the strategy will automatically start an internal time alert. Once the internal GTD time alert is reached, the order will be canceled (if not already *closed*). Some venues (such as Binance Futures) support the GTD time in force, so to avoid conflicts when using `managed_gtd_expiry` you should set `use_gtd=False` for your execution client config. ### Multiple strategies If you intend running multiple instances of the same strategy, with different configurations (such as trading different instruments), then each instance needs a unique strategy ID and order ID tag. If `strategy_id` is not supplied, the platform builds the strategy ID from the strategy class name and an order ID tag. The tag can be supplied with `order_id_tag`; otherwise registration assigns the next numeric tag, starting with `000`. For example, the above config results in a strategy ID of `MyStrategy-001`. If `strategy_id` is supplied with `order_id_tag`, Rust appends the tag to the runtime strategy ID unless the ID already ends with that tag. For example, `strategy_id=MyStrategy-PRIMARY` with `order_id_tag=ABC` becomes `MyStrategy-PRIMARY-ABC`. If `order_id_tag` is omitted, Rust uses the final hyphen-separated part of `strategy_id` as the order ID tag. :::note The platform has built-in safety measures: if two strategies share a duplicated strategy ID, a `RuntimeError` is raised during registration indicating the strategy ID is already registered. ::: The reason for this is that the system must be able to identify which strategy various commands and events belong to. The order ID tag also keeps generated client order IDs unique across strategies for the same trader. :::info Rust implementation Rust treats `StrategyConfig` as immutable construction input. The runtime `StrategyId` carries the order ID tag, matching Python/Cython behavior. This keeps actor registration, client order ID generation, order list ID generation, and position ID generation aligned through `strategy_id.get_tag()`. If `strategy_id` is omitted, `order_id_tag` overrides the generated suffix, for example `MyStrategy-ABC`. ::: See the [`StrategyId` API Reference](/docs/python-api-latest/model/identifiers.html) for further details. ## Related guides - [Actors](actors.md) - Base class that strategies extend. - [Events](events.md) - Event types and handler dispatch. - [Orders](orders/) - Order types and management from strategies. - [Backtesting](backtesting.md) - Test strategies with historical data. # Synthetics Source: https://nautilustrader.io/docs/latest/concepts/synthetics/ Synthetic instruments are locally defined instruments whose prices derive from other instruments. They can combine components from one venue or many venues and expose the result as a standard Nautilus instrument with the synthetic venue code `SYNTH`. Synthetic instruments are useful for: - Enabling `Actor` and `Strategy` components to subscribe to quote or trade feeds. - Triggering emulated orders from derived prices. - Constructing bars from synthetic quotes or trades. Synthetic instruments cannot be traded directly. They exist locally within the platform and serve as analytical tools. In the future, Nautilus may support trading component instruments based on synthetic instrument behavior. ## Formula language Each synthetic instrument defines a derivation formula. Nautilus evaluates this formula with its built-in numeric expression engine and converts the final numeric result to the synthetic `Price`. ### Supported syntax Formulas can reference component `InstrumentId` values directly, including IDs that contain `/` and `-`. | Construct | Example | Notes | |---------------------|------------------------------------------------|-----------------------------------------------------------------------| | Component reference | `BTCUSDT.BINANCE` | Use the raw `InstrumentId` text. | | Component reference | `AUD/USD.SIM` | IDs containing `/` are valid. | | Component reference | `ETH-USDT-SWAP.OKX` | IDs containing `-` are valid. | | Numeric literal | `1`, `0.5`, `1.2e-3` | Evaluated with `f64` semantics. | | Boolean literal | `true`, `false` | Used in conditions and logical expressions. | | Parentheses | `(a + b) / 2` | Use parentheses to override precedence. | | Unary operators | `-x`, `!flag` | Unary `-` negates numbers. Unary `!` negates booleans. | | Binary operators | `+ - * / % ^`, `== !=`, `< <= > >=`, `&& \|\|` | Arithmetic is numeric. Logical operators are boolean. | | Local assignment | `spread = a - b; spread / 2` | Statements run from left to right. The formula must end with a value. | | Comments | `// line`, `/* block */` | Comments are ignored. | :::note New formulas should use raw `InstrumentId` values. For backward compatibility, formulas that replace `-` with `_` in component IDs remain accepted. ::: ### Operator precedence The expression engine evaluates operators in the following order, from highest precedence to lowest precedence: | Level | Operators | Notes | |---------|----------------------|--------------------------------------------------------------| | Highest | `^` | Exponentiation. Right associative. | | | Unary `-`, unary `!` | `-2 ^ 2` evaluates as `-(2 ^ 2)`. | | | `*`, `/`, `%` | Multiplication, division, and modulo. | | | `+`, `-` | Addition and subtraction. | | | `<`, `<=`, `>`, `>=` | Numeric comparisons. | | | `==`, `!=` | Equality and inequality. Both sides must have the same type. | | Lowest | `&&`, `\|\|` | Boolean operators. | Assignments are not expression operators. Separate statements with `;`, and make the last statement the value you want the synthetic to produce. ### Built-in functions | Function | Signature | Notes | |----------|----------------------------------------|------------------------------------------------------| | `abs` | `abs(x)` | Absolute value. | | `ceil` | `ceil(x)` | Ceiling. | | `floor` | `floor(x)` | Floor. | | `round` | `round(x)` | Round to the nearest integer using Rust `f64` rules. | | `min` | `min(x1, x2, ...)` | Accepts one or more numeric arguments. | | `max` | `max(x1, x2, ...)` | Accepts one or more numeric arguments. | | `if` | `if(condition, when_true, when_false)` | The condition must be boolean. Both branches match. Only the selected branch evaluates. | ### Type rules - Component inputs are numeric. - Arithmetic operators require numeric operands and return numeric results. - `<`, `<=`, `>`, `>=` require numeric operands and return boolean results. - `==` and `!=` accept any matching type (both numeric or both boolean) and return boolean results. - `&&`, `||`, and unary `!` require boolean operands. - `&&` and `||` short-circuit. The right-hand side evaluates only when needed. - Local variables must be assigned before use. - Local variable names must start with a letter or `_` and then use letters, digits, or `_`. - The final formula result must be numeric. A formula that ends with an assignment or produces a boolean result is invalid for a synthetic instrument. ### Limits The expression engine enforces the following compile-time limits. Formulas that exceed them produce a clear error at construction time. | Limit | Value | Description | |------------------|-------|----------------------------------------------------------------| | Stack depth | 32 | Maximum number of intermediate values on the evaluation stack. | | Local variables | 16 | Maximum number of distinct local variable names. | These limits are generous for any realistic pricing formula. A weighted sum of 8 components uses a peak stack depth of 3 and zero locals. ### Examples ```python # Simple spread formula = "BTCUSDT.BINANCE - ETHUSDT.BINANCE" # Average of two FX pairs formula = "(AUD/USD.SIM + NZD/USD.SIM) / 2" # Reuse an intermediate value formula = "spread = BTCUSDT.BINANCE - ETHUSDT.BINANCE; spread / 2" # Conditional output formula = "if(BTCUSDT.BINANCE > ETHUSDT.BINANCE, BTCUSDT.BINANCE, ETHUSDT.BINANCE)" ``` ## Creating a synthetic instrument Before defining a new synthetic instrument, make sure all component instruments already exist in the cache. The following example creates a synthetic instrument with an actor or strategy. This synthetic represents a simple spread between Bitcoin and Ethereum spot prices on Binance. It assumes that `BTCUSDT.BINANCE` and `ETHUSDT.BINANCE` already exist in the cache. ```python from nautilus_trader.model.instruments import SyntheticInstrument btcusdt_binance_id = InstrumentId.from_str("BTCUSDT.BINANCE") ethusdt_binance_id = InstrumentId.from_str("ETHUSDT.BINANCE") synthetic = SyntheticInstrument( symbol=Symbol("BTC-ETH:BINANCE"), price_precision=8, components=[ btcusdt_binance_id, ethusdt_binance_id, ], formula=f"{btcusdt_binance_id} - {ethusdt_binance_id}", ts_event=self.clock.timestamp_ns(), ts_init=self.clock.timestamp_ns(), ) self._synthetic_id = synthetic.id self.add_synthetic(synthetic) self.subscribe_quote_ticks(self._synthetic_id) ``` :::note The synthetic `instrument_id` in the example above is `{symbol}.SYNTH`, which produces `BTC-ETH:BINANCE.SYNTH`. ::: ## Updating formulas You can update a synthetic formula at any time. ```python synthetic = self.cache.synthetic(self._synthetic_id) new_formula = "(BTCUSDT.BINANCE + ETHUSDT.BINANCE) / 2" synthetic.change_formula(new_formula) self.update_synthetic(synthetic) ``` ## Trigger instrument IDs You can trigger emulated orders from synthetic prices. In the following example, a synthetic instrument releases an emulated order once the synthetic price reaches the trigger condition. ```python order = self.strategy.order_factory.limit( instrument_id=ETHUSDT_BINANCE.id, order_side=OrderSide.BUY, quantity=Quantity.from_str("1.5"), price=Price.from_str("30000.00000000"), emulation_trigger=TriggerType.DEFAULT, trigger_instrument_id=self._synthetic_id, ) self.strategy.submit_order(order) ``` ## Performance Formulas compile once at construction time and evaluate on every incoming component price tick. The expression engine uses a compile-once/eval-many architecture with a zero-allocation f64 stack, so evaluation adds negligible overhead to the tick-processing path. Measured on Apple M4 Pro, rustc 1.94.1, release profile (opt-level 3): ### Evaluation (hot path) | Formula pattern | Time | |-----------------------------------------|-------| | `(A + B) / 2.0` | 12 ns | | `A * 0.4 + B * 0.3 + C * 0.2 + D * 0.1` | 18 ns | | `if(A > B, A - B, B - A)` | 12 ns | | `spread = A - B; mid = ...; mid + ...` | 19 ns | | `max(min(A, B * 20), abs(A - B))` | 15 ns | ### Evaluation scaling (weighted sum) | Components | Time | |------------|-------| | 2 | 14 ns | | 4 | 18 ns | | 8 | 28 ns | ### Compilation (cold path) | Formula pattern | Time | |--------------------|--------| | Simple average | 675 ns | | 4-input weighted | 1.4 us | | Conditional | 1.0 us | | With locals | 1.3 us | | Hyphenated IDs | 755 ns | ## Error handling Nautilus validates synthetic instruments at every boundary. Formula compilation rejects unknown symbols, type errors, and capacity overflows. Evaluation rejects wrong input counts and non-finite prices (NaN, Infinity) before they reach the formula. See the [`SyntheticInstrument` API Reference](/docs/python-api-latest/model/instruments.html#nautilus_trader.model.instruments.synthetic.SyntheticInstrument) for input requirements and exceptions. ## Related guides - [Instruments](instruments/) - Instrument definitions and venue-specific instrument types. - [Data](data.md) - Market data types that reference instruments. - [Orders](orders/) - Orders can use synthetic instrument IDs for emulation triggers. # Value Types Source: https://nautilustrader.io/docs/latest/concepts/value_types/ NautilusTrader provides specialized value types for representing core trading concepts: `Price`, `Quantity`, and `Money`. These types use fixed-point arithmetic internally for performant, deterministic calculations across different platforms and environments. ## Overview | Type | Purpose | Signed | Currency | |------------|------------------------------------------|--------|----------| | `Quantity` | Trade sizes, order amounts, positions. | No | - | | `Price` | Market prices, quotes, price levels. | Yes | - | | `Money` | Monetary amounts, P&L, account balances. | Yes | Yes | ## Immutability All value types are **immutable**. Once a value is constructed, it cannot be changed. Operations do not mutate the original object. ```python from nautilus_trader.model.objects import Quantity qty1 = Quantity(100, precision=0) qty2 = Quantity(50, precision=0) # This creates a NEW Quantity; qty1 and qty2 are unchanged result = qty1 + qty2 print(qty1) # 100 print(qty2) # 50 print(result) # 150 ``` This design provides several benefits: - **Thread safety**: Immutable values can be safely shared across threads without synchronization. - **Predictability**: Values never change unexpectedly, making debugging easier. - **Hashability**: Immutable types can be used as dictionary keys and in sets. ## Arithmetic operations Value types support standard arithmetic operators (`+`, `-`, `*`, `/`, `%`, `//`) and unary operators (`-`, `+`, `abs`). The return type depends on the operator and the operand types. ### Same-type binary operations Addition and subtraction of the same value type return that type, preserving domain meaning (a price plus a price is still a price): | Operation | Result | |-----------------------|------------| | `Quantity + Quantity` | `Quantity` | | `Quantity - Quantity` | `Quantity` | | `Price + Price` | `Price` | | `Price - Price` | `Price` | | `Money + Money` | `Money` | | `Money - Money` | `Money` | ```python from nautilus_trader.model.objects import Price price1 = Price(100.50, precision=2) price2 = Price(0.25, precision=2) result = price1 + price2 # Returns Price(100.75, precision=2) print(type(result)) # ``` Multiplication, division, floor division, and modulo between two values of the same type return `Decimal`: | Operation | Result | |-----------------------|-----------| | `Price * Price` | `Decimal` | | `Price / Price` | `Decimal` | | `Price // Price` | `Decimal` | | `Price % Price` | `Decimal` | The same pattern applies to `Quantity` and `Money`. These operations do not return the original type because the result has different dimensional meaning. Multiplying a price by a price produces "price squared", not a price. Dividing a quantity by a quantity produces a dimensionless ratio, not a quantity. Returning `Decimal` makes the unit change explicit and prevents misinterpretation of the result as a value with the original unit. ### Unary operations Unary operators preserve the value type where the result is valid for that type: | Operation | `Price` | `Quantity` | `Money` | |--------------|-----------|------------|-----------| | `-x` (neg) | `Price` | `Decimal` | `Money` | | `+x` (pos) | `Price` | `Quantity` | `Money` | | `abs(x)` | `Price` | `Quantity` | `Money` | | `int(x)` | `int` | `int` | `int` | | `float(x)` | `float` | `float` | `float` | | `round(x)` | `Decimal` | `Decimal` | `Decimal` | `Quantity.__neg__` returns `Decimal` rather than `Quantity` because `Quantity` is unsigned and cannot represent a negative value. ```python from nautilus_trader.model.objects import Price, Quantity, Money from nautilus_trader.model.currencies import USD price = Price(100.50, precision=2) print(-price) # -100.50 print(type(-price)) # money = Money(-50.00, USD) print(abs(money)) # 50.00 USD print(type(abs(money))) # qty = Quantity(10, precision=0) print(+qty) # 10 print(type(+qty)) # ``` ### Mixed-type operations When operating with other numeric types, the result type follows Python's [numeric tower](https://docs.python.org/3/library/numbers.html) conventions. The general principle is that operations widen to the more general type: `float` operations return `float`, while `int` and `Decimal` operations return `Decimal` for precision preservation. This applies to all six binary operators (`+`, `-`, `*`, `/`, `//`, `%`) and works in both directions (`value op scalar` and `scalar op value`): | Left operand | Right operand | Result type | |--------------|---------------|-------------| | Value type | `int` | `Decimal` | | Value type | `float` | `float` | | Value type | `Decimal` | `Decimal` | | `int` | Value type | `Decimal` | | `float` | Value type | `float` | | `Decimal` | Value type | `Decimal` | ```python from decimal import Decimal from nautilus_trader.model.objects import Quantity qty = Quantity(100, precision=0) # Quantity + int -> Decimal result1 = qty + 50 print(type(result1)) # # Quantity + float -> float result2 = qty + 50.5 print(type(result2)) # # Quantity + Decimal -> Decimal result3 = qty + Decimal("50") print(type(result3)) # ``` ## Precision handling Each value type stores a precision field indicating the number of decimal places. Precision is set at construction and is immutable. There is no "unspecified" precision. ### Fixed-point representation Value types are stored internally as integers scaled to a global fixed precision (e.g., 10^16 in high-precision mode), not floating-point numbers. The `precision` field tracks the number of decimal places used at construction, controlling display formatting and serialization, but the underlying raw value always uses the global scale. ```python from nautilus_trader.model.objects import Price p1 = Price(1.23, precision=2) # displays as "1.23" p2 = Price(1.230, precision=3) # displays as "1.230" p1 == p2 # True: same underlying value str(p1) # "1.23" str(p2) # "1.230" ``` **Precision controls display, not identity.** Two prices with the same decimal value but different precisions are equal. The `precision` field determines string formatting and how many decimal places are shown, but equality is based on the underlying numeric value. **Market data serialization uses precision metadata.** When market data types (quotes, trades, order book deltas) are written to Parquet or Arrow format, precision is stored in the file metadata so that values can be correctly decoded. All market data values within a single file must share the same precision. :::note If a venue changes an instrument's tick size (and thus its precision), data files written before and after the change will have different precision metadata and should not be consolidated into a single file. ::: For how instrument-level precision constrains valid prices and quantities, see the [Precision](instruments/index.md#precision) section of the Instruments guide. ### Arithmetic precision When performing arithmetic between values with different precisions, the result uses the maximum precision of the operands. ```python from nautilus_trader.model.objects import Price price1 = Price(100.5, precision=1) # 1 decimal place price2 = Price(0.125, precision=3) # 3 decimal places result = price1 + price2 print(result) # 100.625 print(result.precision) # 3 (max of 1 and 3) ``` ## Type-specific constraints ### Quantity `Quantity` represents non-negative amounts. Attempting to create a negative quantity or subtract a larger quantity from a smaller one raises an error: ```python from nautilus_trader.model.objects import Quantity # This raises ValueError: Quantity cannot be negative qty = Quantity(-100, precision=0) # This also raises ValueError qty1 = Quantity(50, precision=0) qty2 = Quantity(100, precision=0) result = qty1 - qty2 # Would be -50, which is invalid ``` ### Money `Money` values include a currency. Addition and subtraction between `Money` values require matching currencies: ```python from nautilus_trader.model.objects import Money from nautilus_trader.model.currencies import USD, EUR usd_amount = Money(100.00, USD) eur_amount = Money(50.00, EUR) # This works - same currency result = usd_amount + Money(25.00, USD) # This raises ValueError - currency mismatch result = usd_amount + eur_amount ``` ## Common patterns ### Accumulating values Since value types are immutable, accumulate by reassigning: ```python from nautilus_trader.model.objects import Money from nautilus_trader.model.currencies import USD total = Money(0.00, USD) amounts = [Money(100.00, USD), Money(50.00, USD), Money(25.00, USD)] for amount in amounts: total = total + amount # Reassign to new Money instance print(total) # 175.00 USD ``` ### Converting to other types Value types provide conversion methods: ```python from nautilus_trader.model.objects import Price price = Price(123.456, precision=3) # Convert to Decimal (preserves precision) decimal_value = price.as_decimal() # Convert to float float_value = price.as_double() # Convert to string string_value = str(price) # "123.456" ``` ### Creating from strings Parse value types from string representations: ```python from nautilus_trader.model.objects import Quantity, Price, Money qty = Quantity.from_str("100.5") price = Price.from_str("99.95") money = Money.from_str("1000.00 USD") ``` # Visualization Source: https://nautilustrader.io/docs/latest/concepts/visualization/ NautilusTrader provides interactive HTML tearsheets for analyzing backtest results through an extensible visualization system built on Plotly. You can generate reports with minimal code and add custom charts and themes. ## Overview The visualization system has three parts: 1. **Chart Registry** - Decoupled chart definitions that can be extended with custom visualizations. 2. **Theme System** - Consistent styling with built-in and custom themes. 3. **Configuration** - Declarative specification of what to render and how to display it. All visualization outputs are self-contained HTML files that can be viewed in any modern browser, shared with stakeholders, or archived for future reference. :::note The visualization system requires the `visualization` extra. It installs Pandas for DataFrame handling, Plotly for interactive figures, and Kaleido for static image export: ```bash uv pip install "nautilus_trader[visualization]" ``` ::: ## Tearsheets A tearsheet is a performance report that combines multiple charts and statistics into a single interactive visualization. Tearsheets are generated after completing a backtest run and provide immediate visual feedback on strategy performance. ### Quick start Generate a tearsheet with default settings: ```python from nautilus_trader.analysis import create_tearsheet from nautilus_trader.backtest.engine import BacktestEngine # After running your backtest engine.run() # Generate tearsheet create_tearsheet( engine=engine, output_path="backtest_results.html", ) ``` This produces an HTML file with all default charts, using the light theme and automatic layout. Open `backtest_results.html` in your browser to view the interactive tearsheet. ### Customization Control which charts appear and how they're styled: ```python from nautilus_trader.analysis import TearsheetConfig from nautilus_trader.analysis import TearsheetDrawdownChart from nautilus_trader.analysis import TearsheetEquityChart from nautilus_trader.analysis import TearsheetRunInfoChart from nautilus_trader.analysis import TearsheetStatsTableChart config = TearsheetConfig( charts=[ TearsheetRunInfoChart(), TearsheetStatsTableChart(), TearsheetEquityChart(), TearsheetDrawdownChart(), ], theme="nautilus_dark", height=2000, ) create_tearsheet( engine=engine, output_path="custom_tearsheet.html", config=config, ) ``` ### Currency filtering For multi-currency backtests, filter statistics to a specific currency: ```python from nautilus_trader.model.currencies import USD create_tearsheet( engine=engine, output_path="usd_only.html", currency=USD, # Currency object, shows only USD statistics ) ``` When `currency` is `None` (default), statistics for all currencies are displayed separately in the tearsheet. Return-based charts are reconstructed from account reports only when the accounts share one currency; pass `currency` for multi-currency backtests so return charts use the selected currency. ## Available charts The tearsheet can include any combination of the following built-in charts: | Chart Name | Type | Description | |--------------------|--------------|----------------------------------------------------------| | `run_info` | Table | Run metadata and account balances. | | `stats_table` | Table | Performance statistics (PnL, returns, general metrics). | | `equity` | Line | Cumulative returns over time with optional benchmark. | | `drawdown` | Area | Drawdown percentage from peak equity. | | `monthly_returns` | Heatmap | Monthly portfolio return percentages organized by year. | | `distribution` | Histogram | Distribution of individual return values. | | `rolling_sharpe` | Line | 60-day rolling Sharpe ratio. | | `yearly_returns` | Bar | Annual return percentages. | | `bars_with_fills` | Candlestick | Price bars (OHLC) with order fills overlaid as markers. | All charts are registered in the chart registry and are configured via chart objects in `TearsheetConfig.charts` (each chart object maps to a built-in chart name). ### Run information table The `run_info` chart displays key metadata about the backtest run: - Run ID, start time, finish time - Backtest period (start/end dates) - Total iterations processed - Event, order, and position counts - Account starting and ending balances (per currency) This table appears in the top-left position by default. ### Performance statistics table The `stats_table` chart displays performance metrics organized into sections: - **PnL Statistics** (per currency): Total PnL, win rate, profit factor, etc. - **Returns Statistics**: Sharpe ratio, Sortino ratio, max drawdown, etc. - **General Statistics**: Total trades, average trade duration, etc. This table appears in the top-right position by default. ### Equity curve The `equity` chart plots cumulative returns over the backtest period. When `benchmark_returns` is provided to `create_tearsheet()`, the benchmark is overlaid for comparison. ```python import pandas as pd # Load benchmark returns (e.g., from a market index) # Index should be datetime, aligned with strategy returns timeframe benchmark_returns = pd.read_csv("sp500_returns.csv", index_col=0, parse_dates=True)["return"] create_tearsheet( engine=engine, output_path="with_benchmark.html", benchmark_returns=benchmark_returns, benchmark_name="S&P 500", ) ``` The benchmark series is plotted as-is; ensure the index aligns with your strategy's return dates for accurate comparison. ### Monthly and yearly returns The `monthly_returns` and `yearly_returns` charts default to compounded (time-weighted) returns: each cell measures the period's gain against the running start-of-period balance, and the periods compound to the total return. Set `compounding=False` to report simple, non-compounding returns measured against fixed initial capital. Each cell then measures the period's gain as a percentage of the starting capital, so the periods sum to the total return instead of compounding to it. This is the nominal rate of return, the convention used for constant-capital strategies that trade fixed size and withdraw profits. ```python config = TearsheetConfig( charts=[ TearsheetMonthlyReturnsChart(compounding=False), TearsheetYearlyReturnsChart(compounding=False), ], ) create_tearsheet(engine=engine, config=config) ``` The standalone `create_monthly_returns_heatmap()` and `create_yearly_returns()` functions accept the same `compounding` argument. For the non-compounding figures to faithfully represent constant capital, size positions at a fixed quantity rather than as a fraction of current equity; otherwise later periods inflate as the running balance grows. ## Themes Themes control the visual styling of charts including colors, fonts, and backgrounds. NautilusTrader provides four built-in themes: | Theme Name | Description | Use Case | |-----------------|------------------------------------------------|--------------------------------| | `plotly_white` | Clean light theme with dark gray headers. | Default, professional reports. | | `plotly_dark` | Dark background with standard Plotly colors. | Low‑light environments. | | `nautilus` | Light theme with NautilusTrader brand colors. | Official light mode. | | `nautilus_dark` | Dark theme with teal/cyan signature colors. | Official dark mode. | ### Selecting a theme Specify the theme in `TearsheetConfig`: ```python config = TearsheetConfig(theme="nautilus_dark") create_tearsheet(engine=engine, config=config) ``` ### Custom themes Register a custom theme for consistent branding across all visualizations: ```python from nautilus_trader.analysis import register_theme register_theme( name="corporate", template="plotly_white", # Base Plotly template colors={ "primary": "#003366", # Navy blue "positive": "#2e8b57", # Sea green "negative": "#c41e3a", # Cardinal red "neutral": "#808080", # Gray "background": "#ffffff", # White "grid": "#e5e5e5", # Light gray # Optional table colors (defaults will be provided if omitted) "table_section": "#e5e5e5", "table_row_odd": "#f8f8f8", "table_row_even": "#ffffff", "table_text": "#000000", } ) # Use the custom theme config = TearsheetConfig(theme="corporate") ``` The theme system automatically provides sensible defaults for `table_*` colors based on the `background` and `grid` colors, ensuring backward compatibility with themes registered before table-specific colors were introduced. ## Configuration The `TearsheetConfig` class provides declarative control over tearsheet generation: ```python from nautilus_trader.analysis import GridLayout from nautilus_trader.analysis import TearsheetConfig from nautilus_trader.analysis import TearsheetDrawdownChart from nautilus_trader.analysis import TearsheetEquityChart from nautilus_trader.analysis import TearsheetStatsTableChart config = TearsheetConfig( charts=[ TearsheetEquityChart(), TearsheetDrawdownChart(), TearsheetStatsTableChart(), ], theme="nautilus_dark", title="Q4 2024 Strategy Performance", height=1800, include_benchmark=True, benchmark_name="SPY", layout=GridLayout( rows=2, cols=2, heights=[0.60, 0.40], vertical_spacing=0.08, horizontal_spacing=0.12, ), ) ``` ### Configuration parameters | Parameter | Type | Default | Description | |---------------------|------------------------|------------------|-------------------------------------| | `charts` | `list[TearsheetChart]` | Built‑ins | Charts to include, in order. | | `theme` | `str` | `"plotly_white"` | Theme name for styling. | | `layout` | `GridLayout` | `None` | Custom subplot grid layout. | | `title` | `str` | Auto‑generated | Tearsheet title. | | `include_benchmark` | `bool` | `True` | Show benchmark when provided. | | `benchmark_name` | `str` | `"Benchmark"` | Display name for benchmark. | | `height` | `int` | `1500` | Total height in pixels. | | `show_logo` | `bool` | `True` | Reserved for future logo rendering. | When `layout` is `None`, the grid dimensions and row heights are automatically calculated based on the number of charts. For 8 charts (the default), a 4x2 grid is used with heights `[0.50, 0.22, 0.16, 0.12]` to give more space to the top row tables. ## Custom charts The registry pattern lets you add custom charts. Charts are functions that render traces onto a Plotly figure object. ### Registering a custom chart ```python from nautilus_trader.analysis.tearsheet import register_chart import plotly.graph_objects as go def my_custom_chart(returns, output_path=None, title="Custom Chart", theme="plotly_white"): """ Create a custom visualization. This function signature matches the built-in chart functions for consistency. """ from nautilus_trader.analysis.themes import get_theme theme_config = get_theme(theme) # Create your visualization fig = go.Figure() fig.add_trace(go.Scatter( x=returns.index, y=returns.cumsum(), mode="lines", name="Custom Metric", line={"color": theme_config["colors"]["primary"]}, )) fig.update_layout( title=title, template=theme_config["template"], xaxis_title="Date", yaxis_title="Value", ) if output_path: fig.write_html(output_path) return fig # Register the chart for standalone use (via `get_chart()` / `list_charts()`) register_chart("my_custom", my_custom_chart) ``` ### Tearsheet integration For tearsheet integration with proper grid placement, use `register_tearsheet_chart`. Unlike `register_chart` (which registers a standalone function that returns its own figure), a tearsheet renderer draws traces directly onto a shared subplot grid cell, so its signature takes the target `fig` plus the `row` and `col` to render into. ```python from nautilus_trader.analysis import TearsheetConfig from nautilus_trader.analysis import TearsheetCustomChart from nautilus_trader.analysis import TearsheetEquityChart from nautilus_trader.analysis import TearsheetStatsTableChart from nautilus_trader.analysis import register_tearsheet_chart def _render_my_metric(fig, row, col, returns, theme_config, **kwargs): """ Render custom metric directly onto a subplot. Parameters ---------- fig : go.Figure The figure to add traces to. row : int Subplot row position. col : int Subplot column position. returns : pd.Series Strategy returns series supplied to the renderer. theme_config : dict Theme configuration dictionary. **kwargs : dict Additional parameters (stats_pnls, stats_returns, benchmark_returns, etc.). """ metric_values = returns.rolling(30).std() * 100 # Example metric fig.add_trace( go.Scatter( x=returns.index, y=metric_values, mode="lines", name="30-Day Volatility", line={"color": theme_config["colors"]["neutral"]}, ), row=row, col=col, ) fig.update_xaxes(title_text="Date", row=row, col=col) fig.update_yaxes(title_text="Volatility (%)", row=row, col=col) # Register for tearsheet use register_tearsheet_chart( name="volatility", subplot_type="scatter", title="Rolling Volatility (30-day)", renderer=_render_my_metric, ) # Now "volatility" can be used in TearsheetConfig.charts: config = TearsheetConfig( charts=[ TearsheetStatsTableChart(), TearsheetEquityChart(), TearsheetCustomChart(chart="volatility"), ], ) ``` The renderer function receives all necessary data (returns, statistics, theme configuration) and renders directly onto the specified subplot position. ## Offline analysis For situations where you have precomputed statistics but not a `BacktestEngine` instance, use the lower-level API: ```python import pandas as pd from nautilus_trader.analysis.tearsheet import create_tearsheet_from_stats # Load precomputed data. The structure matches BacktestResult stats fields. stats_pnls = {"USD": {"PnL (total)": 1500.0, "Win Rate": 0.55, ...}} # Per-currency stats_returns = {"Sharpe Ratio (252 days)": 1.2, "Max Drawdown": -0.15, ...} stats_general = {"Avg Winner": 100.0, "Avg Loser": -50.0, ...} returns = pd.Series(...) # Daily returns with datetime index create_tearsheet_from_stats( stats_pnls=stats_pnls, stats_returns=stats_returns, stats_general=stats_general, returns=returns, output_path="offline_analysis.html", ) ``` The dictionary keys should match those returned by `engine.get_result().stats_pnls`, `engine.get_result().stats_returns`, and `engine.get_result().stats_general`. This approach is useful for: - Analyzing results from multiple backtest runs stored separately. - Comparing strategies using precomputed metrics. - Integrating with external analysis pipelines. ## Best practices ### Chart selection - Use default charts for exploratory analysis to see all available metrics. - Customize charts when you know which metrics matter for your strategy. - Remove irrelevant charts to reduce visual clutter and file size. ### Theme usage - Use `plotly_white` for professional reports and presentations. - Use `nautilus_dark` for official materials or low-light viewing. - Create custom themes to match internal guidelines or personal preferences. ### Performance considerations - Tearsheet HTML files contain all data inline and can be several megabytes for long backtests. - Consider generating separate tearsheets for different analysis timeframes. - For very large datasets, use the individual chart functions instead of full tearsheets. ### Custom statistics integration Custom charts work best when paired with statistics supplied through the same `stats_pnls`, `stats_returns`, and `stats_general` dictionaries used by the built-in tearsheet charts. For live `BacktestEngine` usage these values come from `engine.get_result()`; for offline analysis, pass compatible dictionaries directly to `create_tearsheet_from_stats()`: ```python stats_returns = { "Sharpe Ratio (252 days)": 1.2, "Custom Volatility Score": 0.42, } ``` ## API levels The visualization system provides two API levels: ### High-level API Recommended for most use cases: ```python create_tearsheet(engine=engine, config=config) ``` Automatically extracts data from the `BacktestEngine`, generates all configured charts, and produces a complete HTML tearsheet. ### Low-level API For advanced customization or offline analysis: ```python create_tearsheet_from_stats( stats_pnls=stats_pnls, stats_returns=stats_returns, stats_general=stats_general, returns=returns, run_info=run_info, account_info=account_info, config=config, ) ``` Provides fine-grained control over data inputs and allows analysis of precomputed statistics. ### Standalone chart functions Individual chart functions can be used independently to generate single-purpose HTML visualizations or Plotly figures for custom analysis workflows. #### Price bars with fills The `create_bars_with_fills` function generates a candlestick chart with order fills overlaid, useful for visually analyzing strategy execution within price action. It can be used standalone or included in tearsheets: ```python from nautilus_trader.analysis import create_bars_with_fills from nautilus_trader.analysis import create_tearsheet from nautilus_trader.analysis import TearsheetBarsWithFillsChart from nautilus_trader.analysis import TearsheetConfig from nautilus_trader.analysis import TearsheetEquityChart from nautilus_trader.analysis import TearsheetStatsTableChart from nautilus_trader.model.data import BarType # Standalone usage bar_type = BarType.from_str("ESM4.XCME-1-MINUTE-LAST-EXTERNAL") fig = create_bars_with_fills( engine=engine, bar_type=bar_type, title="ES Futures - Entry/Exit Analysis", ) fig.show() # Display in Jupyter fig.write_html("bars_with_fills.html") # Or save to file # Include in tearsheet config = TearsheetConfig( charts=[ TearsheetStatsTableChart(), TearsheetEquityChart(), TearsheetBarsWithFillsChart( bar_type="ESM4.XCME-1-MINUTE-LAST-EXTERNAL", title="Bars with Fills", ), ], ) create_tearsheet(engine=engine, config=config) # Multiple bars-with-fills charts in one tearsheet config = TearsheetConfig( charts=[ TearsheetStatsTableChart(), TearsheetEquityChart(), TearsheetBarsWithFillsChart( bar_type=f"{instrument.id}-5-MINUTE-MID-INTERNAL", title=f"Bars with Order Fills - {instrument.id}", ), TearsheetBarsWithFillsChart( bar_type=f"{other_instrument.id}-5-MINUTE-MID-INTERNAL", title=f"Bars with Order Fills - {other_instrument.id}", ), ], ) create_tearsheet(engine=engine, config=config) ``` The visualization shows candlesticks for OHLC price action with triangle markers representing order fills (green up-triangles for buys, red down-triangles for sells). Charts that need extra configuration (like `bar_type`) take those parameters directly on the chart object (e.g. `TearsheetBarsWithFillsChart(bar_type=...)`). Other individual chart functions include `create_equity_curve`, `create_drawdown_chart`, `create_monthly_returns_heatmap`, and more. See the API reference for the complete list. ## Related guides - [Backtesting](backtesting.md) - Learn how to run backtests that generate tearsheets. - [Reports](reports.md) - Understand the underlying statistics displayed in tearsheets. - [Portfolio](portfolio.md) - Explore portfolio tracking and performance metrics. # Adapters Source: https://nautilustrader.io/docs/latest/developer_guide/adapters/ ## Introduction This developer guide provides specifications for how to build an integration adapter for the NautilusTrader platform. Adapters connect to trading venues and data providers, translating their native APIs into the platform’s unified interface and normalized domain model. ## Structure of an adapter NautilusTrader adapters follow a layered architecture pattern with: - **Rust core** for networking clients and performance-sensitive operations. - **Python layer** for integrating Rust clients into the platform's data and execution engines. ### Rust core (`crates/adapters/your_adapter/`) The Rust layer handles: - **HTTP client**: Raw API communication, request signing, rate limiting. - **WebSocket client**: Low-latency streaming connections, message parsing. - **Parsing**: Fast conversion of venue data to Nautilus domain models. - **Python bindings**: PyO3 exports to make Rust functionality available to Python. Typical Rust structure: ``` crates/adapters/your_adapter/ ├── src/ │ ├── common/ # Shared types and utilities │ │ ├── consts.rs # Venue constants / broker IDs │ │ ├── credential.rs # API key storage and signing helpers │ │ ├── enums.rs # Venue enums mirrored in REST/WS payloads │ │ ├── error.rs # Adapter-level error aggregation (when applicable) │ │ ├── models.rs # Shared model types │ │ ├── parse.rs # Shared parsing helpers │ │ ├── retry.rs # Retry classification (when applicable) │ │ ├── urls.rs # Environment & product aware base-url resolvers │ │ └── testing.rs # Fixtures reused across unit tests │ ├── http/ # HTTP client implementation │ │ ├── client.rs # HTTP client with authentication │ │ ├── error.rs # HTTP-specific error types │ │ ├── models.rs # Structs for REST payloads │ │ ├── parse.rs # Response parsing functions │ │ └── query.rs # Request and query builders │ ├── websocket/ # WebSocket implementation │ │ ├── client.rs # WebSocket client │ │ ├── dispatch.rs # Execution event dispatch and order routing │ │ ├── enums.rs # WebSocket-specific enums │ │ ├── error.rs # WebSocket-specific error types │ │ ├── handler.rs # Feed handler (I/O boundary) │ │ ├── messages.rs # Frame and message enums │ │ ├── parse.rs # Message parsing functions │ │ └── subscription.rs # Subscription topic helpers (optional) │ ├── python/ # PyO3 Python bindings │ │ ├── enums.rs # Python-exposed enums │ │ ├── http.rs # Python HTTP client bindings │ │ ├── urls.rs # Python URL helpers │ │ ├── websocket.rs # Python WebSocket client bindings │ │ └── mod.rs # Module exports │ ├── config.rs # Configuration structures │ ├── data.rs # Data client implementation │ ├── execution.rs # Execution client implementation │ ├── factories.rs # Factory functions │ └── lib.rs # Library entry point ├── tests/ # Integration tests with mock servers │ ├── data_client.rs # Data client integration tests │ ├── exec_client.rs # Execution client integration tests │ ├── http.rs # HTTP client integration tests │ └── websocket.rs # WebSocket client integration tests └── test_data/ # Canonical venue payloads ``` ### Python layer (`nautilus_trader/adapters/your_adapter`) The Python layer provides the integration interface through these components: 1. **Instrument Provider**: Supplies instrument definitions via `InstrumentProvider`. 2. **Data Client**: Handles market data feeds and historical data requests via `LiveDataClient` and `LiveMarketDataClient`. 3. **Execution Client**: Manages order execution via `LiveExecutionClient`. 4. **Factories**: Converts venue-specific data to Nautilus domain models. 5. **Configuration**: User-facing configuration classes for client settings. Typical Python structure: ``` nautilus_trader/adapters/your_adapter/ ├── config.py # Configuration classes ├── constants.py # Adapter constants ├── data.py # LiveDataClient/LiveMarketDataClient ├── execution.py # LiveExecutionClient ├── factories.py # Instrument factories ├── providers.py # InstrumentProvider └── __init__.py # Package initialization ``` ## Adapter implementation sequence Follow this dependency-driven order when building an adapter. Each phase builds on the previous one. Implement the Rust core before any Python layer. ### Phase 1: Rust core infrastructure Build the low-level networking and parsing foundation. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 1.1 | HTTP error types | Define HTTP‑specific error enum with retryable/non‑retryable variants (`http/error.rs`). | | 1.2 | HTTP client | Implement credentials, request signing, rate limiting, and retry logic. | | 1.3 | HTTP API models | Define request/response structs for REST endpoints (`http/models.rs`, `http/query.rs`). | | 1.4 | HTTP parsing | Convert venue responses to Nautilus domain models (`http/parse.rs`, `common/parse.rs`). | | 1.5 | WebSocket error types | Define WebSocket‑specific error enum (`websocket/error.rs`). | | 1.6 | WebSocket client | Implement connection lifecycle, authentication, heartbeat, and reconnection. | | 1.7 | WebSocket messages | Define streaming payload types (`websocket/messages.rs`). | | 1.8 | WebSocket parsing | Convert stream messages to Nautilus domain models (`websocket/parse.rs`). | | 1.9 | Python bindings | Expose Rust functionality via PyO3 (`python/mod.rs`). | **Milestone**: Rust crate compiles, unit tests pass, HTTP/WebSocket clients can authenticate and stream/request raw data. ### Phase 2: Instrument definitions Instruments are the foundation: both data and execution clients depend on them. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 2.1 | Instrument parsing | Parse venue instrument definitions into Nautilus types (spot, perpetual, future, option). | | 2.2 | Instrument provider | Implement `InstrumentProvider` to load, filter, and cache instruments. | | 2.3 | Symbol mapping | Handle venue‑specific symbol formats and Nautilus `InstrumentId` conversion. | **Milestone**: `InstrumentProvider.load_all_async()` returns valid Nautilus instruments. ### Phase 3: Market data Build data subscriptions and historical data requests. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 3.1 | Public WebSocket streams | Subscribe to order books, trades, tickers, and other public channels. | | 3.2 | Historical data requests | Fetch historical bars, trades, and order book snapshots via HTTP. | | 3.3 | Data client (Python) | Implement `LiveDataClient` or `LiveMarketDataClient` wiring Rust clients to the data engine. | **Milestone**: Data client connects, subscribes to instruments, and emits market data to the platform. ### Phase 4: Order execution Build order management and account state. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 4.1 | Private WebSocket streams | Subscribe to order updates, fills, positions, and account balance changes. | | 4.2 | Basic order submission | Implement market and limit orders via HTTP or WebSocket. | | 4.3 | Order modification/cancel | Implement order amendment and cancellation. | | 4.4 | Execution client (Python) | Implement `LiveExecutionClient` wiring Rust clients to the execution engine. | | 4.5 | Execution reconciliation | Generate order, fill, and position status reports for startup reconciliation. | **Milestone**: Execution client submits orders, receives fills, and reconciles state on connect. ### Phase 5: Advanced features Extend coverage based on venue capabilities. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 5.1 | Advanced order types | Conditional orders, stop‑loss, take‑profit, trailing stops, iceberg, etc. | | 5.2 | Batch operations | Batch order submission, batch cancellation, mass cancel. | | 5.3 | Venue‑specific features | Options chains, funding rates, liquidations, or other venue‑specific data. | ### Phase 6: Configuration and factories Wire everything together for production usage. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 6.1 | Configuration classes | Create `LiveDataClientConfig` and `LiveExecClientConfig` subclasses. | | 6.2 | Factory functions | Implement factory functions to instantiate clients from configuration. | | 6.3 | Environment variables | Support credential resolution from environment variables. | ### Phase 7: Testing and documentation Validate the integration and document usage. | Step | Component | Description | |------|----------------------------|----------------------------------------------------------------------------------------------| | 7.1 | Rust unit tests | Test parsers, signing helpers, and business logic in `#[cfg(test)]` blocks. | | 7.2 | Rust integration tests | Test HTTP/WebSocket clients against mock Axum servers in `tests/`. | | 7.3 | Python integration tests | Test data/execution clients in `tests/integration_tests/adapters//`. | | 7.4 | Example scripts | Provide runnable examples demonstrating data subscription and order execution. | See the [Testing](#testing) section for detailed test organization guidelines. --- ## Rust adapter patterns ### Common code (`common/`) Group venue constants, credential helpers, enums, and reusable parsers under `src/common`. Adapters such as OKX keep submodules like `consts`, `credential`, `enums`, and `urls` alongside a `testing` module for fixtures, providing a single place for cross-cutting pieces. When an adapter has multiple environments or product categories, add a dedicated `common::urls` helper so REST/WebSocket base URLs stay in sync with the Python layer. ### Symbol normalization (`common/symbol.rs`) When a venue uses a different symbol format than Nautilus `InstrumentId`, place bidirectional conversion helpers in `common/symbol.rs`. Two functions form the standard interface: - `format_instrument_id(venue_symbol, product_type)` converts a venue symbol string to a Nautilus `InstrumentId`, appending or transforming product-type suffixes as needed (e.g., `"BTCUSDT"` + `Linear` becomes `"BTCUSDT-LINEAR.BYBIT"`). - `format_venue_symbol(instrument_id)` strips Nautilus suffixes to recover the venue-native symbol for API calls. Common patterns across adapters: - **Suffix-based product types**: Bybit appends `-SPOT`, `-LINEAR`, `-INVERSE`, `-OPTION`. A `BybitSymbol` wrapper validates the suffix and normalizes to uppercase on construction. - **Implicit product mapping**: Binance USD-M futures append `-PERP` at the Nautilus layer while COIN-M keeps the venue's existing `_PERP` suffix. - **Case normalization**: Convert to uppercase on input when venues are case-insensitive. - **`Ustr` interning**: Store normalized symbols as `Ustr` for zero-cost comparison. For venues where the raw symbol maps 1:1 to an `InstrumentId` (no suffix gymnastics), inline helpers in `common/parse.rs` are sufficient and a dedicated `symbol.rs` is not needed. ### URL resolution Define URL constants and resolution functions in `common/urls.rs`: ```rust const VENUE_WS_URL: &str = "wss://stream.venue.com/ws"; const VENUE_TESTNET_WS_URL: &str = "wss://testnet-stream.venue.com/ws"; pub const fn get_ws_base_url(testnet: bool) -> &'static str { if testnet { VENUE_TESTNET_WS_URL } else { VENUE_WS_URL } } ``` Config structs should provide override fields (`base_url_http`, `base_url_ws`, etc.) that fall back to these defaults when unset. ### Configurations (`config.rs`) Expose typed config structs in `src/config.rs` so Python callers toggle venue-specific behaviour (see how OKX wires demo URLs, retries, and channel flags). Keep defaults minimal and delegate URL selection to helpers in `common::urls`. For the user-facing design rationale, see the [Configuration](../concepts/configuration.md) concept guide. #### Builder and Default Config structs derive `bon::Builder` and implement `Default`. The builder owns all default values via `#[builder(default = value)]` annotations. The `Default` impl delegates to the builder so defaults are defined in exactly one place: ```rust #[derive(Clone, Debug, bon::Builder)] pub struct VenueDataClientConfig { pub api_key: Option, #[builder(default = 60)] pub http_timeout_secs: u64, #[builder(default = 3)] pub max_retries: u32, } impl Default for VenueDataClientConfig { fn default() -> Self { Self::builder().build() } } ``` This prevents drift between builder defaults and `Default` output. Never duplicate default values in the `Default` impl body. Bon always defaults `Option` fields to `None`. For the rare case where an `Option` field should default to `Some(value)`, override it in the `Default` impl and delegate everything else to the builder: ```rust impl Default for VenueDataClientConfig { fn default() -> Self { Self { poll_interval_secs: Some(60), ..Self::builder().build() } } } ``` #### Field type rules Use plain `T` with `#[builder(default = value)]` when a field always has a sensible default and downstream code consumes the value directly: ```rust #[builder(default = 60)] pub http_timeout_secs: u64, ``` Use `Option` (no builder annotation) when `None` carries distinct meaning such as "feature disabled", "unbounded", or "inherit from environment": ```rust /// Interval in seconds between open order checks. /// When `None`, open order polling is disabled. pub open_check_interval_secs: Option, ``` Choose the type based on the config's own semantics, not downstream function signatures. If `None` means "this feature is off" at the config level, use `Option`. If the field always resolves to a concrete value, use plain `T` even when a downstream constructor still accepts `Option` and the call site wraps with `Some(config.field)`. #### Python constructors The `py_new` constructor accepts `Option` for all configurable fields (Python callers pass `None` to mean "use default"). For plain `T` fields, unwrap against the default: ```rust fn py_new(http_timeout_secs: Option) -> Self { let defaults = Self::default(); Self { http_timeout_secs: http_timeout_secs.unwrap_or(defaults.http_timeout_secs), .. } } ``` For `Option` fields, use `.or()` to fall back to the default option value. When the default is `None`, this preserves the caller's `None`. When the default is `Some(value)`, this fills in the default if the caller passed `None`: ```rust open_check_interval_secs: open_check_interval_secs.or(defaults.open_check_interval_secs), ``` #### Default values Use sensible production defaults: credentials as `None` (resolved from environment at runtime), mainnet URLs, standard timeouts. For `trader_id` and `account_id`, use placeholder values like `TraderId::from("TRADER-001")` and `AccountId::from("VENUE-001")`. The `..Default::default()` pattern keeps examples and tests focused on fields that differ from defaults: ```rust let config = VenueExecClientConfig { trader_id, account_id, environment: VenueEnvironment::Testnet, ..Default::default() }; ``` ### Error taxonomy (`common/error.rs`) For adapters with multiple client types, define an adapter-level error enum in `common/error.rs` that aggregates component errors: ```rust #[derive(Debug, thiserror::Error)] pub enum VenueError { #[error("HTTP error: {0}")] Http(#[from] VenueHttpError), #[error("WebSocket error: {0}")] WebSocket(#[from] VenueWsError), #[error("Build error: {0}")] Build(#[from] VenueBuildError), } ``` This enables unified error handling at the adapter boundary while preserving component-specific error details for debugging. ### Retry classification (`common/retry.rs`) When an adapter needs sophisticated retry logic, define a retry classification module in `common/retry.rs` that distinguishes between retryable, non-retryable, and fatal errors: ```rust #[derive(Debug, thiserror::Error)] pub enum VenueError { #[error("Retryable error: {source}")] Retryable { #[source] source: VenueRetryableError, retry_after: Option, }, #[error("Non-retryable error: {source}")] NonRetryable { #[source] source: VenueNonRetryableError, }, #[error("Fatal error: {source}")] Fatal { #[source] source: VenueFatalError, }, } ``` Include helper methods like `from_http_status()`, `from_rate_limit_headers()`, `is_retryable()`, `is_fatal()`, and `retry_after()` to enable consistent error classification across the adapter. See BitMEX and Bybit adapters for reference implementations. ### Python exports (`python/mod.rs`) Mirror the Rust surface area through PyO3 modules by re-exporting clients, enums, and helper functions. When new functionality lands in Rust, add it to `python/mod.rs` so the Python layer stays in sync (the OKX adapter is a good reference). ### Python bindings (`python/`) Expose Rust functionality to Python through PyO3. Mark venue-specific structs that need Python access with `#[pyclass]` and implement `#[pymethods]` blocks with `#[getter]` attributes for field access. For async methods in the HTTP client, use `pyo3_async_runtimes::tokio::future_into_py` to convert Rust futures into Python awaitables. When returning lists of custom types, map each item with `Py::new(py, item)` before constructing the Python list. Register all exported classes and enums in `python/mod.rs` using `m.add_class::()` so they're available to Python code. Follow the pattern established in other adapters: prefixing Python-facing methods with `py_*` in Rust while using `#[pyo3(name = "method_name")]` to expose them without the prefix. When delivering instruments from WebSocket to Python, use `instrument_any_to_pyobject()` which returns PyO3 types for caching. For the reverse direction (Python->Rust), use `pyobject_to_instrument_any()` in `cache_instrument()` methods. Never call `.into_py_any()` directly on `InstrumentAny` as it doesn't implement the required trait. ### Type qualification Adapter-specific types (enums, structs) and Nautilus domain types should not be fully qualified. Import them at the module level and use short names (e.g., `OKXContractType` instead of `crate::common::enums::OKXContractType`, `InstrumentId` instead of `nautilus_model::identifiers::InstrumentId`). This keeps code concise and readable. Only fully qualify types from `anyhow` and `tokio` to avoid ambiguity with similarly-named types from other crates. ### String interning Use `ustr::Ustr` for any non-unique strings the platform stores repeatedly (venues, symbols, instrument IDs) to minimise allocations and comparisons. ### Instrument cache standardization All clients that cache instruments must implement three methods with standardized names: `cache_instruments()` (plural, bulk replace), `cache_instrument()` (singular, upsert), and `get_instrument()` (retrieve by symbol). WebSocket clients store instruments in `Arc>` on the outer client for thread-safe access across clones. ### Testing helpers (`common/testing.rs`) Store shared fixtures and payload loaders in `src/common/testing.rs` for use across HTTP and WebSocket unit tests. This keeps `#[cfg(test)]` helpers out of production modules and encourages reuse. ### Instrument status diffing (`common/status.rs`) When a data client polls instrument status via REST, place the diff logic in `common/status.rs` rather than inlining it in the data client. The standard function signature is: ```rust pub fn diff_and_emit_statuses( new_statuses: &AHashMap, cached_statuses: &mut AHashMap, subscriptions: Option<&DashSet>, sender: &tokio::sync::mpsc::UnboundedSender, ts_event: UnixNanos, ts_init: UnixNanos, ) ``` The function compares each entry in `new_statuses` against `cached_statuses`, emitting an `InstrumentStatus` event for any instrument whose `MarketStatusAction` changed. Instruments present in the cache but absent from the new snapshot are treated as removed and emit `NotAvailableForTrading`. The cache always reflects the full API state. Pass `subscriptions` as `Some(&set)` to restrict emissions to subscribed instruments, or `None` to emit all changes unconditionally. The data client stores the cache in an `Arc>>` and calls this function on each poll cycle. ### Factory module (`factories.rs`) Complex adapters may define a `factories.rs` module for converting venue data to Nautilus types. This centralizes transformation logic that would otherwise be scattered across HTTP and WebSocket parsers: ```rust // factories.rs pub fn create_instrument( venue_instrument: &VenueInstrument, ts_init: UnixNanos, ) -> anyhow::Result { match venue_instrument.instrument_type { InstrumentType::Perpetual => parse_perpetual(venue_instrument, ts_init), InstrumentType::Future => parse_future(venue_instrument, ts_init), InstrumentType::Option => parse_option(venue_instrument, ts_init), } } ``` Use this pattern when the same venue data structures are parsed in multiple places (HTTP responses, WebSocket updates, historical data). ### Connection lifecycle (`connect`) Both data and execution clients follow a strict initialization order during `connect()` to prevent race conditions with reconciliation and strategy startup. The platform waits for all clients to signal connected before running reconciliation or starting strategies, so all initialization must complete within `connect()`. #### Data event emission Data clients emit events to the platform through an unbounded channel obtained at construction: ```rust let data_sender = get_data_event_sender(); ``` The `DataEvent` enum carries all data types the client produces: | Variant | Usage | |-------------------------------|-------------------------------------------------------| | `DataEvent::Instrument` | Instrument definitions during bootstrap and updates. | | `DataEvent::InstrumentStatus` | Market status changes from polling or WS streams. | | `DataEvent::Data` | Market data (trades, quotes, book deltas, bars). | | `DataEvent::Response` | Responses to historical data requests. | | `DataEvent::FundingRate` | Funding rate updates for derivatives. | Send events with `self.data_sender.send(DataEvent::Instrument(instrument))`. Log warnings on send failure but do not propagate the error since a closed receiver means the system is shutting down. Clone the sender for spawned tasks that emit data from async work. #### Data client 1. **Fetch instruments via REST** - call `bootstrap_instruments()` or equivalent. 2. **Cache locally** - populate the client's internal instrument map and HTTP client cache. 3. **Emit to data engine** - send each instrument as `DataEvent::Instrument` via `data_sender`. These events are queued during startup and processed before reconciliation runs. 4. **Cache to WebSocket** - call `ws.cache_instruments()` so the handler can parse messages. 5. **Connect WebSocket** - establish the streaming connection. ```rust async fn connect(&mut self) -> anyhow::Result<()> { let instruments = self.bootstrap_instruments().await?; ws.cache_instruments(instruments); ws.connect().await?; ws.wait_until_active(10.0).await?; // ... } ``` #### Execution client 1. **Initialize instruments** - call `ensure_instruments_initialized_async()` which checks `self.core.instruments_initialized()` and returns early if instruments are already cached. Otherwise it fetches instruments via REST and caches them to the HTTP client, WebSocket client, and any broadcaster clients. 2. **Connect WebSocket** - establish the private streaming connection. 3. **Subscribe to channels** - orders, executions, positions, wallet/margin. 4. **Start WebSocket stream handler** - begin processing incoming messages. 5. **Fetch account state** - call `refresh_account_state()` which requests balances and margins via REST, builds an `AccountState`, and emits it through the `ExecutionEventEmitter`. 6. **Await account registered** - call `await_account_registered(timeout_secs)` which polls `self.core.cache().account(&account_id)` at 10ms intervals until the account appears or the timeout expires. This step blocks connect so the portfolio can process orders during reconciliation. 7. **Signal connected** - call `self.core.set_connected()`. ```rust async fn connect(&mut self) -> anyhow::Result<()> { self.ensure_instruments_initialized_async().await?; self.ws_client.connect().await?; self.ws_client.wait_until_active(10.0).await?; // ... subscribe channels, start stream ... self.refresh_account_state().await?; self.await_account_registered(30.0).await?; self.core.set_connected(); Ok(()) } ``` #### Account state emission The `ExecutionEventEmitter` provides two methods for emitting account state: - `emit_account_state(balances, margins, reported, ts_event)` builds an `AccountState` from raw parameters using the internal `OrderEventFactory`, then dispatches it. Use this when the adapter has individual balance and margin values to combine. - `send_account_state(state)` dispatches a pre-built `AccountState`. Use this when the adapter already has a fully constructed state from parsing an HTTP or WebSocket payload. ## HTTP client patterns Adapters use a two-layer HTTP client architecture: a raw client for low-level API operations and a domain client for high-level logic. The split also enables efficient cloning for Python bindings. ### Client structure The architecture consists of two complementary clients: 1. **Raw client** (`MyRawHttpClient`) - Low-level API methods matching venue endpoints. 2. **Domain client** (`MyHttpClient`) - High-level methods using Nautilus domain types. ```rust use std::sync::Arc; use nautilus_network::http::HttpClient; // Raw HTTP client - low-level API methods matching venue endpoints pub struct MyRawHttpClient { base_url: String, client: HttpClient, // Use nautilus_network::http::HttpClient, not reqwest directly credential: Option, retry_manager: RetryManager, cancellation_token: CancellationToken, } // Domain HTTP client - wraps raw client with Arc, provides high-level API pub struct MyHttpClient { pub(crate) inner: Arc, // Additional domain-specific state (e.g., instrument cache) instruments: DashMap, } ``` **Key points**: - **Raw client** (`MyRawHttpClient`) contains low-level HTTP methods named to match venue endpoints (e.g., `get_instruments`, `get_balance`, `place_order`). These methods take venue-specific query objects and return venue-specific response types. - **Domain client** (`MyHttpClient`) wraps the raw client in an `Arc` for efficient cloning (required for Python bindings). It provides high-level methods that accept Nautilus domain types (e.g., `InstrumentId`, `ClientOrderId`) and return domain objects. It may also cache instruments or other venue metadata. - Use `nautilus_network::http::HttpClient` instead of `reqwest::Client` directly for rate limiting, retry logic, and consistent error handling. - Both clients are exposed to Python, but the domain client is the primary interface. ### Parser functions Parser functions convert venue-specific data structures into Nautilus domain objects. Place them in `common/parse.rs` for cross-cutting conversions (instruments, trades, bars) or `http/parse.rs` for REST-specific transformations. Each parser takes venue data plus context (account IDs, timestamps, instrument references) and returns a Nautilus domain type wrapped in `Result`. **Standard patterns:** - Handle string-to-numeric conversions with proper error context using `.parse::()` and `anyhow::Context`. - Check for empty strings before parsing optional fields - venues often return `""` instead of omitting fields. - Map venue enums to Nautilus enums explicitly with `match` statements rather than implementing automatic conversions that could hide mapping errors. - Accept instrument references when precision or other metadata is required for constructing Nautilus types (quantities, prices). - Use descriptive function names: `parse_position_status_report`, `parse_order_status_report`, `parse_trade_tick`. Place parsing helpers (`parse_price_with_precision`, `parse_timestamp`) in the same module as private functions when they're reused across multiple parsers. ### Timestamp conventions Nautilus uses `UnixNanos` (nanoseconds since epoch). Most venues deliver `ms`. Convert at the parser boundary using `nautilus_core::datetime::millis_to_nanos`; document the wire unit on the struct field. `ts_event` is the converted venue timestamp; `ts_init` is `clock.get_time_ns()`. For records with no venue timestamp (instruments), use `clock.get_time_ns()` for both. ### Method naming and organization The raw client mirrors venue endpoints with venue-specific parameter and response types. The domain client wraps it and exposes high-level methods that accept Nautilus domain types. **Naming conventions:** - **Raw client methods**: Named to match venue endpoints as closely as possible (e.g., `get_instruments`, `get_balance`, `place_order`). These methods are internal to the raw client and take venue-specific types (builders, JSON values). - **Domain client methods**: Named based on operation semantics (e.g., `request_instruments`, `submit_order`, `cancel_order`). These are the methods exposed to Python and take Nautilus domain objects (InstrumentId, ClientOrderId, OrderSide, etc.). **Domain method flow:** Domain methods follow a three-step pattern: build venue-specific parameters from Nautilus types, call the corresponding raw client method, then parse the response. For endpoints returning domain objects (positions, orders, trades), call parser functions from `common/parse`. For endpoints returning raw venue data (fee rates, balances), extract the result directly from the response envelope. Methods prefixed with `request_*` indicate they return domain data, while methods like `submit_*`, `cancel_*`, or `modify_*` perform actions and return acknowledgments. The domain client wraps the raw client in an `Arc` for efficient cloning required by Python bindings. ### Query parameter builders Use the `derive_builder` crate with proper defaults and ergonomic Option handling: ```rust use derive_builder::Builder; #[derive(Clone, Debug, Deserialize, Serialize, Builder)] #[serde(rename_all = "camelCase")] #[builder(setter(into, strip_option), default)] pub struct InstrumentsInfoParams { pub category: ProductType, #[serde(skip_serializing_if = "Option::is_none")] pub symbol: Option, #[serde(skip_serializing_if = "Option::is_none")] pub limit: Option, } impl Default for InstrumentsInfoParams { fn default() -> Self { Self { category: ProductType::Linear, symbol: None, limit: None, } } } ``` **Key attributes:** - `#[builder(setter(into, strip_option), default)]` - enables clean API: `.symbol("BTCUSDT")` instead of `.symbol(Some("BTCUSDT".to_string()))`. - `#[serde(skip_serializing_if = "Option::is_none")]` - omits optional fields from query strings. - Always implement `Default` for builder parameters. ### Request signing and authentication Keep signing logic in a `Credential` struct under `common/credential.rs`: - Store API keys using `Ustr` for efficient comparison, secrets in `Box<[u8]>` with `#[zeroize]`. - Implement `sign()` and `sign_bytes()` methods that compute HMAC-SHA256 signatures. - Pass the credential to the raw HTTP client; the domain client delegates signing through the inner client. For WebSocket authentication, the handler constructs login messages using the same `Credential::sign()` method with a WebSocket-specific timestamp format. ### Credential module structure Each adapter's `common/credential.rs` must provide two things: 1. **`credential_env_vars()` free function**: returns environment variable names as a tuple. 2. **`Credential::resolve()` method**: resolves credentials from config values or environment variables using `resolve_env_var_pair` from `nautilus_core::env`. Config structs are DTOs and must not contain credential resolution logic. All resolution belongs in `credential.rs`. **Standard layout:** ```rust use nautilus_core::env::resolve_env_var_pair; /// Returns the environment variable names for API credentials. pub fn credential_env_vars(is_testnet: bool) -> (&'static str, &'static str) { if is_testnet { ("{VENUE}_TESTNET_API_KEY", "{VENUE}_TESTNET_API_SECRET") } else { ("{VENUE}_API_KEY", "{VENUE}_API_SECRET") } } impl Credential { /// Resolves credentials from provided values or environment variables. pub fn resolve( api_key: Option, api_secret: Option, is_testnet: bool, ) -> Option { let (key_var, secret_var) = credential_env_vars(is_testnet); let (k, s) = resolve_env_var_pair(api_key, api_secret, key_var, secret_var)?; Some(Self::new(k, s)) } } ``` ### Environment variable conventions Adapters load API credentials from environment variables when not provided directly, avoiding hardcoded secrets. **Naming conventions:** | Environment | API Key Variable | API Secret Variable | |--------------|---------------------------|---------------------| | Mainnet/Live | `{VENUE}_API_KEY` | `{VENUE}_API_SECRET` | | Testnet | `{VENUE}_TESTNET_API_KEY` | `{VENUE}_TESTNET_API_SECRET` | | Demo | `{VENUE}_DEMO_API_KEY` | `{VENUE}_DEMO_API_SECRET` | Some venues require additional credentials: - OKX: `OKX_API_PASSPHRASE` **Key principles:** - Environment variable names must be centralized in `credential_env_vars()`, never duplicated as string literals across files. - Environment variable resolution should happen in core Rust code, not Python bindings. - Use `get_or_env_var_opt` for optional credentials (returns `None` if missing). - Use `get_or_env_var` when credentials are required (returns error if missing). - Invalid credentials (e.g. malformed keys) must fail fast with an error, never silently degrade to unauthenticated mode. ### Error handling and retry logic Use the `RetryManager` from `nautilus_network` for consistent retry behavior. ### Rate limiting Configure rate limiting through `HttpClient` using `LazyLock` static variables. **Naming conventions:** - REST quotas: `{VENUE}_REST_QUOTA` (e.g., `OKX_REST_QUOTA`, `BYBIT_REST_QUOTA`) - WebSocket quotas: `{VENUE}_WS_{OPERATION}_QUOTA` (e.g., `OKX_WS_CONNECTION_QUOTA`, `OKX_WS_ORDER_QUOTA`) - Rate limit keys: `{VENUE}_RATE_LIMIT_KEY_{OPERATION}` (e.g., `OKX_RATE_LIMIT_KEY_SUBSCRIPTION`, `OKX_RATE_LIMIT_KEY_ORDER`) **Standard rate limit keys for WebSocket:** | Key | Operations | |---------------------------------|----------------------------------| | `*_RATE_LIMIT_KEY_SUBSCRIPTION` | Subscribe, unsubscribe, login. | | `*_RATE_LIMIT_KEY_ORDER` | Place orders (regular and algo). | | `*_RATE_LIMIT_KEY_CANCEL` | Cancel orders, mass cancel. | | `*_RATE_LIMIT_KEY_AMEND` | Amend/modify orders. | **Example:** ```rust pub static OKX_REST_QUOTA: LazyLock = LazyLock::new(|| Quota::per_second(NonZeroU32::new(250).unwrap())); pub static OKX_WS_SUBSCRIPTION_QUOTA: LazyLock = LazyLock::new(|| Quota::per_hour(NonZeroU32::new(480).unwrap())); pub const OKX_RATE_LIMIT_KEY_ORDER: &str = "order"; ``` Pass rate limit keys when sending WebSocket messages to enforce per-operation quotas: ```rust self.send_with_retry(payload, Some(vec![OKX_RATE_LIMIT_KEY_ORDER.to_string()])).await ``` **Policy:** Adapters should converge on the following rate-limiting principles. - Map each quota to the scope the venue meters it against (per IP, account, API key, connection, URL, transport, or operation class). Draw every client that shares a scope (data, execution, pollers) from one limiter keyed by that scope; separate limiters for a shared cap silently double the effective rate. Add distinct keys only for sub-caps the venue meters independently. - Bucket data and execution traffic apart only when the venue meters them apart. The split still matters for recovery: a data-path trip (subscribe, unsubscribe, control frames) rejects a subscription, surfaces as missing market data, and recovers through adapter retry or reconnect, usually without the strategy knowing; an execution-path trip is strategy-visible and governed by the outcome policy below. - Pace to the venue's actual metering, not just its headline number. Match the window shape, burst, and any endpoint weights: a token bucket at the documented rate can still overrun a strict rolling window after an idle burst. Wire latency does not create rate headroom, since a constant delay shifts arrival times without changing the rate and jitter bunches messages as readily as it spreads them. Add headroom only when window semantics or shared external traffic require it, never as a round-number buffer. - Bound inflight with a closed-loop gate, not the limiter. When a venue caps concurrent unacknowledged messages separately from the send rate, gate dispatch on a count that releases a slot on every terminal outcome: acknowledgement, rejection, send failure, and reconnect. A send-rate limiter cannot do this, because inflight tracks send rate times acknowledgement latency, which it never observes. - Treat an execution-path rate-limit response as an unknown outcome, not a rejection. Per the [order command outcome policy](#order-command-outcome-policy), a rate-limited command may still have reached the venue, so retry only when the command is idempotent or the venue proves it was not processed; otherwise leave the order in flight and reconcile. ## WebSocket client patterns WebSocket clients handle real-time streaming data. They manage connection state, authentication, subscriptions, and reconnection logic. ### Client structure WebSocket adapters use a **two-layer architecture** to separate Python-accessible state from high-performance async I/O: #### Connection state tracking Track connection state using `Arc>` to provide lock-free, race-free visibility across all clones: ```rust use arc_swap::ArcSwap; pub struct MyWebSocketClient { connection_mode: Arc>, // Shared connection mode (lock-free) signal: Arc, // Cancellation signal for graceful shutdown // ... } ``` **Pattern breakdown:** - **Outer `Arc`**: Shared across all clones (Python bindings clone clients before async operations). - **`ArcSwap`**: Enables atomic pointer replacement via `.store()` without replacing the outer Arc. - **Inner `Arc`**: The actual connection state from `WebSocketClient::connection_mode_atomic()`. Initialize with a placeholder atomic (`ConnectionMode::Closed`), then in `connect()` call `.store(client.connection_mode_atomic())` to atomically swap to the real client's state. All clones see updates instantly through lock-free `.load()` calls in `is_active()`. The underlying `WebSocketClient` sends a `RECONNECTED` sentinel message when reconnection completes, triggering resubscription logic in the handler. **Outer client** (`{Venue}WebSocketClient`): - Orchestrates connection lifecycle, authentication, subscriptions. - Maintains state for Python access using `Arc>`. - Tracks subscription state for reconnection logic. - Stores instruments cache for replay on reconnect. - Sends commands to handler via `cmd_tx` channel. - Receives venue events via `out_rx` channel. **Inner handler** (`{Venue}WsFeedHandler`): - Runs in dedicated Tokio task as stateless I/O boundary. - Owns `WebSocketClient` exclusively (no `RwLock` needed). - Processes commands from `cmd_rx` -> serializes to JSON -> sends via WebSocket. - Receives raw WebSocket messages -> deserializes into `{Venue}WsFrame` -> converts to `{Venue}WsMessage` -> emits via `out_tx`. - Owns pending request state using `AHashMap` (single-threaded, no locking). - Uses `VecDeque<{Venue}WsMessage>` to buffer multi-message yields from a single frame parse. Some venues expose separate WebSocket endpoints for market data and order management (different URLs, authentication flows, or message protocols). In this case, split into two client+handler pairs under `websocket/data/` and `websocket/orders/` subdirectories, each following the same two-layer pattern. Name them `{Venue}MdWebSocketClient` / `{Venue}MdWsFeedHandler` and `{Venue}OrdersWebSocketClient` / `{Venue}OrdersWsFeedHandler`. **Communication pattern:** ```mermaid flowchart LR subgraph client["Client (orchestrator)"] cmd_tx["cmd_tx
├ Subscribe { args }
├ PlaceOrder { params }
└ MassCancel { id }"] out_rx["out_rx
← {Venue}WsMessage
← Authenticated
← ChannelData"] end subgraph handler["Handler (I/O boundary)"] cmd_rx[cmd_rx] out_tx[out_tx] ws[WebSocket] end cmd_tx --> cmd_rx cmd_rx -->|"serialize"| ws ws -->|"parse -> transform"| out_tx out_tx --> out_rx ``` **Key principles:** - **No shared locks on hot path**: Handler owns `WebSocketClient`, client sends commands via lock-free mpsc channel. - **Command pattern for all sends**: Subscriptions, orders, cancellations all route through `HandlerCommand` enum. - **Event pattern for state**: Handler emits `{Venue}WsMessage` events (including `Authenticated`), client maintains state from events. - **Pending state ownership**: Handler owns `AHashMap` for matching responses (no `Arc` between layers). - **Message buffering**: Handler uses `VecDeque<{Venue}WsMessage>` for frames that produce multiple output messages. The `next()` method drains the queue before polling channels. - **Python constraint**: Client uses `Arc` only for state Python might query; handler uses `AHashMap` for internal matching. #### Handler initialization handshake (`SetClient`) The handler does not own its `WebSocketClient` at construction time. `WebSocketClient` is not `Clone` and is awkward to move into an already-spawned task constructor; several adapters use a deferred handoff through the command channel. Lighter uses this stricter ordering: 1. The outer client calls `WebSocketClient::connect(...)` and obtains the live client. 2. The outer client creates the local `cmd_tx`/`cmd_rx` and `out_tx`/`out_rx` channels. The connection-mode atomic is captured into a local before `client` is moved. 3. The outer client sends `HandlerCommand::SetClient(client)` on the local `cmd_tx` first, followed by any cache-replay commands (e.g. `InitializeInstruments`). 4. Only after `SetClient` is queued does the outer client publish the new command channel (swap `self.cmd_tx`) and store the captured connection mode (transition `is_active()` to true). Doing this in the opposite order races: a clone observing `is_active()` could enqueue a Subscribe on the published `cmd_tx` before SetClient lands, and the handler would drop it because `inner == None`. 5. The outer client spawns the handler task with `cmd_rx`. The handler constructor takes `inner: Option` initialized to `None`; the first command it processes is `SetClient`, which moves the client into `self.inner`. 6. Any subscribe/order commands queued by clones after step 4 land behind `SetClient` and `InitializeInstruments` in `cmd_rx`, so they reach a fully wired handler in queue order. ```rust pub enum HandlerCommand { SetClient(WebSocketClient), Disconnect, Subscribe { /* ... */ }, // ... other commands } pub(super) struct {Venue}WsFeedHandler { inner: Option, // None until SetClient cmd_rx: tokio::sync::mpsc::UnboundedReceiver, // ... } // In the cmd_rx match arm: HandlerCommand::SetClient(client) => { self.inner = Some(client); } ``` BitMEX, OKX, Bybit, Hyperliquid, and Lighter all use `SetClient` to hand the connected `WebSocketClient` to the handler. Lighter queues `SetClient` before it publishes the new `cmd_tx` or marks the connection active; older adapters may publish the command channel first. ### Authentication Authentication state is managed through events: - Handler processes `Login` response -> **returns** `{Venue}WsMessage::Authenticated` immediately. - Client receives event -> updates local auth state -> proceeds with subscriptions. - `AuthTracker` (from `nautilus_network::websocket::auth`) tracks auth state across threads. The `AuthTracker` struct from `nautilus_network` provides thread-safe authentication state: ```rust pub struct AuthTracker { tx: Arc>>, authenticated: Arc, } ``` `AuthTracker` is internally `Arc`-based, so cloning shares state. Both client and handler store `auth_tracker: AuthTracker` and receive a `.clone()` of the same instance. The tracker exposes a four-method lifecycle: `begin()` starts an attempt and returns a one-shot receiver, `succeed()` sets the authenticated flag and notifies the receiver, `fail(message)` clears the flag with an error, and `invalidate()` clears the flag on disconnect. Downstream consumers query `is_authenticated()` for lock-free reads via the internal `AtomicBool`. **Note**: The `Authenticated` message is consumed in the client's spawn loop for reconnection flow coordination and is not forwarded to downstream consumers (data/execution clients). Downstream consumers can query authentication state via `AuthTracker` if needed. The execution client's `Authenticated` handler only logs at debug level with no important logic depending on this event. #### Auth-token rotation (single-endpoint mixed-trust adapters) Some venues run public market data and authenticated account channels through a single WebSocket endpoint, gated by a short-lived bearer token attached to each subscribe request. Lighter is the canonical example: the token is a Schnorr signature over `(deadline, account_index, api_key_index)` with a venue-imposed hard cap of 8 hours (`LIGHTER_AUTH_TOKEN_MAX_TTL`). `build_auth_token_for(...)` currently emits a 7-hour token, and the execution client refreshes account channel subscriptions every 6 hours. This contrasts with the per-message-signature pattern (Hyperliquid) and the session-login pattern (BitMEX, Bybit); neither needs in-session token rotation. **Token lifecycle** The outer client owns the schedule; the handler owns the wire send. The flow is: 1. **Mint before account subscribes**: The execution client calls `build_auth_token_for(...)` after the WebSocket reaches active state, then uses that token for the initial account-channel subscriptions. 2. **Distribute on subscribe**: `subscribe_account(...)` attaches the token to `HandlerCommand::Subscribe`. The WebSocket client stores the exact `(channel, auth)` pair in `subscription_args` for reconnect replay. 3. **Schedule refresh**: After the execution WebSocket consumer starts, the execution client spawns a refresh task on `get_runtime()`. The task sleeps for `AUTH_TOKEN_REFRESH_INTERVAL` (6 hours), mints a new token, and re-issues `subscribe_account(...)` for every account channel. 4. **Stop on disconnect**: The refresh task observes the execution client's cancellation token and exits when the client stops or disconnects. **Where the schedule lives** Place the rotation timer in the outer client, not the handler. The execution client owns the credential and decides when to mint; the handler remains an I/O boundary that sends the supplied token and signs nothing on its own. ```rust fn spawn_auth_token_refresh(&self, credential: Credential) { let ws_client = self.ws_client.clone(); let cancellation_token = self.cancellation_token.clone(); let account_index = credential.account_index(); let channels = [ LighterWsChannel::AccountAllOrders(account_index), LighterWsChannel::AccountAllTrades(account_index), LighterWsChannel::AccountAllPositions(account_index), LighterWsChannel::AccountAllAssets(account_index), ]; get_runtime().spawn(async move { loop { tokio::select! { () = cancellation_token.cancelled() => break, () = tokio::time::sleep(AUTH_TOKEN_REFRESH_INTERVAL) => { if let Ok(token) = build_auth_token_for(&credential) { for channel in channels.clone() { let _ = ws_client .subscribe_account(channel, token.clone()) .await; } } } } } }); } ``` **Reconnect interaction** On `Reconnected`, the Lighter WebSocket client replays the tracked `subscription_args` through `HandlerCommand::Subscribe`. It does not mint a fresh token on reconnect; fresh account-channel tokens come from the scheduled execution-client refresh task. **Failure handling** Subscription send failures call `mark_failure(topic)` so reconnect replay keeps the topic pending. Lighter does not currently implement an immediate auth-token refresh path on a mid-session venue rejection. ### Subscription management #### Shared `SubscriptionState` pattern The `SubscriptionState` struct from `nautilus_network::websocket` is shared between client and handler using `Arc>` internally for thread-safe access: - **`SubscriptionState` is shared via `Arc`**: Both client and handler receive `.clone()` of the same instance (shallow clone of Arc pointers). - **Responsibility split**: Client tracks user intent (`mark_subscribe`, `mark_unsubscribe`), handler tracks server confirmations (`confirm_subscribe`, `confirm_unsubscribe`, `mark_failure`). - **Why both need it**: Single source of truth with lock-free concurrent access, no synchronization overhead. #### Subscription lifecycle A **subscription** represents any topic in one of two states: | State | Description | |---------------|-------------| | **Pending** | Subscription request sent to venue, awaiting acknowledgment. | | **Confirmed** | Venue acknowledged subscription and is actively streaming data. | State transitions follow this lifecycle: | Trigger | Method Called | From State | To State | Notes | |-------------------|----------------------|------------|-----------|-------| | User subscribes | `mark_subscribe()` | | Pending | Topic added to pending set. | | Venue confirms | `confirm()` | Pending | Confirmed | Moved from pending to confirmed. | | Venue rejects | `mark_failure()` | Pending | Pending | Stays pending for retry on reconnect. | | User unsubscribes | `mark_unsubscribe()` | Confirmed | Pending | Temporarily pending until ack. | | Unsubscribe ack | `clear_pending()` | Pending | Removed | Topic fully removed. | **Key principles**: - `subscription_count()` reports **only confirmed subscriptions**, not pending ones. - Failed subscriptions remain pending and are automatically retried on reconnect. - Both confirmed and pending subscriptions are restored after reconnection. - Unsubscribe operations must check the `op` field in acknowledgments to avoid re-confirming topics. #### Confirmation timing The handler is responsible for transitioning topics from Pending to Confirmed. Two patterns are established, chosen per venue based on what the wire format provides: **Explicit ack (preferred when the venue supports it)**: used by BitMEX, OKX, Bybit, and Lighter. The venue sends a dedicated subscribe/unsubscribe acknowledgment frame (typically `{ "event": "subscribe", "arg": ..., "code": ... }` or similar). The handler matches that frame, derives the topic from its `arg`/`req_id` field, and dispatches: - success -> `confirm_subscribe(topic)` / `confirm_unsubscribe(topic)`. - failure -> `mark_failure(topic)` (stays pending; retried on reconnect). - unsubscribe failures should be treated as still-subscribed and reconfirmed. Keep the ack handling in one branch or function invoked from the wire-frame match arm so the lifecycle is auditable in one place. **Implicit-on-first-frame (fallback)**: used by Lighter as a backstop and by any venue that either omits subscribe acks or makes them unreliable. The handler calls `confirm_subscribe(topic)` when the first inbound data frame for that topic arrives. The topic is recovered from the frame's `channel` field. This doubles as a backstop when an explicit ack is dropped or arrives after the first data frame. ```rust // Inside the data-frame match arm: let topic = frame_topic(&frame); self.subscriptions.confirm_subscribe(&topic); // ... then parse and emit ``` The two patterns can coexist: a venue that sometimes sends acks and sometimes doesn't can use the explicit handler for ack frames and the implicit backstop on data frames; `confirm_subscribe()` is idempotent. Failure paths must call `mark_failure(topic)` (not silently drop the subscription). `mark_failure()` keeps the topic pending so the reconnect replay restores it. #### Topic format patterns Adapters use venue-specific delimiters to structure subscription topics: | Adapter | Delimiter | Example | Pattern | |--------------|-----------|------------------------|------------------------------| | **BitMEX** | `:` | `trade:XBTUSD` | `{channel}:{symbol}` | | **OKX** | `:` | `trades:BTC-USDT-SWAP` | `{channel}:{symbol}` | | **Bybit** | `.` | `orderbook.50.BTCUSDT` | `{channel}.{depth}.{symbol}` | | **Lighter** | `:` / `/` | `order_book:0` | `{channel}:{market_index}` | Parse topics using `split_once()` with the appropriate delimiter to extract channel and symbol components. ##### Asymmetric inbound vs outbound delimiters Some venues use different separators for outbound subscribe payloads versus inbound frame `channel` fields. Lighter is the canonical example: outbound subscribe uses `order_book/0` (slash), inbound frames carry `"channel": "order_book:0"` (colon). Established workaround: - Pick the inbound separator for `SubscriptionState::new(delimiter)` so the handler can confirm subscriptions directly against the `channel` field of every received frame. - Expose two methods on the venue's channel/subscription enum: - `subscription_channel()` returns the outbound-formatted payload (used when serializing the subscribe/unsubscribe request). - `topic_key()` returns the canonical topic key (matching the inbound form) used to key `SubscriptionState` and the reconnection replay map. This keeps a single canonical topic identity throughout the handler while honoring the venue's wire format on the way out. ### Reconnection logic On reconnection, restore authentication and subscriptions: 1. **Track subscriptions**: Preserve original subscription arguments in collections (e.g., `Arc`) to avoid parsing topics back to arguments. 2. **Reconnection flow**: - Receive `{Venue}WsMessage::Reconnected` from handler. - If authenticated: Re-authenticate and wait for confirmation. - Restore all tracked subscriptions via handler commands. - Forward `{Venue}WsMessage::Reconnected` to downstream consumers via `out_tx` when those consumers need to reset local state. BitMEX, OKX, Bybit, and Lighter forward this event after restore is initiated; Hyperliquid currently consumes it in the WebSocket client after resubscribing. For adapters with an `Authenticated` event, the client spawn loop can consume it for reconnection coordination instead of forwarding it. Downstream consumers can query `AuthTracker` when they need authentication state. **Preserving subscription arguments:** Store original subscription arguments in a separate collection to enable deterministic reconnection replay without parsing topics back into arguments: ```rust pub struct MyWebSocketClient { subscription_state: Arc, subscription_args: Arc>, // topic -> original args // ... } impl MyWebSocketClient { async fn subscribe(&self, args: SubscriptionArgs) -> Result<(), Error> { let topic = args.to_topic(); self.subscription_state.mark_subscribe(&topic); self.subscription_args.insert(topic.clone(), args.clone()); self.send_cmd(HandlerCommand::Subscribe(args)).await } async fn unsubscribe(&self, topic: &str) -> Result<(), Error> { self.subscription_state.mark_unsubscribe(topic); self.subscription_args.remove(topic); self.send_cmd(HandlerCommand::Unsubscribe(topic.to_string())).await } async fn restore_subscriptions(&self) { for entry in self.subscription_args.iter() { let _ = self.send_cmd(HandlerCommand::Subscribe(entry.value().clone())).await; } } } ``` This avoids complex topic parsing and ensures subscriptions are replayed exactly as originally requested. ### Ping/Pong handling Support both WebSocket control frame pings and application-level text pings: - **Control frame pings**: Handled automatically by `WebSocketClient` via the `PingHandler` callback. - **Text pings**: Some venues (e.g., OKX) use `"ping"`/`"pong"` text messages. Configure `heartbeat_msg: Some(TEXT_PING.to_string())` in `WebSocketConfig` and respond to incoming `TEXT_PING` with `TEXT_PONG` in the handler. The handler should check for ping messages early in the message processing loop and respond immediately to maintain connection health. ### Disconnection lifecycle (`close`) The `close()` method follows a three-step shutdown sequence: signal, command, await. ```rust impl MyWebSocketClient { pub async fn close(&mut self) -> Result<(), MyWsError> { tracing::debug!("Starting close process"); // 1. Send disconnect command so handler can clean up gracefully if let Err(e) = self.cmd_tx.read().await.send(HandlerCommand::Disconnect) { tracing::warn!("Failed to send disconnect command to handler: {e}"); } // 2. Set stop signal so handler loop exits after processing disconnect self.signal.store(true, Ordering::Release); // 3. Await task handle with timeout, abort if stuck if let Some(task_handle) = self.task_handle.take() { match Arc::try_unwrap(task_handle) { Ok(handle) => { let abort_handle = handle.abort_handle(); match tokio::time::timeout(Duration::from_secs(2), handle).await { Ok(Ok(())) => tracing::debug!("Handler task completed"), Ok(Err(e)) => tracing::error!("Handler task error: {e:?}"), Err(_) => { tracing::warn!("Timeout waiting for handler task, aborting"); abort_handle.abort(); } } } Err(arc_handle) => { tracing::debug!("Cannot unwrap task handle, aborting"); arc_handle.abort(); } } } Ok(()) } } ``` **Key points:** - Send `Disconnect` before setting the stop signal so the handler processes it before exiting. - Return `Result<(), {Venue}WsError>` so callers can handle failures. - Use `Ordering::Release` on the signal store so the handler sees the write. - Extract `abort_handle` before awaiting so it remains available after timeout. - When `Arc::try_unwrap` fails (other clones exist), abort directly. ### Stream consumption (`stream`) The outer client exposes a `stream()` method that hands ownership of `out_rx` to the caller as an async stream. Data and execution clients call this once to drive their message processing loop: ```rust impl MyWebSocketClient { pub fn stream(&mut self) -> impl Stream + 'static { let rx = self .out_rx .take() .expect("Stream receiver already taken or not connected"); let mut rx = Arc::try_unwrap(rx) .expect("Cannot take ownership - other references exist"); async_stream::stream! { while let Some(msg) = rx.recv().await { yield msg; } } } } ``` The data/execution client consumes the stream in a `tokio::select!` loop with a cancellation token or stop signal, matching on `{Venue}WsMessage` variants and calling parse functions to produce Nautilus domain types. ### Subscription topic helpers (`subscription.rs`) When a venue's subscription topics have complex structure (multiple parameter types, instrument type / family / ID variants, candle width encoding), extract topic building and parsing into `websocket/subscription.rs`. This keeps `client.rs` focused on connection lifecycle and `handler.rs` focused on I/O. For venues with simple `{channel}:{symbol}` topics, inline helpers in the client are sufficient and a separate module is not needed. ### Handler configuration constants Define handler-specific tuning constants for consistent behavior: | Constant | Purpose | Typical value | |----------------------------|--------------------------------------------------|---------------| | `DEFAULT_HEARTBEAT_SECS` | Interval for sending keep‑alive messages. | 15-30 | | `WEBSOCKET_AUTH_WINDOW_MS` | Maximum age for authentication timestamps. | 5000-30000 | | `BATCH_PROCESSING_LIMIT` | Maximum messages processed per event loop cycle. | 100-1000 | Place these in `websocket/handler.rs` or `common/consts.rs` depending on scope. ### Message routing The handler uses two message enums to separate wire deserialization from emitted events. The data and execution client layers convert emitted events into Nautilus domain types. Define two enums: 1. **`{Venue}WsFrame`**: Serde-deserialized wire frames. Contains every JSON shape the venue can send (login responses, subscription acks, channel data, order responses, errors, pings). Typically `pub(super)` since only the handler uses it. 2. **`{Venue}WsMessage`**: Handler output events emitted on `out_tx`. Contains the subset of wire data the client needs plus synthetic control variants (`Reconnected`, `Authenticated`, `SendFailed`) that have no wire representation. This is the `pub` type consumers match on. The handler deserializes raw text into `{Venue}WsFrame`, handles control frames internally (subscription acks, login, pings), and converts relevant frames into `{Venue}WsMessage` events sent via `out_tx`. The client receives from `out_rx` and routes to data/execution callbacks, which convert venue types to Nautilus domain types using parse functions. #### Message type naming convention Types prefixed with the venue name (e.g., `OKX`, `Bitmex`) contain raw exchange-specific types. Types prefixed with `Nautilus` contain normalized domain types ready for the trading system. **Wire frame enum (serde-deserialized, handler-internal):** ```rust pub(super) enum MyWsFrame { Login { event, code, msg, conn_id }, Subscription { event, arg, conn_id, code, msg }, OrderResponse { id, op, code, msg, data }, BookData { arg, action, data: Vec }, Data { arg, data: Value }, Error { code, msg }, Ping, Reconnected, } ``` **Handler output enum (emitted to client):** ```rust pub enum MyWsMessage { BookData { arg, action, data: Vec }, ChannelData { channel, inst_id, data: Value }, Orders(Vec), OrderResponse { id, op, code, msg, data }, SendFailed { request_id, client_order_id, op, error }, Instruments(Vec), Error(MyWebSocketError), Reconnected, Authenticated, } ``` The frame enum includes every wire shape (login acks, subscription acks, pings) for deserialization. The output enum drops shapes the handler consumes internally and adds synthetic variants (`Authenticated`, `SendFailed`) that originate in handler logic, not on the wire. Include `OrderResponse` for venue acknowledgements (place, cancel, amend) and `SendFailed` for WebSocket send failures after retries are exhausted. The execution client dispatch layer may convert `OrderResponse` into Nautilus rejection events when the venue explicitly rejects the command. It must treat `SendFailed` as an unknown outcome and leave the order state open to reconciliation. **Conversion in data/exec client:** The data client's message loop matches on `{Venue}WsMessage` variants and calls parse functions to produce Nautilus domain types (`Data`, `OrderBookDeltas`, etc.). The execution client's dispatch layer handles `OrderResponse`, `SendFailed`, and `Orders` variants. `SendFailed` records that the venue outcome is unknown; it is not a rejection. This keeps the handler focused on I/O and deserialization while the client layers own domain conversion. The execution dispatch converts order and fill messages using a two-tier routing contract: 1. The handler emits venue-specific order types (e.g., `Orders(Vec)`). 2. The client dispatch layer tracks which orders were submitted through this client. 3. **Tracked order**: convert venue types to order events (`OrderAccepted`, `OrderCanceled`, `OrderFilled`, etc.) and synthesize any missing lifecycle events (e.g., `OrderAccepted` before a fast fill). 4. **External/unknown order**: convert to reports (`OrderStatusReport` or `FillReport`) for downstream reconciliation. When the venue outcome is unconfirmed (a command times out, a stream message races the HTTP response, or a partial state arrives mid-modify), leave the order in its pending state and let the live execution engine resolve it from venue state. Do not add adapter code to cover every such race: the engine already owns truth-from-venue resolution through the inflight check and open-order queries, whose reports reconcile the order. Direct events carry the live happy path, reports carry external orders and reconciliation, and the engine reconciles the races. For example, a Betfair `replaceOrders` timeout leaves the order `PendingUpdate`, and the inflight check resolves it from venue state rather than the adapter adding bespoke timeout-recovery logic. #### `WsDispatchState` Execution dispatch state lives in a `WsDispatchState` struct defined in `websocket/dispatch.rs`. It tracks which lifecycle events have already been emitted to prevent duplicates across reconnections and fast-fill races: ```rust #[derive(Debug, Default)] pub struct WsDispatchState { pub order_identities: DashMap, pub emitted_accepted: DashSet, pub triggered_orders: DashSet, pub filled_orders: DashSet, clearing: AtomicBool, } ``` | Field | Purpose | |---------------------|-----------------------------------------------------------------| | `order_identities` | Maps client order ID to identity metadata set at submission. | | `emitted_accepted` | Prevents duplicate `OrderAccepted` events. | | `triggered_orders` | Tracks conditional orders that have triggered. | | `filled_orders` | Prevents duplicate `OrderFilled` events on reconnect replay. | | `clearing` | Guards concurrent eviction when sets reach capacity. | Each `DashSet` is bounded by a `DEDUP_CAPACITY` constant (typically 10,000). When a set reaches capacity, `evict_if_full()` clears it atomically using a compare-exchange on the `clearing` flag to prevent concurrent clears. The `dispatch_ws_message()` free function in the same module routes `{Venue}WsMessage` variants to the appropriate order event builders, using `WsDispatchState` for dedup and `OrderIdentity` for tracked-vs-external classification. #### Cross-source fill deduplication `WsDispatchState` prevents duplicate lifecycle events within a single stream. When an adapter receives fills from multiple sources (WebSocket user data and HTTP reconciliation), a separate trade-ID-level dedup is needed to prevent the same fill from being emitted twice. The `BoundedDedup` pattern addresses this with a fixed-capacity set backed by a `VecDeque` for insertion order and an `AHashSet` for O(1) lookup. When the set reaches capacity, the oldest entry is evicted (FIFO). The `insert()` method returns `true` if the value was already present, signaling a duplicate: ```rust struct BoundedDedup { order: VecDeque, set: AHashSet, capacity: usize, } ``` Use this in the execution client to track trade IDs (typically as `(Ustr, i64)` tuples of symbol and trade ID). A capacity of 10,000 provides sufficient coverage for most venues without unbounded memory growth. #### Cancel-replace modifies and in-flight fills Some venues (e.g. Hyperliquid) implement a modify as a cancel-replace: the venue assigns a new venue order ID and emits `ACCEPTED(new_voi)` with `CANCELED(old_voi)`. The dispatch promotes the new leg to `OrderUpdated` and suppresses the stale cancel. A fill carrying the replacement venue order ID drives the same promotion (falling back to buffering only when the order has no price), so a dropped `ACCEPTED` does not strand the fill. Fills on the old venue order ID count separately, so the replacement is sized at the remaining (`target - filled`), not the total. That subtraction runs when the modify is dispatched, so a fill landing after the request is sent leaves the replacement oversized and the venue can overfill the order. To prevent this, the cancel-replace promotion: - re-reads the cumulative filled quantity; - queues a corrective reduce of the new venue order to the true remaining when oversized; - re-arms the in-flight marker on the new venue order ID to suppress the corrective's cancel leg. The reduce posts off the receive loop. It narrows but does not close the race: if the replacement fills before the reduce lands, the engine's overfill guard is the backstop. ### Error handling #### Order command outcome policy Adapters must emit these rejection events only from definitive command-failure evidence: - `OrderRejected`. - `OrderModifyRejected`. - `OrderCancelRejected`. Positive venue evidence includes structured order responses, per-order batch responses, or order status messages that explicitly report a rejection. Positive local evidence includes prepare failures that prove a cancel or modify command cannot be sent and can be attributed to a single order command. Local validation is not automatically a venue rejection: - Validate submit commands before `OrderSubmitted` and emit `OrderDenied` when validation fails. - If submit validation fails after `OrderSubmitted`, log the failure and leave the order in flight. - If cancel or modify prepare fails before the command is sent, emit `OrderCancelRejected` or `OrderModifyRejected` only when the adapter can attribute the failure to that command. Otherwise log a warning and do not emit a rejection event. Do not emit rejection events for errors that leave the venue outcome unknown. Unknown outcomes include transport errors, WebSocket send failures, request timeouts, disconnects, canceled local tasks, missing acknowledgements, HTTP 5xx responses, rate limits, retry exhaustion, parse failures after a request may have reached the venue, and whole-batch request failures without per-order venue results. When the outcome is unknown, leave the order in its current in-flight state and let WebSocket updates, in-flight checks, open-order polling, startup reconciliation, or explicit query commands resolve the final state. For batch commands, emit rejection events only for per-order venue results that unambiguously reject the command; a whole-request failure must not become one rejection per order. Cancel and modify errors need venue-specific allowlists. A generic venue error such as "not found", "already closed", or "unknown order" can mean the order filled or canceled before the request was processed. Emit `OrderCancelRejected` or `OrderModifyRejected` only when the venue semantics make the command rejection unambiguous. #### Client-side error propagation Channel send failures (client -> handler) should propagate loudly as `Result<(), Error>`: ```rust impl MyWebSocketClient { async fn send_cmd(&self, cmd: HandlerCommand) -> Result<(), Error> { self.cmd_tx.read().await.send(cmd) .map_err(|e| Error::ClientError(format!("Handler not available: {e}"))) } pub async fn submit_order(...) -> Result<(), Error> { let cmd = HandlerCommand::PlaceOrder { ... }; self.send_cmd(cmd).await // Propagates channel failures } } ``` #### Handler-side retry logic WebSocket send failures (handler -> network) should be retried by the handler using `RetryManager`: ```rust pub struct MyWsFeedHandler { inner: Option, retry_manager: RetryManager, // ... } impl MyWsFeedHandler { async fn send_with_retry(&self, payload: String, rate_limit_keys: Option>) -> Result<(), MyWsError> { if let Some(client) = &self.inner { self.retry_manager.execute_with_retry( "websocket_send", || async { client.send_text(payload.clone(), rate_limit_keys.clone()) .await .map_err(|e| MyWsError::ClientError(format!("Send failed: {e}"))) }, should_retry_error, create_timeout_error, ).await } else { Err(MyWsError::ClientError("No active WebSocket client".to_string())) } } async fn handle_place_order(...) -> anyhow::Result<()> { let payload = serde_json::to_string(&request)?; match self.send_with_retry(payload, Some(vec![RATE_LIMIT_KEY])).await { Ok(()) => Ok(()), Err(e) => { // Emit SendFailed so dispatch can record an unknown outcome. let _ = self.out_tx.send(MyWsMessage::SendFailed { request_id: request_id.clone(), client_order_id: Some(client_order_id), op: Some(MyWsOperation::Order), error: e.to_string(), }); Err(anyhow::anyhow!("Failed to send order: {e}")) } } } } fn should_retry_error(error: &MyWsError) -> bool { match error { MyWsError::NetworkError(_) | MyWsError::Timeout(_) => true, MyWsError::AuthenticationError(_) | MyWsError::ParseError(_) => false, } } ``` **Key principles:** - Client propagates channel failures immediately (handler unavailable). - Handler retries transient WebSocket failures (network issues, timeouts). - Handler emits `SendFailed` when retries are exhausted; the exec client dispatch records an unknown outcome and waits for reconciliation or a later venue update. - Use `RetryManager` from `nautilus_network::retry` for consistent backoff. ### Naming conventions Adapters follow standardized naming conventions for consistency across all venue integrations. #### Channel naming: `raw` -> `out` WebSocket message channels follow a two-stage transformation pipeline within the handler: | Stage | Type | Description | Example | |-------|------|-------------|---------| | `raw` | Raw WebSocket frames | Bytes/text from the network layer. | `raw_rx: UnboundedReceiver` | | `out` | Venue‑specific messages | Parsed venue message types. | `out_tx: UnboundedSender` | The handler deserializes raw frames into venue-specific types and emits them on `out_tx`. The data and execution client layers then convert venue types into Nautilus domain types. **Example flow:** ```rust // Client creates output channel for venue messages let (out_tx, out_rx) = tokio::sync::mpsc::unbounded_channel(); // Venue messages (MyWsMessage) // Handler receives raw frames, outputs venue messages let handler = MyWsFeedHandler::new( cmd_rx, raw_rx, // Input: Message (raw WebSocket frames) out_tx, // Output: MyWsMessage // ... ); ``` Channel names reflect the data transformation stage, not the destination. Use `raw_*` for raw WebSocket frames (`Message`) and `out_*` for venue-specific message types. ### Backpressure strategy WebSocket channels on latency-sensitive paths are intentionally **unbounded**. The platform prioritizes latency and prefers an explicit crash (OOM) over delaying or dropping data. :::note Do not add bounded channels, buffering limits, or backpressure unless the latency requirement changes. ::: #### Field naming: `inner` and command channels Structs holding references to lower-level components follow these conventions: | Field | Type | Description | |---------------|-----------------------------------------------------|-------------| | `inner` | `Option` | Network‑level WebSocket client (handler only, exclusively owned). | | `cmd_tx` | `Arc>>` | Command channel to handler (client side). | | `cmd_rx` | `UnboundedReceiver` | Command channel from client (handler side). | | `out_tx` | `UnboundedSender<{Venue}WsMessage>` | Output channel to client (handler side). | | `out_rx` | `Option>>` | Output channel from handler (client side). | | `task_handle` | `Option>>` | Handler task handle. | **Example:** ```rust // Client struct pub struct MyWebSocketClient { cmd_tx: Arc>>, out_rx: Option>>, task_handle: Option>>, connection_mode: Arc>, // Lock-free connection state // ... } impl MyWebSocketClient { async fn send_cmd(&self, cmd: HandlerCommand) -> Result<(), Error> { self.cmd_tx.read().await.send(cmd) .map_err(|e| Error::ClientError(format!("Handler not available: {e}"))) } } // Handler struct pub(super) struct MyWsFeedHandler { inner: Option, // Exclusively owned - no RwLock cmd_rx: UnboundedReceiver, raw_rx: UnboundedReceiver, out_tx: UnboundedSender, pending_requests: AHashMap, // Single-threaded - no locks pending_messages: VecDeque, // Multi-message buffer // ... } ``` The handler exclusively owns `WebSocketClient` without locks. The client sends commands via `cmd_tx` (wrapped in `RwLock` to allow reconnection channel replacement) and receives events via `out_rx`. Use a `send_cmd()` helper to standardize command sending. #### Type naming: `{Venue}Ws{TypeSuffix}` All WebSocket-related types follow a standardized naming pattern: `{Venue}Ws{TypeSuffix}` - `{Venue}`: Capitalized venue name (e.g., `OKX`, `Bybit`, `Bitmex`, `Hyperliquid`). - `Ws`: Abbreviated "WebSocket" (not fully spelled out). - `{TypeSuffix}`: Full type descriptor (e.g., `Message`, `Error`, `Request`, `Response`). **Examples:** ```rust // Correct - abbreviated Ws, full type suffix pub enum OKXWsMessage { ... } pub enum BybitWsError { ... } pub struct HyperliquidWsRequest { ... } ``` **Standard type suffixes:** - `Message`: WebSocket message enums. - `Error`: WebSocket error types. - `Request`: Request message types. - `Response`: Response message types. **Tokio channel qualification:** Always fully qualify tokio channel types as `tokio::sync::mpsc::` to avoid ambiguity with similarly-named types from other crates. Never import `mpsc` directly at module level. ```rust // Correct let (tx, rx) = tokio::sync::mpsc::unbounded_channel::(); ``` ### Split WebSocket architectures Some venues expose multiple WebSocket endpoints with distinct protocols or encodings. When a venue requires separate connections for market data and order management, split the `websocket/` module into submodules that mirror the connection boundaries: ``` src/ ├── websocket/ │ ├── mod.rs # Re-exports from submodules │ ├── streams/ # Market data pub/sub connection │ │ ├── client.rs # Streams client │ │ ├── handler.rs # Streams feed handler │ │ ├── messages.rs # Streams message types │ │ └── mod.rs │ └── trading/ # Order management + user data (authenticated WS API) │ ├── client.rs # Trading client │ ├── handler.rs # Trading handler │ ├── messages.rs # Trading message types │ ├── user_data.rs # User data stream venue types (execution reports, etc.) │ ├── parse.rs # Parse functions for user data -> Nautilus types │ ├── error.rs # Trading error types │ └── mod.rs ``` Each submodule follows the same two-layer client/handler pattern described above. The parent `websocket/mod.rs` re-exports the public client types. The `trading/` module handles both order operations (place, cancel, modify) and the user data stream (execution reports, account updates). When the venue's authenticated WebSocket API supports `session.logon` and inline user data subscriptions, both concerns share a single authenticated connection. This avoids a separate `execution/` module and the deprecated REST listenKey lifecycle. For venues where user data events arrive on a separate stream connection (e.g., futures APIs that return a listenKey for a dedicated stream URL), the `streams/` handler dispatches both market data and user data events from the combined connection. #### Naming conventions for split architectures Type names include the submodule qualifier to avoid ambiguity: | Submodule | Command type | Message type | |--------------|--------------------------------------|-------------------------------------| | `streams/` | `{Venue}WsStreamsCommand` | `{Venue}WsMessage` (venue types) | | `trading/` | `{Venue}WsTradingCommand` | `{Venue}WsTradingMessage` | The `{Venue}Ws` prefix follows the standard type naming convention. The qualifier (`Streams`, `Trading`) distinguishes types that would otherwise collide across submodules. #### When to split Split the WebSocket module when the venue has: - Different endpoints with different protocols (e.g., SBE binary for market data, JSON for trading) - A dedicated order management WebSocket API (`ws-api` style) alongside pub/sub streams - User data delivered inline on the authenticated trading connection rather than via a separate listenKey stream Do not split when a single connection handles all message types through channel-based multiplexing (the common pattern for OKX, Bybit, and similar venues). ### Multi-product WebSocket management Some venues use the same WebSocket protocol for all product types but serve them on separate endpoints (e.g., Bybit provides distinct URLs for Linear, Spot, and Inverse). In this case the data client creates one WebSocket client per product type and manages them in a map: ```rust pub struct MyDataClient { ws_clients: AHashMap, } ``` Each client follows the same two-layer client/handler pattern. Subscription routing inspects the instrument's product type to select the correct client. On connect, the data client iterates the map to connect all clients; on disconnect, it closes them all. This differs from the split architecture (`streams/` vs `trading/`) which separates by protocol or purpose. Multi-product management separates by product type while sharing the same protocol. ## Modeling venue payloads Use the following conventions when mirroring upstream schemas in Rust. ### REST models (`http::models` and `http::query`) - Put request and response representations in `src/http/models.rs` and derive `serde::Deserialize` (add `serde::Serialize` when the adapter sends data back). - Mirror upstream payload names with blanket casing attributes such as `#[serde(rename_all = "camelCase")]` or `#[serde(rename_all = "snake_case")]`; only add per-field renames when the upstream key would be an invalid Rust identifier or collide with a keyword (for example `#[serde(rename = "type")] pub order_type: String`). - Keep helper structs for query parameters in `src/http/query.rs`, deriving `serde::Serialize` to remain type-safe and reusing constants from `common::consts` instead of duplicating literals. ### WebSocket messages (`websocket::messages`) - Define streaming payload types in `src/websocket/messages.rs`, giving each venue topic a struct or enum that mirrors the upstream JSON. - Apply the same naming guidance as REST models: rely on blanket casing renames and keep field names aligned with the venue unless syntax forces a change; consider serde helpers such as `#[serde(tag = "op")]` or `#[serde(flatten)]` and document the choice. - Note any intentional deviations from the upstream schema in code comments and module docs so other contributors can follow the mapping quickly. ### Enum hardening for open venue value sets A strict enum with no fallback hard-fails the whole message the first time the venue sends a value it does not model. Choose the behavior by what the field drives: - Reference/descriptive (instrument or product type, market status, ticker type, public trade type): add an `Unknown` fallback so a new value degrades gracefully. Use `#[serde(other)] Unknown`, or a `deserialize_{field}_or_unknown` shim (see `coinbase/src/common/parse.rs`) when the enum also derives `strum::EnumString`. Warn and map `Unknown` to a skip or safe default; never fabricate a value. - Order/fill/position state (order/position status, order/fill type, liquidity, time-in-force, trigger fields): keep strict, no catch-all. An unmodeled value must fail deserialization so the handler logs it loudly rather than let the engine run out of sync with the venue. Model known values explicitly instead of via a catch-all: a documented `"unknown"` sentinel becomes one variant mapped to a safe default with a warning (see `kraken/src/common/enums.rs`); a changelog-named value like `FillOrKill` becomes a real variant. --- ## Task management ### Spawning async tasks (`spawn_task`) Data and execution clients spawn background tasks for WebSocket stream processing, periodic polling, and order submission. Wrap all spawned work with a `spawn_task()` method that provides error logging and handle tracking: ```rust fn spawn_task(&self, description: &'static str, fut: F) where F: Future> + Send + 'static, { let runtime = get_runtime(); let handle = runtime.spawn(async move { if let Err(e) = fut.await { log::warn!("{description} failed: {e:?}"); } }); let mut tasks = self.pending_tasks.lock().expect(MUTEX_POISONED); tasks.retain(|handle| !handle.is_finished()); tasks.push(handle); } ``` Store task handles in `pending_tasks: Mutex>>`. Each call to `spawn_task` prunes finished handles before pushing the new one, preventing unbounded growth. On disconnect, abort all remaining handles. ### Never use `block_on` in trait methods The live runner calls sync `ExecutionClient` and `DataClient` trait methods from within a tokio runtime. Using `runtime.block_on()` in these methods panics with *"Cannot start a runtime from within a runtime"*. Use `spawn_task` instead: ```rust // Wrong: panics at runtime fn query_order(&self, cmd: &QueryOrder) -> anyhow::Result<()> { get_runtime().block_on(async { self.http_client.get_order(&id).await }) } // Correct: clone what you need, spawn, return immediately fn query_order(&self, cmd: &QueryOrder) -> anyhow::Result<()> { let http_client = self.http_client.clone(); let emitter = self.emitter.clone(); self.spawn_task("query_order", async move { let report = http_client.get_order(&id).await?; emitter.send_order_status_report(report); Ok(()) }); Ok(()) } ``` `block_on` is valid in contexts that run outside a tokio runtime: | Context | Why safe | |------------------------------|------------------------------------------------| | PyO3 `#[pymethods]` | Called from Python, no ambient runtime | | Binary `main()` functions | Top‑level entry point, runtime not yet started | | Dedicated background threads | Thread created outside tokio's worker pool | | `block_in_place` wrapper | Moves the thread out of the worker pool first | | Test code with own runtime | `Runtime::new()` creates an isolated runtime | ### Graceful shutdown with `CancellationToken` Use `tokio_util::sync::CancellationToken` to coordinate shutdown across multiple spawned tasks. The client creates a token at construction and passes clones to each spawned task. Tasks select on the token alongside their primary work: ```rust tokio::select! { msg = stream.next() => { /* process */ } _ = cancellation_token.cancelled() => { break; } } ``` On disconnect, the client cancels the token, which signals all tasks to exit their loops. This complements the handler-level `signal: Arc` pattern: `AtomicBool` gates the handler's I/O loop, while `CancellationToken` coordinates shutdown of tasks the client spawned outside the handler (polling loops, reconciliation tasks, stream consumers). Reset the token on reconnect by replacing it with a fresh `CancellationToken::new()` so subsequent tasks are not born cancelled. --- ## Testing Adapters should ship two layers of coverage: the Rust crate that talks to the venue and the Python glue that exposes it to the wider platform. Keep the suites deterministic and colocated with the production code they protect. **Key principle:** The `tests/` directory is reserved for integration tests that require external infrastructure (mock Axum servers, simulated network conditions). Unit tests for parsing, serialization, and business logic belong in `#[cfg(test)]` blocks within source modules. ### Rust testing #### Layout ``` crates/adapters/your_adapter/ ├── src/ │ ├── http/ │ │ ├── client.rs # HTTP client + unit tests │ │ └── parse.rs # REST payload parsers + unit tests │ └── websocket/ │ ├── client.rs # WebSocket client + unit tests │ └── parse.rs # Streaming parsers + unit tests ├── tests/ # Integration tests (mock servers) │ ├── data_client.rs # Data client integration tests │ ├── exec_client.rs # Execution client integration tests │ ├── http.rs # HTTP client integration tests │ └── websocket.rs # WebSocket client integration tests └── test_data/ # Canonical venue payloads used by the suites ├── http_{method}_{endpoint}.json # Full venue responses with retCode/result/time └── ws_{message_type}.json # WebSocket message samples ``` #### Test file organization | File | Purpose | |----------------------|---------------------------------------------------------------------------------------------------------------------------| | `tests/data_client.rs` | Integration tests for the data client. Validates data subscriptions, historical data requests, and market data parsing. | | `tests/exec_client.rs` | Integration tests for the execution client. Validates order submission, modification, cancellation, and execution reports. | | `tests/http.rs` | Low‑level HTTP client tests. Validates request signing, error handling, and response parsing against mock Axum servers. | | `tests/websocket.rs` | WebSocket client tests. Validates connection lifecycle, authentication, subscriptions, and message routing. | **Guidelines:** - Place unit tests next to the module they exercise (`#[cfg(test)]` blocks). Use `src/common/testing.rs` (or an equivalent helper module) for shared fixtures so production files stay tidy. - Keep Axum-based integration suites under `crates/adapters//tests/`, mirroring the public APIs (HTTP client, WebSocket client, data client, execution client). - Data and execution client tests (`data_client.rs`, `exec_client.rs`) should focus on higher-level behavior: subscription workflows, order lifecycle, and domain model transformations. HTTP and WebSocket tests (`http.rs`, `websocket.rs`) focus on transport-level concerns. - Store upstream payload samples (snapshots, REST replies) under `test_data/` and reference them from both unit and integration tests. Name test data files consistently: `http_get_{endpoint_name}.json` for REST responses, `ws_{message_type}.json` for WebSocket messages. Include complete venue response envelopes (status codes, timestamps, result wrappers) rather than just the data payload. Provide multiple realistic examples in each file - for instance, position data should include long, short, and flat positions to exercise all parser branches. - **Test data sourcing**: Test data must be obtained from either official API documentation examples or directly from the live API via network calls. Never fabricate or generate test data manually, as this risks missing edge cases (e.g., negative precision values, scientific notation, unexpected field types) that only appear in real venue responses. #### Unit tests Unit tests belong in `#[cfg(test)]` blocks within source modules, not in the `tests/` directory. **What to test (in source modules):** - Deserialization of venue JSON payloads into Rust structs. - Parsing functions that convert venue types to Nautilus domain models. - Request signing and authentication helpers. - Enum conversions and mapping logic. - Price, quantity, and precision calculations. **What NOT to test:** - Standard library behavior (Vec operations, HashMap lookups, string parsing). - Third-party crate functionality (chrono date arithmetic, serde attributes). - Test helper code itself (fixture loaders, mock builders). Tests should exercise production code paths. If a test only verifies that `Vec::extend()` works or that chrono can parse a date string, it provides no value. ##### WebSocket unit test coverage WebSocket unit tests exercise three areas: message deserialization, parse dispatch, and handler logic. Each area lives in a `#[cfg(test)]` block within the module it tests. **Message types (`messages.rs`):** - Deserialize every message variant from fixture JSON files in `test_data/`. - Round-trip tests: serialize a constructed struct, deserialize the output, and assert equality. Round-trip tests catch field renames, missing `skip_serializing_if` attributes, and precision loss that deserialization-only tests miss. - Cover edge cases in venue payloads: null optional fields, empty arrays, zero quantities. **Parse functions (`parse.rs`):** - Exercise the fast-path byte scanner for each type tag or discriminant value. - Exercise the slow-path fallback (fields not at expected byte positions). - Verify unknown type tags produce a descriptive error, not a panic. **Handler logic (`handler.rs`):** - Verify the handler filters internal messages (heartbeats, subscription acks, pong frames) and does not forward them to consumers. - Verify reconnect signals trigger re-authentication and emit the `Reconnected` variant. - Verify multi-message buffering: when a single raw frame produces multiple output messages, all messages appear in the correct order from `next()`. - Verify pending-order cleanup on error and success responses. #### Integration tests Integration tests belong in the `tests/` directory and exercise the public API against mock infrastructure. **What to test (in tests/ directory):** - HTTP client requests against mock Axum servers. - WebSocket connection lifecycle, authentication, and message routing. - Data client subscription workflows and historical data requests. - Execution client order submission, modification, and cancellation flows. - Error handling and retry behavior with simulated failures. At a minimum, review existing adapter test suites for reference patterns and verify every adapter proves the same core behaviours. ##### HTTP client integration coverage - **Happy paths** – fetch a representative public resource (e.g., instruments or mark price) and verify the response is converted into Nautilus domain models. - **Credential guard** – call a private endpoint without credentials and assert a structured error; repeat with credentials to prove success. - **Rate limiting / retry mapping** – surface venue-specific rate-limit responses and assert the adapter produces the correct `OkxError`/`BitmexHttpError` variant so the retry policy can react. - **Query builders** – exercise builders for paginated/time-bounded endpoints (historical trades, candles) and assert the emitted query string matches the venue specification (`after`, `before`, `limit`, etc.). - **Error translation** – verify non-2xx upstream responses map to adapter error enums with the original code/message attached. ##### WebSocket client integration coverage - **Login handshake** – confirm a successful login flips the internal auth state and test failure cases where the server returns a non-zero code; the client should surface an error and avoid marking itself as authenticated. - **Ping/Pong** – prove both text-based and control-frame pings trigger immediate pong responses. - **Subscription lifecycle** – assert subscription requests/acks are emitted for public and private channels, and that unsubscribe calls remove entries from the cached subscription sets. - **Reconnect behaviour** – simulate a disconnect and verify the client re-authenticates, restores public channels, and skips private channels that were explicitly unsubscribed pre-disconnect. - **Message routing** – feed representative data/ack/error payloads through the socket and assert they arrive on the public stream as the correct `{Venue}WsMessage` variant. - **Quota tagging** – (optional but recommended) validate that order/cancel/amend operations are tagged with the appropriate quota label so rate limiting can be enforced independently of subscription traffic. **CI robustness:** - Never use bare `tokio::time::sleep()` with arbitrary durations. Tests become flaky under CI load and slower than necessary. - Use the `wait_until_async` test helper to poll for conditions with timeout. Tests return immediately when the condition is met and fail deterministically on timeout rather than relying on arbitrary sleep durations. - Prefer event-driven assertions with shared state (for example, collect `subscription_events`, track pending/confirmed topics, wait for `connection_count` transitions). - Use adapter-specific helpers to gate on explicit signals such as "auth confirmed" or "reconnection finished" so suites remain deterministic under load. ##### Data and execution client integration testing Data (`tests/data_client.rs`) and execution (`tests/exec_client.rs`) client integration tests verify the full message flow from WebSocket through parsing to event emission. **Test infrastructure:** | Component | Purpose | |------------------------------|------------------------------------------------------------------------------------| | Mock Axum server | Serves HTTP endpoints (instruments, fee rates, positions) and WebSocket channels. | | `TestServerState` | Tracks connections, subscriptions, and authentication state for assertions. | | Thread‑local event channels | `set_data_event_sender()` / `set_exec_event_sender()` for capturing emitted events.| | `wait_until_async` | Polls conditions with timeout for deterministic async assertions. | **Data client coverage:** | Test scenario | Validates | |------------------------------|----------------------------------------------------------------| | Connect/disconnect | Connection lifecycle, WebSocket establishment, clean shutdown. | | Subscribe trades | Trade tick events emitted to data channel. | | Subscribe quotes | Quote events from ticker (LINEAR) or orderbook (SPOT). | | Subscribe book deltas | OrderBookDeltas events from orderbook snapshots/updates. | | Subscribe mark/index prices | Filtered by subscription state (only emit when subscribed). | | Reset state | Subscription tracking cleared, connection terminated. | | Instruments on connect | Instrument events emitted during connection setup. | **Execution client coverage:** | Test scenario | Validates | |------------------------------|----------------------------------------------------------------| | Connect/disconnect | Auth handshake, private + trade WS connections, subscriptions. | | Demo mode | Only private WS connects (trade WS skipped for HTTP fallback). | | Order submission | Order accepted/rejected events, venue ID correlation. | | Order modification/cancel | Update and cancel acknowledgment events. | | Position/wallet updates | PositionStatusReport and AccountState events. | **Key patterns:** - Each `#[tokio::test]` runs on a fresh thread, ensuring thread-local channel isolation. - Use `wait_until_async` for subscription/connection state instead of arbitrary sleeps. - Drain instrument events before subscription tests to isolate assertions. - Verify subscription state in `TestServerState` before asserting on emitted events. ### Python testing #### Layout ``` tests/integration_tests/adapters/your_adapter/ ├── conftest.py # Shared fixtures (mock clients, test instruments) ├── test_data.py # Data client integration tests ├── test_execution.py # Execution client integration tests ├── test_providers.py # Instrument provider tests ├── test_factories.py # Factory and configuration tests └── __init__.py # Package initialization ``` #### Test file organization | File | Purpose | |---------------------|--------------------------------------------------------------------------------------------------------------------| | `test_data.py` | Tests for `LiveDataClient` and `LiveMarketDataClient`. Validates subscriptions, data parsing, and message handling. | | `test_execution.py` | Tests for `LiveExecutionClient`. Validates order submission, modification, cancellation, and execution reports. | | `test_providers.py` | Tests for `InstrumentProvider`. Validates instrument loading, filtering, and caching behavior. | | `test_factories.py` | Tests for factory functions. Validates client instantiation and configuration wiring. | **Guidelines:** - Exercise the adapter's Python surface (instrument providers, data/execution clients, factories) inside `tests/integration_tests/adapters//`. - Mock the PyO3 boundary (`nautilus_pyo3` shims, stubbed Rust clients) so tests stay fast while verifying that configuration, factory wiring, and error handling match the exported Rust API. - Mirror the Rust integration coverage: when the Rust suite adds a new behaviour (e.g., reconnection replay, error propagation), assert the Python layer performs the same sequence (connect/disconnect, submit/amend/cancel translations, venue ID hand-off, failure handling). BitMEX's Python tests provide the target level of detail. --- ## Documentation All adapter documentation (module-level docs, doc comments, and inline comments) should follow the [Documentation Style Guide](docs.md). ### Rust documentation requirements Every Rust module, struct, and public method must have documentation comments. Use third-person declarative voice (e.g., "Returns the account ID" not "Return the account ID"). - **Modules**: Use `//!` doc comments at the top of each file (after the license header) to describe the module's purpose. - **Structs**: Use `///` doc comments above struct definitions. Keep descriptions concise; one sentence is often sufficient. - **Public methods**: Every `pub fn` and `pub async fn` must have a `///` doc comment describing what the method does. Do not document individual parameters in a separate `# Arguments` section. The type signatures and names should be self-explanatory. Parameters may be mentioned in the description when behavior is complex or non-obvious. **What NOT to document**: - Private methods and fields (unless complex logic warrants it). - Individual parameters/arguments (use descriptive names instead). - Implementation details that are obvious from the code. - Files in the `python/` module (PyO3 bindings). Documentation conventions are TBD (*may* use numpydoc specification). --- ## Python adapter layer Step-by-step guide to building the Python layer of an adapter using the provided template. ### Method ordering convention When implementing adapter classes, group methods by category in this order: 1. **Connection handlers**: `_connect`, `_disconnect` 2. **Subscribe handlers**: `_subscribe`, `_subscribe_*` 3. **Unsubscribe handlers**: `_unsubscribe`, `_unsubscribe_*` 4. **Request handlers**: `_request`, `_request_*` This keeps related functionality together rather than interleaving subscribe/unsubscribe pairs. ### InstrumentProvider The `InstrumentProvider` loads instrument definitions from the venue: all instruments, specific instruments by ID, or a filtered subset. ```python from nautilus_trader.common.providers import InstrumentProvider from nautilus_trader.model import InstrumentId class TemplateInstrumentProvider(InstrumentProvider): """Example `InstrumentProvider` showing the minimal overrides required for a complete integration.""" async def load_all_async(self, filters: dict | None = None) -> None: raise NotImplementedError("implement `load_all_async` in your adapter subclass") async def load_ids_async(self, instrument_ids: list[InstrumentId], filters: dict | None = None) -> None: raise NotImplementedError("implement `load_ids_async` in your adapter subclass") async def load_async(self, instrument_id: InstrumentId, filters: dict | None = None) -> None: raise NotImplementedError("implement `load_async` in your adapter subclass") ``` | Method | Description | |------------------|----------------------------------------------------------------| | `load_all_async` | Loads all instruments asynchronously, optionally with filters. | | `load_ids_async` | Loads specific instruments by their IDs. | | `load_async` | Loads a single instrument by its ID. | ### DataClient The `LiveDataClient` handles data feeds that are not market data: news feeds, custom data streams, or other non-market sources. ```python from nautilus_trader.data.messages import RequestData from nautilus_trader.data.messages import SubscribeData from nautilus_trader.data.messages import UnsubscribeData from nautilus_trader.live.data_client import LiveDataClient from nautilus_trader.model import DataType class TemplateLiveDataClient(LiveDataClient): """Example `LiveDataClient` showing the overridable abstract methods.""" async def _connect(self) -> None: raise NotImplementedError("implement `_connect` in your adapter subclass") async def _disconnect(self) -> None: raise NotImplementedError("implement `_disconnect` in your adapter subclass") async def _subscribe(self, command: SubscribeData) -> None: raise NotImplementedError("implement `_subscribe` in your adapter subclass") async def _unsubscribe(self, command: UnsubscribeData) -> None: raise NotImplementedError("implement `_unsubscribe` in your adapter subclass") async def _request(self, request: RequestData) -> None: raise NotImplementedError("implement `_request` in your adapter subclass") ``` | Method | Description | |----------------|------------------------------------------------| | `_connect` | Establishes a connection to the data provider. | | `_disconnect` | Closes the connection to the data provider. | | `_subscribe` | Subscribes to a specific data type. | | `_unsubscribe` | Unsubscribes from a specific data type. | | `_request` | Requests data from the provider. | ### MarketDataClient The `MarketDataClient` handles market-specific data: order books, top-of-book quotes and trades, instrument status updates, and historical data requests. ```python from nautilus_trader.data.messages import RequestBars from nautilus_trader.data.messages import RequestData from nautilus_trader.data.messages import RequestInstrument from nautilus_trader.data.messages import RequestInstruments from nautilus_trader.data.messages import RequestOrderBookDeltas from nautilus_trader.data.messages import RequestOrderBookDepth from nautilus_trader.data.messages import RequestOrderBookSnapshot from nautilus_trader.data.messages import RequestQuoteTicks from nautilus_trader.data.messages import RequestTradeTicks from nautilus_trader.data.messages import SubscribeBars from nautilus_trader.data.messages import SubscribeData from nautilus_trader.data.messages import SubscribeFundingRates from nautilus_trader.data.messages import SubscribeIndexPrices from nautilus_trader.data.messages import SubscribeInstrument from nautilus_trader.data.messages import SubscribeInstrumentClose from nautilus_trader.data.messages import SubscribeInstruments from nautilus_trader.data.messages import SubscribeInstrumentStatus from nautilus_trader.data.messages import SubscribeMarkPrices from nautilus_trader.data.messages import SubscribeOrderBook from nautilus_trader.data.messages import SubscribeQuoteTicks from nautilus_trader.data.messages import SubscribeTradeTicks from nautilus_trader.data.messages import UnsubscribeBars from nautilus_trader.data.messages import UnsubscribeData from nautilus_trader.data.messages import UnsubscribeFundingRates from nautilus_trader.data.messages import UnsubscribeIndexPrices from nautilus_trader.data.messages import UnsubscribeInstrument from nautilus_trader.data.messages import UnsubscribeInstrumentClose from nautilus_trader.data.messages import UnsubscribeInstruments from nautilus_trader.data.messages import UnsubscribeInstrumentStatus from nautilus_trader.data.messages import UnsubscribeMarkPrices from nautilus_trader.data.messages import UnsubscribeOrderBook from nautilus_trader.data.messages import UnsubscribeQuoteTicks from nautilus_trader.data.messages import UnsubscribeTradeTicks from nautilus_trader.live.data_client import LiveMarketDataClient class TemplateLiveMarketDataClient(LiveMarketDataClient): """Example `LiveMarketDataClient` showing the overridable abstract methods.""" async def _connect(self) -> None: raise NotImplementedError("implement `_connect` in your adapter subclass") async def _disconnect(self) -> None: raise NotImplementedError("implement `_disconnect` in your adapter subclass") async def _subscribe(self, command: SubscribeData) -> None: raise NotImplementedError("implement `_subscribe` in your adapter subclass") async def _subscribe_instruments(self, command: SubscribeInstruments) -> None: raise NotImplementedError("implement `_subscribe_instruments` in your adapter subclass") async def _subscribe_instrument(self, command: SubscribeInstrument) -> None: raise NotImplementedError("implement `_subscribe_instrument` in your adapter subclass") async def _subscribe_order_book_deltas(self, command: SubscribeOrderBook) -> None: raise NotImplementedError("implement `_subscribe_order_book_deltas` in your adapter subclass") async def _subscribe_order_book_depth(self, command: SubscribeOrderBook) -> None: raise NotImplementedError("implement `_subscribe_order_book_depth` in your adapter subclass") async def _subscribe_quote_ticks(self, command: SubscribeQuoteTicks) -> None: raise NotImplementedError("implement `_subscribe_quote_ticks` in your adapter subclass") async def _subscribe_trade_ticks(self, command: SubscribeTradeTicks) -> None: raise NotImplementedError("implement `_subscribe_trade_ticks` in your adapter subclass") async def _subscribe_mark_prices(self, command: SubscribeMarkPrices) -> None: raise NotImplementedError("implement `_subscribe_mark_prices` in your adapter subclass") async def _subscribe_index_prices(self, command: SubscribeIndexPrices) -> None: raise NotImplementedError("implement `_subscribe_index_prices` in your adapter subclass") async def _subscribe_bars(self, command: SubscribeBars) -> None: raise NotImplementedError("implement `_subscribe_bars` in your adapter subclass") async def _subscribe_funding_rates(self, command: SubscribeFundingRates) -> None: raise NotImplementedError("implement `_subscribe_funding_rates` in your adapter subclass") async def _subscribe_instrument_status(self, command: SubscribeInstrumentStatus) -> None: raise NotImplementedError("implement `_subscribe_instrument_status` in your adapter subclass") async def _subscribe_instrument_close(self, command: SubscribeInstrumentClose) -> None: raise NotImplementedError("implement `_subscribe_instrument_close` in your adapter subclass") async def _subscribe_option_greeks(self, command: SubscribeOptionGreeks) -> None: raise NotImplementedError("implement `_subscribe_option_greeks` in your adapter subclass") async def _unsubscribe(self, command: UnsubscribeData) -> None: raise NotImplementedError("implement `_unsubscribe` in your adapter subclass") async def _unsubscribe_instruments(self, command: UnsubscribeInstruments) -> None: raise NotImplementedError("implement `_unsubscribe_instruments` in your adapter subclass") async def _unsubscribe_instrument(self, command: UnsubscribeInstrument) -> None: raise NotImplementedError("implement `_unsubscribe_instrument` in your adapter subclass") async def _unsubscribe_order_book_deltas(self, command: UnsubscribeOrderBook) -> None: raise NotImplementedError("implement `_unsubscribe_order_book_deltas` in your adapter subclass") async def _unsubscribe_order_book_depth(self, command: UnsubscribeOrderBook) -> None: raise NotImplementedError("implement `_unsubscribe_order_book_depth` in your adapter subclass") async def _unsubscribe_quote_ticks(self, command: UnsubscribeQuoteTicks) -> None: raise NotImplementedError("implement `_unsubscribe_quote_ticks` in your adapter subclass") async def _unsubscribe_trade_ticks(self, command: UnsubscribeTradeTicks) -> None: raise NotImplementedError("implement `_unsubscribe_trade_ticks` in your adapter subclass") async def _unsubscribe_mark_prices(self, command: UnsubscribeMarkPrices) -> None: raise NotImplementedError("implement `_unsubscribe_mark_prices` in your adapter subclass") async def _unsubscribe_index_prices(self, command: UnsubscribeIndexPrices) -> None: raise NotImplementedError("implement `_unsubscribe_index_prices` in your adapter subclass") async def _unsubscribe_bars(self, command: UnsubscribeBars) -> None: raise NotImplementedError("implement `_unsubscribe_bars` in your adapter subclass") async def _unsubscribe_funding_rates(self, command: UnsubscribeFundingRates) -> None: raise NotImplementedError("implement `_unsubscribe_funding_rates` in your adapter subclass") async def _unsubscribe_instrument_status(self, command: UnsubscribeInstrumentStatus) -> None: raise NotImplementedError("implement `_unsubscribe_instrument_status` in your adapter subclass") async def _unsubscribe_instrument_close(self, command: UnsubscribeInstrumentClose) -> None: raise NotImplementedError("implement `_unsubscribe_instrument_close` in your adapter subclass") async def _unsubscribe_option_greeks(self, command: UnsubscribeOptionGreeks) -> None: raise NotImplementedError("implement `_unsubscribe_option_greeks` in your adapter subclass") async def _request(self, request: RequestData) -> None: raise NotImplementedError("implement `_request` in your adapter subclass") async def _request_instrument(self, request: RequestInstrument) -> None: raise NotImplementedError("implement `_request_instrument` in your adapter subclass") async def _request_instruments(self, request: RequestInstruments) -> None: raise NotImplementedError("implement `_request_instruments` in your adapter subclass") async def _request_order_book_deltas(self, request: RequestOrderBookDeltas) -> None: raise NotImplementedError("implement `_request_order_book_deltas` in your adapter subclass") async def _request_order_book_depth(self, request: RequestOrderBookDepth) -> None: raise NotImplementedError("implement `_request_order_book_depth` in your adapter subclass") async def _request_order_book_snapshot(self, request: RequestOrderBookSnapshot) -> None: raise NotImplementedError("implement `_request_order_book_snapshot` in your adapter subclass") async def _request_quote_ticks(self, request: RequestQuoteTicks) -> None: raise NotImplementedError("implement `_request_quote_ticks` in your adapter subclass") async def _request_trade_ticks(self, request: RequestTradeTicks) -> None: raise NotImplementedError("implement `_request_trade_ticks` in your adapter subclass") async def _request_bars(self, request: RequestBars) -> None: raise NotImplementedError("implement `_request_bars` in your adapter subclass") ``` | Method | Description | |------------------------------------|---------------------------------------------------------| | `_connect` | Establishes a connection to the venue APIs. | | `_disconnect` | Closes the connection to the venue APIs. | | `_subscribe` | Subscribes to generic data (base for custom types). | | `_subscribe_instruments` | Subscribes to market data for multiple instruments. | | `_subscribe_instrument` | Subscribes to market data for a single instrument. | | `_subscribe_order_book_deltas` | Subscribes to order book delta updates. | | `_subscribe_order_book_depth` | Subscribes to order book depth updates. | | `_subscribe_quote_ticks` | Subscribes to top‑of‑book quote updates. | | `_subscribe_trade_ticks` | Subscribes to trade tick updates. | | `_subscribe_mark_prices` | Subscribes to mark price updates. | | `_subscribe_index_prices` | Subscribes to index price updates. | | `_subscribe_bars` | Subscribes to bar/candlestick updates. | | `_subscribe_funding_rates` | Subscribes to funding rate updates. | | `_subscribe_instrument_status` | Subscribes to instrument status updates. | | `_subscribe_instrument_close` | Subscribes to instrument close price updates. | | `_subscribe_option_greeks` | Subscribes to option greeks updates. | | `_unsubscribe` | Unsubscribes from generic data (base for custom types). | | `_unsubscribe_instruments` | Unsubscribes from market data for multiple instruments. | | `_unsubscribe_instrument` | Unsubscribes from market data for a single instrument. | | `_unsubscribe_order_book_deltas` | Unsubscribes from order book delta updates. | | `_unsubscribe_order_book_depth` | Unsubscribes from order book depth updates. | | `_unsubscribe_quote_ticks` | Unsubscribes from quote tick updates. | | `_unsubscribe_trade_ticks` | Unsubscribes from trade tick updates. | | `_unsubscribe_mark_prices` | Unsubscribes from mark price updates. | | `_unsubscribe_index_prices` | Unsubscribes from index price updates. | | `_unsubscribe_bars` | Unsubscribes from bar updates. | | `_unsubscribe_funding_rates` | Unsubscribes from funding rate updates. | | `_unsubscribe_instrument_status` | Unsubscribes from instrument status updates. | | `_unsubscribe_instrument_close` | Unsubscribes from instrument close price updates. | | `_unsubscribe_option_greeks` | Unsubscribes from option greeks updates. | | `_request` | Requests generic data (base for custom types). | | `_request_instrument` | Requests historical data for a single instrument. | | `_request_instruments` | Requests historical data for multiple instruments. | | `_request_order_book_snapshot` | Requests an order book snapshot. | | `_request_order_book_depth` | Requests order book depth. | | `_request_order_book_deltas` | Requests historical order book deltas. | | `_request_quote_ticks` | Requests historical quote tick data. | | `_request_trade_ticks` | Requests historical trade tick data. | | `_request_bars` | Requests historical bar data. | | `_request_funding_rates` | Requests historical funding rate data. | #### Order book delta flag requirements When implementing `_subscribe_order_book_deltas` or streaming order book data, adapters **must** set `RecordFlag` flags correctly on each `OrderBookDelta`. See also [Delta flags and event boundaries](../concepts/data.md#delta-flags-and-event-boundaries). - **`F_LAST`**: Set on the last delta of every logical event group. The `DataEngine` uses this flag as the flush signal when `buffer_deltas` is enabled. Without it, deltas accumulate indefinitely and are never published to subscribers. - **`F_SNAPSHOT`**: Set on all deltas that belong to a snapshot sequence (a `Clear` action followed by `Add` actions reconstructing the book). - **Empty book snapshots**: When emitting a snapshot for an empty book, the `Clear` delta must have `F_SNAPSHOT | F_LAST`. Otherwise buffered consumers never receive it. - **Incremental updates**: Each venue update message ends with a delta that has `F_LAST` set. If the venue batches multiple updates into one message, terminate each logical group with `F_LAST`. ```python from nautilus_trader.model.enums import RecordFlag # Incremental update (single event) delta = OrderBookDelta( instrument_id=instrument_id, action=BookAction.UPDATE, order=order, flags=RecordFlag.F_LAST, # Last (and only) delta in this event sequence=sequence, ts_event=ts_event, ts_init=ts_init, ) # Snapshot sequence clear_delta = OrderBookDelta( instrument_id=instrument_id, action=BookAction.CLEAR, order=NULL_ORDER, flags=RecordFlag.F_SNAPSHOT, # Not the last delta ... ) last_add_delta = OrderBookDelta( instrument_id=instrument_id, action=BookAction.ADD, order=last_order, flags=RecordFlag.F_SNAPSHOT | RecordFlag.F_LAST, # End of snapshot ... ) ``` :::warning A missing `F_LAST` is a silent bug: no error is raised, but subscribers never receive the data when buffering is enabled. ::: ### ExecutionClient The `ExecutionClient` manages order submission, modification, and cancellation against the venue trading system. ```python from nautilus_trader.execution.messages import BatchCancelOrders from nautilus_trader.execution.messages import CancelAllOrders from nautilus_trader.execution.messages import CancelOrder from nautilus_trader.execution.messages import GenerateFillReports from nautilus_trader.execution.messages import GenerateOrderStatusReport from nautilus_trader.execution.messages import GenerateOrderStatusReports from nautilus_trader.execution.messages import GeneratePositionStatusReports from nautilus_trader.execution.messages import ModifyOrder from nautilus_trader.execution.messages import SubmitOrder from nautilus_trader.execution.messages import SubmitOrderList from nautilus_trader.execution.reports import ExecutionMassStatus from nautilus_trader.execution.reports import FillReport from nautilus_trader.execution.reports import OrderStatusReport from nautilus_trader.execution.reports import PositionStatusReport from nautilus_trader.live.execution_client import LiveExecutionClient class TemplateLiveExecutionClient(LiveExecutionClient): """Example `LiveExecutionClient` outlining the required overrides.""" async def _connect(self) -> None: raise NotImplementedError("implement `_connect` in your adapter subclass") async def _disconnect(self) -> None: raise NotImplementedError("implement `_disconnect` in your adapter subclass") async def generate_order_status_report( self, command: GenerateOrderStatusReport, ) -> OrderStatusReport | None: raise NotImplementedError("method `generate_order_status_report` must be implemented in the subclass") async def generate_order_status_reports( self, command: GenerateOrderStatusReports, ) -> list[OrderStatusReport]: raise NotImplementedError("method `generate_order_status_reports` must be implemented in the subclass") async def generate_fill_reports( self, command: GenerateFillReports, ) -> list[FillReport]: raise NotImplementedError("method `generate_fill_reports` must be implemented in the subclass") async def generate_position_status_reports( self, command: GeneratePositionStatusReports, ) -> list[PositionStatusReport]: raise NotImplementedError("method `generate_position_status_reports` must be implemented in the subclass") async def generate_mass_status( self, lookback_mins: int | None = None, ) -> ExecutionMassStatus | None: raise NotImplementedError("method `generate_mass_status` must be implemented in the subclass") async def _submit_order(self, command: SubmitOrder) -> None: raise NotImplementedError("implement `_submit_order` in your adapter subclass") async def _submit_order_list(self, command: SubmitOrderList) -> None: raise NotImplementedError("implement `_submit_order_list` in your adapter subclass") async def _modify_order(self, command: ModifyOrder) -> None: raise NotImplementedError("implement `_modify_order` in your adapter subclass") async def _cancel_order(self, command: CancelOrder) -> None: raise NotImplementedError("implement `_cancel_order` in your adapter subclass") async def _cancel_all_orders(self, command: CancelAllOrders) -> None: raise NotImplementedError("implement `_cancel_all_orders` in your adapter subclass") async def _batch_cancel_orders(self, command: BatchCancelOrders) -> None: raise NotImplementedError("implement `_batch_cancel_orders` in your adapter subclass") ``` | Method | Description | |------------------------------------|-----------------------------------------------------------| | `_connect` | Establishes a connection to the venue APIs. | | `_disconnect` | Closes the connection to the venue APIs. | | `generate_order_status_report` | Generates a report for a specific order on the venue. | | `generate_order_status_reports` | Generates reports for all orders on the venue. | | `generate_fill_reports` | Generates reports for filled orders on the venue. | | `generate_position_status_reports` | Generates reports for position status on the venue. | | `generate_mass_status` | Generates execution mass status reports. | | `_submit_order` | Submits a new order to the venue. | | `_submit_order_list` | Submits a list of orders to the venue. | | `_modify_order` | Modifies an existing order on the venue. | | `_cancel_order` | Cancels a specific order on the venue. | | `_cancel_all_orders` | Cancels all orders for an instrument on the venue. | | `_batch_cancel_orders` | Cancels a batch of orders for an instrument on the venue. | ### Configuration Configuration classes hold adapter-specific settings like API keys and connection details. ```python from nautilus_trader.config import LiveDataClientConfig from nautilus_trader.config import LiveExecClientConfig class TemplateDataClientConfig(LiveDataClientConfig): """Configuration for `TemplateDataClient` instances.""" api_key: str api_secret: str base_url: str class TemplateExecClientConfig(LiveExecClientConfig): """Configuration for `TemplateExecClient` instances.""" api_key: str api_secret: str base_url: str ``` **Key attributes**: - `api_key`: The API key for authenticating with the data provider. - `api_secret`: The API secret for authenticating with the data provider. - `base_url`: The base URL for connecting to the data provider's API. ## Common test scenarios Exercise adapters across every venue behaviour they claim to support. Incorporate these scenarios into the Rust and Python suites. ### Product coverage Test each supported product family. - Spot instruments - Derivatives (perpetuals, futures, swaps) - Options and structured products ### Order flow - Cover each supported order type (limit, market, stop, conditional, etc.) under every venue time-in-force option, expiries, and rejection handling. - Submit buy and sell market orders and assert balance, position, and average-price updates align with venue responses. - Submit representative buy and sell limit orders, verifying acknowledgements, execution reports, full and partial fills, and cancel flows. ### State management - Start sessions with existing open orders to verify the adapter reconciles state on connect before issuing new commands. - Seed preloaded positions and confirm position snapshots, valuation, and PnL agree with the venue prior to trading. --- ## Data testing spec See the full [Data Testing Spec](spec_data_testing.md) for the `DataTester` test matrix. --- ## Execution testing spec See the full [Execution Testing Spec](spec_exec_testing.md) for the `ExecTester` test matrix. # Benchmarking Source: https://nautilustrader.io/docs/latest/developer_guide/benchmarking/ This document is the practitioner reference for writing and running NautilusTrader benchmarks. It covers tooling specifics, directory layout, example code, local execution, and flamegraph profiling. For policy (what we benchmark, when, with what rigor, how it ties into CI), see [`/BENCHMARKING.md`](../../BENCHMARKING.md) at the repository root. --- ## Tooling overview NautilusTrader uses two complementary Rust benchmarking frameworks: | Framework | What it measures | When to prefer it | |--------------------------------------------------------------|-------------------------------------------|------------------------------------------------------| | [**Criterion**](https://docs.rs/criterion/latest/criterion/) | Wall‑clock time with confidence bands | Anything ≥ 100 ns; absolute measurement; comparison. | | [**iai**](https://docs.rs/iai/latest/iai/) | Retired CPU instructions (via Cachegrind) | Sub‑100 ns functions; CI regression detection. | Most hot code paths benefit from both. Criterion gives the user-visible number; iai gives a noise-free regression signal. :::note iai is deterministic (immune to system noise) but results are machine-specific. Use it for regression detection within CI, not for cross-machine comparisons. ::: --- ## Directory layout Each crate keeps its benchmarks in a local `benches/` folder: ```text crates// └── benches/ ├── foo_criterion.rs └── foo_iai.rs ``` Register each benchmark explicitly in the crate's `Cargo.toml` so `cargo bench` discovers it: ```toml [[bench]] name = "foo_criterion" path = "benches/foo_criterion.rs" harness = false [[bench]] name = "foo_iai" path = "benches/foo_iai.rs" harness = false ``` To opt into the nightly CI performance workflow, add the crate to the `cargo-ci-benches` recipe in the workspace `Makefile`. --- ## Writing Criterion benchmarks 1. **Set up outside the timing loop.** All work that doesn't change between iterations belongs in the surrounding code or in `iter_batched_ref`'s setup closure, not in the body passed to `iter`. 2. **Wrap inputs in `black_box`** so the optimizer doesn't fold them away. 3. **Use `iter_batched_ref` for mutating benches.** It excludes input `Drop` from the timed region, which otherwise dominates the measurement on benches that own large structures. 4. **Add `Throughput::Elements(n)`** to size-parameterized groups so Criterion reports per-element throughput. 5. **Comment intent.** State what the benchmark is measuring (the hot path, the worst case, the cache-cold case) so a future reader understands what regressing it would mean. ```rust use std::hint::black_box; use criterion::{BatchSize, BenchmarkId, Criterion, Throughput, criterion_group, criterion_main}; const SIZES: &[usize] = &[10, 100, 1_000]; fn bench_my_op(c: &mut Criterion) { let mut group = c.benchmark_group("module/my_op"); for &n in SIZES { group.throughput(Throughput::Elements(n as u64)); group.bench_with_input(BenchmarkId::from_parameter(n), &n, |b, &n| { b.iter_batched_ref( || populate(n), |state| state.run(black_box(n)), BatchSize::SmallInput, ); }); } group.finish(); } criterion_group!(benches, bench_my_op); criterion_main!(benches); ``` --- ## Writing iai benchmarks `iai` requires functions that take no parameters. Keep them small so the instruction count is meaningful and so changes outside the function don't leak into the measurement. ```rust use std::hint::black_box; fn bench_add() -> i64 { let a = black_box(123); let b = black_box(456); a + b } iai::main!(bench_add); ``` Setup that varies between runs (allocations, randomness, system calls) will inflate instruction counts in misleading ways. iai is best for pure, allocation-free functions. --- ## Running benches locally | Goal | Command | |-------------------------------------|----------------------------------------------------------------------| | All benches in one crate | `cargo bench -p nautilus-execution` | | One bench module | `cargo bench -p nautilus-execution --bench matching_core` | | One specific bench by name pattern | `cargo bench -p nautilus-execution --bench matching_core -- iterate` | | Quick smoke run (low sample count) | `cargo bench ... -- --quick` | | All CI-tracked benches | `make cargo-ci-benches` | Criterion writes HTML reports to `target/criterion/`. Open `target/criterion/report/index.html`. The report includes per-bench violin plots, confidence intervals, and comparisons against the previous run's saved baseline. --- ## Generating a flamegraph `cargo-flamegraph` produces a sampled call-stack profile for one bench. Useful when a bench shows a regression but it's not obvious which inner call is responsible. 1. Install once per machine: ```bash cargo install flamegraph ``` 2. Run a specific bench with the `bench` profile: ```bash cargo flamegraph --bench matching -p nautilus-common --profile bench ``` 3. Open `flamegraph.svg` in a browser and zoom into hot paths. ### Linux `perf` must be available. On Debian/Ubuntu: ```bash sudo apt install linux-tools-common linux-tools-$(uname -r) ``` If `perf_event_paranoid` blocks the run: ```bash sudo sh -c 'echo 1 > /proc/sys/kernel/perf_event_paranoid' ``` A value of `1` is usually enough. Set it back to `2` (default) afterwards or persist via `/etc/sysctl.conf`. ### macOS `DTrace` requires root, so `cargo flamegraph` must be run with `sudo`. :::warning Running with `sudo` creates files in `target/` owned by root, causing permission errors with subsequent `cargo` commands. You may need to remove root-owned files manually or run `sudo cargo clean`. ::: ```bash sudo cargo flamegraph --bench matching -p nautilus-common --profile bench ``` The `bench` profile keeps full debug symbols, so flamegraphs render with readable function names without bloating production binaries (which still use `panic = "abort"` and are built via `[profile.release]`). > **Note** Benchmark binaries are compiled with the custom `[profile.bench]` > defined in the workspace `Cargo.toml`. That profile inherits from > `release` and sets `debug = "full"`, preserving full optimisation *and* > debug symbols so tools like `cargo flamegraph` or `perf` produce > human-readable stack traces. --- ## Templates Ready-to-copy starter files live in [`docs/dev_templates/`](../dev_templates/): - **Criterion**: [`criterion_template.rs`](../dev_templates/criterion_template.rs) - **iai**: [`iai_template.rs`](../dev_templates/iai_template.rs) Copy the template into the target crate's `benches/`, adjust imports and group names, register in `Cargo.toml`, and start measuring. # Coding Standards Source: https://nautilustrader.io/docs/latest/developer_guide/coding_standards/ ## Code Style The current codebase can be used as a guide for formatting conventions. Additional guidelines are provided below. ### Universal formatting rules The following applies to **all** source files (Rust, Python, Cython, shell, etc.): - Use **spaces only**, never hard tab characters. - Lines should generally stay below **100 characters**; wrap thoughtfully when necessary. - Prefer American English spelling (`color`, `serialize`, `behavior`). ### Shell script portability Shell scripts in this repository use **bash** (not POSIX sh) and must be portable across **Linux** and **macOS**. User-facing scripts (e.g., `scripts/cli/install.sh`) must also work on **Windows** via Git Bash or WSL. **Shebang**: Always use `#!/usr/bin/env bash` for portability. **Common pitfalls**: GNU and BSD utilities differ between Linux and macOS: | Command | Linux (GNU) | macOS (BSD) | Portable solution | |----------------------|-------------------|-------------------|------------------------------------------| | `sed -i` | `sed -i 's/…'` | `sed -i '' 's/…'` | Use backup extension: `sed -i.bak 's/…'` | | `stat` (file size) | `stat -c%s file` | `stat -f%z file` | Detect with `stat --version` | | `sha256sum` | `sha256sum file` | N/A | Use `shasum -a 256` or detect | | `readlink -f` | Works | N/A | Avoid, or use `realpath` | | `grep -P` (PCRE) | Works | N/A | Use `-E` (extended regex) instead | | `date` (nanoseconds) | `date +%N` | N/A | Use `$RANDOM` for cache‑busting | **Bash version**: macOS ships with bash 3.2; avoid bash 4+ features in user-facing scripts: | Feature | Bash version | Alternative | |-----------------------------------|--------------|----------------------------------| | Associative arrays (`declare -A`) | 4.0+ | Use files or simple arrays | | `readarray` / `mapfile` | 4.0+ | Use `while read` loops | | `${var,,}` / `${var^^}` (case) | 4.0+ | Use `tr '[:upper:]' '[:lower:]'` | **CI scripts** (`scripts/ci/*`) run on Linux runners, so bash 4+ and GNU tools are acceptable there. ### Comment conventions 1. Generally leave **one blank line above** every comment block or docstring so it is visually separated from code. 2. Use *sentence case* – capitalize the first letter, keep the rest lowercase unless proper nouns or acronyms. 3. Do not use double spaces after periods. 4. **Single-line comments** *must not* end with a period *unless* the line ends with a URL or inline Markdown link – in those cases leave the punctuation exactly as the link requires. 5. **Multi-line comments** should separate sentences with commas (not period-per-line). The final line *should* end with a period. 6. Keep comments concise; favor clarity and only explain the non-obvious – *less is more*. 7. Avoid emoji symbols in text. ### Doc comment mood **Rust** doc comments should be written in the **indicative mood** – e.g. *"Returns a cached client."* This convention aligns with the prevailing style of the Rust ecosystem and makes generated documentation feel natural to end-users. ### Terminology and phrasing 1. **Error messages**: Avoid using ", got" in error messages. Use more descriptive alternatives like ", was", ", received", or ", found" depending on context. - ❌ `"Expected string, got {type(value)}"` - ✅ `"Expected string, was {type(value)}"` 2. **Spelling**: Use "hardcoded" (single word) rather than "hard-coded" or "hard coded" – this is the more modern and accepted spelling. 3. **Error variable naming**: Use single-letter `e` for caught errors/exceptions: - Rust: `Err(e)` not `Err(err)` or `Err(error)`, and `|e|` not `|err|` in closures - Python: `except SomeError as e:` not `as err:` or `as error:` ### Naming conventions 1. **Internal fields**: Abbreviations are acceptable for private/internal fields (e.g., `_price_prec`, `_size_prec`) to keep hot-path code concise. 2. **User-facing API**: Use full, descriptive names for public properties, function parameters, return types, and metric names/labels (e.g., `price_precision`, `size_precision`). This prevents abbreviated terminology from leaking into dashboards or alerts. 3. **Error messages and logs**: Use full words for clarity (e.g., "price precision" not "price prec"). The user should never see abbreviated terminology. ### Formatting 1. For longer lines of code, and when passing more than a couple of arguments, you should take a new line which aligns at the next logical indent (rather than attempting a hanging 'vanity' alignment off an opening parenthesis). This practice conserves space to the right, keeps important code more central in view, and survives function/method name changes. 2. The closing parenthesis should be located on a new line, aligned at the logical indent. 3. Multiple hanging parameters or arguments should end with a trailing comma: ```python long_method_with_many_params( some_arg1, some_arg2, some_arg3, # <-- trailing comma ) ``` ## Commit messages Here are some guidelines for the style of your commit messages: 1. Limit subject titles to 60 characters or fewer. Capitalize subject line and do not end with period. 2. Use 'imperative voice', i.e. the message should describe what the commit will do if applied. 3. Optional: Use the body to explain change. Separate from subject with a blank line. Keep under 100 character width. You can use bullet points with or without terminating periods. 4. Optional: Provide # references to relevant issues or tickets. 5. Optional: Provide any hyperlinks which are informative. ### Gitlint (optional) Gitlint is available to help enforce commit message standards automatically. It checks that commit messages follow the guidelines above (character limits, formatting, etc.). This is **opt-in** and not enforced in CI. **Benefits**: Encourages concise yet expressive commit messages, helps develop clear explanations of changes. **Installation**: First install gitlint to run it locally: ```bash uv pip install gitlint ``` To enable gitlint as an automatic commit-msg hook: ```bash prek install --hook-type commit-msg ``` **Manual usage**: Check your last commit message: ```bash gitlint ``` Configuration is in `.gitlint` at the repository root: - **60-character title limit**: Ensures clear rendering on GitHub and encourages brevity while remaining descriptive. - **79-character body width**: Aligns with Python's PEP 8 conventions and the traditional limit for git tooling. :::note Gitlint may be enforced in CI in the future, so adopting these practices early eases the transition. ::: # Design Principles Source: https://nautilustrader.io/docs/latest/developer_guide/design_principles/ ## Message immutability Once a message (request, response, event, or command) is created, its fields must not be mutated. See [Message Bus: message integrity](../concepts/message_bus.md#message-integrity) for the ownership rules that follow from this. The invariant protects several properties the system depends on: - **Determinism**: Every consumer sees the same input. Behavior is easier to reason about, replay, and test. - **Temporal integrity**: A message preserves what was true when the system emitted it. Events and commands remain factual records instead of containers of drifting state. - **Safer concurrency**: Readers do not need coordination to protect message payloads from later rewrites. This removes a common source of races around shared state. - **Easier debugging**: Logs, traces, replay tools, and dead-letter inspection remain useful because the message still reflects the original payload. - **Reliable replay and simulation**: Replaying a sequence yields the same logical inputs as the original run. This supports backtesting, incident reconstruction, and regression testing. - **Clear ownership boundaries**: Components treat incoming messages as input. If a component needs a different representation, it derives new local state or a new message explicitly. - **Better auditability**: The system can answer what it knew, when it knew it, and what it did from that information. - **More robust distribution**: Serialized messages already cross process and service boundaries as copies. The same ownership rule keeps the in-memory model aligned with that reality. # Docs Style Source: https://nautilustrader.io/docs/latest/developer_guide/docs/ This guide outlines the style conventions and best practices for writing documentation for NautilusTrader. ## General principles - We favor simplicity over complexity, less is more. - We favor concise yet readable prose and documentation. - We value standardization in conventions, style, patterns, etc. - Documentation should be accessible to users of varying technical backgrounds. ## Documentation types Most pages should fit one of four types ([Divio documentation system](https://docs.divio.com/documentation-system/)). Mixing types in a single page makes it harder to read and harder to maintain. | Type | Purpose | Section | |------------------|----------------------------------|------------------| | **Tutorial** | Teach by walking through a task | `tutorials/` | | **How‑to guide** | Solve a specific problem | `how_to/` | | **Explanation** | Clarify design and architecture | `concepts/` | | **Reference** | Describe the machinery | `api_reference/` | Two sections are exceptions: `getting_started/` is an onboarding path that combines tutorial-style walkthroughs with setup instructions, and `integrations/` pages mix reference (capabilities, symbology) with how-to content (setup, configuration) so each venue page is self-contained. Standalone how-to content that is not venue-specific belongs in `how_to/`. ### Choosing the right type - **Does your page walk a newcomer through a learning experience?** Tutorial. - **Does it answer "How do I...?" for someone who already knows the system?** How-to guide. - **Does it explain why something works the way it does?** Explanation. - **Does it list classes, config fields, enums, or capabilities?** Reference. A tutorial says "do this, then this, then this." The author picks the path. A how-to guide says "here is how to achieve X." The reader already knows they want X. Keep these distinct: - Tutorials should not assume prior knowledge. - How-to guides should not teach background concepts. When one type needs to reference another, link to it instead of inlining. For example, a how-to guide that configures `TradingNodeConfig` should link to the API reference for field definitions rather than listing them again. ## Language and tone - Use active voice when possible ("Configure the adapter" vs "The adapter should be configured"). - Write in present tense for describing current functionality. - Use future tense only for planned features. - Avoid unnecessary jargon; define technical terms on first use. - Be direct and concise; avoid filler words like "basically", "simply", "just". - Use parallel structure in lists; keep grammatical patterns consistent across items. ## Markdown tables ### Column alignment and spacing - Use symmetrical column widths based on the space dictated by the widest content in each column. - Align column separators (`|`) vertically for better readability. - Use consistent spacing around cell content. ### Notes and descriptions - All notes and descriptions should have terminating periods. - Keep notes concise but informative. - Use sentence case (capitalize only the first letter and proper nouns). ### Example ```markdown | Order Type | Spot | Margin | USDT Futures | Coin Futures | Notes | |------------------------|------|--------|--------------|--------------|-------------------------| | `MARKET` | ✓ | ✓ | ✓ | ✓ | | | `STOP_MARKET` | - | ✓ | ✓ | ✓ | Not supported for Spot. | | `MARKET_IF_TOUCHED` | - | - | ✓ | ✓ | Futures only. | ``` ### Support indicators - Use `✓` for supported features. - Use `-` for unsupported features (not `✗` or other symbols). - When adding notes for unsupported features, emphasize with italics: `*Not supported*`. - Make unsupported notes specific when the reason matters: use `*Not supported by *` for venue gaps, or `*Not currently implemented*` for adapter gaps. - Leave cells empty when no content is needed. ## Code references - Use backticks for inline code, method names, class names, and configuration options. - Use code blocks for multi-line examples. - When referencing code locations, use `file_path::function_name` or `file_path::ClassName` rather than line numbers, which become stale as code changes. ## Headings We follow modern documentation conventions that prioritize readability and accessibility: - Use title case for the main page heading (# Level 1 only). - Use sentence case for all subheadings (## Level 2 and below). - Always capitalize proper nouns regardless of heading level (product names, technologies, companies, acronyms). - Use proper heading hierarchy (don't skip levels). This convention aligns with industry standards used by major technology companies including Google Developer Documentation, Microsoft Docs, and Anthropic's documentation. It improves readability, reduces cognitive load, and is more accessible for international users and screen readers. ### Examples ```markdown # NautilusTrader Developer Guide ## Getting started with Python ## Using the Binance adapter ## REST API implementation ## WebSocket data streaming ## Testing with pytest ``` ## Lists - Use hyphens (`-`) for unordered list bullets; avoid `*` or `+` to keep the Markdown style consistent across the project. - Use numbered lists only when order matters. - Maintain consistent indentation for nested lists. - End list items with periods when they are complete sentences. ## Links and references - Use descriptive link text (avoid "click here" or "this link"). - Reference external documentation when appropriate. - Keep all internal links relative and accurate. ## Technical terminology - Base capability matrices on the Nautilus domain model, not exchange-specific terminology. - Mention exchange-specific terms in parentheses or notes when necessary for clarity. - Use consistent terminology throughout the documentation. ## Examples and code samples - Provide practical, working examples. - Include necessary imports and context. - Use realistic variable names and values. - Add comments to explain non-obvious parts of examples. ## Admonitions Use admonition blocks to highlight important information: | Admonition | Purpose | |--------------|---------------------------------------------------------------| | `:::note` | Supplementary context that clarifies but isn't essential. | | `:::info` | Important information the reader should be aware of. | | `:::tip` | Helpful suggestions or best practices. | | `:::warning` | Potential pitfalls or important caveats. | | `:::danger` | Critical issues that could cause data loss or system failure. | Avoid overusing admonitions; too many diminish their impact. ## MDX components The docs site (fumadocs) provides built-in MDX components available in all `.md` files. No imports are needed. ### Tabs Use tabs for language-specific or variant content. List Rust before Python so Rust is the default (left-most) tab. For code examples, add `tab="..."` to consecutive fenced code blocks: ```markdown \`\`\`rust tab="Rust" let params = Params::from([("close_position", true.into())]); \`\`\` \`\`\`python tab="Python" strategy.submit_order(order, params={"close_position": True}) \`\`\` ``` For tables or other content, wrap each variant in `` and ``. The instrument Fields tables use this so each language shows a single type column instead of side-by-side Rust and Python columns. Leave a blank line above and below the inner content so the Markdown renders. ```markdown | Field | Type | Required/default | Notes | |-----------------|----------------|------------------|-------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | Field | Type | Required/default | Notes | |-----------------|----------------|------------------|-------| | `instrument_id` | `InstrumentId` | Required | | ``` ### Steps Use `Steps` and `Step` for sequential procedures. ```markdown Configure the adapter. Start the trading node. ``` ### Accordions Use `Accordions` and `Accordion` for collapsible content. ```markdown Content here. ``` ### Files Use `Files`, `Folder`, and `File` for directory tree visualizations. ```markdown ``` ### Cards Use `Cards` and `Card` for linked content grids. ```markdown ``` ### TypeTable Use `TypeTable` for parameter or type documentation tables. ## Line length and wrapping - Wrap lines at no more than ~100-120 characters for better readability and diff reviews. - Break long sentences at natural points (after commas, conjunctions, or phrases). - Avoid orphaned words on new lines when possible. - Code blocks and URLs can exceed the line limit when necessary. ## API documentation - Document parameters and return types clearly. - Include usage examples for complex APIs. - Explain any side effects or important behavior. - Keep parameter descriptions concise but complete. # Environment Setup Source: https://nautilustrader.io/docs/latest/developer_guide/environment_setup/ For development we recommend using the PyCharm *Professional* edition IDE, as it interprets Cython syntax. Alternatively, you could use Visual Studio Code with a Cython extension. [uv](https://docs.astral.sh/uv) is the preferred tool for handling all Python virtual environments and dependencies. [prek](https://github.com/j178/prek) is used to automatically run various pre-commit checks, auto-formatters and linting tools at commit. NautilusTrader uses increasingly more [Rust](https://www.rust-lang.org), so Rust should be installed on your system as well ([installation guide](https://www.rust-lang.org/tools/install)). [Cap'n Proto](https://capnproto.org/) is required for serialization schema compilation. The required version is specified in `tools.toml` in the repository root. Ubuntu's default package is typically too old, so you may need to install from source (see below). :::info NautilusTrader *must* compile and run on **Linux, macOS, and Windows**. Please keep portability in mind (use `std::path::Path`, avoid Bash-isms in shell scripts, etc.). ::: ## Setup The following steps are for UNIX-like systems, and only need to be completed once. ### Quick setup Use this as a compact setup path for a new Linux or macOS development machine. The detailed sections below explain each step and cover alternatives. Install platform tools first: ```bash tab="Ubuntu" sudo apt-get update sudo apt-get install -y build-essential clang lld curl git make pkg-config ``` ```bash tab="macOS" xcode-select --install ``` Then clone the repository and install the pinned project tools: ```bash git clone --branch develop https://github.com/nautechsystems/nautilus_trader cd nautilus_trader curl https://sh.rustup.rs -sSf | sh source "$HOME/.cargo/env" curl -LsSf https://astral.sh/uv/install.sh | sh export PATH="$HOME/.local/bin:$PATH" cargo install cargo-binstall --locked make install-tools ./scripts/install-capnp.sh uv sync --all-groups --all-extras source .venv/bin/activate export PYO3_PYTHON="$PWD/.venv/bin/python" if [ "$(uname -s)" = "Linux" ]; then PYTHON_LIB_DIR="$("$PYO3_PYTHON" -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')" export LD_LIBRARY_PATH="$PYTHON_LIB_DIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" fi export PYTHONHOME="$("$PYO3_PYTHON" -c 'import sys; print(sys.base_prefix)')" prek install make build-debug ``` Windows users should follow the source installation steps in the [installation guide](../getting_started/installation.md#from-source), then use the relevant commands from this guide. ### 1. Install dependencies Follow the [installation guide](../getting_started/installation.md) to set up the project with a modification to the final command to install development and test dependencies: ```bash tab="uv" uv sync --active --all-groups --all-extras ``` ```bash tab="make" make install ``` If you're developing and iterating frequently, then compiling in debug mode is often sufficient and *significantly* faster than a fully optimized build. To install in debug mode, use: ```bash make install-debug ``` ### 2. Install development tools NautilusTrader pins every development tool so that all contributors and CI run identical versions. A single Makefile target installs the full set: ```bash make install-tools ``` This installs: - **Cargo CLIs** pinned in `Cargo.toml` under `[workspace.metadata.tools]`: `cargo-audit`, `cargo-deny`, `cargo-edit`, `cargo-fuzz`, `cargo-llvm-cov`, `cargo-machete`, `cargo-nextest`, `cargo-vet`, `flamegraph`, `lychee`. - **Prebuilt binaries** pinned in `tools.toml`: `prek` (pre-commit runner) and `osv-scanner` (vulnerability scanner). - **uv**, synced to the version required by `pyproject.toml`. Cap'n Proto is also pinned in `tools.toml` but installs separately; see the [Cap'n Proto](#capn-proto) section below. Fuzz targets also require a Rust nightly toolchain at runtime because `cargo-fuzz` uses `libfuzzer-sys` and unstable compiler flags: ```bash rustup toolchain install nightly ``` #### One-off prerequisite: cargo-binstall `make install-tools` uses [`cargo-binstall`](https://github.com/cargo-bins/cargo-binstall) to fetch `prek` as a prebuilt binary instead of compiling it from source. Install `cargo-binstall` once per machine: ```bash cargo install cargo-binstall --locked ``` This is a one-time step. Subsequent runs of `make install-tools` reuse the installed `cargo-binstall`. #### Single source of truth for versions The repository manifests are the canonical source for dependency and tool versions. Do not copy current version numbers into docs, runner images, or scripts unless there is no manifest-backed way to read them. | Source file or section | Defines | |----------------------------------------------|-------------------------------------------------------| | `rust-toolchain.toml` | Rust toolchain. | | `Cargo.toml` and `Cargo.lock` | Rust workspace dependencies and exact resolution. | | `Cargo.toml` `[workspace.metadata.tools]` | Cargo‑installable development tools. | | `pyproject.toml` and `python/pyproject.toml` | Python dependencies, supported Python range, and uv. | | `uv.lock` and `python/uv.lock` | Exact Python dependency resolutions. | | `tools.toml` | External CLIs and binaries without a native manifest. | The external tool pins in `tools.toml` include `prek`, `pip-audit`, `pypi-attestations`, `osv-scanner`, and `capnp`. The Makefile reads these via `scripts/cargo-tool-version.sh`, `scripts/tool-version.sh`, and `scripts/uv-version.sh`, so bumping a version in the source file is the only required version change. To check the pinned cargo tool versions against crates.io, run: ```bash make outdated ``` ### 3. Set up pre-commit Set up the pre-commit hook which will then run automatically at commit: ```bash prek install ``` Before opening a pull-request run the formatting and lint suite locally so that CI passes on the first attempt: ```bash make format make pre-commit ``` Make sure the Rust compiler reports **zero errors** -- broken builds slow everyone down. ### 4. Configure environment variables **Required for Rust/PyO3 (Linux and macOS)**: When using Python installed via `uv` on Linux or macOS, set the following environment variables from the repository root after `uv sync`: ```bash # Set the Python executable path for PyO3 export PYO3_PYTHON="$PWD/.venv/bin/python" # Linux only: Set the library path for the uv-managed Python runtime PYTHON_LIB_DIR="$("$PYO3_PYTHON" -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')" export LD_LIBRARY_PATH="$PYTHON_LIB_DIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" # Set the Python home path (required for Rust tests) export PYTHONHOME="$("$PYO3_PYTHON" -c 'import sys; print(sys.base_prefix)')" ``` :::note The `LD_LIBRARY_PATH` export is Linux-specific and not needed on macOS or Windows. - `PYO3_PYTHON` tells PyO3 which Python interpreter to use, reducing unnecessary recompilation. - `PYTHONHOME` is required when running `make cargo-test` with a `uv`-installed Python. Without it, tests that depend on PyO3 may fail to locate the Python runtime. ::: To verify your environment is configured correctly: ```bash python -c "import sys; print('Python:', sys.executable, sys.version)" echo "PYO3_PYTHON: $PYO3_PYTHON" echo "PYTHONHOME: $PYTHONHOME" ``` ## Dependency management Python dependencies are managed by [uv](https://docs.astral.sh/uv). The `[tool.uv]` section in `pyproject.toml` enforces three supply chain safety settings: - **`required-version`**: all developers and CI use the same uv version. The version is extracted by `scripts/uv-version.sh` for Makefile, CI, and Docker builds. If your local uv drifts off the pin, `uv lock`/`uv sync` will fail with `Required uv version ... does not match the running version ...`. Run `make update-uv` to install the pinned version (or follow uv's own `uv self update ` hint). - **`exclude-newer = "3 days"`**: `uv lock` ignores package versions published within the last 3 days. This gives the community time to detect and quarantine compromised releases before they enter the lockfile. The value accepts an RFC 3339 timestamp (`"2026-03-30T00:00:00Z"`), a friendly duration (`"3 days"`, `"1 week"`, `"24 hours"`), or an ISO 8601 duration (`"P3D"`, `"P1W"`, `"PT24H"`). uv 0.11.8+ stores the friendly/ISO form as `exclude-newer-span` inside `uv.lock` and emits a sentinel `exclude-newer` timestamp alongside it for backwards compatibility; both lockfiles in this repo use that format. - **`no-build-package`**: explicit list of every third-party package locked in `uv.lock`. `uv` refuses to build any of them from source. In normal operation uv prefers wheels, so the setting is a no-op; it triggers only if a listed package stops publishing wheels for the target platform, in which case `uv lock` fails rather than silently building from an sdist. The local workspace package is intentionally not in the list because it must be built by the workspace's own build backend. The list is kept in sync with `uv.lock` by `scripts/check-no-build-packages.sh`, which also runs as a pre-commit hook on changes to `uv.lock` or `pyproject.toml`. ### Bypassing the cooldown When a security patch or critical bug fix must be pulled in immediately, override `exclude-newer` on the command line. All forms accept a timestamp, friendly duration, or ISO duration; package overrides additionally accept `false` to exempt a package from the cooldown entirely. ```bash # Shorten the cooldown for a single package (friendly duration) uv lock --exclude-newer-package "somepackage=1 day" # Pin a single package to an absolute cutoff uv lock --exclude-newer-package "somepackage=2026-03-30T00:00:00Z" # Exempt a single package from the cooldown entirely uv lock --exclude-newer-package "somepackage=false" # Disable the cooldown for the whole resolution uv lock --exclude-newer "0 seconds" ``` The CLI flag overrides the `pyproject.toml` value for that invocation only. The config remains unchanged for subsequent runs. ### Updating uv To update the pinned uv version, change `required-version` in both `pyproject.toml` and `python/pyproject.toml`, then update the `rev` in `.pre-commit-config.yaml` to match. Run `make update-uv` to install the new pinned version locally. ## Builds Following any changes to `.rs`, `.pyx` or `.pxd` files, you can re-compile by running: ```bash tab="uv" uv run --no-sync python build.py ``` ```bash tab="make" make build ``` If you're developing and iterating frequently, then compiling in debug mode is often sufficient and *significantly* faster than a fully optimized build. To compile in debug mode, use: ```bash make build-debug ``` ## Cap'n Proto [Cap'n Proto](https://capnproto.org/) is required for serialization schema compilation. The required version is defined in `tools.toml` in the repository root. Install the correct version for your platform: ```bash tab="Script (Linux/macOS)" ./scripts/install-capnp.sh ``` ```bash tab="macOS (Homebrew)" brew install capnp ``` ```bash tab="Linux (source)" CAPNP_VERSION=$(bash scripts/tool-version.sh capnp) cd ~ wget https://capnproto.org/capnproto-c++-${CAPNP_VERSION}.tar.gz tar xzf capnproto-c++-${CAPNP_VERSION}.tar.gz cd capnproto-c++-${CAPNP_VERSION} ./configure make -j$(nproc) sudo make install sudo ldconfig ``` ```bash tab="Windows (Chocolatey)" choco install capnproto ``` Verify the installed version matches `tools.toml`: ```bash capnp --version ``` The install script ensures the pinned version is installed. If Homebrew or Chocolatey provides an older version, install from source or see the [Cap'n Proto installation guide](https://capnproto.org/install.html). ## Faster builds The cranelift backends reduces build time significantly for dev, testing and IDE checks. However, cranelift is available on the nightly toolchain and needs extra configuration. Install the nightly toolchain ``` rustup install nightly rustup override set nightly rustup component add rust-analyzer # install nightly lsp rustup override set stable # reset to stable ``` Activate the nightly feature and use "cranelift" backend for dev and testing profiles in workspace `Cargo.toml`. You can apply the below patch using `git apply `. You can remove it using `git apply -R ` before pushing changes. :::warning Do not commit these changes. The cranelift patch is for local development only and will break CI if pushed. ::: ``` diff --git a/Cargo.toml b/Cargo.toml index 62b78cd8d0..beb0800211 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,3 +1,6 @@ +# This line needs to come before anything else in Cargo.toml +cargo-features = ["codegen-backend"] + [workspace] resolver = "2" members = [ @@ -140,6 +143,7 @@ lto = false panic = "unwind" incremental = true codegen-units = 256 +codegen-backend = "cranelift" [profile.test] opt-level = 0 @@ -150,11 +154,13 @@ strip = false lto = false incremental = true codegen-units = 256 +codegen-backend = "cranelift" [profile.nextest] inherits = "test" debug = false # Improves compile times strip = "debuginfo" # Improves compile times +codegen-backend = "cranelift" [profile.release] opt-level = 3 ``` Pass `RUSTUP_TOOLCHAIN=nightly` when running `make build-debug` like commands and include it in all [rust analyzer settings](#rust-analyzer-settings) for faster builds and IDE checks. ## Services You can use `docker-compose.yml` file located in `.docker` directory to bootstrap the Nautilus working environment. This will start the following services: ```bash docker-compose up -d ``` If you only want specific services running (like `postgres` for example), you can start them with command: ```bash docker-compose up -d postgres ``` Used services are: - `postgres`: Postgres database with root user `POSTGRES_USER` which defaults to `postgres`, `POSTGRES_PASSWORD` which defaults to `pass` and `POSTGRES_DB` which defaults to `postgres`. - `redis`: Redis server. - `pgadmin`: PgAdmin4 for database management and administration. :::info Please use this as development environment only. For production, use a proper and more secure setup. ::: After the services has been started, you must log in with `psql` cli to create `nautilus` Postgres database. To do that you can run, and type `POSTGRES_PASSWORD` from docker service setup ```bash psql -h localhost -p 5432 -U postgres ``` After you have logged in as `postgres` administrator, run `CREATE DATABASE` command with target db name (we use `nautilus`): ``` psql (16.2, server 15.2 (Debian 15.2-1.pgdg110+1)) Type "help" for help. postgres=# CREATE DATABASE nautilus; CREATE DATABASE ``` ## Nautilus CLI developer guide ## Introduction The Nautilus CLI is a command-line interface tool for interacting with the NautilusTrader ecosystem. It offers commands for managing the PostgreSQL database and handling various trading operations. :::warning On Linux systems with GNOME desktop, the `nautilus` command typically refers to the GNOME file manager (`/usr/bin/nautilus`). After installing the NautilusTrader CLI, you may need to ensure the Cargo binary takes precedence by either: - Adding an alias to your shell config: `alias nautilus="$HOME/.cargo/bin/nautilus"` - Using the full path: `~/.cargo/bin/nautilus` - Ensuring `~/.cargo/bin` appears before `/usr/bin` in your `PATH` ::: :::note The Nautilus CLI command is only supported on UNIX-like systems. ::: ## Install You can install the Nautilus CLI using the below Makefile target, which uses `cargo install` under the hood. This will place the nautilus binary in your system's PATH, assuming Rust's `cargo` is properly configured. ```bash make install-cli ``` ## Commands You can run `nautilus --help` to view the CLI structure and available command groups: ### Database These commands handle bootstrapping the PostgreSQL database. To use them, you need to provide the correct connection configuration, either through command-line arguments or a `.env` file located in the root directory or the current working directory. - `--host` or `POSTGRES_HOST` for the database host - `--port` or `POSTGRES_PORT` for the database port - `--user` or `POSTGRES_USERNAME` for the root administrator (typically the postgres user) - `--password` or `POSTGRES_PASSWORD` for the root administrator's password - `--database` or `POSTGRES_DATABASE` for both the database **name and the new user** with privileges to that database (e.g., if you provide `nautilus` as the value, a new user named nautilus will be created with the password from `POSTGRES_PASSWORD`, and the `nautilus` database will be bootstrapped with this user as the owner). Example of `.env` file ``` POSTGRES_HOST=localhost POSTGRES_PORT=5432 POSTGRES_USERNAME=postgres POSTGRES_PASSWORD=pass POSTGRES_DATABASE=nautilus ``` List of commands are: 1. `nautilus database init`: Will bootstrap schema, roles and all sql files located in `schema` root directory (like `tables.sql`). 2. `nautilus database drop`: Will drop all tables, roles and data in target Postgres database. ## Rust analyzer settings Rust analyzer is a popular language server for Rust and has integrations for many IDEs. It is recommended to configure rust analyzer to have same environment variables as `make build-debug` for faster compile times. Below tested configurations for VSCode and Astro Nvim are provided. For more information see [PR](https://github.com/nautechsystems/nautilus_trader/pull/2524) or rust analyzer [config docs](https://rust-analyzer.github.io/book/configuration.html). ```json tab="VSCode" { "rust-analyzer.restartServerOnConfigChange": true, "rust-analyzer.linkedProjects": [ "Cargo.toml" ], "rust-analyzer.cargo.features": "all", "rust-analyzer.check.workspace": false, "rust-analyzer.check.extraEnv": { "VIRTUAL_ENV": "/.venv", "CC": "clang", "CXX": "clang++" }, "rust-analyzer.cargo.extraEnv": { "VIRTUAL_ENV": "/.venv", "CC": "clang", "CXX": "clang++" }, "rust-analyzer.runnables.extraEnv": { "VIRTUAL_ENV": "/.venv", "CC": "clang", "CXX": "clang++" }, "rust-analyzer.check.features": "all", "rust-analyzer.testExplorer": true } ``` ```lua tab="Neovim (AstroLSP)" config = { rust_analyzer = { settings = { ["rust-analyzer"] = { restartServerOnConfigChange = true, linkedProjects = { "Cargo.toml" }, cargo = { features = "all", extraEnv = { VIRTUAL_ENV = "/.venv", CC = "clang", CXX = "clang++", }, }, check = { workspace = false, command = "check", features = "all", extraEnv = { VIRTUAL_ENV = "/.venv", CC = "clang", CXX = "clang++", }, }, runnables = { extraEnv = { VIRTUAL_ENV = "/.venv", CC = "clang", CXX = "clang++", }, }, testExplorer = true, }, }, }, } ``` # FFI Memory Contract Source: https://nautilustrader.io/docs/latest/developer_guide/ffi/ NautilusTrader exposes several **C-compatible** types so that compiled Rust code can be consumed from C-extensions generated by Cython or by other native languages. The most important of these is `CVec` – a *thin* wrapper around a Rust `Vec` that is passed across the FFI boundary **by value**. The rules below are *strict*; violating them results in undefined behaviour (usually a double-free or a memory leak). ## Fail-fast panics at the FFI boundary Rust panics must never unwind across `extern "C"` functions. Unwinding into C or Python is undefined behaviour and can corrupt the foreign stack or leave partially-dropped resources behind. To enforce the fail-fast architecture we wrap every exported symbol in `crate::ffi::abort_on_panic`, which executes the body and calls `process::abort()` if a panic occurs. The panic message is still logged before the abort, so debugging output is preserved while avoiding undefined behaviour. When adding new FFI functions, call `abort_on_panic(|| { … })` around the implementation (or use a helper that does so) to maintain this guarantee. ## CVec lifecycle | Step | Owner | Action | |-------|-------------------------------|--------| | **1** | Rust | Build a `Vec` and convert it with `into()` – this *leaks* the vector and transfers ownership of the raw allocation to foreign code. | | **2** | Foreign (Python / Cython / C) | Use the data while the `CVec` value is in scope. **Do not modify the fields `ptr`, `len`, `cap`.** | | **3** | Foreign | Exactly once, call the *type‑specific* drop helper exported by Rust (for example `vec_drop_book_levels`, `vec_drop_book_orders`, `vec_time_event_handlers_drop`). The helper reconstructs the original `Vec` with `Vec::from_raw_parts` and lets it drop, freeing the memory. | :::warning If step **3** is forgotten the allocation is leaked for the remainder of the process; if it is performed **twice** the program will double-free and likely crash. ::: ## Typed CVec wrappers and Send `CVec` is untyped ownership metadata. Do not implement `Send` for the raw `CVec` type: it can represent a `Vec` for any `T`, including non-`Send` element types. When PyO3 requires `Send` for a capsule payload, introduce a narrow wrapper for the concrete payload type and put the `unsafe impl Send` on that wrapper only after documenting the payload invariant. For example, DataFFI streaming capsules use `DataFfiCVec`, a transparent wrapper around `CVec` whose allocation always comes from `Vec`. ## Capsules created on the Python side Several Cython routines allocate temporary C buffers with `PyMem_Malloc`, wrap them into a `CVec`, and return the address inside a `PyCapsule`. **Every such capsule is created with a destructor** (`capsule_destructor` or `capsule_destructor_deltas`) that frees both the buffer and the `CVec`. Callers must therefore *not* free the memory manually – doing so would double free. ## Capsules created on the Rust side *(PyO3 bindings)* When Rust code pushes a heap-allocated value into Python and Python becomes the final owner, it **must** use `PyCapsule::new_with_destructor` so that Python knows how to free the allocation once the capsule becomes unreachable. The closure/destructor is responsible for reconstructing the original `Box` or `Vec` and letting it drop. ```rust use pyo3::types::PyCapsule; Python::attach(|py| { // Allocate the value on the heap let my_data = Box::new(MyStruct::new()); let ptr = Box::into_raw(my_data); // Move it into the capsule and register a destructor that frees the memory let capsule = PyCapsule::new_with_destructor( py, ptr, None, |ptr, _| { // Reconstruct the Box and let it drop, freeing the allocation let _ = unsafe { Box::from_raw(ptr) }; }, ) .expect("capsule creation failed"); // ... pass `capsule` back to Python ... }); ``` Do **not** use `PyCapsule::new(…, None)`; that variant registers *no* destructor and will leak memory unless the recipient manually extracts and frees the pointer. ### Rust-owned CVec capsules with explicit drop Rust-owned `CVec` batch capsules are an explicit exception to the destructor-owned pattern above. Use this pattern only when the Python/Cython consumer must first extract the batch into Python objects and then release the Rust allocation explicitly. Requirements for this pattern: 1. Wrap the raw `CVec` in a type-specific capsule payload, such as `DataFfiCVec`. 2. Mark that wrapper `#[repr(transparent)]` over `CVec`, or use `#[repr(C)]` with `CVec` as the first field, before casting capsule pointers back to `*mut CVec`. 3. Give the capsule a stable, explicit name, such as `nautilus.DataFFI.CVec`. Do not use the default unnamed capsule for this pattern. 4. Require all consumers to check the same capsule name before reading the pointer. 5. Expose one type-specific drop function, such as `drop_cvec_pycapsule`. 6. Call that drop function only for capsules created as `CVec` batches. Never pass a single-value capsule, such as one created by `data_to_pycapsule`, to a `CVec` drop function. 7. Validate `len <= cap`, reject null non-empty pointers, and handle empty `CVec` values. 8. Reset the stored `CVec` metadata to `CVec::empty()` before calling `Vec::from_raw_parts`, so cleanup paths can call the drop function more than once without double-freeing. 9. Add tests for wrong capsule names, invalid metadata, empty capsules, and repeated drops. ## Why there is no generic `cvec_drop` anymore Earlier versions of the codebase shipped a generic `cvec_drop` function that always treated the buffer as `Vec`. Using it with any other element type causes a size-mismatch during deallocation and corrupts the allocator's bookkeeping. Because the helper was not referenced anywhere inside the project it has been removed to avoid accidental misuse. Instead, use the **type-specific** drop helper for your element type (e.g., `vec_drop_book_levels`, `vec_drop_book_orders`). If no helper exists for your type, add one following the pattern in `crates/core/src/ffi/cvec.rs`. ## Box-backed `*_API` wrappers (owned Rust objects) When the Rust core needs to hand a *complex* value (for example an `OrderBook`, `SyntheticInstrument`, or `TimeEventAccumulator`) to foreign code it allocates the value on the heap with `Box::new` and returns a small `repr(C)` wrapper whose only field is that `Box`. ```rust #[repr(C)] pub struct OrderBook_API(Box); #[unsafe(no_mangle)] pub extern "C" fn orderbook_new(id: InstrumentId, book_type: BookType) -> OrderBook_API { OrderBook_API(Box::new(OrderBook::new(id, book_type))) } #[unsafe(no_mangle)] pub extern "C" fn orderbook_drop(book: OrderBook_API) { drop(book); // frees the heap allocation } ``` Memory-safety requirements are therefore: 1. Every constructor (`*_new`) **must** have a matching `*_drop` exported next to it. 2. Validate parameters before heap allocation to fail fast and avoid allocating invalid objects. 3. The *Python/Cython* binding must guarantee that `*_drop` is invoked exactly once. Two approaches exist: • **Preferred for new code**: Wrap the pointer in a `PyCapsule` created with `PyCapsule::new_with_destructor`, passing a destructor that calls the drop helper. • **Legacy pattern** (v1 Cython modules only): Call the helper explicitly in `__del__`/`__dealloc__` on the Python side: ```python cdef class OrderBook: cdef OrderBook_API _mem def __cinit__(self, ...): self._mem = orderbook_new(...) def __del__(self): if self._mem._0 != NULL: orderbook_drop(self._mem) ``` Whichever style is used, remember: **forgetting the drop call leaks the entire structure**, while calling it twice will double-free and crash. New FFI code must use `PyCapsule` with destructors and follow this template before it can be merged. # Developer Guide Source: https://nautilustrader.io/docs/latest/developer_guide/ Guidance on developing and extending NautilusTrader to meet your trading needs or to contribute improvements back to the project. The core is written in Rust. Python serves as the control plane for strategy logic, configuration, and orchestration. PyO3 bridges the two, exposing Rust functionality to Python with minimal overhead. } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> } /> # Python Source: https://nautilustrader.io/docs/latest/developer_guide/python/ The [Python](https://www.python.org/) programming language is used for the majority of user-facing code in NautilusTrader. Python provides a rich ecosystem of libraries and frameworks, making it ideal for strategy development, data analysis, and system integration. ## Code style ### PEP-8 The codebase generally follows the PEP-8 style guide. One notable departure is that Python truthiness is not always taken advantage of to check if an argument is `None` for everything other than collections. As per the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html), it's discouraged to use truthiness to check if an argument is/is not `None`, when there is a chance an unexpected object could be passed into the function or method which will yield an unexpected truthiness evaluation (which could result in a logical error type bug). *"Always use if foo is None: (or is not None) to check for a None value. E.g., when testing whether a variable or argument that defaults to None was set to some other value. The other value might be a value that's false in a boolean context!"* :::note Use truthiness to check for empty collections (e.g., `if not my_list:`) rather than comparing explicitly to `None` or empty. ::: We welcome all feedback on where the codebase departs from PEP-8 for no apparent reason. ### Type hints All function and method signatures *must* include type annotations: ```python def __init__(self, config: EMACrossConfig) -> None: def on_bar(self, bar: Bar) -> None: def on_save(self) -> dict[str, bytes]: def on_load(self, state: dict[str, bytes]) -> None: ``` **Union syntax**: Use PEP 604 union syntax for optional types: ```python # Preferred def get_instrument(self, id: InstrumentId) -> Instrument | None: # Avoid def get_instrument(self, id: InstrumentId) -> Optional[Instrument]: ``` **Generic types**: Use `TypeVar` for reusable components: ```python T = TypeVar("T") class ThrottledEnqueuer(Generic[T]): ``` ### Docstrings The [NumPy docstring spec](https://numpydoc.readthedocs.io/en/latest/format.html) is used throughout the codebase. This needs to be followed consistently so the docs build correctly. **Python** docstrings should be written in the **imperative mood** – e.g. *"Return a cached client."* This convention aligns with the prevailing style of the Python ecosystem and makes generated documentation feel natural to end-users. #### Private methods Do not add docstrings to private methods (prefixed with `_`): - Docstrings generate public-facing API documentation. - Docstrings on private methods incorrectly imply they are part of the public API. - Private methods are implementation details not intended for end-users. Exceptions where docstrings are acceptable: - Very complex methods with non-trivial logic, multiple steps, or important edge cases. - Methods requiring detailed parameter or return value documentation due to complexity. When a private method needs context (such as a tricky precondition or side effect), prefer a short inline comment (`#`) near the relevant logic rather than a docstring. ### Properties vs methods (PyO3 bindings) When exposing Rust types to Python via PyO3, use `#[getter]` (property) or a plain method based on what the call site communicates, not whether the value can change: - **Property (`#[getter]`):** cheap, side-effect-free, attribute-like view of current state. Scalar fields, predicates, and lightweight derived values belong here even if they change over the object's lifetime. Examples: `status`, `side`, `quantity`, `price`, `is_open`, `has_inputs`, `realized_pnl`, `venue_order_id`. - **Method (no `#[getter]`):** actions, mutations, nontrivial work, allocations/copies, I/O, or anything that takes arguments. Examples: `apply(fill)`, `unrealized_pnl(price)`, `calculate_pnl(...)`. - **Gray area (prefer method):** getters that clone or allocate a collection each call. Using a method signals the cost to the caller. Examples: `events()`, `adjustments()`, `client_order_ids()`, `trade_ids()`. ## Python v2 live callback routing Python v2 live nodes keep one runtime invariant: Tokio worker threads do not run Python code during live trading. `LiveNode::py_run` releases the GIL while the Rust async runtime runs. Worker-side work that must trigger Python uses existing live runner event channels instead of calling `Python::attach` on the worker. Timer callbacks use the time-event channel. The runner drains that channel during startup buffering and the main select loop, then executes callbacks on the live event loop thread. This path is a boundary for unavoidable user Python callback work. It is not a place to move adapter, provider, data, or execution logic into Python. Python v2 adapter modules configure Rust adapters and register factories; Rust owns adapter operations. If worker-side Rust work needs a Python callback, route it through a specific event type that belongs in the live runner. When adding Python-aware live code: - Prefer an existing runner event channel. - Keep callback bodies short because they run synchronously on the live event loop. - Do not call `Python::attach` from Tokio worker tasks in Python v2 live trading. - Do not add adapter business logic in Python to fit callback routing. Legacy Cython `LiveClock` callbacks are a separate FFI path. They use capsule-style callback arguments for v1 compatibility and can be created without a live runner sender. Keep that ABI distinct until time event dispatch can be unified across v1 and v2. ### Test naming Descriptive names explaining the scenario: ```python def test_currency_with_negative_precision_raises_overflow_error(self): def test_sma_with_no_inputs_returns_zero_count(self): def test_sma_with_single_input_returns_expected_value(self): ``` ### Ruff [ruff](https://astral.sh/ruff) is used to lint the codebase. Ruff rules can be found in the top-level `pyproject.toml`, with ignore justifications typically commented. ## Cython (legacy) :::note This section covers Cython conventions for `.pyx` and `.pxd` files. ::: For `.pyx` and `.pxd` files, make sure all functions and methods returning `void` or a primitive C type (such as `bint`, `int`, `double`) include the `except *` keyword in the signature. Without it, Python exceptions are silently ignored. For more information, see the [Cython docs](https://cython.readthedocs.io/en/latest/index.html). # Release Security Architecture Source: https://nautilustrader.io/docs/latest/developer_guide/release_security/ This page describes the security model for the NautilusTrader release pipeline. It explains how release artifacts are built, published, attested, and verified. Use this page with: - [Releases](releases.md), which documents the release workflow and checklist. - [Security Policy](https://github.com/nautechsystems/nautilus_trader/blob/develop/SECURITY.md), which gives consumer-facing verification commands. - [.github/OVERVIEW.md](https://github.com/nautechsystems/nautilus_trader/blob/develop/.github/OVERVIEW.md#security), which documents CI/CD controls. ## Security goals The release pipeline has four goals: - Build every official artifact from a reviewed repository commit. - Publish Python and Rust packages without long-lived package registry tokens. - Attach checksums, manifests, and provenance before publishing the GitHub release. - Give users enough public data to verify that downloaded artifacts match the release. The GitHub release anchors package integrity. Stable releases attach wheel and sdist assets to a draft GitHub release before any package index publish starts. The pipeline publishes package indexes, verifies those indexes against the GitHub release assets, attaches final integrity assets, then publishes the GitHub release. ## Threat model The pipeline defends against: - Compromised or mutable third-party GitHub Actions, by pinning actions to commit SHAs. - Accidental release from the wrong workflow, branch, or environment, by binding OIDC publishers to `nautechsystems/nautilus_trader`, `build.yml`, and the `release` environment. - Long-lived package registry token theft, by using PyPI and crates.io Trusted Publishing. - Registry propagation lag or partial re-runs, by making publish and verify scripts idempotent and retry-tolerant. - Registry substitution or upload drift, by comparing PyPI and crates.io artifacts against release manifests and registry metadata. - Silent manual crate recovery, by requiring explicit `CRATES_IO_MANUAL_PUBLISH_EXCEPTIONS` entries and recording those exceptions in `crates-manifest.json`. The pipeline does not defend against: - A malicious maintainer with permission to change release workflows and approve releases. - A compromise of GitHub, PyPI, crates.io, or Sigstore that can forge the trust roots users rely on. - A compromised end-user machine before verification runs. - Runtime compromise of an exchange, broker, data provider, or user trading strategy. - Bit-identical rebuild drift for wheels and sdists. The current guarantee is provenance and digest verification, not reproducible builds. ## Trust roots - GitHub repository rules protect reviewed source, release branches, and release tags. Protected `master` and immutable `v*` release tags are the relevant records. - GitHub Actions OIDC issuer provides short-lived workflow identities from `https://token.actions.githubusercontent.com`. - GitHub `release` environment gates package publishing and release approvals. The environment restricts deployment to `master` and requires reviewer approval. - PyPI Trusted Publishing publishes wheels and sdist without a persistent token. It binds to repository `nautechsystems/nautilus_trader`, workflow `build.yml`, and environment `release`. - crates.io Trusted Publishing publishes Rust crates without a persistent token. It binds to owner `nautechsystems`, repository `nautilus_trader`, workflow `build.yml`, and environment `release`. - Sigstore Fulcio, Rekor, and TUF bind artifacts to OIDC identities and the transparency log. GitHub artifact attestations, PyPI publish attestations, and Docker cosign signatures rely on this root. - GitHub release immutability prevents post-publish asset and tag replacement. Published release assets and the release tag become immutable. ## Release flow ```mermaid flowchart TD source["Reviewed commit on master"] gates["Release gates
cargo-deny + cargo-vet"] build["Build wheels and sdist"] draft["Create tag and draft GitHub release"] assets["Attach wheels and sdist to draft release"] registries["Publish PyPI and crates.io
Trusted Publishing"] verify["Verify registries against release assets"] integrity["Attach SHA256SUMS, manifests,
Sigstore bundles, DSSE envelopes"] publish["Publish GitHub release"] release_attest["Verify GitHub release attestation"] docker["Build, sign, and attest Docker images"] source --> gates source --> build gates --> draft build --> draft draft --> assets assets --> registries registries --> verify verify --> integrity integrity --> publish publish --> release_attest source --> docker ``` The Docker workflow is separate from the package release workflow, but it follows the same identity model: image signatures and SBOM attestations bind the image digest to the expected GitHub Actions workflow identity. ## Artifact records - Python wheels are published to GitHub Releases, PyPI, and the Nautech Systems package index (`packages.nautechsystems.io`). `SHA256SUMS`, per-asset `.sha256` files, and `dist-manifest.json` record integrity. GitHub artifact attestations, PyPI publish attestations, `.sigstore` bundles, and `.intoto.jsonl` envelopes record provenance. - Python sdists use the same public locations, integrity records, and provenance records as wheels. - Rust crates are published to crates.io. The crates.io checksum and `crates-manifest.json` record integrity. crates.io `trustpub_data` records provenance unless an explicit manual exception is present. - Docker images are published to GitHub Container Registry. The image digest is the integrity record. Sigstore cosign signatures and SPDX SBOM attestations record provenance. - The GitHub release record is published through GitHub Releases. Published release assets and the immutable tag record integrity. The GitHub release attestation records provenance. ## Consumer verification map Detailed commands live in [Verifying releases](https://github.com/nautechsystems/nautilus_trader/blob/develop/SECURITY.md#verifying-releases). The checks below show the public data each consumer should verify. ### Python wheels and sdist Verify: - The artifact digest matches `SHA256SUMS`, the per-asset `.sha256` file, or `dist-manifest.json`. - The GitHub artifact attestation identity matches `nautechsystems/nautilus_trader/.github/workflows/build.yml` on `master` or `nightly`. - The PyPI publish attestation reports repository `nautechsystems/nautilus_trader`, workflow `build.yml`, and environment `release`. Example: ```bash export TAG=v1.228.0 export REPO=nautechsystems/nautilus_trader export ARTIFACT=nautilus_trader-1.228.0.tar.gz export ISSUER=https://token.actions.githubusercontent.com export IDENTITY='^https://github\.com/nautechsystems/nautilus_trader/\.github/workflows/build\.yml@refs/heads/(master|nightly)$' gh release download "$TAG" --repo "$REPO" --pattern "$ARTIFACT" --pattern "$ARTIFACT.sha256" sha256sum -c "$ARTIFACT.sha256" gh attestation verify "$ARTIFACT" \ --repo "$REPO" \ --cert-identity-regex "$IDENTITY" \ --cert-oidc-issuer "$ISSUER" ``` ### PyPI publish provenance Verify: - PyPI file hashes match `dist-manifest.json`. - PyPI provenance exposes the expected GitHub publisher identity. - `pypi-attestations verify` accepts the downloaded file URL. Example: ```bash export VERSION=1.228.0 export ARTIFACT=nautilus_trader-1.228.0.tar.gz export PYPI_URL=$(curl -sS "https://pypi.org/pypi/nautilus_trader/$VERSION/json" | \ jq -r --arg artifact "$ARTIFACT" '.urls[] | select(.filename == $artifact) | .url') uv run --no-project --no-build --with pypi-attestations -- \ pypi-attestations verify pypi \ --repository https://github.com/nautechsystems/nautilus_trader \ "$PYPI_URL" ``` ### Rust crates Verify: - The crates.io version checksum matches the downloaded `.crate` file. - `trustpub_data.provider` is `github`. - `trustpub_data.repository` is `nautechsystems/nautilus_trader`. - `published_by` is `null`, unless `crates-manifest.json` records an explicit `manual_token_publish` exception. Example: ```bash export CRATE=nautilus-core export VERSION=0.58.0 export REPO=nautechsystems/nautilus_trader export VERSION_JSON=$(curl -sS "https://crates.io/api/v1/crates/$CRATE/versions" | \ jq -c --arg version "$VERSION" '.versions[] | select(.num == $version)') export CRATE_SHA256=$(printf '%s\n' "$VERSION_JSON" | jq -r '.checksum') printf '%s\n' "$VERSION_JSON" | jq -e --arg repo "$REPO" \ '.trustpub_data.provider == "github" and .trustpub_data.repository == $repo and .published_by == null' curl -sSL "https://static.crates.io/crates/$CRATE/$CRATE-$VERSION.crate" -o "$CRATE-$VERSION.crate" test "$(sha256sum "$CRATE-$VERSION.crate" | cut -d ' ' -f 1)" = "$CRATE_SHA256" ``` ### Docker images Verify: - The mutable tag resolves to the digest you intend to run. - The cosign signature identity matches the Docker workflow. - The SPDX SBOM attestation is bound to the same image digest. Example: ```bash export IMAGE_BASE=ghcr.io/nautechsystems/nautilus_trader export DIGEST=$(crane digest "$IMAGE_BASE:latest") export IMAGE=$IMAGE_BASE@$DIGEST export ISSUER=https://token.actions.githubusercontent.com export IDENTITY='^https://github\.com/nautechsystems/nautilus_trader/\.github/workflows/docker\.yml@refs/heads/(master|nightly)$' cosign verify "$IMAGE" --certificate-identity-regexp "$IDENTITY" --certificate-oidc-issuer "$ISSUER" cosign verify-attestation \ --type https://spdx.dev/Document/v2.3 \ "$IMAGE" \ --certificate-identity-regexp "$IDENTITY" \ --certificate-oidc-issuer "$ISSUER" ``` ## Manual recovery posture Normal releases use Trusted Publishing only. Manual package publishing is a last-resort recovery path after a partial release. Rules for manual recovery: - Prefer re-running the failed job or workflow when a registry or Sigstore verifier fails. - Do not replace a release tag or GitHub release assets after publication. - Do not silently accept manually published crates. - If a crate must be recovered with a token, list each `crate@version` in `CRATES_IO_MANUAL_PUBLISH_EXCEPTIONS`. - Record the exception in release notes and in `crates-manifest.json` with `release_status: "manual_token_publish"`. No routine release path depends on a long-lived PyPI or crates.io token. ## Incident response posture - PyPI publisher drift is detected by the PyPI provenance verifier. Stop publishing, fix the PyPI Trusted Publisher, and rerun verification. - crates.io publisher drift is detected by the trusted-publishing check or registry verifier. Fix crate publisher settings and rerun. Use a manual exception only for partial recovery. - GitHub release asset mismatch is detected by checksum or manifest verification. Stop the release before publication, or publish an advisory if assets already shipped. - Sigstore, Rekor, or TUF lag is detected by retryable transparency errors. Retry with bounded backoff, then pause release sealing if lag persists. - Sigstore trust root concern appears when attestation verification becomes ambiguous. Pause releases, verify against registry records, and rotate trust roots when supported. - Workflow identity mismatch is detected by GitHub, PyPI, or cosign identity checks. Treat it as configuration drift or compromise until reviewed. - Manual crate publish exceptions are detected when crates.io shows `published_by` instead of `trustpub_data`. Record the explicit exception, document affected crates, and preserve the audit trail. ## SLSA posture Python release artifacts carry build provenance through GitHub artifact attestations and PyPI publish attestations. Docker images carry Sigstore signatures and SPDX SBOM attestations. Rust crates rely on crates.io Trusted Publishing metadata and the release `crates-manifest.json`. This page does not assert a named SLSA level for all artifact classes. Any future SLSA level claim must cite this architecture, name the artifact classes it covers, and include CI validation that the published provenance parses as the claimed predicate type. # Releases Source: https://nautilustrader.io/docs/latest/developer_guide/releases/ This guide covers the release process and the standards for writing release notes. ## Overview NautilusTrader uses a three-branch model: - **`develop`**: active development; publishes dev wheels to Cloudflare R2 on every push. - **`nightly`**: pre-release testing; publishes alpha wheels and CLI binaries. - **`master`**: stable releases; triggers the full release pipeline. Pushing to `master` automatically tags the version from `pyproject.toml`, creates a draft GitHub release, uploads release assets, publishes Cargo crates to crates.io, publishes wheels and sdist to PyPI, publishes the GitHub release, builds Docker images, and triggers a docs rebuild. ## Stable release workflow The `build` workflow treats the GitHub release as the anchor for stable releases. It creates the release as a draft first, uploads the wheel and sdist assets to that draft release, and only then publishes those packages to package indexes. The workflow publishes the GitHub release only after the registry verification and final integrity assets are complete. ```mermaid flowchart TD push["Push to master"] wheels["Build wheel artifacts
Linux x86/ARM, macOS, Windows"] audits["Release gates
cargo-deny + cargo-vet"] tag["tag-release
Create tag and draft GitHub release"] wheel_assets["publish-wheels-master
Upload wheels to GitHub release and R2
release env"] build_sdist["build-sdist
Build sdist workflow artifact"] sdist_asset["upload-sdist-release
Upload sdist to GitHub release"] crates["publish-cargo-crates
crates.io Trusted Publishing
release env"] wheel_pypi["publish-wheels-pypi
Attest and publish wheels to PyPI
release env"] sdist_pypi["publish-sdist-pypi
Attest and publish sdist to PyPI
release env"] integrity["publish-release-integrity
Checksums and registry verification
Attestation siblings and cleanup"] publish_release["publish-github-release
Publish draft release
Verify release attestation"] push --> wheels push --> audits wheels --> tag audits --> tag tag --> build_sdist build_sdist --> sdist_asset tag --> sdist_asset tag --> wheel_assets wheels --> wheel_assets sdist_asset --> wheel_assets wheel_assets --> wheel_pypi wheel_assets --> crates wheel_pypi --> sdist_pypi sdist_asset --> sdist_pypi crates --> integrity wheel_pypi --> integrity sdist_pypi --> integrity tag --> integrity integrity --> publish_release tag --> publish_release ``` Keep these sequencing rules intact when editing `.github/workflows/build.yml`: - The draft GitHub release must exist before any release asset upload or package registry publish. - Wheel and sdist assets must be attached to the GitHub release before package index publishing starts (`packages.nautechsystems.io`, PyPI, crates.io). - PyPI and crates.io Trusted Publishing jobs must keep `environment: release` and `id-token: write`; those registrations depend on the `release` environment. - Non-OIDC integrity and asset-upload jobs should avoid `environment: release` unless they need release environment secrets or approvals. - `publish-release-integrity` must run after PyPI and crates.io publishing. It generates the release manifest first, verifies registries against that manifest, then attaches final integrity assets only after verification passes. - `publish-github-release` must be the final stable release job. GitHub recommends creating a draft release, attaching all assets, then publishing the draft before enabling release immutability. Once GitHub release immutability is enabled for the repo, published release assets and the release tag cannot be changed; only the title and release notes remain editable. The job verifies the final draft asset set before publishing and verifies GitHub's release attestation after publishing the draft. ## Versioning The project maintains two version numbers: | File | Scope | Example | |--------------------------|----------------|-----------| | `pyproject.toml` | Python package | `1.223.0` | | `Cargo.toml` (workspace) | Rust crates | `0.55.0` | These are bumped independently. The Python version drives the release tag (`v1.223.0`). ## Crates.io publishing The `build` workflow publishes Cargo crates from the `publish-cargo-crates` job. The job uses crates.io Trusted Publishing through GitHub Actions OIDC, so it does not use a persistent cargo token. Configure each crate on crates.io with: | Field | Value | |-------------|-------------------| | Owner | `nautechsystems` | | Repository | `nautilus_trader` | | Workflow | `build.yml` | | Environment | `release` | Enable Trusted Publishing Only for crates after their trusted publisher is configured. Crates that have never been published still need an initial manual publish before crates.io allows the trusted publisher configuration. Do not use `cargo publish --workspace` for CI releases. The release job runs `scripts/ci/publish-cargo-crates.sh`, which publishes crates one at a time in dependency order, skips versions already present on crates.io, and waits for each new version to appear in the crates.io API and sparse index before publishing dependents. The script fails before uploading if a publishable crate depends on a local `publish = false` crate that is absent from crates.io. Optional local dependencies count as blockers because publishing a public feature that resolves to an absent crate would leave that feature unusable. Post-publish verification treats an existing crate version as `previously_published` only when crates.io shows it was trusted-published by this repository. It still fails for user-published crate versions unless `CRATES_IO_MANUAL_PUBLISH_EXCEPTIONS` names each recovered `crate@version` entry for emergency token-publish recovery. Accepted manual entries are recorded in `crates-manifest.json` with `release_status: "manual_token_publish"`, and malformed or unused exception entries fail the job. Wrong trusted-publishing repositories and checksum or sparse-index mismatches also fail. ## Release checklist ### Pre-release (on `develop`) - [ ] Finalize `RELEASES.md`: review all items, remove empty sections - [ ] Ensure versions are set in `pyproject.toml` and `Cargo.toml` workspace - [ ] Ensure crates.io Trusted Publishing is configured for every crate that CI publishes: `bash scripts/ci/check-crates-io-trusted-publishing.sh` - [ ] Ensure all CI checks pass on `develop` ### Release - [ ] Merge `develop` into `nightly`, verify nightly CI passes - [ ] Merge `nightly` into `master` - [ ] Verify the `build` workflow completes: - Wheels built for Linux x86/ARM, macOS, Windows - `cargo-deny` and `cargo-vet` pass - Release docs/features and Cargo publish preflights pass before tagging - Tag and draft GitHub release created - Wheels and sdist attached to the GitHub release before package registry publishing - Cargo crates published to crates.io or skipped because the version already exists - Wheels and sdist published to PyPI - Registry verification passes before release checksums, crates manifest, and attestation siblings are attached - GitHub release published after all release assets and integrity assets are attached - [ ] Verify the `docker` workflow completes (images built and pushed) - [ ] Verify the `build-docs` workflow completes (docs rebuild triggered) ### Post-release (on `develop`) - [ ] Update the release date in `RELEASES.md` for the published version - [ ] Add horizontal separator `---` below the completed release - [ ] Add the next version template at the top of `RELEASES.md` (see below) - [ ] Bump `pyproject.toml` version to the next release number - [ ] Bump crate versions in tutorial and how-to `Cargo.toml` snippets (`docs/concepts/rust.md`, `docs/how_to/run_rust_backtest.md`, `docs/how_to/run_rust_live_trading.md`) ## Release notes This section documents the standards for writing release notes in `RELEASES.md`. ### Sections Use the following sections in this order: 1. Enhancements 2. Breaking Changes 3. Security 4. Fixes 5. Internal Improvements 6. Documentation Updates 7. Deprecations Omit sections that have no items for a given release. ### Enhancements New features and user-visible improvements. **Format**: ```markdown - Added `subscribe_order_fills(...)` and `unsubscribe_order_fills(...)` for `Actor` - Added BitMEX conditional orders support - Added support for `OrderBookDepth10` requests (#2955), thanks @faysou ``` **Guidelines**: - Start with "Added". - Use backticks for code elements. - Be specific about what was added, not how. ### Breaking Changes Changes that may break existing code. **Format**: ```markdown - Removed `nautilus_trader.analysis.statistics` subpackage - must import from `nautilus_trader.analysis` - Renamed `BinanceAccountType.USDT_FUTURE` to `USDT_FUTURES` - Changed `start` parameter to required for `Actor` data request methods ``` **Guidelines**: - Start with "Removed", "Renamed", or "Changed". - Explain migration path briefly. ### Security Security hardening and fixes that prevent crashes, undefined behavior, or data corruption. Includes significant hardening improvements elevated from Internal Improvements. **Format**: ```markdown - Fixed non-executable stack for Cython extensions to support hardened Linux systems - Fixed divide-by-zero and overflow bugs in model crate that could cause crashes - Fixed core arithmetic operations to reject NaN/Infinity values and improve overflow handling ``` **Guidelines**: - Include overflow/underflow fixes, memory safety improvements, FFI guards, data integrity fixes. - Focus on user impact: what could have happened. - Exclude routine dependency updates, minor hardening, or test-only fixes. - Omit this section entirely if there are no security items for the release. ### Fixes Bug fixes that improve correctness but don't qualify as security issues. **Format**: ```markdown - Fixed reduce-only order panic when quantity exceeds position - Fixed Binance order status parsing for external orders (#3006), thanks for reporting @bmlquant ``` **Guidelines**: - Start with "Fixed". ### Internal Improvements Implementation details and infrastructure changes. **Format**: ```markdown - Added ARM64 support to Docker builds - Ported `PortfolioAnalyzer` to Rust - Improved clock and timer thread safety - Upgraded Rust (MSRV) to 1.90.0 - Upgraded `pyo3` crates to v0.26.0 ``` **Guidelines**: - Use "Added", "Implemented", "Improved", "Optimized", "Upgraded", "Refined", "Standardized". - Include version numbers for dependency upgrades. ### Documentation Updates Changes to guides and examples. **Format**: ```markdown - Added rate limit tables with links to official docs - Improved dark and light themes for readability - Fixed broken links ``` ### Deprecations Features marked for removal. **Format**: ```markdown - Deprecated `some_config_option`; disable (`False`) to maintain consistent behaviour. Will be removed in future version ``` **Guidelines**: - Explain migration path and provide alternatives. ## Attribution - Credit external contributors: `thanks @username` or `thanks for reporting @username`. - Include issue/PR numbers for community contributions and complex features: `(#1234)`. ## Style - Use sentence case (capitalize first word only). - Do not end with periods. - Use backticks for code elements. - Focus on **what** changed, not how. **Be specific**: ```markdown ❌ Improved Binance adapter ✅ Improved Binance fill handling when instrument not cached ``` ## Security classification Include in Security if the change addresses: - Memory safety (overflow, underflow, divide-by-zero that threatens stability). - Undefined behavior or crashes that could corrupt state. - Data integrity (NaN/Infinity propagation, race conditions leading to corruption). - Input validation preventing injection or exploitation (SQL injection, command injection, path traversal). - Build hardening (non-exec stack, FFI guards). - Significant hardening that users should know about. Otherwise use Fixes (for logic bugs and panics) or Internal Improvements (for minor hardening). Note: Plain logic panics belong in Fixes unless they threaten system stability or data corruption. ## Examples **Security** (could cause crashes/corruption): ```markdown - Fixed divide-by-zero in margin calculations that could crash the engine - Fixed non-executable stack for Cython extensions to support hardened systems ``` **Fixes** (incorrect but safe): ```markdown - Fixed Binance order status parsing for external orders - Fixed position purge logic to prevent purging re-opened position ``` **Enhancements** (user-facing): ```markdown - Added BitMEX conditional orders support ``` **Internal** (implementation): ```markdown - Implemented BitMEX ping/pong handling ``` ## Release notes template ```markdown # NautilusTrader Beta Released on TBD (UTC). ### Enhancements ### Breaking Changes ### Security ### Fixes ### Internal Improvements ### Documentation Updates ### Deprecations --- ``` # Rust Source: https://nautilustrader.io/docs/latest/developer_guide/rust/ The [Rust](https://www.rust-lang.org/learn) programming language is an ideal fit for implementing the mission-critical core of the platform and systems. Its strong type system, ownership model, and compile-time checks eliminate memory errors and data races by construction, while zero-cost abstractions and the absence of a garbage collector deliver C-like performance, important for high-frequency trading workloads. ## Cargo manifest conventions - In `[dependencies]`, list internal crates (`nautilus-*`) first in alphabetical order, insert a blank line, then external required dependencies alphabetically, followed by another blank line and the optional dependencies (those with `optional = true`) in alphabetical order. Preserve inline comments with their dependency. - Add `"python"` to every `extension-module` feature list that builds a Python artefact, keeping it adjacent to `"pyo3/extension-module"` so the full Python stack is obvious. - When a manifest groups adapters separately (for example `crates/pyo3`), keep the `# Adapters` block immediately below the internal crate list so downstream consumers can scan adapter coverage quickly. - Always include a blank line before `[dev-dependencies]` and `[build-dependencies]` sections. - Apply the same layout across related manifests when the feature or dependency sets change to avoid drift between crates. - Use snake_case filenames for `bin/` sources (for example `bin/ws_data.rs`) and reflect those paths in each `[[bin]]` section. - Keep `[[bin]] name` entries in kebab-case (for example `name = "hyperliquid-ws-data"`) so the compiled binaries retain their intended CLI names. ## Versioning guidance - Use workspace inheritance for shared dependencies (for example `serde = { workspace = true }`). - Only pin versions directly for crate-specific dependencies that are not part of the workspace. - Group workspace-provided dependencies before crate-only dependencies so the inheritance is easy to audit. - Keep related dependencies aligned: `capnp`/`capnpc` (exact), `arrow`/`parquet` (major.minor), `datafusion`/`object_store`, and `dydx-proto`/`prost`/`tonic`. Pre-commit enforces this. - Adapter-only dependencies belong in the "Adapter dependencies" section of the workspace `Cargo.toml`. Pre-commit prevents core crates from using them. ## Feature flag conventions - Prefer additive feature flags. Enabling a feature must not break existing functionality. - Use descriptive flag names that explain what capability is enabled. - Document every feature in the crate-level documentation so consumers know what they toggle. - Common patterns: - `high-precision`: switches the value-type backing (64-bit or 128-bit integers) to support domains that require extra precision. - `default = []`: keep defaults minimal. - `python`: enables Python bindings. - `extension-module`: builds a Python extension module (always include `python`). - `ffi`: enables C FFI bindings. - `stubs`: exposes testing stubs. ## Build configurations To avoid unnecessary rebuilds during development, align cargo features, profiles, and flags across different build targets. Cargo's build cache is keyed by the exact combination of features, profiles, and flags. Any mismatch triggers a full rebuild. ### Aligned targets (testing and linting) | Target | Features | Profile | `--all-targets` | `--no-deps` | Purpose | |-----------------------------|----------------------------------|-----------|-----------------|-------------|----------------| | `cargo-test` | `ffi,python,high-precision,defi` | `nextest` | ✓ (implicit) | n/a | Run tests. | | `cargo-clippy` (pre‑commit) | `ffi,python,high-precision,defi` | `nextest` | ✓ | n/a | Lint all code. | These targets share the same feature set and profile, allowing cargo to reuse compiled artifacts between linting and testing without rebuilds. The `nextest` profile is used to align with the workflow of the majority of core maintainers who use cargo-nextest for running tests. ### Documentation builds Documentation is built separately using `make docs-rust`, which runs: ```bash cargo +nightly doc --all-features --no-deps --workspace ``` This uses the nightly toolchain and `--all-features` rather than the aligned feature set above, so it does not share build artifacts with testing/linting. ### Separate target (Python extension building) | Target | Features | Profile | Notes | |---------------|--------------------------------------|-----------|-------| | `build` | Includes `extension-module` + subset | `release` | Requires different features for PyO3 extension module. | | `build-debug` | Includes `extension-module` + subset | `dev` | Requires different features for PyO3 extension module. | Python extension building intentionally uses different features (`extension-module` is required) and will trigger rebuilds. This is expected and unavoidable. ### Rebuild triggers to avoid Mismatches in any of these cause full rebuilds: - Different feature combinations (e.g., `--features "a,b"` vs `--features "a,c"`). - Different `--no-default-features` usage (enables/disables default features). - Different profiles (e.g., `dev` vs `nextest` vs `release`). When adding new build targets or modifying existing ones, maintain alignment with the testing/linting group to preserve fast incremental builds. ### Generated FFI bindings and precision mode The `nautilus-model` build script regenerates `nautilus_trader/core/includes/model.h` and `nautilus_trader/core/rust/model.pxd` when the `ffi` feature is enabled. Those files encode whether the generated C/Cython bindings use high precision. The committed generated files use high precision. Local cargo commands that compile `nautilus-model` with `ffi` should either include the `high-precision` feature or avoid regenerating those files. Make targets that use `BASE_FEATURES`, such as `make build-debug-v2`, already include `high-precision`. The drift risk mainly comes from ad-hoc cargo commands that enable `ffi` without the aligned feature set. Use the Rust feature for narrow checks that do not include the full aligned feature set. Keep the environment override in the command so a stale shell value cannot force standard-precision bindings: ```bash env HIGH_PRECISION=true cargo check -p nautilus-model --features ffi,python,high-precision ``` Before committing FFI-related work, verify those generated files did not drift: ```bash git diff -- nautilus_trader/core/includes/model.h nautilus_trader/core/rust/model.pxd ``` If they changed only because a command ran without high precision, rerun the cargo command with `HIGH_PRECISION=true`. Do not hand-edit the generated files. ## Module organization - Keep modules focused on a single responsibility. - Use `mod.rs` as the module root when defining submodules. - Prefer relatively flat hierarchies over deep nesting to keep paths manageable. - Re-export commonly used items from the crate root for convenience. ## Code style and conventions ### File header requirements All Rust files must include the standardized copyright header: ```rust // ------------------------------------------------------------------------------------------------- // Copyright (C) 2015-2026 Nautech Systems Pty Ltd. All rights reserved. // https://nautechsystems.io // // Licensed under the GNU Lesser General Public License Version 3.0 (the "License"); // You may not use this file except in compliance with the License. // You may obtain a copy of the License at https://www.gnu.org/licenses/lgpl-3.0.en.html // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // ------------------------------------------------------------------------------------------------- ``` :::info[Automated enforcement] The `check_copyright_year.sh` pre-commit hook verifies copyright headers include the current year. ::: ### Code formatting Import formatting is automatically handled by rustfmt when running `make format`. The tool organizes imports into groups (standard library, external crates, local imports) and sorts them alphabetically within each group. Within this section, follow these spacing rules: - Leave **one blank line between functions** (including tests) – this improves readability and mirrors the default behavior of `rustfmt`. - Leave **one blank line above every doc comment** (`///` or `//!`) so that the comment is clearly detached from the previous code block. #### String formatting Prefer inline format strings over positional arguments: ```rust // Preferred - inline format with variable names anyhow::bail!("Failed to subtract {n} months from {datetime}"); // Instead of - positional arguments anyhow::bail!("Failed to subtract {} months from {}", n, datetime); ``` This makes messages more readable and self-documenting, especially when there are multiple variables. ### Type qualification Follow these conventions for qualifying types in code: - **anyhow**: Always fully qualify `anyhow` macros (`anyhow::bail!`, `anyhow::anyhow!`) and the Result type (`anyhow::Result`). - **Nautilus domain types**: Do not fully qualify Nautilus domain types. Use them directly after importing (e.g., `Symbol`, `InstrumentId`, `Price`). - **tokio**: Generally fully qualify `tokio` types as they can have equivalents in std library and other crates (e.g., `tokio::spawn`, `tokio::time::timeout`). ```rust use nautilus_model::identifiers::Symbol; pub fn process_symbol(symbol: Symbol) -> anyhow::Result<()> { if !symbol.is_valid() { anyhow::bail!("Invalid symbol: {symbol}"); } tokio::spawn(async move { // Process symbol asynchronously }); Ok(()) } ``` :::info[Automated enforcement] The `check_anyhow_usage.sh` pre-commit hook enforces these anyhow conventions automatically. ::: ### Logging - Fully qualify logging macros so the backend is explicit: - Use `log::…` (`log::debug!`, `log::info!`, `log::warn!`, etc.) for all Rust components. - Start messages with a capitalised word, prefer complete sentences, and omit terminal periods (e.g. `"Processing batch"`, not `"Processing batch."`). :::info[Automated enforcement] The `check_logging_conventions.sh` pre-commit hook enforces fully qualified logging macros. ::: ### Error handling Use structured error handling patterns consistently: 1. **Primary Pattern**: Use `anyhow::Result` for fallible functions: ```rust pub fn calculate_balance(&mut self) -> anyhow::Result { // Implementation } ``` 2. **Custom Error Types**: Use `thiserror` for domain-specific errors: ```rust #[derive(Error, Debug)] pub enum NetworkError { #[error("Connection failed: {0}")] ConnectionFailed(String), #[error("Timeout occurred")] Timeout, } ``` 3. **Error Propagation**: Use the `?` operator for clean error propagation. 4. **Error Creation**: Prefer `anyhow::bail!` for early returns with errors: ```rust // Preferred - using bail! for early returns pub fn process_value(value: i32) -> anyhow::Result { if value < 0 { anyhow::bail!("Value cannot be negative: {value}"); } Ok(value * 2) } // Instead of - verbose return statement if value < 0 { return Err(anyhow::anyhow!("Value cannot be negative: {value}")); } ``` **Note**: Use `anyhow::bail!` for early returns, but `anyhow::anyhow!` in closure contexts like `ok_or_else()` where early returns aren't possible. 5. **Error Context**: Use lowercase for `.context()` messages to support error chaining (except proper nouns/acronyms): ```rust // Good - lowercase chains naturally parse_timestamp(value).context("failed to parse timestamp")?; // Exception - proper nouns stay capitalized connect().context("BitMEX websocket did not become active")?; ``` :::info[Automated enforcement] The `check_error_conventions.sh` and `check_anyhow_usage.sh` pre-commit hooks enforce these error handling patterns. ::: ### Async patterns Use consistent async/await patterns: 1. **Async function naming**: No special suffix is required; prefer natural names. 2. **Tokio usage**: Fully qualify tokio types (e.g., `tokio::time::timeout`). See [Adapter runtime patterns](#adapter-runtime-patterns) for spawn rules. 3. **Error handling**: Return `anyhow::Result` from async functions to match the synchronous conventions. 4. **Cancellation safety**: Call out whether the function is cancellation-safe and what invariants still hold when it is cancelled. 5. **Stream handling**: Use `tokio_stream` (or `futures::Stream`) for async iterators to make back-pressure explicit. 6. **Timeout patterns**: Wrap network or long-running awaits with timeouts (`tokio::time::timeout`) and propagate or handle the timeout error. ### Adapter runtime patterns Adapter crates (under `crates/adapters/`) require special handling for spawning async tasks due to Python FFI compatibility: 1. **Use `get_runtime().spawn()` instead of `tokio::spawn()`**: When called from Python threads (which have no Tokio context), `tokio::spawn()` panics because it relies on thread-local storage. The global runtime pattern provides an explicit reference accessible from any thread. ```rust use nautilus_common::live::get_runtime; // Correct - works from Python threads get_runtime().spawn(async move { // async work }); // Incorrect - panics from Python threads tokio::spawn(async move { // async work }); ``` 2. **Use the shorter import path**: Import `get_runtime` from the `live` module re-export, not the full path: ```rust // Preferred - shorter path via re-export use nautilus_common::live::get_runtime; // Avoid - unnecessarily verbose use nautilus_common::live::runtime::get_runtime; ``` 3. **Use `get_runtime().block_on()` for sync-to-async bridges**: When synchronous code needs to call async functions in adapters: ```rust fn sync_method(&self) -> anyhow::Result<()> { get_runtime().block_on(self.async_implementation()) } ``` 4. **Install custom runtimes before first use**: Rust-native binaries that own `main()` may call `set_runtime()` before `LiveNode::build()` or any adapter/client usage. Build custom runtimes with `tokio::runtime::Builder::new_multi_thread().enable_all()`; current-thread runtimes and runtimes without I/O or timer drivers do not satisfy adapter assumptions. If the `python` feature is enabled, prepare Python before building the runtime or keep the default initializer. 5. **Tests are exempt**: Test code using `#[tokio::test]` creates its own runtime context, so `tokio::spawn()` works correctly. The enforcement hook skips test files and test modules. :::info[Automated enforcement] The `check_tokio_usage.sh` pre-commit hook enforces these adapter runtime patterns automatically. ::: ### Attribute patterns Consistent attribute usage and ordering: ```rust #[repr(C)] #[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, PartialOrd, Ord)] #[cfg_attr( feature = "python", pyo3::pyclass(module = "nautilus_trader.model") )] #[cfg_attr( feature = "python", pyo3_stub_gen::derive::gen_stub_pyclass(module = "nautilus_trader.model") )] pub struct Symbol(Ustr); ``` For enums with extensive derive attributes: ```rust #[repr(C)] #[derive( Copy, Clone, Debug, Display, Hash, PartialEq, Eq, PartialOrd, Ord, AsRefStr, FromRepr, EnumIter, EnumString, )] #[strum(ascii_case_insensitive)] #[strum(serialize_all = "SCREAMING_SNAKE_CASE")] #[cfg_attr( feature = "python", pyo3::pyclass( frozen, eq, eq_int, module = "nautilus_trader.model", from_py_object, rename_all = "SCREAMING_SNAKE_CASE", ) )] #[cfg_attr( feature = "python", pyo3_stub_gen::derive::gen_stub_pyclass_enum(module = "nautilus_trader.model") )] pub enum AccountType { /// An account with unleveraged cash assets only. Cash = 1, /// An account which facilitates trading on margin, using account assets as collateral. Margin = 2, } ``` ### Type stub annotations Python type stubs (`.pyi` files) are generated from Rust source using [pyo3-stub-gen](https://github.com/Jij-Inc/pyo3-stub-gen). Every type and function exposed to Python needs a matching stub annotation so the generated stubs stay in sync with the bindings. **Annotation types:** | PyO3 construct | Stub annotation | | ----------------- | ------------------------------------------------ | | `#[pyclass]` | `pyo3_stub_gen::derive::gen_stub_pyclass` | | enum `#[pyclass]` | `pyo3_stub_gen::derive::gen_stub_pyclass_enum` | | `#[pymethods]` | `pyo3_stub_gen::derive::gen_stub_pymethods` | | `#[pyfunction]` | `pyo3_stub_gen::derive::gen_stub_pyfunction` | **Placement rules:** - On structs and enums, use `#[cfg_attr(feature = "python", ...)]` and place the stub annotation directly below the `pyo3::pyclass` attribute. - On `#[pymethods]` impl blocks, place `#[pyo3_stub_gen::derive::gen_stub_pymethods]` directly below `#[pymethods]`. - On functions, place the stub annotation directly above `#[pyfunction]`, after any doc comments. Fully qualify the path rather than importing it. ```rust /// Converts a list of `Bar` into Arrow IPC bytes. #[pyo3_stub_gen::derive::gen_stub_pyfunction(module = "nautilus_trader.serialization")] #[pyfunction(name = "bars_to_arrow")] pub fn py_bars_to_arrow(data: Vec) -> PyResult> { // ... } ``` ```rust #[pymethods] #[pyo3_stub_gen::derive::gen_stub_pymethods] impl AccountState { #[staticmethod] #[pyo3(name = "from_dict")] pub fn py_from_dict(values: &Bound<'_, PyDict>) -> PyResult { // ... } } ``` **Module parameter:** set `module = "nautilus_trader."` to match the Python package where the type is imported. For example, model types use `nautilus_trader.model` and serialization functions use `nautilus_trader.serialization`. **Cargo.toml:** add `pyo3-stub-gen` as an optional dependency and include it in the `python` feature list: ```toml [features] python = ["pyo3", "pyo3-stub-gen"] [dependencies] pyo3-stub-gen = { workspace = true, optional = true } ``` **Regenerating stubs:** run `make py-stubs-v2` (or `python python/generate_stubs.py`) after changing annotations. The post-processor handles `py_` prefix stripping, `@property`/`@staticmethod`/`@classmethod` decoration, keyword escaping, deduplication, and ruff formatting. ### Constructor patterns Use the `new()` vs `new_checked()` convention consistently: ```rust /// Creates a new [`Symbol`] instance with correctness checking. /// /// # Errors /// /// Returns an error if `value` is not a valid string. /// /// # Notes /// /// PyO3 requires a `Result` type for proper error handling and stacktrace printing in Python. pub fn new_checked>(value: T) -> CorrectnessResult { // Implementation } /// Creates a new [`Symbol`] instance. /// /// # Panics /// /// Panics if `value` is not a valid string. pub fn new>(value: T) -> Self { Self::new_checked(value).expect_display(FAILED) } ``` Always use the `FAILED` constant for `.expect_display()` messages on `CorrectnessResult`, and import the trait that provides it: ```rust use nautilus_core::correctness::{CorrectnessResult, CorrectnessResultExt, FAILED}; ``` #### Fluent builders for many-optional constructors Types with large constructors dominated by optional fields (the `instruments` domain types) also expose a fluent `bon` builder, so callers set only the fields they need instead of passing a long run of `None`. Put `#[bon::bon]` on the inherent impl and add a builder method that delegates to `new_checked`, which keeps a single validated construction path: ```rust #[bon::bon] impl CryptoPerpetual { // new_checked / new as above /// Returns a fluent builder for a [`CryptoPerpetual`] instance. /// /// # Errors /// /// Returns an error if any input validation fails (see [`CryptoPerpetual::new_checked`]). #[builder(start_fn = builder, finish_fn = build)] pub fn build_checked(/* same parameters as new_checked */) -> CorrectnessResult { Self::new_checked(/* forward verbatim */) } } ``` Callers write `CryptoPerpetual::builder().instrument_id(..)..build()?`. Required (non-`Option`) parameters are enforced at compile time by bon's typestate; `Option` parameters are omittable and default exactly as `new_checked` applies them. `build()` returns the same `CorrectnessResult` as `new_checked`, so every correctness check still runs. Unlike the test-only event specs, this builder lives on the production type, ships in production builds, and returns a `Result` rather than the value. Keep `new()` and `new_checked()` in place; the builder is additive. ### Type conversion patterns For types that parse from strings, provide both fallible and infallible conversions: 1. **`FromStr`**: Fallible parsing via `.parse()` or `from_str()`. Returns `Result`. 2. **`From>`**: Ergonomic infallible conversion that accepts `&str`, `String`, `Cow`, etc. directly without requiring `.as_str()`. ```rust impl FromStr for Symbol { type Err = SymbolParseError; fn from_str(s: &str) -> Result { // parsing logic } } impl> From for Symbol { fn from(value: T) -> Self { Self::from_str(value.as_ref()).expect(FAILED) } } ``` **Design note**: The `From` impl may panic on invalid input. This is intentional for API ergonomics. Use `FromStr` / `.parse()` when error handling is needed. The `From` impl provides convenience for cases where the input is known to be valid. **Constraint**: This pattern cannot be used for types that implement `AsRef` themselves (e.g., string wrapper types), as it would conflict with the blanket `impl From for T`. For such types, provide separate `From<&str>` and `From` impls instead. ### Constants and naming conventions Use SCREAMING_SNAKE_CASE for constants with descriptive names: ```rust /// Number of nanoseconds in one second. pub const NANOSECONDS_IN_SECOND: u64 = 1_000_000_000; /// Bar specification for 1-minute last price bars. pub const BAR_SPEC_1_MINUTE_LAST: BarSpecification = BarSpecification { step: NonZero::new(1).unwrap(), aggregation: BarAggregation::Minute, price_type: PriceType::Last, }; ``` ### Hash collections Three concerns drive the choice of hash collection: - **Iteration-order determinism** (the primary filter) - **Performance** - **Thread safety** Answer the determinism question first, then pick from the remaining options on performance grounds. #### Iteration-order determinism `AHash` randomizes its hasher per process, so `AHashMap` / `AHashSet` iteration order varies between runs. When the iteration order of a collection feeds observable state on the deterministic simulation testing (DST) path (events emitted on the message bus, ordered `Vec`s returned from public methods, the sequence in which a seeded RNG is consumed, the order in which downstream effects fire), use `IndexMap` / `IndexSet` from the `indexmap` crate instead. They preserve insertion order and are a drop-in replacement for the `AHash*` collections. ```rust use indexmap::{IndexMap, IndexSet}; // Insertion-order iteration; deterministic across runs let mut commissions: IndexMap = IndexMap::new(); let mut subscribed: IndexSet = IndexSet::new(); ``` The pre-commit hook `check-dst-conventions` enforces `IndexMap` / `IndexSet` in `crates/live/src/manager.rs` and `crates/execution/src/matching_engine/engine.rs` because both files were audited as load-bearing for fill ordering and reconciliation. Other call sites are reviewed individually; the closed sites and remaining allowed patterns are listed under "Implementation notes" in [../concepts/dst.md](../concepts/dst.md). When the collection is **lookup-only** (no `.iter()`, `.values()`, `.keys()`, `.into_iter()`, `.drain()`, or `for x in map { ... }`), iteration order is irrelevant and `AHashMap` / `AHashSet` is the right choice on performance grounds. Borderline cases (e.g. a public getter that clones the map and lets callers iterate) should be reviewed against the inventory's classification rules. #### Performance For lookup-heavy hot paths where iteration order does not feed observable state, prefer `AHashMap` / `AHashSet` over the standard library: ```rust use ahash::{AHashMap, AHashSet}; let mut symbols: AHashSet = AHashSet::new(); let mut prices: AHashMap = AHashMap::new(); ``` For non-performance-critical, non-iteration-sensitive cases (factory registries, configuration maps, test fixtures), standard `HashMap` / `HashSet` is acceptable and often preferred for simplicity: ```rust use std::collections::{HashMap, HashSet}; let mut symbols: HashSet = HashSet::new(); let mut prices: HashMap = HashMap::new(); ``` **Why use `ahash`?** - **Superior performance**: AHash uses AES-NI hardware instructions when available, providing 2-3x faster hashing compared to the default SipHash. - **Low collision rates**: Despite being non-cryptographic, AHash provides excellent distribution and low collision rates for typical data. - **Drop-in replacement**: Fully compatible API with standard library collections. **When to use standard `HashMap`/`HashSet`:** - **Non-performance-critical code**: For simple cases where performance is not critical (e.g., factory registries, configuration maps, test fixtures), standard `HashMap`/`HashSet` are acceptable and even preferred for simplicity. - **Cryptographic security required**: Use standard `HashMap` when hash flooding attacks are a concern (e.g., handling untrusted user input in network protocols). - **Network clients**: Prefer standard `HashMap` for network-facing components where security considerations outweigh performance benefits. - **External library boundaries**: Use standard `HashMap` when interfacing with external libraries that expect it (e.g., Arrow serialization metadata). #### AHashMap vs IndexMap microbenchmarks The numbers below come from `crates/core/benches/hash_map.rs` (release profile). Times are per operation; ratio is `IndexMap` relative to `AHashMap` (values below 1.0 favour `IndexMap`). | Pattern | Size | AHashMap | IndexMap | Ratio | |-----------------------|-----:|---------:|---------:|------:| | Insert (build map) | 4 | 40.8 ns | 49.8 ns | 1.22x | | Insert (build map) | 32 | 192.4 ns | 348.2 ns | 1.81x | | Insert (build map) | 256 | 1.01 us | 2.74 us | 2.72x | | Lookup (random get) | 4 | 2.56 ns | 9.36 ns | 3.66x | | Lookup (random get) | 32 | 2.49 ns | 7.95 ns | 3.19x | | Lookup (random get) | 256 | 3.00 ns | 9.48 ns | 3.16x | | `.values().collect()` | 4 | 8.08 ns | 6.61 ns | 0.82x | | `.values().collect()` | 32 | 22.8 ns | 14.8 ns | 0.65x | | `.values().collect()` | 256 | 145 ns | 109 ns | 0.75x | | `.keys().collect()` | 4 | 7.90 ns | 6.24 ns | 0.79x | | `.keys().collect()` | 32 | 23.0 ns | 12.6 ns | 0.55x | | `.keys().collect()` | 256 | 145 ns | 101 ns | 0.70x | | Clone | 4 | 8.48 ns | 17.8 ns | 2.10x | | Clone | 32 | 25.3 ns | 62.5 ns | 2.47x | | Clone | 256 | 71.0 ns | 247 ns | 3.48x | | Entry accumulate | 4 | 122 ns | 159 ns | 1.30x | | Entry accumulate | 32 | 439 ns | 1.10 us | 2.51x | | Entry accumulate | 256 | 2.21 us | 7.83 us | 3.54x | For one-key removal, `IndexMap` exposes two methods: `shift_remove` preserves insertion order at `O(n)` cost; `swap_remove` is `O(1)` but swaps the last entry into the removed slot, breaking iteration order. | Pattern | Size | AHashMap.remove | IndexMap.shift_remove | IndexMap.swap_remove | |------------|-----:|----------------:|----------------------:|---------------------:| | Remove one | 4 | 9.89 ns | 37.8 ns | 37.1 ns | | Remove one | 32 | 62.0 ns | 117 ns | 53.4 ns | | Remove one | 256 | 70.3 ns | 355 ns | 269 ns | How to read the table: - `AHashMap` is roughly 3x faster on pure lookup. Keep `AHashMap` on hot lookup paths where iteration order does not flow into observable state. - `IndexMap` is 25 to 45 percent faster on `.values().collect()` and `.keys().collect()`. Where iteration drives observable state, the flip to `IndexMap` is a small performance win as well as a determinism win. - `IndexMap` is 1.3 to 3.5x slower on insert, clone, and entry-modify-or-insert. Keep `AHashMap` on construction-heavy or per-fill accumulation paths. - Prefer `swap_remove` over `shift_remove` when iteration order does not matter after the removal; it stays competitive with `AHashMap` removal. ### Thread-safe hash map patterns `AHashMap` is not thread-safe. Wrapping it in `Arc` only enables sharing the pointer across threads but does not coordinate mutation. Use `Arc` only when the map is immutable after construction, otherwise add proper synchronization. ```rust // Avoid: Data races when multiple threads mutate let cache = Arc::new(AHashMap::new()); let cache_clone = Arc::clone(&cache); tokio::spawn(async move { cache_clone.insert(key, value); // Data race }); cache.insert(other_key, other_value); // Data race ``` **Patterns:** 1. **Immutable after construction** – Build the map once, then share it read-only: ```rust let mut map = AHashMap::new(); map.insert(key1, value1); map.insert(key2, value2); let shared_map = Arc::new(map); // Now immutable // Multiple threads can safely read let map_clone = Arc::clone(&shared_map); tokio::spawn(async move { if let Some(value) = map_clone.get(&key1) { // Safe read-only access } }); ``` 2. **Concurrent reads and writes** – Use `DashMap`: ```rust use dashmap::DashMap; let cache: Arc> = Arc::new(DashMap::new()); // Multiple threads can safely read and write concurrently cache.insert(key, value); if let Some(entry) = cache.get(&key) { // Safe concurrent access } ``` `DashMap` internally uses sharding and fine-grained locking for efficient concurrent access. 3. **Single-threaded hot paths** – Use plain `AHashMap` in single-threaded contexts: ```rust struct Handler { instruments: AHashMap, } impl Handler { async fn next(&mut self) -> Option<()> { // Handler runs on a single task, no concurrent access self.instruments.insert(key, value); Ok(()) } } ``` **Decision tree:** 1. Iteration order observable on the DST path? Use `IndexMap` / `IndexSet` 2. Otherwise, by access pattern: - Immutable after construction: use `Arc>` - Concurrent access needed: use `Arc>` - Single-threaded access: use plain `AHashMap` ### Shared mutability storage Code ported from Cython often cloned values out of a container before mutating them. That pattern produces silent staleness: the local clone diverges from the canonical entry the moment another code path applies an event to it. Reach for `Rc>` (single-threaded) or `Arc>` (multi-threaded) storage only when all three hold: - The value is mutated after insertion. - Multiple holders need to observe each other's writes. - A handle must outlive the container's borrow scope. Orders in `Cache` use this shape internally for per-key borrow tracking. Storage is `AHashMap>`; the smart-pointer leak stays internal. Public accessors return scoped newtypes that hide it: `Cache::order` returns `OrderRef<'_>` (read borrow), `Cache::order_mut` returns `OrderRefMut<'_>` (exclusive write borrow, requires `&mut Cache`), and `Cache::order_owned` returns an owned `OrderAny` snapshot when a value must cross a boundary. Use `Cache::try_order` or `Cache::try_order_owned` when a missing order is an error; they return `OrderLookupError` instead of forcing each caller to build an ad hoc not-found error. Engines drop the borrow before dispatching events and re-read the cache for post-event state, which keeps the dispatch a clean transaction boundary. `Cache::order_mut` takes `&mut Cache`, which means strategies and adapters receiving a `CacheView` (which only exposes immutable cache borrows) cannot reach it. Order mutation is reserved for the data and execution engines that hold the cache directly; the type system enforces that contract. Otherwise prefer the simpler shape: - Read-mostly and set once: `Rc` or `Arc` (no interior mutability). - Owned snapshots suffice for callers: store `T`, clone on read. - Single owner, no mutation: plain field. Costs of `Rc>` worth weighing before adopting it: - Every access pays a runtime borrow check. - The smart-pointer type leaks at write boundaries. - Misuse panics at runtime instead of failing to compile. - `Rc>` is `!Send`; cross-thread storage needs `Arc>` (or `Arc>` when reads are rare). **Decision tree:** 1. Mutable, multi-observer, and handle outlives container borrow? - Single-threaded: `Rc>`. - Multi-threaded: `Arc>` (or `Arc>` when reads are rare). 2. Read-mostly and set once: `Rc` or `Arc`. 3. Owned snapshots fine: store `T`, clone on read. 4. Single owner, no mutation: plain field. ### Re-export patterns Organize re-exports alphabetically and place at the end of lib.rs files: ```rust // Re-exports pub use crate::{ nanos::UnixNanos, time::AtomicTime, uuid::UUID4, }; // Module-level re-exports pub use crate::identifiers::{ account_id::AccountId, actor_id::ActorId, client_id::ClientId, }; ``` ### Documentation standards Use third-person declarative voice for all doc comments (e.g., "Returns the account ID" not "Return the account ID"). #### Section header casing Rustdoc section headers use Title Case, matching the Rust standard library convention: - `# Examples` - `# Errors` - `# Panics` - `# Safety` - `# Notes` - `# Thread Safety` - `# Feature Flags` #### Module-Level documentation All modules must have module-level documentation starting with a brief description: ```rust //! Functions for correctness checks similar to the *design by contract* philosophy. //! //! This module provides validation checking of function or method conditions. //! //! A condition is a predicate which must be true just prior to the execution of //! some section of code - for correct behavior as per the design specification. ``` For modules with feature flags, document them clearly: ```rust //! # Feature flags //! //! This crate provides feature flags to control source code inclusion during compilation, //! depending on the intended use case: //! //! - `ffi`: Enables the C foreign function interface (FFI) from [cbindgen](https://github.com/mozilla/cbindgen). //! - `python`: Enables Python bindings from [PyO3](https://pyo3.rs). //! - `extension-module`: Builds as a Python extension module (used with `python`). //! - `stubs`: Enables type stubs for use in testing scenarios. ``` #### Field documentation All struct and enum fields must have documentation with terminating periods: ```rust pub struct Currency { /// The currency code as an alpha-3 string (e.g., "USD", "EUR"). pub code: Ustr, /// The currency decimal precision. pub precision: u8, /// The ISO 4217 currency code. pub iso4217: u16, /// The full name of the currency. pub name: Ustr, /// The currency type, indicating its category (e.g. Fiat, Crypto). pub currency_type: CurrencyType, } ``` #### Function documentation Document all public functions with: - Purpose and behavior - Explanation of input argument usage - Error conditions (if applicable) - Panic conditions (if applicable) ```rust /// Returns a reference to the `AccountBalance` for the specified currency, or `None` if absent. /// /// # Panics /// /// Panics if `currency` is `None` and `self.base_currency` is `None`. pub fn base_balance(&self, currency: Option) -> Option<&AccountBalance> { // Implementation } ``` #### Errors and panics documentation format For single line errors and panics documentation, use sentence case with the following convention: ```rust /// Returns a reference to the `AccountBalance` for the specified currency, or `None` if absent. /// /// # Errors /// /// Returns an error if the currency conversion fails. /// /// # Panics /// /// Panics if `currency` is `None` and `self.base_currency` is `None`. pub fn base_balance(&self, currency: Option) -> anyhow::Result> { // Implementation } ``` For multi-line errors and panics documentation, use sentence case with bullets and terminating periods: ```rust /// Calculates the unrealized profit and loss for the position. /// /// # Errors /// /// Returns an error if: /// - The market price for the instrument cannot be found. /// - The conversion rate calculation fails. /// - Invalid position state is encountered. /// /// # Panics /// /// This function panics if: /// - The instrument ID is invalid or uninitialized. /// - Required market data is missing from the cache. /// - Internal state consistency checks fail. pub fn calculate_unrealized_pnl(&self, market_price: Price) -> anyhow::Result { // Implementation } ``` #### Safety documentation format For Safety documentation, use the `SAFETY:` prefix followed by a short description explaining why the unsafe operation is valid: ```rust /// Creates a new instance from raw components without validation. /// /// # Safety /// /// The caller must ensure that all input parameters are valid and properly initialized. pub unsafe fn from_raw_parts(ptr: *const u8, len: usize) -> Self { // SAFETY: Caller guarantees ptr is valid and len is correct Self { data: std::slice::from_raw_parts(ptr, len), } } ``` For inline unsafe blocks, use the `SAFETY:` comment directly above the unsafe code: ```rust impl Send for MessageBus { fn send(&self) { // SAFETY: Message bus is not meant to be passed between threads unsafe { // unsafe operation here } } } ``` ## Python bindings Python bindings are provided via [PyO3](https://pyo3.rs), allowing users to import NautilusTrader crates directly in Python without a Rust toolchain. ### PyO3 naming conventions When exposing Rust functions to Python **via PyO3**: 1. The Rust symbol **must** be prefixed with `py_*` to make its purpose explicit inside the Rust codebase. 2. Use the `#[pyo3(name = "…")]` attribute to publish the *Python* name **without** the `py_` prefix so the Python API remains clean. ```rust #[pyo3(name = "do_something")] pub fn py_do_something() -> PyResult<()> { // … } ``` :::info[Automated enforcement] The `check_pyo3_conventions.sh` pre-commit hook enforces the `py_` prefix for PyO3 functions. ::: ### PyO3 enum conventions Enums exposed to Python should use the following `pyclass` attributes: - `frozen`: enums are immutable value types. - `eq, eq_int`: enables equality with other enum instances and integer discriminants. - `rename_all = "SCREAMING_SNAKE_CASE"`: standardizes Python variant names. - `from_py_object`: enables conversion from Python objects. :::warning[Do not use the `hash` pyclass attribute with `eq_int` enums] PyO3's auto-generated `__hash__` uses Rust's `DefaultHasher`, which produces different values than Python's `hash()` on the equivalent integer. Since `eq_int` makes `MyEnum.VARIANT == 1` true, the hash contract (`a == b` implies `hash(a) == hash(b)`) would be violated. Instead, provide a manual `__hash__` returning the discriminant directly: ::: ```rust #[pymethods] impl MyEnum { const fn __hash__(&self) -> isize { *self as isize } } ``` ### Testing conventions - Use `mod tests` as the standard test module name unless you need to specifically compartmentalize. - Use `#[rstest]` attributes consistently, this standardization reduces cognitive overhead. - Do *not* use Arrange, Act, Assert separator comments in Rust tests. :::info[Automated enforcement] The `check_testing_conventions.sh` pre-commit hook enforces the use of `#[rstest]` over `#[test]`. ::: #### Parameterized testing Use the `rstest` attribute consistently, and for parameterized tests: ```rust #[rstest] #[case("AUDUSD", false)] #[case("AUD/USD", false)] #[case("CL.FUT", true)] fn test_symbol_is_composite(#[case] input: &str, #[case] expected: bool) { let symbol = Symbol::new(input); assert_eq!(symbol.is_composite(), expected); } ``` #### Test specs (bon builders) For events with many constructor arguments, the canonical test builder is a fluent spec defined alongside the event under `events//spec/.rs` (see `crates/model/src/events/order/spec/filled.rs` for the reference implementation). Gate the spec module with `#[cfg(any(test, feature = "stubs"))]` so it is available to in-crate tests and to downstream crates that opt in with the `stubs` feature, but compiled out of production builds. Specs must not be referenced from production code. Why a custom spec instead of `derive_builder::Builder` with `builder(default)`: the latter bypasses the production constructor, so invariants added later are not exercised by tests. A spec funnels through the production constructor on every `build()`. Anatomy: - Derive `bon::Builder` with `finish_fn = into_spec` so the generated finish method does not collide with the custom `build()`. - Mark every required field `#[builder(default = ...)]` with a literal or a `TestDefault::test_default()` call. Leave optional fields as `Option` without a default so callers either set them or accept `None`. - Default event ID fields to `test_uuid()` from `crate::stubs`. This yields distinct, reproducible UUIDs without callers managing state. - Implement `build()` on the generated builder so it calls `into_spec()` and forwards through the production constructor (e.g. `OrderFilled::new`). The return type is the event itself, not a `Result`, because spec defaults are valid by construction. Caller usage: ```rust let fill = OrderFilledSpec::builder() .last_qty(Quantity::from(50_000)) .trade_id(TradeId::from("TRADE-1")) .build(); ``` Override only the fields the test cares about; the rest take spec defaults. Do not write `.unwrap()` after `build()`. Determinism: under `cargo nextest` each test runs in a fresh process, so the per-thread UUID sequence resets automatically. Under plain `cargo test`, call `reset_test_uuid_rng()` from `crate::stubs` at the start of any test that compares UUID sequences across draws. Pin spec defaults with a single test in the spec module so accidental drift in any field surfaces there rather than as silent behavior change in downstream tests. #### Property-based testing Use the `proptest` crate for property-based tests. Place these in a separate `property_tests` module (not inside `mod tests`) to keep deterministic unit tests separate from randomized property tests: ```rust #[cfg(test)] mod property_tests { use proptest::prelude::*; use rstest::rstest; use super::*; // Define strategies for generating test inputs fn my_strategy() -> impl Strategy { prop_oneof![ Just(MyType::VariantA), Just(MyType::VariantB), ] } fn value_strategy() -> impl Strategy { prop_oneof![ -1000.0..1000.0, Just(0.0), ] } // Group all property tests inside the proptest! macro proptest! { #[rstest] fn prop_construction_roundtrip( value in value_strategy(), variant in my_strategy() ) { // Test invariants that should hold for all generated inputs } } } ``` Conventions: - Name the module `property_tests`, separate from `mod tests`. - Import `proptest::prelude::*` and `rstest::rstest`. - Define strategy functions returning `impl Strategy`. - Combine value ranges with edge cases using `prop_oneof!`. - Filter invalid combinations with `prop_filter_map`. - Prefix test names with `prop_`. - Mark each test inside `proptest!` with `#[rstest]`. #### Test naming Use descriptive test names that explain the scenario: ```rust fn test_sma_with_no_inputs() fn test_sma_with_single_input() fn test_symbol_is_composite() ``` ### Box-style banner comments Do not use box-style banner or separator comments. If code requires visual separation, consider splitting it into separate modules or files. Instead use: - Clear function names that convey purpose. - Module structure for logical groupings (`mod tests { mod fixtures { } }`). - Impl blocks to group related methods. - Doc comments (`///`) for semantic documentation. - IDE navigation and code folding. Patterns to avoid: ```rust // ============================================================================ // Some Section // ============================================================================ // ========== Test Fixtures ========== ``` ## Rust-Python memory management When working with PyO3 bindings, it's critical to understand and avoid reference cycles between Rust's `Arc` reference counting and Python's garbage collector. This section documents best practices for handling Python objects in Rust callback-holding structures. ### The reference cycle problem **Problem**: Using `Arc` in callback-holding structs creates circular references: 1. **Rust `Arc` holds Python objects** -> increases Python reference count. 2. **Python objects might reference Rust objects** -> creates cycles. 3. **Neither side can be garbage collected** -> memory leak. **Example of problematic pattern**: ```rust // AVOID: This creates reference cycles struct CallbackHolder { handler: Option>, // ❌ Arc wrapper causes cycles } ``` ### The solution: GIL-based cloning **Solution**: Use plain `PyObject` with proper GIL-based cloning via `clone_py_object()`: ```rust use nautilus_core::python::clone_py_object; // CORRECT: Use plain PyObject without Arc wrapper struct CallbackHolder { handler: Option, // ✅ No Arc wrapper } // Manual Clone implementation using clone_py_object impl Clone for CallbackHolder { fn clone(&self) -> Self { Self { handler: self.handler.as_ref().map(clone_py_object), } } } ``` ### Best practices #### 1. Use `clone_py_object()` for Python object cloning ```rust // When cloning Python callbacks let cloned_callback = clone_py_object(&original_callback); // In manual Clone implementations self.py_handler.as_ref().map(clone_py_object) ``` #### 2. Remove `#[derive(Clone)]` from callback-holding structs ```rust // BEFORE: Automatic derive causes issues with PyObject #[derive(Clone)] // ❌ Remove this struct Config { handler: Option, } // AFTER: Manual implementation with proper cloning struct Config { handler: Option, } impl Clone for Config { fn clone(&self) -> Self { Self { // Clone regular fields normally url: self.url.clone(), // Use clone_py_object for Python objects handler: self.handler.as_ref().map(clone_py_object), } } } ``` #### 3. Update function signatures to accept `PyObject` ```rust // BEFORE: Arc wrapper in function signatures fn spawn_task(handler: Arc) { ... } // ❌ // AFTER: Plain PyObject fn spawn_task(handler: PyObject) { ... } // ✅ ``` #### 4. Avoid `Arc::new()` when creating Python callbacks ```rust // BEFORE: Wrapping in Arc let callback = Arc::new(py_function); // ❌ // AFTER: Use directly let callback = py_function; // ✅ ``` ### Why this works The `clone_py_object()` function: - **Acquires the Python GIL** before performing clone operations. - **Uses Python's native reference counting** via `clone_ref()`. - **Avoids Rust Arc wrappers** that interfere with Python GC. - **Maintains thread safety** through proper GIL management. This approach allows both Rust and Python garbage collectors to work correctly, eliminating memory leaks from reference cycles. ## Design by contract Design by contract states the obligations between a function and its callers: - **Preconditions**: what the function requires from callers. - **Postconditions**: what the function guarantees in return. - **Invariants**: what properties its type maintains across calls. Prefer the type system first. Ownership, lifetimes, `Send`/`Sync`, `Result`/`Option`, exhaustive matching, newtypes, and visibility encode most contracts at compile time and cost nothing at runtime. Use runtime checks only where the type system cannot. For most preconditions, use the `nautilus_core::correctness` module: it is the project's design-by-contract mechanism and should be the default. `check_*` functions (`check_predicate_true`, `check_valid_string_ascii`, `check_positive_u64`, `check_in_range_inclusive_f64`, `check_equal_usize`, `check_key_in_map`, ...) return a typed `CorrectnessResult<()>` whose `CorrectnessError` variants name each kind of violation. Pair `new_checked()` (fallible, returns `CorrectnessResult`) with a `new()` wrapper that panics via `.expect_display(FAILED)` for validated types; this is the [Constructor patterns](#constructor-patterns) convention and produces panic messages prefixed with `Condition failed: ...`. Use `debug_assert!` (and `debug_assert_eq!`/`_ne!`) for *internal* invariants the correctness module does not model: field relationships, monotonic sequences, CAS postconditions, encode/decode round-trips, provably in-range indices, and preconditions on internal helpers that trusted upstream validation. Release builds strip the check, so never use `debug_assert!` for public API input. For `unsafe` code, use always-on `assert!` for soundness-critical preconditions (null, alignment, provenance) and reserve `debug_assert!` for hot-path preconditions upheld by design. Choosing a mechanism: | Situation | Use | |--------------------------------------------------------------------|---------------------------------------------------| | Public API input against named preconditions | `check_*` from `nautilus_core::correctness` | | Validated constructors (fallible + panic pair) | `new_checked()` / `new()` | | Recoverable non‑validation errors (I/O, parse, network) | `Result` | | Internal invariant the compiler cannot prove | `debug_assert!` | | Always‑on internal invariant without a matching `CorrectnessError` | `assert!` | | Soundness‑critical `unsafe` precondition | `assert!` (always on) | | Hot‑path `unsafe` precondition upheld by design | `debug_assert!` plus a documented `Safety` clause | Style: - Prefix `debug_assert!` messages with `Invariant:` and state the positive rule, not the failure: `debug_assert!(next > last, "Invariant: time is strictly monotonic across CAS")`. - `Condition failed: ...` (from the `FAILED` constant) marks a caller-supplied input violation; `Invariant: ...` marks an internal contract bug. - Place assertions where the invariant is first assumed. When an invariant holds across a hot loop, assert once at the boundary rather than inside the loop. ## Common anti-patterns 1. **Avoid `.clone()` in hot paths** – favour borrowing or shared ownership via `Arc`. 2. **Avoid `.unwrap()` in production code** – generally propagate errors with `?` or map them into domain errors, but unwrapping lock poisoning is acceptable because it signals a severe program state that should abort fast. 3. **Avoid `String` when `&str` suffices** – minimise allocations on tight loops. 4. **Avoid exposing interior mutability** – hide mutexes/`RefCell` behind safe APIs. 5. **Avoid large structs in `Result`** – box large error payloads (`Box`). ## Unsafe Rust It will be necessary to write `unsafe` Rust code to be able to achieve the value of interoperating between Cython and Rust. The ability to step outside the boundaries of safe Rust is what makes it possible to implement many of the most fundamental features of the Rust language itself, just as C and C++ are used to implement their own standard libraries. Great care will be taken with the use of Rusts `unsafe` facility - which enables a small set of additional language features, thereby changing the contract between the interface and caller, shifting some responsibility for guaranteeing correctness from the Rust compiler, and onto us. The goal is to realize the advantages of the `unsafe` facility, whilst avoiding *any* undefined behavior. The definition for what the Rust language designers consider undefined behavior can be found in the [language reference](https://doc.rust-lang.org/stable/reference/behavior-considered-undefined.html). ### Safety policy To maintain correctness, any use of `unsafe` Rust must follow our policy: - If a function is `unsafe` to call, there *must* be a `Safety` section in the documentation explaining why the function is `unsafe`, covering the invariants which the function expects the callers to uphold, and how to meet their obligations in that contract. - Document why each function is `unsafe` in its doc comment's Safety section, and cover all `unsafe` blocks with unit tests. - Always include a `SAFETY:` comment explaining why the unsafe operation is valid. - **Crate-level lint** – every crate that exposes FFI symbols enables `#![deny(unsafe_op_in_unsafe_fn)]`. Even inside an `unsafe fn`, each pointer dereference or other dangerous operation must be wrapped in its own `unsafe { … }` block. - **CVec contract** – for raw vectors that cross the FFI boundary read the [FFI Memory Contract](ffi.md). Foreign code becomes the owner of the allocation and **must** call the matching `vec_drop_*` function exactly once. ### Categories of unsafe code The codebase uses unsafe Rust in these categories: 1. **FFI boundaries** – Raw pointer operations for C interop. See [FFI documentation](ffi.md). 2. **Interior mutability** – `UnsafeCell` for thread-local registries with controlled access patterns. 3. **Unsafe Send/Sync** – Types that are not inherently thread-safe but satisfy trait bounds through runtime invariants (e.g., single-threaded access guaranteed by architecture). ### Unsafe Send/Sync requirements When implementing `Send` or `Sync` unsafely: 1. Document exactly which fields violate the trait requirements. 2. Explain the runtime mechanism that ensures safety (e.g., single-threaded event loop). 3. Include a `WARNING` stating that violating the invariant is undefined behavior. 4. Prefer runtime enforcement (assertions, `Result` returns) over documentation-only guarantees. ```rust // SAFETY: Contains Rc> which is not thread-safe. // Single-threaded access guaranteed by the backtest engine architecture. // WARNING: Actually sending across threads is undefined behavior. #[allow(unsafe_code)] unsafe impl Send for BacktestDataClient {} ``` ### Defense in depth Where unsafe code relies on invariants, add defense mechanisms: - **Type verification**: Check types at runtime before casting (e.g., `TypeId` comparison). - **Debug assertions**: Catch memory corruption early in debug builds. - **RAII guards**: Ensure cleanup on both normal return and panic paths. - **Runtime checks**: Fail fast when invariants are violated rather than proceeding unsafely. ### Runtime invariants Several core subsystems rely on runtime invariants rather than compile-time guarantees. Tests verify the first three contracts below. The guard usage rules are enforced by convention. Any PR that touches `UnsafeCell`, registries, `unsendable`, or live-node threading should confirm the invariant tests still pass. #### Thread-local registries The actor registry, component registry, and message bus each use `thread_local!` storage. An object registered on one thread is never visible from another. The live node event loop runs on a single thread, and all registry and message bus access happens on that thread. `LiveNodeHandle` is the only intended cross-thread control surface. It uses `Arc` for stop signaling and `Arc` for state, both with `Ordering::Relaxed`. #### Actor registry vs component registry Both registries store `Rc>` in thread-local maps but differ in how they handle aliased access: | Property | Actor registry | Component registry | |-------------------|------------------------------------|------------------------------------| | Aliasing | Allowed (multiple guards) | Prevented (`BorrowGuard` + set) | | Re‑entrant access | Yes, required for callbacks | No, lifecycle ops are sequential | | Error handling | Panic or `None` on lookup failure | Returns `anyhow::Result` on error | | Guard type | `ActorRef` (Rc‑backed) | Stack‑local `BorrowGuard` | The actor registry chooses re-entrant access over aliasing prevention because message handlers frequently call back into the registry to look up other actors. The component registry can enforce strict aliasing because lifecycle operations (start, stop, reset, dispose) are non-re-entrant. #### `ActorRef` usage rules `ActorRef` guards must be: - Obtained and dropped within a single synchronous scope. - Never stored in a struct field. - Never held across an `.await` point. - Never sent to another thread. The canonical pattern captures an actor's `Ustr` ID in a closure and looks up the actor each time the callback fires: ```rust let actor_id = actor.actor_id().inner(); let handler = TypedHandler::from(move |quote: &QuoteTick| { if let Some(mut actor) = try_get_actor_unchecked::(&actor_id) { actor.handle_quote(quote); } }); ``` ## Tooling configuration The project uses several tools for code quality: - **rustfmt**: Automatic code formatting (see `rustfmt.toml`). - **clippy**: Linting and best practices (see `clippy.toml`). When suppressing `missing_panics_doc` or `missing_errors_doc`, include a `reason` explaining why the lint does not apply: ```rust #[allow(clippy::missing_panics_doc, reason = "mutex poisoning is not expected")] ``` - **cbindgen**: C header generation for FFI. ## Rust version management The project pins to a specific Rust version via `rust-toolchain.toml`. **Keep your toolchain synchronized with CI:** ```bash rustup update # Update to latest stable Rust rustup show # Verify correct toolchain is active ``` If pre-commit passes locally but fails in CI, clear the prek cache and re-run: ```bash prek clean # Clear cached environments make pre-commit # Re-run all checks ``` This ensures you're using the same Rust and clippy versions as CI. ## Resources - [The Rustonomicon](https://doc.rust-lang.org/nomicon/) – The Dark Arts of Unsafe Rust. - [The Rust Reference – Unsafety](https://doc.rust-lang.org/stable/reference/unsafety.html). - [Safe Bindings in Rust – Russell Johnston](https://www.abubalay.com/blog/2020/08/22/safe-bindings-in-rust). - [Google – Rust and C interoperability](https://www.chromium.org/Home/chromium-security/memory-safety/rust-and-c-interoperability/). ## Cap'n Proto serialization The `nautilus-serialization` crate provides optional Cap'n Proto serialization support for efficient data interchange. This feature is opt-in to avoid requiring the Cap'n Proto compiler for standard builds. ### Installing Cap'n Proto Install the Cap'n Proto compiler before working with schemas. The required version is specified in `tools.toml` in the repository root. See the [Environment Setup](environment_setup.md#capn-proto) guide for detailed installation instructions for each platform. :::warning Ubuntu's default `capnproto` package is too old. Linux users must install from source. ::: Verify installation: ```bash capnp --version # Should match the version in tools.toml ``` ### Schema development workflow Schema files live in `crates/serialization/schemas/capnp/`: - `common/` - Base types, identifiers, enums. - `commands/` - Trading commands. - `events/` - Order and position events. - `data/` - Market data types. When modifying schemas: 1. Edit the `.capnp` schema file in the appropriate subdirectory. 2. Regenerate Rust bindings: ```bash make regen-capnp # or ./scripts/regen-capnp.sh ``` 3. Review changes: ```bash git diff crates/serialization/generated/capnp ``` 4. Update conversions in `crates/serialization/src/capnp/conversions.rs` if needed. 5. Run tests: ```bash make cargo-test EXTRA_FEATURES="capnp" ``` ### Generated code Generated Rust files are checked into `crates/serialization/generated/capnp/` for these reasons: - **docs.rs compatibility**: The documentation build environment lacks the Cap'n Proto compiler. - **Contributor convenience**: Most developers don't need to install capnp for standard development. - **Build reproducibility**: Ensures consistent code generation across environments. The generated files are automatically created during builds via `build.rs` when the `capnp` feature is enabled, but we commit them to the repository to support builds without the compiler installed. ### Verifying schema consistency Before committing schema changes, ensure generated files are up-to-date: ```bash make check-capnp-schemas ``` This target: 1. Skips with a warning if `capnp` is not installed (acceptable for local development). 2. Fails if regeneration errors occur (e.g., version mismatch). 3. Regenerates schemas and fails if generated files differ from committed versions. CI runs this check automatically to catch drift (capnp is always installed in CI). ### Testing with capnp feature ```bash # Run workspace tests with capnp make cargo-test EXTRA_FEATURES="capnp" # Run specific crate tests with capnp make cargo-test-crate-nautilus-serialization FEATURES="capnp" # Run specific test cargo test -p nautilus-serialization --features capnp test_price_roundtrip ``` ### Schema evolution guidelines When evolving schemas: - **Additive changes only**: Add new fields at the end. - **Never remove fields**: Mark deprecated fields in comments. - **Never reuse field numbers**: Even after deprecation. - **Test roundtrip compatibility**: Ensure old and new versions interoperate. Cap'n Proto's evolution rules allow schema changes without breaking binary compatibility, but you must follow these constraints to maintain forward/backward compatibility. # Data Testing Spec Source: https://nautilustrader.io/docs/latest/developer_guide/spec_data_testing/ This section defines a rigorous test matrix for validating adapter data functionality using the `DataTester` actor. Both Python (`nautilus_trader.test_kit.strategies.tester_data`) and Rust (`nautilus_testkit::testers`) provide the `DataTester`. Each test case is identified by a prefixed ID (e.g. TC-D01) and grouped by functionality. **Each adapter must pass the subset of tests matching its supported data types.** Test groups are ordered from least derived to most derived data: instruments and raw book data first, then quotes, trades, bars, and derivatives data. An adapter that passes groups 1–4 is considered baseline data compliant. Document adapter-specific data behavior (custom channels, throttling, snapshot semantics, etc.) in the adapter's own guide, not here. ## Prerequisites Before running data tests: - Target instrument available and loadable via the instrument provider. - API credentials set via environment variables (`{VENUE}_API_KEY`, `{VENUE}_API_SECRET`) when the venue requires authentication for the data being tested. - If the venue offers a demo/testnet mode, use credentials created for that environment. Demo and production API keys are typically separate and not interchangeable; using the wrong credentials produces authentication errors (e.g. HTTP 401). **Python node setup**: Legacy examples still use `nautilus_trader.live.node.TradingNode`, but new Rust-backed PyO3 adapters should prefer `nautilus_trader.live.LiveNode`. Use `LiveNode.builder(...)` when you need to register adapter client factories before the node is built. ```python from nautilus_trader.common import Environment from nautilus_trader.live import LiveDataEngineConfig, LiveNode from nautilus_trader.model import TraderId node = ( LiveNode.builder("TESTER-001", TraderId("TESTER-001"), Environment.SANDBOX) .with_data_engine_config( LiveDataEngineConfig(time_bars_build_with_no_updates=False) ) .add_data_client(None, adapter_data_client_factory, data_client_config) .build() ) node.add_actor_from_config(importable_actor_config) # Register remaining components, then start or run ``` **Rust node setup** (reference: `crates/adapters/{adapter}/examples/node_data_tester.rs`): ```rust use nautilus_testkit::testers::{DataTester, DataTesterConfig}; let tester_config = DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_quotes(true) .build()?; let tester = DataTester::new(tester_config); node.add_actor(tester)?; node.run().await?; ``` Each group below begins with a summary table, followed by detailed test cards. Test IDs use spaced numbering to allow insertion without renumbering. --- ## Group 1: Instruments Verify instrument loading and subscription before testing market data streams. | TC | Name | Description | Skip when | |---------|-----------------------------|------------------------------------------------------|----------------------| | TC-D01 | Request instruments | Load all instruments for a venue. | Never. | | TC-D02 | Subscribe instrument | Subscribe to instrument updates. | No instrument sub. | | TC-D03 | Load specific instrument | Load a single instrument by ID. | Never. | ### TC-D01: Request instruments | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected. | | **Action** | DataTester requests all instruments for the venue on start. | | **Event sequence** | `on_instruments` callback receives instrument list. | | **Pass criteria** | At least one instrument received; each has valid symbol, price precision, and size increment. | | **Skip when** | Never. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_instruments=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .request_instruments(true) .build()? ``` ### TC-D02: Subscribe instrument | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to instrument updates. | | **Event sequence** | `on_instrument` callback receives instrument. | | **Pass criteria** | Instrument received with correct `instrument_id`, valid fields. | | **Skip when** | Adapter does not support instrument subscriptions. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_instrument=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_instrument(true) .build()? ``` ### TC-D03: Load specific instrument | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected. | | **Action** | Load a specific instrument by `InstrumentId` via the instrument provider. | | **Event sequence** | Instrument available in cache after load. | | **Pass criteria** | Instrument loaded with correct ID, price precision, size increment, and trading rules. | | **Skip when** | Never. | **Considerations:** - This tests the instrument provider's `load` / `load_async` method directly. - Verify the instrument is cached and available via `self.cache.instrument(instrument_id)`. --- ## Group 2: Order book Test order book subscription modes and snapshot requests. | TC | Name | Description | Skip when | |---------|--------------------------------|----------------------------------------------------|------------------------| | TC-D10 | Subscribe book deltas | Stream `OrderBookDeltas` updates. | No book support. | | TC-D11 | Subscribe book at interval | Periodic `OrderBook` snapshots. | No book support. | | TC-D12 | Subscribe book depth | `OrderBookDepth10` snapshots. | No book depth. | | TC-D13 | Request book snapshot | One‑time book snapshot request. | No book snapshot. | | TC-D14 | Managed book from deltas | Build local book from delta stream. | No book support. | | TC-D15 | Request historical book deltas | Historical book deltas request. | No historical deltas. | ### TC-D10: Subscribe book deltas | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to order book deltas. | | **Event sequence** | `OrderBookDeltas` events received in `on_order_book_deltas`. | | **Pass criteria** | Deltas received with valid instrument ID; at least one delta contains bid/ask updates. | | **Skip when** | Adapter does not support order book data. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_book_deltas=True, book_type=BookType.L2_MBP, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_book_deltas(true) .book_type(BookType::L2_MBP) .build()? ``` ### TC-D11: Subscribe book at interval | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to periodic order book snapshots. | | **Event sequence** | `OrderBook` events received in `on_order_book` at configured interval. | | **Pass criteria** | Book snapshots received with bid/ask levels; updates arrive at approximately the configured interval. | | **Skip when** | Adapter does not support order book data. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_book_at_interval=True, book_type=BookType.L2_MBP, book_depth=10, book_interval_ms=1000, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_book_at_interval(true) .book_type(BookType::L2_MBP) .book_depth(10) .book_interval_ms(1000) .build()? ``` ### TC-D12: Subscribe book depth | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to `OrderBookDepth10` snapshots. | | **Event sequence** | `OrderBookDepth10` events received in `on_order_book_depth`. | | **Pass criteria** | Depth snapshots received with up to 10 bid/ask levels; prices are correctly ordered. | | **Skip when** | Adapter does not support book depth subscriptions. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_book_depth=True, book_type=BookType.L2_MBP, book_depth=10, ) ``` **Rust config:** Not yet supported. Book depth subscription is TODO in the Rust `DataTester`. ### TC-D13: Request book snapshot | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester requests a one‑time order book snapshot. | | **Event sequence** | Book snapshot received via historical data callback. | | **Pass criteria** | Snapshot contains bid/ask levels with valid prices and sizes. | | **Skip when** | Adapter does not support book snapshot requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_book_snapshot=True, book_depth=10, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .request_book_snapshot(true) .book_depth(10) .build()? ``` ### TC-D14: Managed book from deltas | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, book deltas streaming. | | **Action** | DataTester subscribes to deltas with `manage_book=True`; builds local order book from the delta stream. | | **Event sequence** | `OrderBookDeltas` applied to local `OrderBook`; book logged with configured depth. | | **Pass criteria** | Local book builds correctly from deltas; bid levels descend, ask levels ascend; book is not empty after initial snapshot. | | **Skip when** | Adapter does not support order book data. | **Considerations:** - The managed book applies each delta to an `OrderBook` instance maintained by the actor. - Use `book_levels_to_print` to control logging verbosity. **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_book_deltas=True, manage_book=True, book_type=BookType.L2_MBP, book_levels_to_print=10, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_book_deltas(true) .manage_book(true) .book_type(BookType::L2_MBP) .build()? ``` ### TC-D15: Request historical book deltas | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester requests historical order book deltas. | | **Event sequence** | Historical deltas received via callback. | | **Pass criteria** | Deltas received with valid timestamps and book actions. | | **Skip when** | Adapter does not support historical book delta requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_book_deltas=True, ) ``` **Rust config:** Not yet supported. Historical book delta requests are TODO in the Rust `DataTester`. --- ## Group 3: Quotes Test quote tick subscriptions and historical requests. | TC | Name | Description | Skip when | |---------|---------------------------|-------------------------------------------------|------------------------| | TC-D20 | Subscribe quotes | Verify `QuoteTick` events flow after start. | Never. | | TC-D21 | Request historical quotes | Request historical quote ticks. | No historical quotes. | ### TC-D20: Subscribe quotes | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to quotes on start. | | **Event sequence** | `QuoteTick` events received in `on_quote_tick`. | | **Pass criteria** | At least one `QuoteTick` received with valid bid/ask prices and sizes; bid < ask. | | **Skip when** | Never. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_quotes=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_quotes(true) .build()? ``` ### TC-D21: Request historical quotes | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester requests historical quote ticks. | | **Event sequence** | Historical quotes received via `on_historical_data` callback. | | **Pass criteria** | Quotes received with valid timestamps, bid/ask prices and sizes. | | **Skip when** | Adapter does not support historical quote requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_quotes=True, requests_start_delta=pd.Timedelta(hours=1), ) ``` --- ## Group 4: Trades Test trade tick subscriptions and historical requests. | TC | Name | Description | Skip when | |--------|---------------------------|-------------------------------------------------|------------------------| | TC-D30 | Subscribe trades | Verify `TradeTick` events flow after start. | Never. | | TC-D31 | Request historical trades | Request historical trade ticks. | No historical trades. | ### TC-D30: Subscribe trades | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to trades on start. | | **Event sequence** | `TradeTick` events received in `on_trade_tick`. | | **Pass criteria** | At least one `TradeTick` received with valid price, size, and aggressor side. | | **Skip when** | Never. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_trades=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_trades(true) .build()? ``` ### TC-D31: Request historical trades | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester requests historical trade ticks. | | **Event sequence** | Historical trades received via `on_historical_data` callback. | | **Pass criteria** | Trades received with valid timestamps, prices, sizes, and trade IDs. | | **Skip when** | Adapter does not support historical trade requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_trades=True, requests_start_delta=pd.Timedelta(hours=1), ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .request_trades(true) .build()? ``` --- ## Group 5: Bars Test bar subscriptions and historical requests. | TC | Name | Description | Skip when | |---------|-------------------------|---------------------------------------------------|---------------------| | TC-D40 | Subscribe bars | Verify `Bar` events flow after start. | No bar support. | | TC-D41 | Request historical bars | Request historical OHLCV bars. | No historical bars. | ### TC-D40: Subscribe bars | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, bar type configured. | | **Action** | DataTester subscribes to bars for a configured `BarType`. | | **Event sequence** | `Bar` events received in `on_bar`. | | **Pass criteria** | At least one `Bar` received with valid OHLCV values; high >= low, high >= open, high >= close. | | **Skip when** | Adapter does not support bar subscriptions. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], bar_types=[BarType.from_str("BTCUSDT-PERP.VENUE-1-MINUTE-LAST-EXTERNAL")], subscribe_bars=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .bar_types(vec![bar_type]) .subscribe_bars(true) .build()? ``` ### TC-D41: Request historical bars | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, bar type configured. | | **Action** | DataTester requests historical bars for a configured `BarType`. | | **Event sequence** | Historical bars received via callback. | | **Pass criteria** | Bars received with valid OHLCV values and ascending timestamps. | | **Skip when** | Adapter does not support historical bar requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], bar_types=[BarType.from_str("BTCUSDT-PERP.VENUE-1-MINUTE-LAST-EXTERNAL")], request_bars=True, requests_start_delta=pd.Timedelta(hours=1), ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .bar_types(vec![bar_type]) .request_bars(true) .build()? ``` --- ## Group 6: Derivatives data Test derivatives-specific data streams: mark prices, index prices, and funding rates. | TC | Name | Description | Skip when | |--------|----------------------------------|---------------------------------------------|-----------------------| | TC-D50 | Subscribe mark prices | `MarkPriceUpdate` events. | Not a derivative. | | TC-D51 | Subscribe index prices | `IndexPriceUpdate` events. | Not a derivative. | | TC-D52 | Subscribe funding rates | `FundingRateUpdate` events. | Not a perpetual. | | TC-D53 | Request historical funding rates | Historical funding rate data. | Not a perpetual. | ### TC-D50: Subscribe mark prices | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, derivative instrument loaded. | | **Action** | DataTester subscribes to mark price updates. | | **Event sequence** | `MarkPriceUpdate` events received in `on_mark_price`. | | **Pass criteria** | At least one `MarkPriceUpdate` received with valid instrument ID and mark price. | | **Skip when** | Instrument is not a derivative, or adapter does not provide mark prices. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_mark_prices=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_mark_prices(true) .build()? ``` ### TC-D51: Subscribe index prices | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, derivative instrument loaded. | | **Action** | DataTester subscribes to index price updates. | | **Event sequence** | `IndexPriceUpdate` events received in `on_index_price`. | | **Pass criteria** | At least one `IndexPriceUpdate` received with valid instrument ID and index price. | | **Skip when** | Instrument is not a derivative, or adapter does not provide index prices. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_index_prices=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_index_prices(true) .build()? ``` ### TC-D52: Subscribe funding rates | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, perpetual instrument loaded. | | **Action** | DataTester subscribes to funding rate updates. | | **Event sequence** | `FundingRateUpdate` events received in `on_funding_rate`. | | **Pass criteria** | At least one `FundingRateUpdate` received with valid instrument ID and rate. | | **Skip when** | Instrument is not a perpetual, or adapter does not provide funding rates. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_funding_rates=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_funding_rates(true) .build()? ``` ### TC-D53: Request historical funding rates | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, perpetual instrument loaded. | | **Action** | DataTester requests historical funding rates (default 7-day lookback). | | **Event sequence** | Historical funding rates received via callback. | | **Pass criteria** | Funding rates received with valid timestamps and rate values. | | **Skip when** | Instrument is not a perpetual, or adapter does not support historical funding rate requests. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], request_funding_rates=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .request_funding_rates(true) .build()? ``` --- ## Group 7: Instrument status Test instrument status and close event subscriptions. | TC | Name | Description | Skip when | |--------|-----------------------------|------------------------------------------------|-----------------------| | TC-D60 | Subscribe instrument status | `InstrumentStatus` events. | No status support. | | TC-D61 | Subscribe instrument close | `InstrumentClose` events. | No close support. | ### TC-D60: Subscribe instrument status | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to instrument status updates. | | **Event sequence** | `InstrumentStatus` events received in `on_instrument_status`. | | **Pass criteria** | Status events received with valid `MarketStatusAction` (e.g. `Trading`). | | **Skip when** | Adapter does not support instrument status subscriptions. | **Considerations:** - Status events may only fire on state changes (e.g. trading halt -> resume). - During normal trading hours, a `Trading` status may be received on subscribe. **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_instrument_status=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_instrument_status(true) .build()? ``` ### TC-D61: Subscribe instrument close | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded. | | **Action** | DataTester subscribes to instrument close events. | | **Event sequence** | `InstrumentClose` events received in `on_instrument_close`. | | **Pass criteria** | Close event received with valid close price and close type. | | **Skip when** | Adapter does not support instrument close subscriptions. | **Considerations:** - Close events typically fire at end-of-session for traditional markets. - May not fire for 24/7 crypto venues unless the adapter synthesizes a daily close. **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_instrument_close=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_instrument_close(true) .build()? ``` --- ## Group 8: Option greeks Test option greeks and option chain subscriptions. | TC | Name | Description | Skip when | |--------|-----------------------------|------------------------------------------------|------------------------| | TC-D62 | Subscribe option greeks | `OptionGreeks` data for a single instrument. | No greeks support. | | TC-D63 | Subscribe option chain | `OptionChainSlice` snapshots for a series. | No chain support. | ### TC-D62: Subscribe option greeks | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded. | | **Action** | DataTester subscribes to option greeks updates. | | **Event sequence** | `OptionGreeks` events received in `on_option_greeks`. | | **Pass criteria** | Greeks received with valid delta, gamma, vega, theta values. | | **Skip when** | Adapter does not support option greeks subscriptions. | **Considerations:** - Greeks are only available for option instruments. - Values depend on the venue's pricing model and may update on every quote change. - Some venues (Bybit, Deribit) subscribe per instrument; OKX subscribes per instrument family and filters to the requested instruments. - `rho` may be zero when the venue does not provide it (Bybit, OKX). - `underlying_price` and `open_interest` may be `None` depending on the venue channel. **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_option_greeks=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_option_greeks(true) .build()? ``` ### TC-D63: Subscribe option chain | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instruments loaded for the series. | | **Action** | DataTester subscribes to option chain snapshots for a series. | | **Event sequence** | `OptionChainSlice` snapshots received in `on_option_chain`. | | **Pass criteria** | Chain snapshot contains greeks for instruments matching the series. | | **Skip when** | Adapter does not support option chain subscriptions. | **Considerations:** - Option chain subscriptions are managed by the DataEngine, which creates per-instrument quote and greeks subscriptions internally. - ATM-relative strike ranges require a forward price bootstrap before subscriptions begin. - Not yet configurable via `DataTesterConfig`; requires manual actor setup with `subscribe_option_chain` and an `OptionSeriesId`. --- ## Group 9: Lifecycle Test actor lifecycle behavior: unsubscribe handling and custom parameters. | TC | Name | Description | Skip when | |--------|-------------------------|----------------------------------------------------|----------------------| | TC-D70 | Unsubscribe on stop | Unsubscribe from data feeds on actor stop. | No unsub support. | | TC-D71 | Custom subscribe params | Adapter‑specific subscription parameters. | N/A. | | TC-D72 | Custom request params | Adapter‑specific request parameters. | N/A. | ### TC-D70: Unsubscribe on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Active data subscriptions (quotes, trades, book). | | **Action** | Stop the actor with `can_unsubscribe=True` (default). | | **Event sequence** | Data subscriptions removed; no further data events received. | | **Pass criteria** | Clean unsubscribe; no errors in logs; no data events after stop. | | **Skip when** | Adapter does not support unsubscribe. | **Python config:** ```python DataTesterConfig( instrument_ids=[instrument_id], subscribe_quotes=True, subscribe_trades=True, can_unsubscribe=True, ) ``` **Rust config:** ```rust DataTesterConfig::builder() .client_id(client_id) .instrument_ids(vec![instrument_id]) .subscribe_quotes(true) .subscribe_trades(true) .can_unsubscribe(true) .build()? ``` ### TC-D71: Custom subscribe params | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, adapter accepts additional subscription parameters. | | **Action** | Subscribe with `subscribe_params` dict containing adapter‑specific parameters. | | **Event sequence** | Subscription established with custom parameters applied. | | **Pass criteria** | Data flows with adapter‑specific parameters in effect. | | **Skip when** | N/A (adapter‑specific). | **Considerations:** - The `subscribe_params` dict is opaque to the DataTester and passed through to the adapter. - Consult the adapter's guide for supported parameters. ### TC-D72: Custom request params | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, adapter accepts additional request parameters. | | **Action** | Request data with `request_params` dict containing adapter‑specific parameters. | | **Event sequence** | Request fulfilled with custom parameters applied. | | **Pass criteria** | Historical data received with adapter‑specific parameters in effect. | | **Skip when** | N/A (adapter‑specific). | **Considerations:** - The `request_params` dict is opaque to the DataTester and passed through to the adapter. - Consult the adapter's guide for supported parameters. --- ## DataTester configuration reference Quick reference for all `DataTesterConfig` parameters. Defaults shown are for the Python config. Note: the Rust `DataTesterConfig` builder defaults `manage_book` to `true`, while Python defaults it to `False`. | Parameter | Type | Default | Affects groups | |------------------------------|-------------------|-----------------|----------------| | `instrument_ids` | list[InstrumentId]| *required* | All | | `client_id` | ClientId? | None | All | | `bar_types` | list[BarType]? | None | 5 | | `subscribe_book_deltas` | bool | False | 2 | | `subscribe_book_depth` | bool | False | 2 | | `subscribe_book_at_interval` | bool | False | 2 | | `subscribe_quotes` | bool | False | 3 | | `subscribe_trades` | bool | False | 4 | | `subscribe_mark_prices` | bool | False | 6 | | `subscribe_index_prices` | bool | False | 6 | | `subscribe_funding_rates` | bool | False | 6 | | `subscribe_bars` | bool | False | 5 | | `subscribe_instrument` | bool | False | 1 | | `subscribe_instrument_status`| bool | False | 7 | | `subscribe_instrument_close` | bool | False | 7 | | `subscribe_option_greeks` | bool | False | 8 | | `subscribe_params` | dict? | None | 9 | | `can_unsubscribe` | bool | True | 9 | | `request_instruments` | bool | False | 1 | | `request_book_snapshot` | bool | False | 2 | | `request_book_deltas` | bool | False | 2 | | `request_quotes` | bool | False | 3 | | `request_trades` | bool | False | 4 | | `request_bars` | bool | False | 5 | | `request_funding_rates` | bool | False | 6 | | `request_params` | dict? | None | 9 | | `requests_start_delta` | Timedelta? | 1 hour | 3, 4, 5 | | `book_type` | BookType | L2_MBP | 2 | | `book_depth` | PositiveInt? | None | 2 | | `book_interval_ms` | PositiveInt | 1000 | 2 | | `book_levels_to_print` | PositiveInt | 10 | 2 | | `manage_book` | bool | False | 2 | | `use_pyo3_book` | bool | False | 2 | | `log_data` | bool | True | All | --- # Execution Testing Spec Source: https://nautilustrader.io/docs/latest/developer_guide/spec_exec_testing/ This section defines a rigorous test matrix for validating adapter execution functionality using the `ExecTester` strategy. Both Python (`nautilus_trader.test_kit.strategies.tester_exec`) and Rust (`nautilus_testkit::testers`) provide the `ExecTester`. Each test case is identified by a prefixed ID (e.g. TC-E01) and grouped by functionality. **Each adapter must pass the subset of tests matching its supported capabilities.** Tests progress from simple (single market order) to complex (brackets, modification chains, rejection handling). An adapter that passes groups 1–5 is considered baseline compliant. Data connectivity should be verified first using the [Data Testing Spec](spec_data_testing.md). Document adapter-specific behavior (how a venue simulates market orders, handles TIF options, etc.) in the adapter's own guide, not here. Each adapter guide should include a capability matrix showing which order types, time-in-force options, actions, and flags it supports. ## Prerequisites Before running execution tests: - Demo/testnet account with valid API credentials (preferred, not required). - Account funded with sufficient margin for the test instrument and quantities. - Target instrument available and loadable via the instrument provider. - Environment variables set: `{VENUE}_API_KEY`, `{VENUE}_API_SECRET` (or sandbox variants). - If the venue offers a demo/testnet mode, use credentials created for that environment. Demo and production API keys are typically separate and not interchangeable; using the wrong credentials produces authentication errors (e.g. HTTP 401). - Risk engine bypassed (`LiveRiskEngineConfig(bypass=True)`) to avoid interference. - Reconciliation enabled to verify state consistency. **Python node setup**: Legacy examples still use `nautilus_trader.live.node.TradingNode`, but new Rust-backed PyO3 adapters should prefer `nautilus_trader.live.LiveNode`. Use `LiveNode.builder(...)` when you need to register adapter client factories before the node is built. ```python from nautilus_trader.common import Environment from nautilus_trader.live import LiveExecEngineConfig, LiveNode, LiveRiskEngineConfig from nautilus_trader.model import TraderId node = ( LiveNode.builder("TESTER-001", TraderId("TESTER-001"), Environment.SANDBOX) .with_risk_engine_config(LiveRiskEngineConfig(bypass=True)) .with_exec_engine_config(LiveExecEngineConfig(reconciliation=True)) .add_exec_client(None, adapter_exec_client_factory, exec_client_config) .build() ) node.add_strategy_from_config(importable_strategy_config) # Register remaining components, then start or run ``` **Rust node setup** (reference: `crates/adapters/{adapter}/examples/node_exec_tester.rs`): ```rust use nautilus_testkit::testers::{ExecTester, ExecTesterConfig}; use nautilus_trading::strategy::StrategyConfig; let tester_config = ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(order_qty) .build()?; let tester = ExecTester::new(tester_config); node.add_strategy(tester)?; node.run().await?; ``` ## Basic smoke test A quick sanity check that can run at any time, for example after adapter changes or between development iterations. The tester opens a position with a market order on start, places a buy and sell post-only limit order, waits 30 seconds, then stops (cancelling open orders and closing the position). **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.001"), open_position_on_start_qty=Decimal("0.001"), enable_limit_buys=True, enable_limit_sells=True, use_post_only=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.001")) .open_position_on_start_qty(dec!(0.001)) .enable_limit_buys(true) .enable_limit_sells(true) .use_post_only(true) .build()? ``` **Expected behavior:** 1. On start: market order fills, opening a position. 2. Two limit orders placed at `tob_offset_ticks` away from best bid/ask (default 500 ticks). 3. Strategy idles for 30 seconds. Check logs for errors, rejected orders, or disconnections. 4. On stop: open limit orders cancelled, position closed with a market order. **Pass criteria:** No errors in logs, position opened and closed cleanly, limit orders acknowledged by the venue. --- Each group below begins with a summary table, followed by detailed test cards. Test IDs use spaced numbering to allow insertion without renumbering. --- ## Group 1: Market orders Test market order submission and fills. Market orders should execute immediately. | TC | Name | Description | Skip when | |--------|-------------------------------|-----------------------------------------------------|---------------------| | TC-E01 | Market BUY - submit and fill | Open long position via market buy. | No market orders. | | TC-E02 | Market SELL - submit and fill | Open short position via market sell. | No market orders. | | TC-E03 | Market order with IOC TIF | Market order explicitly using IOC time in force. | No IOC. | | TC-E04 | Market order with FOK TIF | Market order explicitly using FOK time in force. | No FOK. | | TC-E05 | Market order with quote qty | Market order using quote currency quantity. | No quote quantity. | | TC-E06 | Close position via market | Close an open position with a market order on stop. | No market orders. | ### TC-E01: Market BUY - submit and fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, market data flowing, no open position. | | **Action** | ExecTester opens a long position via `open_position_on_start_qty`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Position opened with side=LONG, quantity matches config, fill price within market range, `AccountState` updated. | | **Skip when** | Adapter does not support market orders. | **Considerations:** - Some adapters simulate market orders as aggressive limit IOC orders (check adapter guide). - The event sequence from the strategy's perspective should be identical regardless of the venue mechanism. - Fill price should be within the recent bid/ask spread. - Partial fills are valid; verify the cumulative filled quantity matches the order quantity. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` ### TC-E02: Market SELL - submit and fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, market data flowing, no open position. | | **Action** | ExecTester opens a short position via negative `open_position_on_start_qty`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Position opened with side=SHORT, quantity matches config, fill price within market range. | | **Skip when** | Adapter does not support market orders or short selling. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("-0.01"), enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(-1, 2)) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` ### TC-E03: Market order with IOC TIF | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, market data flowing. | | **Action** | Open position with `open_position_time_in_force=IOC`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Same as TC-E01; the IOC TIF is explicitly set on the order. | | **Skip when** | No IOC support. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), open_position_time_in_force=TimeInForce.IOC, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .enable_limit_buys(false) .enable_limit_sells(false) .open_position_time_in_force(TimeInForce::Ioc) .build()? ``` ### TC-E04: Market order with FOK TIF | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, market data flowing. | | **Action** | Open position with `open_position_time_in_force=FOK`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Same as TC-E01; the FOK TIF is explicitly set on the order. | | **Skip when** | No FOK support. | **Considerations:** - FOK requires the entire quantity to be fillable immediately or the order is canceled. - Use small test quantities so book depth is sufficient for a complete fill. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), open_position_time_in_force=TimeInForce.FOK, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .enable_limit_buys(false) .enable_limit_sells(false) .open_position_time_in_force(TimeInForce::Fok) .build()? ``` ### TC-E05: Market order with quote quantity | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, adapter supports quote quantity. | | **Action** | Open position with `use_quote_quantity=True`, quantity in quote currency. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Order submitted with quote currency quantity; fill quantity is in base currency. | | **Skip when** | Adapter does not support quote quantity orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("100.0"), # Quote currency amount open_position_on_start_qty=Decimal("100.0"), use_quote_quantity=True, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("100")) .open_position_on_start_qty(Decimal::from(100)) .use_quote_quantity(true) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` ### TC-E06: Close position via market order on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open position from TC-E01 or TC-E02. | | **Action** | Stop the strategy; ExecTester closes position via market order. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled` (closing order). | | **Pass criteria** | Position closed (net quantity = 0), no open orders remaining. | | **Skip when** | Adapter does not support market orders. | **Considerations:** - This test naturally follows TC-E01 or TC-E02 as part of the same session. - `close_positions_on_stop=True` is the default. - The closing order should be on the opposite side of the position. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), close_positions_on_stop=True, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .close_positions_on_stop(true) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` --- ## Group 2: Limit orders Test limit order submission, acceptance, and behavior across time-in-force options. | TC | Name | Description | Skip when | |--------|----------------------------|--------------------------------------------------|--------------------| | TC-E10 | Limit BUY GTC | Place GTC limit buy below TOB, verify accepted. | Never. | | TC-E11 | Limit SELL GTC | Place GTC limit sell above TOB, verify accepted. | Never. | | TC-E12 | Limit BUY and SELL pair | Both sides simultaneously, verify both accepted. | Never. | | TC-E13 | Limit IOC aggressive fill | Limit IOC at aggressive price, expect fill. | No IOC. | | TC-E14 | Limit IOC passive no fill | Limit IOC away from market, expect cancel. | No IOC. | | TC-E15 | Limit FOK fill | Limit FOK at aggressive price, expect fill. | No FOK. | | TC-E16 | Limit FOK no fill | Limit FOK away from market, expect cancel. | No FOK. | | TC-E17 | Limit GTD | Limit with expiry time, verify accepted. | No GTD. | | TC-E18 | Limit GTD expiry | Wait for GTD expiry, verify `OrderExpired`. | No GTD. | | TC-E19 | Limit DAY | Limit with DAY TIF, verify accepted. | No DAY. | ### TC-E10: Limit BUY GTC - submit and accept | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a limit buy at `best_bid - tob_offset_ticks`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order is open on the venue with correct price, quantity, side=BUY, TIF=GTC. | | **Skip when** | Never. | **Considerations:** - The `tob_offset_ticks` (default 500) places the order well away from the market to avoid accidental fills. - Verify the order appears in the cache with `OrderStatus.ACCEPTED`. - The order should remain open until explicitly canceled. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .build()? ``` ### TC-E11: Limit SELL GTC - submit and accept | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a limit sell at `best_ask + tob_offset_ticks`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order is open on the venue with correct price, quantity, side=SELL, TIF=GTC. | | **Skip when** | Never. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(true) .build()? ``` ### TC-E12: Limit BUY and SELL pair | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places both a limit buy and limit sell. | | **Event sequence** | Two independent sequences: each `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Both orders open on venue, buy below bid, sell above ask. | | **Skip when** | Never. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(true) .build()? ``` ### TC-E13: Limit IOC aggressive fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Submit a limit buy IOC at or above the best ask (aggressive price). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Order fills immediately; position opened. | | **Skip when** | Adapter does not support IOC TIF. | **Considerations:** - This test requires manual order creation or adapter-specific configuration, as the ExecTester's default limit order placement uses GTC TIF. - IOC orders that don't fill immediately are canceled by the venue. ### TC-E14: Limit IOC passive - no fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Submit a limit buy IOC well below the market (passive price). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderCanceled`. | | **Pass criteria** | Order is immediately canceled by venue with no fill. | | **Skip when** | Adapter does not support IOC TIF. | **Considerations:** - The venue should cancel the unfilled IOC order; verify `OrderCanceled` event (not `OrderExpired`). ### TC-E15: Limit FOK fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing, sufficient book depth. | | **Action** | Submit a limit buy FOK at aggressive price with quantity within top‑of‑book depth. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Order fills completely in a single fill event. | | **Skip when** | Adapter does not support FOK TIF. | **Considerations:** - FOK requires the entire quantity to be fillable; use small quantities so book depth is sufficient. ### TC-E16: Limit FOK no fill | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Submit a limit buy FOK at passive price (well below market). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderCanceled`. | | **Pass criteria** | Order is immediately canceled by venue with no fill. | | **Skip when** | Adapter does not support FOK TIF. | ### TC-E17: Limit GTD - submit and accept | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Place limit buy with `order_expire_time_delta_mins` set (e.g., 60 minutes). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted with GTD TIF and correct expiry timestamp. | | **Skip when** | Adapter does not support GTD TIF. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), order_expire_time_delta_mins=60, enable_limit_buys=True, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .order_expire_time_delta_mins(60) .build()? ``` ### TC-E18: Limit GTD expiry | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTD limit order from TC-E17 (or use a very short expiry). | | **Action** | Wait for the GTD expiry time to elapse. | | **Event sequence** | `OrderExpired`. | | **Pass criteria** | Order transitions to expired status; `OrderExpired` event received. | | **Skip when** | Adapter does not support GTD TIF. | **Considerations:** - Use a short `order_expire_time_delta_mins` (e.g., 1–2 minutes) to avoid long waits. - Some venues may report expiry as a cancel; verify the adapter maps this to `OrderExpired`. ### TC-E19: Limit DAY - submit and accept | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, market is in trading hours. | | **Action** | Submit limit buy with DAY TIF. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted with DAY TIF; will be automatically canceled at end of trading day. | | **Skip when** | Adapter does not support DAY TIF. | **Considerations:** - DAY orders may behave differently on 24/7 crypto venues vs traditional markets. - Verify behavior when submitted outside trading hours (if applicable). --- ## Group 3: Stop and conditional orders Test stop and conditional order types. These orders rest on the venue until a trigger condition is met. Adapters that support venue-native conditional orders should also verify that open trigger orders appear in restart reconciliation, not only in the normal open-order endpoint. | TC | Name | Description | Skip when | |--------|------------------------|-------------------------------------------------------|---------------------| | TC-E20 | StopMarket BUY | Stop buy above ask, verify accepted. | No `STOP_MARKET`. | | TC-E21 | StopMarket SELL | Stop sell below bid, verify accepted. | No `STOP_MARKET`. | | TC-E22 | StopLimit BUY | Stop‑limit buy with trigger + limit price. | No `STOP_LIMIT`. | | TC-E23 | StopLimit SELL | Stop‑limit sell with trigger + limit price. | No `STOP_LIMIT`. | | TC-E24 | MarketIfTouched BUY | MIT buy below bid. | No `MIT`. | | TC-E25 | MarketIfTouched SELL | MIT sell above ask. | No `MIT`. | | TC-E26 | LimitIfTouched BUY | LIT buy with trigger + limit price. | No `LIT`. | | TC-E27 | LimitIfTouched SELL | LIT sell with trigger + limit price. | No `LIT`. | ### TC-E20: StopMarket BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a stop‑market buy above the current ask. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Stop order accepted on venue with correct trigger price and side=BUY. | | **Skip when** | Adapter does not support `StopMarket` orders. | **Considerations:** - The trigger price should be above the current ask by `stop_offset_ticks`. - The order should NOT trigger immediately (trigger price is above market). - For venues with long-lived trigger signatures, verify the trigger-order signing expiry uses the venue's trigger-order window rather than the normal order expiry. - Verifying trigger and fill requires the market to move, which may not happen during the test. - After acceptance, restart or force reconciliation and verify the order still appears as an open order report when the venue keeps trigger orders in a separate endpoint. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=False, enable_stop_buys=True, enable_stop_sells=False, stop_order_type=OrderType.STOP_MARKET, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(false) .enable_stop_buys(true) .enable_stop_sells(false) .stop_order_type(OrderType::StopMarket) .build()? ``` ### TC-E21: StopMarket SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a stop‑market sell below the current bid. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Stop order accepted on venue with correct trigger price and side=SELL. | | **Skip when** | Adapter does not support `StopMarket` orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=False, enable_stop_buys=False, enable_stop_sells=True, stop_order_type=OrderType.STOP_MARKET, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(false) .enable_stop_buys(false) .enable_stop_sells(true) .stop_order_type(OrderType::StopMarket) .build()? ``` ### TC-E22: StopLimit BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a stop‑limit buy with trigger price above ask and limit offset. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Stop‑limit order accepted with correct trigger price, limit price, and side=BUY. | | **Skip when** | Adapter does not support `StopLimit` orders. | **Considerations:** - Requires `stop_limit_offset_ticks` to be set for the limit price offset from the trigger price. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=False, enable_stop_buys=True, enable_stop_sells=False, stop_order_type=OrderType.STOP_LIMIT, stop_limit_offset_ticks=50, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(false) .enable_stop_buys(true) .enable_stop_sells(false) .stop_order_type(OrderType::StopLimit) .stop_limit_offset_ticks(50) .build()? ``` ### TC-E23: StopLimit SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places a stop‑limit sell with trigger price below bid. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Stop‑limit order accepted with correct trigger price, limit price, and side=SELL. | | **Skip when** | Adapter does not support `StopLimit` orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=False, enable_stop_buys=False, enable_stop_sells=True, stop_order_type=OrderType.STOP_LIMIT, stop_limit_offset_ticks=50, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(false) .enable_stop_buys(false) .enable_stop_sells(true) .stop_order_type(OrderType::StopLimit) .stop_limit_offset_ticks(50) .build()? ``` ### TC-E24: MarketIfTouched BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Place MIT buy with trigger below current bid (buy on dip). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | MIT order accepted on venue with correct trigger price. | | **Skip when** | Adapter does not support `MarketIfTouched` orders. | ### TC-E25: MarketIfTouched SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Place MIT sell with trigger above current ask (sell on rally). | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | MIT order accepted on venue with correct trigger price. | | **Skip when** | Adapter does not support `MarketIfTouched` orders. | ### TC-E26: LimitIfTouched BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Place LIT buy with trigger below bid and limit price offset. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | LIT order accepted with correct trigger price and limit price. | | **Skip when** | Adapter does not support `LimitIfTouched` orders. | ### TC-E27: LimitIfTouched SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | Place LIT sell with trigger above ask and limit price offset. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | LIT order accepted with correct trigger price and limit price. | | **Skip when** | Adapter does not support `LimitIfTouched` orders. | --- ## Group 4: Order modification Test order modification (amend) and cancel-replace workflows. | TC | Name | Description | Skip when | |-------|------------------------------|-----------------------------------------------------|-----------------------------| | TC-E30 | Modify limit BUY price | Amend open limit buy to new price. | No modify support. | | TC-E31 | Modify limit SELL price | Amend open limit sell to new price. | No modify support. | | TC-E32 | Cancel‑replace limit BUY | Cancel and resubmit limit buy at new price. | Never. | | TC-E33 | Cancel‑replace limit SELL | Cancel and resubmit limit sell at new price. | Never. | | TC-E34 | Modify stop trigger price | Amend stop order trigger price. | No modify or no stop. | | TC-E35 | Cancel‑replace stop order | Cancel and resubmit stop at new trigger price. | No stop orders. | | TC-E36 | Modify rejected | Modify on unsupported adapter. | Adapter supports modify. | ### TC-E30: Modify limit BUY price | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTC limit buy from TC-E10. | | **Action** | ExecTester modifies limit buy to a new price as market moves (`modify_orders_to_maintain_tob_offset=True`). | | **Event sequence** | `OrderPendingUpdate` -> `OrderUpdated`. | | **Pass criteria** | `OrderUpdated` event logged with the new price; order exits `PendingUpdate`. | | **Skip when** | Adapter does not support order modification. | **Considerations:** - Requires market movement to trigger the ExecTester's order maintenance logic. - The modify is triggered when the order price drifts from the target TOB offset. - Verify the `OrderUpdated` log shows the expected price. If the event never arrives, the order stays in `PendingUpdate` and the tester stops modifying it. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, modify_orders_to_maintain_tob_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .modify_orders_to_maintain_tob_offset(true) .build()? ``` ### TC-E31: Modify limit SELL price | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTC limit sell from TC-E11. | | **Action** | ExecTester modifies limit sell to new price as market moves. | | **Event sequence** | `OrderPendingUpdate` -> `OrderUpdated`. | | **Pass criteria** | `OrderUpdated` event logged with the new price; order exits `PendingUpdate`. | | **Skip when** | Adapter does not support order modification. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=True, modify_orders_to_maintain_tob_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(true) .modify_orders_to_maintain_tob_offset(true) .build()? ``` ### TC-E32: Cancel-replace limit BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTC limit buy. | | **Action** | ExecTester cancels and resubmits limit buy at new price as market moves. | | **Event sequence** | `OrderPendingCancel` -> `OrderCanceled` -> `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Original order canceled, new order accepted at updated price. | | **Skip when** | Never (cancel‑replace is always available). | **Considerations:** - This is the universal alternative when the adapter does not support native modify. - Two distinct orders in the cache: the canceled original and the new replacement. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, cancel_replace_orders_to_maintain_tob_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .cancel_replace_orders_to_maintain_tob_offset(true) .build()? ``` ### TC-E33: Cancel-replace limit SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTC limit sell. | | **Action** | ExecTester cancels and resubmits limit sell at new price. | | **Event sequence** | `OrderPendingCancel` -> `OrderCanceled` -> `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Original order canceled, new order accepted at updated price. | | **Skip when** | Never. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=False, enable_limit_sells=True, cancel_replace_orders_to_maintain_tob_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(false) .enable_limit_sells(true) .cancel_replace_orders_to_maintain_tob_offset(true) .build()? ``` ### TC-E34: Modify stop trigger price | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open stop order from TC-E20 or TC-E22. | | **Action** | ExecTester modifies stop trigger price as market moves (`modify_stop_orders_to_maintain_offset=True`). | | **Event sequence** | `OrderPendingUpdate` -> `OrderUpdated`. | | **Pass criteria** | `OrderUpdated` event logged with the new trigger price; order exits `PendingUpdate`. | | **Skip when** | Adapter does not support native stop modify, or no stop order support. | **Considerations:** - Some venues allow limit-order modify but reject trigger-order replace. For those adapters, skip TC-E34 and run TC-E35 instead. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_stop_buys=True, modify_stop_orders_to_maintain_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_stop_buys(true) .modify_stop_orders_to_maintain_offset(true) .build()? ``` ### TC-E35: Cancel-replace stop order | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open stop order. | | **Action** | ExecTester cancels and resubmits stop at new trigger price. | | **Event sequence** | `OrderPendingCancel` -> `OrderCanceled` -> `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Original stop canceled, new stop accepted at updated trigger price. | | **Skip when** | No stop order support. | **Considerations:** - This is the required path for venues that do not support native trigger-order replace. - After the new stop is accepted, restart or force reconciliation and verify exactly one current trigger order remains. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_stop_buys=True, cancel_replace_stop_orders_to_maintain_offset=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_stop_buys(true) .cancel_replace_stop_orders_to_maintain_offset(true) .build()? ``` ### TC-E36: Modify rejected | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open limit order, adapter does NOT support modify. | | **Action** | Attempt to modify the order (programmatically, not via ExecTester auto‑maintain). | | **Event sequence** | `OrderModifyRejected`. | | **Pass criteria** | Modify attempt results in `OrderModifyRejected` event with reason; original order remains unchanged. | | **Skip when** | Adapter supports order modification. | **Considerations:** - This tests the adapter's rejection path, not the ExecTester's cancel-replace logic. - The rejection reason should indicate that modification is not supported. --- ## Group 5: Order cancellation Test order cancellation workflows. | TC | Name | Description | Skip when | |-------|----------------------------|------------------------------------------------------|----------------------| | TC-E40 | Cancel single limit order | Cancel an open limit order. | Never. | | TC-E41 | Cancel all on stop | Strategy stop cancels all open orders (default). | Never. | | TC-E42 | Individual cancels on stop | Cancel orders one‑by‑one on stop. | Never. | | TC-E43 | Batch cancel on stop | Cancel orders via batch API on stop. | No batch cancel. | | TC-E44 | Cancel already‑canceled | Cancel a non‑open order. | Never. | ### TC-E40: Cancel single limit order | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open GTC limit order from TC-E10 or TC-E11. | | **Action** | Stop the strategy; ExecTester cancels the open limit order. | | **Event sequence** | `OrderPendingCancel` -> `OrderCanceled`. | | **Pass criteria** | Order status transitions to CANCELED; no open orders remaining. | | **Skip when** | Never. | **Considerations:** - `cancel_orders_on_stop=True` (default) triggers cancellation when the strategy stops. - Verify the `OrderCanceled` event contains the correct `venue_order_id`. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, cancel_orders_on_stop=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .cancel_orders_on_stop(true) .build()? ``` ### TC-E41: Cancel all on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Multiple open orders (limit buy + limit sell from TC-E12). | | **Action** | Stop the strategy with `cancel_orders_on_stop=True` (default). | | **Event sequence** | For each order: `OrderPendingCancel` -> `OrderCanceled`. | | **Pass criteria** | All open orders canceled; no open orders remaining. | | **Skip when** | Never. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=True, cancel_orders_on_stop=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(true) .cancel_orders_on_stop(true) .build()? ``` ### TC-E42: Individual cancels on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Multiple open orders. | | **Action** | Stop with `use_individual_cancels_on_stop=True`. | | **Event sequence** | Individual `OrderPendingCancel` -> `OrderCanceled` for each order. | | **Pass criteria** | Each order canceled individually; all orders reach CANCELED status. | | **Skip when** | Never. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=True, use_individual_cancels_on_stop=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(true) .use_individual_cancels_on_stop(true) .build()? ``` ### TC-E43: Batch cancel on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Multiple open orders, adapter supports batch cancel. | | **Action** | Stop with `use_batch_cancel_on_stop=True`. | | **Event sequence** | Batch `OrderPendingCancel` -> `OrderCanceled` for all orders. | | **Pass criteria** | All orders canceled via single batch request; all reach CANCELED status. | | **Skip when** | Adapter does not support batch cancel. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=True, use_batch_cancel_on_stop=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(true) .use_batch_cancel_on_stop(true) .build()? ``` ### TC-E44: Cancel already-canceled order | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | A previously canceled order (from TC-E40). | | **Action** | Attempt to cancel the same order again. | | **Event sequence** | `OrderCancelRejected`. | | **Pass criteria** | Cancel attempt is rejected; `OrderCancelRejected` event received with reason. | | **Skip when** | Never. | **Considerations:** - This tests the adapter's error handling for invalid cancel requests. - The rejection reason should indicate the order is not in a cancelable state. --- ## Group 6: Bracket orders Test bracket order submission (entry + take-profit + stop-loss). | TC | Name | Description | Skip when | |-------|-------------------------------|---------------------------------------------------|----------------------| | TC-E50 | Bracket BUY | Entry limit buy + TP limit sell + SL stop sell. | No bracket support. | | TC-E51 | Bracket SELL | Entry limit sell + TP limit buy + SL stop buy. | No bracket support. | | TC-E52 | Bracket entry fill activates | Verify TP/SL become active after entry fill. | No bracket support. | | TC-E53 | Bracket with post‑only entry | Entry order uses post‑only flag. | No bracket or PO. | ### TC-E50: Bracket BUY | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester submits a bracket order: limit buy entry + take‑profit sell + stop‑loss sell. | | **Event sequence** | Entry: `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`; TP and SL: `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Three orders created and accepted: entry below bid, TP above ask, SL below entry. | | **Skip when** | Adapter does not support bracket orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_brackets=True, bracket_entry_order_type=OrderType.LIMIT, bracket_offset_ticks=500, enable_limit_buys=True, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_brackets(true) .bracket_entry_order_type(OrderType::Limit) .bracket_offset_ticks(500) .enable_limit_buys(true) .enable_limit_sells(false) .build()? ``` ### TC-E51: Bracket SELL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester submits bracket: limit sell entry + TP buy + SL buy. | | **Event sequence** | Same pattern as TC-E50 but for sell side. | | **Pass criteria** | Three orders created and accepted on sell side. | | **Skip when** | Adapter does not support bracket orders. | ### TC-E52: Bracket entry fill activates TP/SL | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Bracket order from TC-E50 where entry order fills. | | **Action** | Entry order fills; verify contingent TP and SL orders activate. | | **Event sequence** | Entry: `OrderFilled`; TP and SL transition from contingent to active. | | **Pass criteria** | After entry fill, TP and SL orders are live on the venue. | | **Skip when** | Adapter does not support bracket orders. | **Considerations:** - This requires the entry order to actually fill, which may need aggressive pricing. - The TP/SL activation mechanism varies by venue (some activate immediately, some are OCA groups). ### TC-E53: Bracket with post-only entry | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter supports brackets and post‑only. | | **Action** | Submit bracket with `use_post_only=True` (applied to entry and TP). | | **Event sequence** | Same as TC-E50 with post‑only flag on entry. | | **Pass criteria** | Entry and TP orders accepted as post‑only (maker); SL is not post‑only. | | **Skip when** | No bracket support or no post‑only support. | --- ## Group 7: Order flags Test order-level flags and special parameters. | TC | Name | Description | Skip when | |-------|----------------------|--------------------------------------------------------|----------------------| | TC-E60 | PostOnly accepted | Limit with post‑only, placed away from TOB. | No post‑only. | | TC-E61 | ReduceOnly on close | Close position with reduce‑only flag. | No reduce‑only. | | TC-E62 | Display quantity | Iceberg order with visible quantity < total. | No display quantity. | | TC-E63 | Custom order params | Adapter‑specific params via `order_params`. | N/A. | ### TC-E60: PostOnly accepted | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places limit buy with `use_post_only=True` at passive price. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted as a maker order; post‑only flag acknowledged by venue. | | **Skip when** | Adapter does not support post‑only flag. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, use_post_only=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .use_post_only(true) .build()? ``` ### TC-E61: ReduceOnly on close | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open position (from TC-E01). | | **Action** | Stop strategy with `reduce_only_on_stop=True`; closing order uses reduce‑only flag. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled` (with reduce‑only). | | **Pass criteria** | Closing order has reduce‑only flag; position fully closed. | | **Skip when** | Adapter does not support reduce‑only flag. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), reduce_only_on_stop=True, close_positions_on_stop=True, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .reduce_only_on_stop(true) .close_positions_on_stop(true) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` ### TC-E62: Display quantity (iceberg) | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, adapter supports display quantity. | | **Action** | Place limit order with `order_display_qty` < `order_qty`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted with display quantity set; only display qty visible on the book. | | **Skip when** | Adapter does not support display quantity / iceberg orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("1.0"), order_display_qty=Decimal("0.1"), enable_limit_buys=True, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("1.0")) .enable_limit_buys(true) .enable_limit_sells(false) .order_display_qty(Quantity::from("0.1")) .build()? ``` ### TC-E63: Custom order params | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, adapter accepts additional parameters. | | **Action** | Place order with `order_params` dict containing adapter‑specific parameters. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted; adapter‑specific parameters passed through to venue. | | **Skip when** | N/A (adapter‑specific). | **Considerations:** - The `order_params` dict is opaque to the ExecTester and passed through to the adapter. - Consult the adapter's guide for supported parameters. --- ## Group 8: Rejection handling Test that the adapter correctly handles and reports order rejections. | TC | Name | Description | Skip when | |--------|-------------------------|--------------------------------------------------|------------------| | TC-E70 | PostOnly rejection | Post‑only order that would cross the spread. | No post‑only. | | TC-E71 | ReduceOnly rejection | Reduce‑only order with no position to reduce. | No reduce‑only. | | TC-E72 | Unsupported order type | Submit order type not supported by adapter. | Never. | | TC-E73 | Unsupported TIF | Submit order with unsupported time in force. | Never. | | TC-E74 | Ambiguous submit fail | Transport, timeout, or send failure on submit. | No mock path. | | TC-E75 | Ambiguous cancel fail | Transport, timeout, or send failure on cancel. | No cancel. | | TC-E76 | Ambiguous modify fail | Transport, timeout, or send failure on modify. | No modify. | | TC-E77 | Ambiguous batch fail | Whole‑batch failure without per‑order result. | No batch. | | TC-E78 | Per‑order batch reject | Batch response has explicit per‑order rejection. | No batch. | TC-E74 through TC-E78 are specified collectively below because they usually require a mock HTTP or WebSocket boundary rather than a live venue. ### Ambiguous outcome failures These cases prove that adapter request failures do not turn into terminal rejection events when the venue outcome is unknown. The pass criteria also define the local prepare-failure carve-out: when a command is known not to have been sent and is attributable to one cancel or modify command, the adapter may emit the matching rejection event. **Pass criteria:** - Submit failures from transport errors, timeouts, WebSocket send failures, retry exhaustion, or response parse failures do not emit `OrderRejected`. - Cancel failures from transport errors, timeouts, WebSocket send failures, retry exhaustion, or whole-request server failures do not emit `OrderCancelRejected`. - Modify failures from transport errors, timeouts, WebSocket send failures, retry exhaustion, or whole-request server failures do not emit `OrderModifyRejected`. - Local cancel prepare failures that prove the command cannot be sent may emit `OrderCancelRejected` when the adapter can attribute the failure to one cancel command. - Local modify prepare failures that prove the command cannot be sent may emit `OrderModifyRejected` when the adapter can attribute the failure to one modify command. - Whole-batch request failures do not emit one rejection per order when the venue did not return per-order results. - Explicit per-order venue rejections still emit the matching rejection event with the venue reason. The order remains in the appropriate in-flight state until a venue update, query result, or reconciliation pass resolves it. ### TC-E70: PostOnly rejection | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, quotes flowing. | | **Action** | ExecTester places post‑only order on the wrong side of the book (`test_reject_post_only=True`), causing it to cross the spread. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderRejected`. | | **Pass criteria** | Venue rejects order; `OrderRejected.due_post_only=true`; reason names post‑only violation. | | **Skip when** | Adapter does not support post‑only flag. | **Considerations:** - The ExecTester's `test_reject_post_only` mode intentionally prices the order to cross. - Some venues may partially fill instead of rejecting; behavior is venue-specific. - Adapters that emit `OrderRejected` for a post-only crossing reject should set `due_post_only=true` so strategies can distinguish this from other venue rejections. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), enable_limit_buys=True, enable_limit_sells=False, use_post_only=True, test_reject_post_only=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .enable_limit_buys(true) .enable_limit_sells(false) .use_post_only(true) .test_reject_post_only(true) .build()? ``` ### TC-E71: ReduceOnly rejection | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, no open position for the instrument. | | **Action** | ExecTester opens a market position with `reduce_only=True` via `test_reject_reduce_only=True` and `open_position_on_start_qty`, when no position exists to reduce. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderRejected`. | | **Pass criteria** | Order rejected; `OrderRejected` event with reason indicating reduce‑only violation. | | **Skip when** | Adapter does not support reduce‑only flag. | **Considerations:** - The `test_reject_reduce_only` flag only applies to the opening market order submitted via `open_position_on_start_qty`. - Verify no prior position exists for the instrument before running this test. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), test_reject_reduce_only=True, enable_limit_buys=False, enable_limit_sells=False, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .test_reject_reduce_only(true) .enable_limit_buys(false) .enable_limit_sells(false) .build()? ``` ### TC-E72: Unsupported order type | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, order type not in adapter's supported set. | | **Action** | Submit an order type the adapter does not support. | | **Event sequence** | `OrderDenied` (pre‑submission rejection by adapter). | | **Pass criteria** | Order denied before reaching venue; `OrderDenied` event with reason. | | **Skip when** | Never (every adapter has unsupported order types to test). | **Considerations:** - `OrderDenied` occurs at the adapter level before the order reaches the venue. - This differs from `OrderRejected` which comes from the venue. - Test by configuring a stop order type that the adapter does not support. ### TC-E73: Unsupported TIF | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, TIF not in adapter's supported set. | | **Action** | Submit an order with a TIF the adapter does not support. | | **Event sequence** | `OrderDenied` (pre‑submission rejection by adapter). | | **Pass criteria** | Order denied before reaching venue; `OrderDenied` event with reason. | | **Skip when** | Never (every adapter has unsupported TIF options to test). | **Considerations:** - Similar to TC-E72 but for time-in-force options. - Test with TIF values from the Nautilus enum that the adapter does not map. --- ## Group 9: Lifecycle (start/stop) Test strategy lifecycle behavior and state management on start and stop. | TC | Name | Description | Skip when | |--------|-----------------------------|--------------------------------------------------------|----------------------| | TC-E80 | Open position on start | Open a position immediately when strategy starts. | No market orders. | | TC-E81 | Cancel orders on stop | Cancel all open orders when strategy stops. | Never. | | TC-E82 | Close positions on stop | Close open positions when strategy stops. | No market orders. | | TC-E83 | Unsubscribe on stop | Unsubscribe from data feeds on strategy stop. | No unsub support. | | TC-E84 | Reconcile open orders | Reconcile existing open orders from a prior session. | Never. | | TC-E85 | Reconcile filled orders | Reconcile previously filled orders from a prior session.| Never. | | TC-E86 | Reconcile open long | Reconcile existing open long position. | Never. | | TC-E87 | Reconcile open short | Reconcile existing open short position. | Never. | ### TC-E80: Open position on start | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, instrument loaded, no existing position. | | **Action** | Strategy starts with `open_position_on_start_qty` set. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | Position opened on start; market order submitted and filled before limit order maintenance begins. | | **Skip when** | Adapter does not support market orders. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), open_position_on_start_qty=Decimal("0.01"), ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .open_position_on_start_qty(Decimal::new(1, 2)) .build()? ``` ### TC-E81: Cancel orders on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open limit orders from the strategy session. | | **Action** | Stop the strategy with `cancel_orders_on_stop=True` (default). | | **Event sequence** | For each open order: `OrderPendingCancel` -> `OrderCanceled`. | | **Pass criteria** | All strategy‑owned open orders canceled on stop. | | **Skip when** | Never. | ### TC-E82: Close positions on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open position from the strategy session. | | **Action** | Stop the strategy with `close_positions_on_stop=True` (default). | | **Event sequence** | Closing order: `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled`. | | **Pass criteria** | All strategy‑owned positions closed; net position = 0. | | **Skip when** | Adapter does not support market orders. | ### TC-E83: Unsubscribe on stop | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Active data subscriptions (quotes, trades, book). | | **Action** | Stop the strategy with `can_unsubscribe=True` (default). | | **Event sequence** | Data subscriptions removed. | | **Pass criteria** | No further data events received after stop; clean disconnection. | | **Skip when** | Adapter does not support unsubscribe. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, order_qty=Decimal("0.01"), can_unsubscribe=True, ) ``` **Rust config:** ```rust ExecTesterConfig::builder() .base(StrategyConfig { strategy_id: Some(strategy_id), ..Default::default() }) .instrument_id(instrument_id) .client_id(client_id) .order_qty(Quantity::from("0.01")) .can_unsubscribe(true) .build()? ``` ### TC-E84: Reconcile open orders | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | One or more open limit orders on the venue from a prior session. | | **Action** | Start the node with `reconciliation=True`. | | **Event sequence** | `OrderStatusReport` generated for each open order. | | **Pass criteria** | Each open order is loaded into the cache with correct `venue_order_id`, status=ACCEPTED, price, quantity, side, and order type. | | **Skip when** | Never. | **Considerations:** - Leave limit orders open from a prior test session (do not cancel on stop). - Use `external_order_claims` to claim the instrument so the adapter reconciles orders for it. - Verify that the reconciled order count matches the venue-reported count. ### TC-E85: Reconcile filled orders | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | One or more filled orders on the venue from a prior session. | | **Action** | Start the node with `reconciliation=True`. | | **Event sequence** | `FillReport` generated for each historical fill. | | **Pass criteria** | Each filled order is loaded into the cache with correct `venue_order_id`, status=FILLED, fill price, fill quantity, and commission. | | **Skip when** | Never. | **Considerations:** - Requires orders that filled in a prior session. - Verify fill price, quantity, and commission match the venue's reported values. - Some adapters may only report fills within a lookback window. ### TC-E86: Reconcile open long position | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | An open long position on the venue from a prior session. | | **Action** | Start the node with `reconciliation=True`. | | **Event sequence** | `PositionStatusReport` generated for the long position. | | **Pass criteria** | Position loaded into cache with correct instrument, side=LONG, quantity, and entry price matching the venue. | | **Skip when** | Never. | **Considerations:** - Open a long position in a prior session and stop the strategy without closing it (`close_positions_on_stop=False`). - Verify the reconciled position quantity and average entry price match the venue. - After reconciliation, the strategy should be able to manage or close this position. ### TC-E87: Reconcile open short position | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | An open short position on the venue from a prior session. | | **Action** | Start the node with `reconciliation=True`. | | **Event sequence** | `PositionStatusReport` generated for the short position. | | **Pass criteria** | Position loaded into cache with correct instrument, side=SHORT, quantity, and entry price matching the venue. | | **Skip when** | Never. | **Considerations:** - Open a short position in a prior session and stop the strategy without closing it (`close_positions_on_stop=False`). - Verify the reconciled position quantity and average entry price match the venue. - After reconciliation, the strategy should be able to manage or close this position. --- ## Group 10: Options trading Test options-specific execution behavior. Options instruments typically have different constraints from linear derivatives: venues may restrict order types, support alternative pricing modes, or disallow conditional orders. Exact restrictions vary by venue; consult the adapter guide. These tests require a `CryptoOption` instrument. Use an OTM option with reasonable liquidity for fills. | TC | Name | Description | Skip when | |---------|-------------------------------|------------------------------------------------------------------|------------------------| | TC-E90 | Limit BUY option | Place a limit buy on an option instrument. | No options support. | | TC-E91 | Limit SELL option | Place a limit sell on an option instrument. | No options support. | | TC-E92 | Limit with alt pricing | Place a limit order with adapter‑specific pricing via `order_params`. | No alt pricing. | | TC-E94 | Unsupported order type denied | Submit an order type the adapter rejects for options. | No options support. | | TC-E96 | Conditional order rejected | Submit a stop/conditional order on an option; expect rejection. | No options support. | | TC-E99 | FOK limit option | Place a FOK limit order on an option instrument. | No FOK options. | | TC-E100 | Cancel option order | Cancel an open limit order on an option instrument. | No options support. | | TC-E101 | Reconcile option position | Reconcile an open option position from a prior session. | No options support. | ### TC-E90: Limit BUY option | Field | Value | |--------------------|-----------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded, quotes flowing. | | **Action** | ExecTester places a limit buy on the option at a passive price. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted by venue with correct instrument, side, price, and quantity. | | **Skip when** | Adapter does not support options trading. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, # CryptoOption instrument order_qty=Decimal("1"), enable_limit_buys=True, enable_limit_sells=False, tob_offset_ticks=500, ) ``` ### TC-E91: Limit SELL option | Field | Value | |--------------------|-----------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded, quotes flowing. | | **Action** | ExecTester places a limit sell on the option at a passive price. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted by venue with correct instrument, side, price, and quantity. | | **Skip when** | Adapter does not support options trading. | **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, # CryptoOption instrument order_qty=Decimal("1"), enable_limit_buys=False, enable_limit_sells=True, tob_offset_ticks=500, ) ``` ### TC-E92: Limit with alternative pricing | Field | Value | |--------------------|---------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded. | | **Action** | Place limit order with adapter‑specific pricing via `order_params`. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted`. | | **Pass criteria** | Order accepted; venue acknowledges the alternative pricing mode. | | **Skip when** | Adapter does not support alternative pricing modes for options. | **Considerations:** - The `price` field on the order object may be a placeholder when alternative pricing is active. Consult the adapter guide for supported parameter keys. - Example: OKX supports `px_usd` (USD price) and `px_vol` (implied volatility). - Verify in venue responses that the pricing mode is reflected correctly. **Python config:** ```python ExecTesterConfig( instrument_id=instrument_id, # CryptoOption instrument order_qty=Decimal("1"), enable_limit_buys=True, enable_limit_sells=False, order_params={"px_usd": "100.5"}, # Adapter-specific pricing key ) ``` ### TC-E94: Unsupported order type denied for options | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded. | | **Action** | Submit an order type the venue does not support for options (e.g. market order). | | **Event sequence** | Adapter‑dependent: `OrderDenied` (pre‑submission) or `OrderSubmitted` -> `OrderRejected` (post‑submission). | | **Pass criteria** | Order does not fill. Denial or rejection reason references the unsupported order type. | | **Skip when** | Adapter does not support options. | **Considerations:** - The exact rejection point varies by adapter. Some adapters deny locally before submitting; others submit and relay the venue rejection. - ExecTester can trigger a market order via `open_position_on_start_qty` on an option instrument. Some unsupported types (e.g. `MarketToLimit`) require manual or programmatic submission. - Test each unsupported type the adapter documents. ### TC-E96: Conditional order rejected for options | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded. | | **Action** | Submit a conditional order on an option instrument. | | **Event sequence** | Adapter‑dependent: `OrderDenied` (pre‑submission) or `OrderSubmitted` -> `OrderRejected` (post‑submission). | | **Pass criteria** | Order does not fill. Reason references unsupported conditional order type. | | **Skip when** | Adapter does not support options, or adapter supports conditionals for options. | **Considerations:** - Test each conditional type the adapter documents as unsupported for options (e.g. `STOP_MARKET`, `STOP_LIMIT`, `MARKET_IF_TOUCHED`, `LIMIT_IF_TOUCHED`, `TRAILING_STOP_MARKET`). - ExecTester can trigger conditional orders via `enable_stop_buys`/`enable_stop_sells` with `stop_order_type` on an option instrument. ### TC-E99: FOK limit option | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Adapter connected, option instrument loaded, sufficient book depth. | | **Action** | Place a limit order with `TimeInForce::Fok` on an option instrument. | | **Event sequence** | `OrderInitialized` -> `OrderSubmitted` -> `OrderAccepted` -> `OrderFilled` or `OrderCanceled`. | | **Pass criteria** | Order fills completely or is canceled. No partial fills. | | **Skip when** | Adapter does not support FOK for options. | **Considerations:** - Some venues use a dedicated order type for options FOK orders (e.g. OKX uses `op_fok`). The adapter handles this mapping transparently. - Use small quantities and aggressive pricing to get a fill for the positive case. ### TC-E100: Cancel option order | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open limit order from TC-E90 or TC-E91. | | **Action** | Cancel the open limit order. | | **Event sequence** | `OrderPendingCancel` -> `OrderCanceled`. | | **Pass criteria** | Order canceled; no longer appears in open orders on the venue. | | **Skip when** | Adapter does not support options. | ### TC-E101: Reconcile option position | Field | Value | |--------------------|------------------------------------------------------------------------| | **Prerequisite** | Open option position from a prior session. | | **Action** | Start the node with `reconciliation=True`. | | **Event sequence** | `PositionStatusReport` generated for the option position. | | **Pass criteria** | Position loaded into cache with correct instrument, side, quantity, and entry price. | | **Skip when** | Adapter does not support options. | **Considerations:** - Open an option position in a prior session and stop without closing (`close_positions_on_stop=False`). - Verify the reconciled position matches the venue-reported state. --- ## ExecTester configuration reference Quick reference for all `ExecTesterConfig` parameters. Defaults shown are for the Python config; the Rust builder uses equivalent defaults. | Parameter | Type | Default | Affects groups | |-------------------------------------------------|-------------------|-----------------|----------------| | `instrument_id` | InstrumentId | *required* | All | | `order_qty` | Decimal | *required* | All | | `order_display_qty` | Decimal? | None | 2, 7 | | `order_expire_time_delta_mins` | PositiveInt? | None | 2 | | `order_params` | dict? | None | 7, 10 | | `client_id` | ClientId? | None | All | | `subscribe_quotes` | bool | True | | | `subscribe_trades` | bool | True | | | `subscribe_book` | bool | False | | | `book_type` | BookType | L2_MBP | | | `book_depth` | PositiveInt? | None | | | `book_interval_ms` | PositiveInt | 1000 | | | `book_levels_to_print` | PositiveInt | 10 | | | `open_position_on_start_qty` | Decimal? | None | 1, 9 | | `open_position_time_in_force` | TimeInForce | GTC | 1 | | `enable_limit_buys` | bool | True | 2, 4, 5, 6 | | `enable_limit_sells` | bool | True | 2, 4, 5, 6 | | `enable_stop_buys` | bool | False | 3, 4 | | `enable_stop_sells` | bool | False | 3, 4 | | `limit_time_in_force` | TimeInForce? | None | 2, 6 | | `tob_offset_ticks` | PositiveInt | 500 | 2, 4 | | `stop_order_type` | OrderType | STOP_MARKET | 3 | | `stop_offset_ticks` | PositiveInt | 100 | 3 | | `stop_limit_offset_ticks` | PositiveInt? | None | 3 | | `stop_time_in_force` | TimeInForce? | None | 3 | | `stop_trigger_type` | TriggerType? | None | 3 | | `enable_brackets` | bool | False | 6 | | `bracket_entry_order_type` | OrderType | LIMIT | 6 | | `bracket_offset_ticks` | PositiveInt | 500 | 6 | | `modify_orders_to_maintain_tob_offset` | bool | False | 4 | | `modify_stop_orders_to_maintain_offset` | bool | False | 4 | | `cancel_replace_orders_to_maintain_tob_offset` | bool | False | 4 | | `cancel_replace_stop_orders_to_maintain_offset` | bool | False | 4 | | `use_post_only` | bool | False | 2, 6, 7, 8 | | `use_quote_quantity` | bool | False | 1, 7 | | `emulation_trigger` | TriggerType? | None | 2, 3 | | `cancel_orders_on_stop` | bool | True | 5, 9 | | `close_positions_on_stop` | bool | True | 9 | | `close_positions_time_in_force` | TimeInForce? | None | 9 | | `reduce_only_on_stop` | bool | True | 7, 9 | | `use_individual_cancels_on_stop` | bool | False | 5 | | `use_batch_cancel_on_stop` | bool | False | 5 | | `dry_run` | bool | False | | | `log_data` | bool | True | | | `test_reject_post_only` | bool | False | 8 | | `test_reject_reduce_only` | bool | False | 8 | | `can_unsubscribe` | bool | True | 9 | # Test Datasets Source: https://nautilustrader.io/docs/latest/developer_guide/test_datasets/ Target standards for curating, storing, and consuming external datasets used as test fixtures. New datasets should follow these standards. Existing datasets that predate this policy are documented under [legacy datasets](#legacy-datasets). ## Dataset categories **Small data** (< 1 MB) is checked directly into `tests/test_data//` alongside a `metadata.json` file. These files are always available without network access. **Large data** (> 1 MB) is hosted as Parquet in the R2 test-data bucket. A SHA-256 checksum is recorded in `tests/test_data/large/checksums.json`. The `ensure_test_data_exists()` helper downloads the file on first use and verifies integrity. **User-fetched data** is used when a vendor license, entitlement model, or access control does not allow NautilusTrader to redistribute the data through the public repo or the public R2 bucket. In this model, the repo stores only a manifest, fetch instructions, and the transform code. Each user downloads the source data with their own vendor account and converts it locally. Use the user-fetched model when any of the following apply: - The vendor requires each user to hold their own account, API key, or historical-data license. - The license allows internal use but does not clearly allow redistribution of derived fixtures. - The dataset is suitable for examples or opt-in integration tests, but not for default CI. ## Required metadata Every curated dataset that stores or redistributes a concrete artifact must include a `metadata.json` with at minimum: | Field | Description | |----------------|-------------------------------------------------------------------| | `file` | Filename of the dataset. | | `sha256` | SHA-256 hash of the file. | | `size_bytes` | File size in bytes. | | `original_url` | Download URL of the original source data. | | `licence` | License terms and any redistribution constraints. | | `added_at` | ISO 8601 timestamp when the dataset was curated. | These fields match the output of `scripts/curate-dataset.sh`. Additional recommended fields for richer provenance: | Field | Description | |-----------------|------------------------------------------------------------------| | `instrument` | Instrument symbol(s) covered. | | `date` | Trading date(s) covered. | | `format` | Storage format (e.g., "Nautilus OrderBookDelta Parquet"). | | `original_file` | Original vendor filename before transformation. | | `parser` | Parser used for transformation (e.g., "itchy 0.3.4"). | User-fetched datasets use the same metadata fields where they apply. They should also include: | Field | Description | |-----------------------|-----------------------------------------------------------------------| | `distribution` | Must be `"user-fetch"`. | | `fetch_method` | How the user acquires the source data (API, web portal, CLI, etc.). | | `fetch_reference` | URL or document reference for the user‑facing download flow. | | `auth` | Required credentials or entitlements, if any. | | `transform_version` | Version of the local transform pipeline that builds the final files. | | `redistribution` | Short note describing redistribution limits for the dataset. | | `public_mirror` | Must be `false` for restricted vendor datasets. | For user-fetched datasets without a single committed or mirrored artifact, `file`, `sha256`, and `size_bytes` may be omitted from `metadata.json`. In that case, `target_files` in `manifest.json` is authoritative for the local output files. For user-fetched datasets, `original_url` may point to the vendor download entry point rather than the exact file URL when the exact file is generated per user account or per request. Other metadata fields remain recommended where they apply. In particular, `licence` and `added_at` should still be recorded for user-fetched datasets. ## Storage format New datasets should be stored as **Nautilus Parquet** (not raw vendor formats). This ensures: - Consistent data types across all test datasets. - No vendor format parsing at test time. - Clear derivative-work status for licensing. Use ZSTD compression (level 3) with 1M row groups. User-fetched datasets should also end up as Nautilus Parquet after the local transform step. Raw vendor files should stay outside the repo and outside the public R2 bucket. ## Naming convention ``` ___.parquet ``` Examples: - `itch_AAPL_2019-01-30_deltas.parquet` - `tardis_BTCUSDT_2020-09-01_depth10.parquet` - `histdata_EURUSD.SIM_2020-01_quotes.parquet` ## Curation workflow ### Simple files (single download) Use `scripts/curate-dataset.sh`: ```bash scripts/curate-dataset.sh ``` This creates a versioned directory (`v1//`) with the file, `LICENSE.txt`, and `metadata.json` containing the required fields above. ### Complex pipelines (parse + transform) For datasets requiring format conversion (e.g., binary ITCH to Parquet): 1. Write a curation function in `crates/testkit/src//` gated behind `#[cfg(test)]` or an `#[ignore]` test. 2. The function should: download, parse, filter, convert to NautilusTrader types, write Parquet. 3. Output the Parquet file and `metadata.json` to a local directory. 4. Upload to R2 manually, then add the checksum to `checksums.json`. ### User-fetched pipelines (restricted redistribution) For datasets that NautilusTrader cannot redistribute: 1. Commit a manifest and `metadata.json`, but do not commit the real vendor data or derived Parquet output. 2. Provide a local fetch command or helper that uses the user's own vendor credentials, entitlements, or purchased historical files. 3. Convert the vendor data locally into Nautilus Parquet. 4. Store the resulting files in a local cache path that is ignored by git. 5. Make tests and examples opt in. They should skip cleanly when the dataset is missing. The default distribution order for new datasets is: 1. Checked in small data. 2. Public R2 large data. 3. User-fetched data. Choose user-fetched only when the first two options are not acceptable under the vendor's terms. Do not: - Upload restricted vendor datasets to the public R2 bucket. - Commit real vendor-derived Parquet files to the repo when redistribution rights are unclear. - Make default CI depend on vendor credentials or paid historical-data access. You may maintain a private mirror for internal CI or employees when the license permits internal sharing. Treat this as a separate operational path, not as part of the public test-data standard. ## Adding a new dataset 1. Curate the data following the workflow above. 2. Write `metadata.json` with all required fields. 3. For small data: commit to `tests/test_data//`. 4. For large data: upload Parquet to R2, add checksum to `tests/test_data/large/checksums.json`. 5. For user-fetched data: commit the manifest and fetch instructions only. Keep the source and derived data out of the repo and out of the public R2 bucket. 6. Add path helper functions to `crates/testkit/src/common.rs` when shared testkit access is needed. 7. Write tests that consume the dataset. For user-fetched data, prefer this layout: ```text tests/test_data/// metadata.json manifest.json README.md ``` Use `tests/test_data/local///` as the standard local cache path for generated artifacts. Keep raw vendor downloads in a sibling `vendor/` directory under the same cache path when local retention is needed. The manifest should be machine-readable and stable. It should capture the minimum information needed to reproduce the fetch and transform steps on another machine. `metadata.json` is authoritative for provenance, licensing, and redistribution rules. `manifest.json` is authoritative for fetch inputs, commands, cache locations, and output files. Recommended manifest fields: | Field | Description | |---------------------|--------------------------------------------------------------------| | `slug` | Stable dataset identifier. | | `vendor` | Vendor or venue name. | | `source_type` | `api`, `portal-download`, `purchased-archive`, etc. | | `source_filters` | Symbols, event IDs, market IDs, date ranges, or file names. | | `target_files` | Output Nautilus Parquet files expected after conversion. | | `cache_dir` | Local output location relative to `tests/test_data/local/`. | | `fetch_command` | Suggested command or script entry point. | | `transform_command` | Suggested local conversion command. | | `env` | Required environment variables. | | `notes` | Short operational notes for users. | Tests that rely on user-fetched data should: - Be marked or grouped separately from default CI tests. - Skip with a clear message when the local dataset is absent. - Avoid network access unless the user explicitly opts in. - Reuse a stable local cache path so the fetch happens once per machine. For pytest-based tests, prefer a guard like: ```python if not filepath.exists(): pytest.skip(f"User-fetched test data not found: {filepath}") ``` For Rust tests that require manual dataset preparation, prefer `#[ignore]` when the test is not expected to run in default CI. ## Test runner serialization Tests that download large data files share target paths across test binaries. Because `nextest` runs each binary in a separate process, concurrent downloads to the same path can race. The nextest config at `.config/nextest.toml` defines a `large-data-tests` group with `max-threads = 1` to serialize these binaries. When adding a new test binary that downloads large shared files, add it to the group filter: ```toml [[profile.default.overrides]] filter = 'binary(grid_mm_itch) | binary(orderbook_integration) | binary(your_new_binary)' test-group = 'large-data-tests' ``` ## Regenerating datasets When a schema change invalidates a large Parquet file, regenerate it from the original source data using the curation tests below. After regenerating: 1. `sha256sum /tmp/.parquet` 2. Update `tests/test_data/large/checksums.json` with the new hash. 3. Update the corresponding `metadata.json` (sha256, size_bytes). 4. Upload the Parquet file to R2. 5. Commit `checksums.json` and `metadata.json` (this also busts the CI cache). ### ITCH AAPL L3 deltas Source: `01302019.NASDAQ_ITCH50.gz` (~4.4 GB) from NASDAQ EMI. ```bash # Download source (keep a local copy, this is a large file) wget -O ~/Downloads/01302019.NASDAQ_ITCH50.gz \ "https://emi.nasdaq.com/ITCH/Nasdaq%20ITCH/01302019.NASDAQ_ITCH50.gz" # Curation test expects source at /tmp ln -sf ~/Downloads/01302019.NASDAQ_ITCH50.gz /tmp/01302019.NASDAQ_ITCH50.gz # Regenerate parquet (output: /tmp/itch_AAPL.XNAS_2019-01-30_deltas.parquet) cargo test -p nautilus-testkit --lib test_curate_aapl_itch -- --ignored --nocapture ``` ### Tardis Deribit BTC-PERPETUAL L2 deltas Source: `tardis_deribit_incremental_book_L2_2020-04-01_BTC-PERPETUAL.csv.gz` from [Tardis](https://tardis.dev/). First-of-month data is available as free samples (no API key required). ```bash # Download source (free sample, no API key needed) wget -O tests/test_data/large/tardis_deribit_incremental_book_L2_2020-04-01_BTC-PERPETUAL.csv.gz \ "https://datasets.tardis.dev/v1/deribit/incremental_book_L2/2020/04/01/BTC-PERPETUAL.csv.gz" # Regenerate parquet (output: /tmp/tardis_BTC-PERPETUAL.DERIBIT_2020-04-01_deltas.parquet) cargo test -p nautilus-tardis test_curate_deribit_deltas -- --ignored --nocapture ``` ## Tutorial test data Several tutorials load user-provided market data. The `NAUTILUS_DATA_DIR` environment variable overrides the base data path used by these tutorials. The test suite sets this variable to `tests/test_data/local/` so that tutorials run against small sample files stored locally. ### Directory layout ```text tests/test_data/local/ Binance/ BTCUSDT_T_DEPTH_2022-11-01_depth_snap.csv BTCUSDT_T_DEPTH_2022-11-01_depth_update.csv Bybit/ 2024-12-01_XRPUSDT_ob500.data.zip HISTDATA/ DAT_ASCII_EURUSD_T_202001.csv.gz ``` The `tests/test_data/local/` directory is gitignored. Tests skip when the data is absent. ### Obtaining the data **Binance depth snapshots** are available from the [Binance public data portal](https://data.binance.vision/). Download the BTCUSDT T_DEPTH files for 2022-11-01 and place the snap and update CSVs under `tests/test_data/local/Binance/`. For testing, a subset of rows (e.g. first 10,000) is sufficient. **Bybit ob500 orderbook data** is available from the Bybit CDN: ```bash curl -L "https://quote-saver.bycsi.com/orderbook/linear/XRPUSDT/2024-12-01_XRPUSDT_ob500.data.zip" \ -o tests/test_data/local/Bybit/2024-12-01_XRPUSDT_ob500.data.zip ``` The full file is ~360 MB. For testing, extract the first few hundred lines and repackage as a smaller zip. **HISTDATA tick data** is available from [histdata.com](https://www.histdata.com/). Download EUR/USD ASCII tick data for any month and place the CSV (or `.csv.gz`) under `tests/test_data/local/HISTDATA/`. ### Running the tests ```bash pytest tests/docs_tests/test_tutorials.py::test_tutorial_with_local_data -v ``` Tests skip with a message when the corresponding data subdirectory is empty or missing. ## Legacy datasets These datasets predate this policy and use raw vendor formats (CSV/CSV.gz) without `metadata.json`. They remain valid for existing tests. New datasets should follow the Parquet standard above. | Dataset | Source | Format | Location | Status | |-------------------------------|----------|------------------|---------------------------|----------| | Tardis Deribit L2 deltas | Tardis | Parquet (large) | `tests/test_data/large/` | Curated | | ITCH AAPL L3 deltas | NASDAQ | Parquet (large) | `tests/test_data/large/` | Curated | | HISTDATA EURUSD.SIM quotes | HISTDATA | Parquet (large) | `tests/test_data/large/` | Migrated | | Tardis Deribit L2 | Tardis | CSV (checked in) | `tests/test_data/tardis/` | Legacy | | Tardis Binance snapshots | Tardis | CSV.gz (large) | `tests/test_data/large/` | Legacy | | Tardis Bitmex trades | Tardis | CSV.gz (large) | `tests/test_data/large/` | Legacy | The former `nautechsystems/nautilus_data` catalog maps to the HISTDATA EURUSD.SIM Parquet files above. Raw HISTDATA CSV files remain user-fetched. # Testing Source: https://nautilustrader.io/docs/latest/developer_guide/testing/ Our automated tests serve as executable specifications for the trading platform. A healthy suite documents intended behaviour, gives contributors confidence to refactor, and catches regressions before they reach production. Tests also double as living examples that clarify complex flows and provide rapid CI feedback so issues surface early. The suite covers these categories: - Unit tests - Integration tests - Acceptance tests - Performance tests - Property-based tests - Fuzzing - Memory leak tests ## Testing policy Tests and runtime contracts form one design system. The [Design by contract](rust.md#design-by-contract) ladder pushes invariants into the type system where possible; the testing ladder below escalates the remaining unknowns through larger input spaces and richer execution models. Each layer extends coverage to inputs or execution states the layer below cannot reach. Not every module requires every technique. Use this section to decide which layers apply before adding tests or `debug_assert!` statements. ### Mechanism ladder Runtime contracts are covered in the [Rust guide](rust.md#design-by-contract): prefer the type system first, then `check_*` from `nautilus_core::correctness` at API boundaries, then `debug_assert!` for internal invariants, then `assert!` for soundness-critical or always-on checks. Test layers follow a parallel escalation. Start at the lowest layer that proves what matters; climb only when the layer below stops detecting regressions or when the input space grows beyond hand-picked cases. | Layer | Trigger condition | |--------------------------|---------------------------------------------------------------------------------| | Unit test | A single function or transition has a small, enumerable set of cases. | | Parametrized test | The same shape repeats across discrete inputs (order side, status, instrument). | | Property‑based test | An invariant must hold for a whole class of inputs the mind cannot enumerate. | | Integration test | Multiple modules interact through a real (non‑mocked) engine or runtime. | | Fuzz test | Untrusted or adversarial bytes cross a parser, decoder, or wire‑format handler. | | Spec acceptance test | Behaviour depends on a live venue contract (see `spec_exec_testing.md`). | | Deterministic simulation | Correctness depends on task scheduling, timeouts, or wall‑clock ordering. | | Formal verification | A pure function has crisp invariants and a bounded input space worth a proof. | The formal verification rung is aspirational: no Kani or Prusti harness has landed in the workspace. The row records the escalation condition for when a verifier is adopted, not a current obligation. ### Projection rule Module shape determines which layers pay off. Not every module warrants the full ladder. Apply the rule at module granularity, not crate granularity: an adapter crate contains pure parsers and I/O-bound client loops, and each row applies to a different part. | Module shape | Layers that apply | Example | |-------------------------------------|-----------------------------------------------|----------------------------------------| | Pure function, crisp invariants | Unit, parametrized, property, fuzz | Reconciliation kernels, portfolio math | | Pure function, no stated invariants | Unit, parametrized, property, fuzz | Codecs, adapter parsers, formatters | | Stateful, synchronous | Unit, parametrized, property over transitions | Cache, order book | | Stateful, async | Unit, integration, deterministic simulation | Live engine, execution manager | | I/O bound, venue contract | Integration, spec acceptance, boundary fuzz | Adapter client loops | ### When not to add coverage - Add `debug_assert!` only where a test can reach it. Release builds strip the check, so an unexercised assertion has no signal. A targeted unit test counts as a harness; a proptest or fuzz harness amplifies the signal. - Prefer a proptest over a hand-written edge-case test when the invariant spans a whole class of inputs. Targeted unit tests remain valid for known venue pathologies and as regression reproducers for shrunk counterexamples. - Do not duplicate a live spec acceptance card as an integration test. Link to it instead. - Do not pad coverage with tests that assert language or framework guarantees (`Option::is_some` after `Some(..)`, `Vec::len` after `push`). ### DST readiness Deterministic simulation testing (DST) requires the runtime to be free of ambient non-determinism. Before promoting a module to run under DST, verify the following: - Time, task, runtime, and signal primitives route through `nautilus_common::live::dst` rather than `tokio` directly. Wall-clock reads go through the seam in `nautilus_core::time` rather than `SystemTime::now()` at call sites. - State maps with ordering-dependent iteration use `IndexMap` or `IndexSet`, not the default hash collections. - Every `tokio::select!` on a control-plane path sets `biased` so poll order is fixed. - No calls to `Instant::now()`, `SystemTime::now()`, `tokio::signal::ctrl_c`, `std::thread::spawn`, or `tokio::task::spawn_blocking` escape the seam. Blocking-thread and OS-thread primitives break madsim determinism the same way an ambient clock read does. - Replay-sensitive IDs (`trade_id`, `venue_order_id`) are pure functions of their inputs; see `crates/execution/src/reconciliation/ids.rs`. Ephemeral event UUIDs on other reconciliation paths do not need to be deterministic. The `surface` probe in `crates/common/src/live/dst.rs` only pins the re-export shape; it does not check that callers actually use the seam. Enforcement is by review. Run the audit whenever a new async module enters the workspace or an existing module gains new control-plane scheduling. ## Property-based testing Property testing verifies that logic holds for *all* valid inputs, not just hand-picked examples. We use [`proptest`](https://altsysrq.github.io/proptest-book/intro.html) in Rust to enforce invariants. - **Use cases:** Core domain types (`Price`, `Quantity`, `UnixNanos`), accounting engines, matching engines, and state machines. - **Example invariants:** - Round-trip serialization: `parse(to_string(value)) == value` - Inverse operations: `(A + B) - B == A` - Transitivity: `If A < B and B < C, then A < C` ## Fuzzing Fuzzing introduces unstructured or malicious data to the system to verify it fails gracefully. - **Use cases:** Network boundaries, exchange data parsers (JSON, FIX, WebSocket feeds), and complex state machines. - **Goal:** The system returns a `Result::Err` and never panics, hangs, or leaks memory when encountering malformed data. When building or modifying core types, write property tests to cover the mathematical boundaries. Performance tests help evolve performance-critical components. Run tests with [pytest](https://docs.pytest.org), our primary test runner. Use parametrized tests and fixtures (e.g., `@pytest.mark.parametrize`) to avoid repetitive code and improve clarity. ## Running tests ### v1 legacy Python tests The v1 legacy test suite lives under `tests/` at the repository root and tests the Cython-based package. From the repository root: ```bash make pytest # or uv run --active --no-sync pytest --new-first --failed-first ``` ### Python tests The Python test suite lives under `python/tests/` and tests the Rust-backed PyO3 package. It requires a built extension module (`make build-debug-v2`) and uses its own virtualenv under `python/.venv/`. For new live adapter examples and docs in the v2 path, prefer `nautilus_trader.live.LiveNode`. `nautilus_trader.live.node.TradingNode` remains the legacy v1/Cython runtime used by the root-level `tests/` suite and older examples. ```bash make pytest-v2 ``` The Makefile target isolates certain test modules in separate pytest processes to avoid global Rust state conflicts. Use `make pytest-v2` rather than invoking pytest directly. Local `make pytest-v2` runs use the debug extension from `make build-debug-v2`. CI `build-v2` tests a release wheel. Do not write `python/tests/` cases that probe Rust panic paths in process with `pytest.raises(BaseException)` or similar broad catches. Those tests can appear to pass against the debug build and abort the interpreter against the release wheel. For abort-prone PyO3 or FFI methods, verify the Python signature and parameter names, or isolate the call in a subprocess. For performance tests: ```bash make test-performance # or uv run --active --no-sync pytest tests/performance_tests --benchmark-disable-gc --codspeed ``` The `--benchmark-disable-gc` flag prevents garbage collection from skewing results. Run performance tests in isolation (not with unit tests) to avoid interference. ### Rust tests ```bash make cargo-test # or cargo nextest run --workspace --features "arrow,ffi,python,high-precision,streaming,defi" --cargo-profile nextest --lib --tests ``` #### Testing with optional features Use `EXTRA_FEATURES` to include optional features like `capnp` or `hypersync`: ```bash # Test with capnp feature make cargo-test EXTRA_FEATURES="capnp" # Test with multiple features make cargo-test EXTRA_FEATURES="capnp hypersync" # Legacy shorthand for hypersync make cargo-test HYPERSYNC=true # Test specific crate with features make cargo-test-crate-nautilus-serialization FEATURES="capnp" ``` ### IDE integration - **PyCharm**: Right-click the tests folder or file -> "Run pytest". - **VS Code**: Use the Python Test Explorer extension. ## Test style ### General - Name test functions after what they exercise; you do not need to encode the expected assertions in the name. - Add docstrings when they clarify setup, scenarios, or expectations. - **Group assertions** when possible: perform all setup/act steps first, then assert together to avoid the act-assert-act smell. - Use `unwrap`, `expect`, or direct `panic!`/`assert` calls inside tests; clarity and conciseness matter more than defensive error handling here. - Do not capture log output to assert on log messages. Log capture in tests is fragile because loggers are global state, test execution order is non-deterministic, and the assertions break when log wording changes. Instead, verify the observable behavior (return values, state changes, side effects) that the log message reflects. ### Python tests (`python/tests/`) Use **pytest-style free functions and fixtures**. Do not use test classes. - Write each test as a standalone `def test_*()` function. - Use `@pytest.fixture` for shared setup (instruments, engine instances, data). Prefer `yield` fixtures when teardown is needed (e.g., `engine.dispose()`). - Use `@pytest.mark.parametrize` to cover multiple inputs without duplicating test bodies. - Import model types from `nautilus_trader.model`, not from `nautilus_trader.core.nautilus_pyo3`. - Test providers live in `python/tests/providers.py`. Use `TestInstrumentProvider` and `TestDataProvider` for common instruments and data. - Mark tests that depend on unfinished features with `@pytest.mark.skip(reason="WIP: ")` rather than deleting them. ### v1 legacy Python tests (`tests/`) The v1 legacy test suite uses a mix of test classes and free functions. New tests added to this suite may follow either pattern, but free functions with fixtures are preferred for new files. ### Rust For Rust-specific test conventions (module structure, `#[rstest]`, parameterization), see the [Rust guide](rust.md#testing-conventions). ## Waiting for asynchronous effects When waiting for background work to complete, prefer the polling helpers `await eventually(...)` from `nautilus_trader.test_kit.functions` and `wait_until_async(...)` from `nautilus_common::testing` instead of arbitrary sleeps. They surface failures faster and reduce flakiness in CI because they stop as soon as the condition is satisfied or time out with a useful error. ## Mocks Prefer hand-written stubs that return fixed values over mocking frameworks. Use `MagicMock` only when you need to assert call counts/arguments or simulate complex state changes. Avoid mocking the objects you're actually testing. ## Code coverage We generate coverage reports with `coverage` and publish them to [codecov](https://about.codecov.io/). Aim for high coverage without sacrificing appropriate error handling or causing "test induced damage" to the architecture. Some branches remain untestable without modifying production behaviour. For example, a final condition in a defensive if-else block may only trigger for unexpected values; leave these checks in place so future changes can exercise them if needed. Design-time exceptions can also be impractical to test, so 100% coverage is not the target. ## Excluded code coverage We use `pragma: no cover` comments to [exclude code from coverage](https://coverage.readthedocs.io/en/coverage-4.3.3/excluding.html) when tests would be redundant. Typical examples include: - Asserting an abstract method raises `NotImplementedError` when called. - Asserting the final condition check of an if-else block when impossible to test (as above). Such tests are expensive to maintain because they must track refactors while providing little value. Keep concrete implementations of abstract methods fully covered. Remove `pragma: no cover` when it no longer applies and restrict its use to the cases above. ## Debugging Rust tests Use the default test configuration to debug Rust tests. To run the full suite with debug symbols for later, run `make cargo-test-debug` instead of `make cargo-test`. In IntelliJ IDEA, adjust the run configuration for parametrised `#[rstest]` cases so it reads `test --package nautilus-model --lib data::bar::tests::test_get_time_bar_start::case_1` (remove `-- --exact` and append `::case_n` where `n` starts at 1). This workaround matches the behaviour explained [here](https://github.com/rust-lang/rust-analyzer/issues/8964#issuecomment-871592851). In VS Code you can pick the specific test case to debug directly. ## Python + Rust Mixed Debugging This workflow lets you debug Python and Rust code simultaneously from a Jupyter notebook inside VS Code. ### Setup Install these VS Code extensions: Rust Analyzer, CodeLLDB, Python, Jupyter. ### Step 0: Compile `nautilus_trader` with debug symbols ```bash cd nautilus_trader && make build-debug-pyo3 ``` ### Step 1: Set up debugging configuration ```python from nautilus_trader.test_kit.debug_helpers import setup_debugging setup_debugging() ``` This command creates the required VS Code debugging configurations and starts a `debugpy` server for the Python debugger. By default `setup_debugging()` expects the `.vscode` folder one level above the `nautilus_trader` root directory. Adjust the target location if your workspace layout differs. ### Step 2: Set breakpoints - **Python breakpoints:** Set in VS Code in the Python source files. - **Rust breakpoints:** Set in VS Code in the Rust source files. ### Step 3: Start mixed debugging 1. In VS Code select the **"Debug Jupyter + Rust (Mixed)"** configuration. 2. Start debugging (F5) or press the green run arrow. 3. Both Python and Rust debuggers attach to your Jupyter session. ### Step 4: Execute code Run Jupyter notebook cells that call Rust functions. The debugger stops at breakpoints in both Python and Rust code. ### Available configurations `setup_debugging()` creates these VS Code configurations: - **`Debug Jupyter + Rust (Mixed)`** - Mixed debugging for Jupyter notebooks. - **`Jupyter Mixed Debugging (Python)`** - Python-only debugging for notebooks. - **`Rust Debugger (for Jupyter debugging)`** - Rust-only debugging for notebooks. ### Example Open and run the example notebook: `debug_mixed_jupyter.ipynb`. ### Reference - [PyO3 debugging](https://pyo3.rs/v0.25.1/debugging.html?highlight=deb#debugging-from-jupyter-notebooks) ## Data type testing Each data type flows through multiple layers of the platform. The table below shows where existing types are tested, so new types can follow the same pattern. ### Test layer matrix | Layer | Location | What it covers | |------------------------|---------------------------------------------|------------------------------------------------------------| | DataEngine subscribe | `crates/data/tests/engine.rs` | Engine processes subscribe/unsubscribe commands correctly. | | DataEngine publish | `crates/data/tests/engine.rs` | Engine routes published data to the message bus. | | DataActor subscribe | `crates/common/src/actor/tests.rs` | Actor subscribes and receives data via typed publish. | | DataActor unsubscribe | `crates/common/src/actor/tests.rs` | Actor stops receiving data after unsubscribe. | | PyO3 actor dispatch | `crates/common/src/python/actor.rs` | Rust handler dispatches to Python `on_*` method. | | Python Actor subscribe | `tests/unit_tests/common/test_actor.py` | Python actor subscribes; command count increments. | | Python Actor unsub | `tests/unit_tests/common/test_actor.py` | Python actor unsubscribes; subscription list clears. | | Backtest client | `nautilus_trader/backtest/data_client.pyx` | Backtest client overrides base subscribe/unsubscribe. | | Adapter live tests | `docs/developer_guide/spec_data_testing.md` | Live data acceptance tests (DataTester). | ### Coverage per data type The following table shows which layers have test coverage for each data type. Use this as a checklist when adding a new type. | Data type | Engine | Actor (Rust) | PyO3 dispatch | Actor (Python) | Backtest client | Adapter spec | |---------------------|--------|--------------|---------------|----------------|-----------------|--------------| | `InstrumentAny` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `OrderBookDeltas` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `OrderBook` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `QuoteTick` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `TradeTick` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `Bar` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `MarkPriceUpdate` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `IndexPriceUpdate` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `FundingRateUpdate` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `InstrumentStatus` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `InstrumentClose` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `OptionGreeks` | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | `OptionChainSlice` | - | ✓ | ✓ | ✓ | - | ✓ | | `CustomData` | ✓ | ✓ | ✓ | ✓ | ✓ | - | `OptionChainSlice` is assembled by the DataEngine's `OptionChainManager` from per-instrument greeks and quote subscriptions. It does not have its own engine subscribe command or backtest client override. ### Adding a new data type When introducing a new data type, add tests at each layer: 1. **DataEngine** (`crates/data/tests/engine.rs`): Add `test_execute_subscribe_` and `test_execute_unsubscribe_` tests. Follow the pattern in existing subscribe tests: register client, build command, call `engine.execute`, assert subscription list. 2. **DataActor Rust** (`crates/common/src/actor/tests.rs`): - Add `received_: Vec` field to `TestDataActor`. - Implement the `on_` handler in the `DataActor` trait impl. - Add `test_subscribe_and_receive_` and `test_unsubscribe_` tests. - Use the typed publish function (`msgbus::publish_`), not `publish_any`, for types that use `TypedHandler` routing. 3. **PyO3 actor dispatch** (`crates/common/src/python/actor.rs`): - Add `dispatch_on_` method that calls `py_self.call_method1("on_", ...)`. - Add `on_` in the `DataActor` trait impl that calls the dispatch method. - Add `#[pyo3(name = "on_")]` method in the `#[pymethods]` block. - Add `on_` to `RustTestDataActor` wrapper and the inline Python test class. - Add handler test and dispatch test. 4. **Python Actor** (`tests/unit_tests/common/test_actor.py`): - Add `test_subscribe_` and `test_unsubscribe_` tests. - Assert `actor.subscribed_()` returns expected entries after subscribe and is empty after unsubscribe. 5. **Backtest client** (`nautilus_trader/backtest/data_client.pyx`): Override `subscribe_` and `unsubscribe_` if the base `MarketDataClient` raises `NotImplementedError` for the method. 6. **Documentation**: Add entries to `actors.md` callback table, `strategies.md` handler signatures, `adapters.md` subscribe method stubs, and `spec_data_testing.md` test cards. :::tip Search for an existing type like `instrument_close` or `funding_rate` across all six layers to find concrete examples of the patterns described above. ::: # Backtest (High-Level API) Source: https://nautilustrader.io/docs/latest/getting_started/backtest_high_level/ Use `BacktestNode` for config-driven backtesting with the Parquet data catalog. This is the recommended path for production workflows because the strategies, actors, and execution algorithms you build here carry forward to live trading with `TradingNode`. This tutorial loads FX quote tick data, writes it to a catalog, and backtests an EMA cross strategy on a simulated FX ECN venue. [View source on GitHub](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/getting_started/backtest_high_level.py). ## Prerequisites - Python 3.12+ - [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install nautilus_trader`) ``` import os import shutil from decimal import Decimal from pathlib import Path import pandas as pd from nautilus_trader.backtest.node import BacktestDataConfig from nautilus_trader.backtest.node import BacktestEngineConfig from nautilus_trader.backtest.node import BacktestNode from nautilus_trader.backtest.node import BacktestRunConfig from nautilus_trader.backtest.node import BacktestVenueConfig from nautilus_trader.config import ImportableStrategyConfig from nautilus_trader.core.datetime import dt_to_unix_nanos from nautilus_trader.model import QuoteTick from nautilus_trader.persistence.catalog import ParquetDataCatalog from nautilus_trader.persistence.wranglers import QuoteTickDataWrangler from nautilus_trader.test_kit.providers import CSVTickDataLoader from nautilus_trader.test_kit.providers import TestInstrumentProvider ``` ## Download sample data This example uses FX tick data from [histdata.com](https://www.histdata.com/download-free-forex-historical-data/?/ascii/tick-data-quotes/). Select an FX pair and one or more months to download. Downloaded files look like: - `DAT_ASCII_EURUSD_T_202410.csv` (EUR/USD for October 2024) - `DAT_ASCII_EURUSD_T_202411.csv` (EUR/USD for November 2024) Extract the CSV files into `~/Downloads/Data/HISTDATA/` (or set the `NAUTILUS_DATA_DIR` environment variable to the parent directory containing a `HISTDATA` subfolder). ``` DATA_DIR = Path(os.environ.get("NAUTILUS_DATA_DIR", "~/Downloads/Data")).expanduser() / "HISTDATA" ``` ``` path = DATA_DIR raw_files = [ f for f in path.iterdir() if f.is_file() and (f.suffix == ".csv" or f.name.endswith(".csv.gz")) ] assert raw_files, f"Unable to find any CSV files in directory {path}" raw_files ``` ## Load data into the catalog Histdata CSV files contain `timestamp, bid_price, ask_price` fields. Load the raw data into a DataFrame, then process it into Nautilus `QuoteTick` objects with a `QuoteTickDataWrangler`. ``` # Load the first CSV file into a pandas DataFrame df = CSVTickDataLoader.load( file_path=raw_files[0], index_col=0, header=None, names=["timestamp", "bid_price", "ask_price", "volume"], usecols=["timestamp", "bid_price", "ask_price"], parse_dates=["timestamp"], date_format="%Y%m%d %H%M%S%f", ) df = df.sort_index() df.head(2) ``` ``` # Process quotes using a wrangler EURUSD = TestInstrumentProvider.default_fx_ccy("EUR/USD") wrangler = QuoteTickDataWrangler(EURUSD) ticks = wrangler.process(df) # Preview: see first 2 ticks ticks[0:2] ``` See the [Loading data](../concepts/data) guide for more details. Instantiate a `ParquetDataCatalog` with a storage directory (here we use the current directory). Write the instrument and tick data to the catalog. ``` CATALOG_PATH = Path.cwd() / "catalog" # Clear if it already exists, then create fresh if CATALOG_PATH.exists(): shutil.rmtree(CATALOG_PATH) CATALOG_PATH.mkdir(parents=True) # Create a catalog instance catalog = ParquetDataCatalog(CATALOG_PATH) # Write instrument to the catalog catalog.write_data([EURUSD]) # Write ticks to catalog catalog.write_data(ticks) ``` ## Query the catalog The catalog provides methods like `.instruments()` and `.quote_ticks()` to query stored data and determine the available time range. ``` # Get list of all instruments in catalog catalog.instruments() ``` ``` # See 1st instrument from catalog instrument = catalog.instruments()[0] instrument ``` ``` # Query quote ticks from catalog to determine the data range all_ticks = catalog.quote_ticks(instrument_ids=[EURUSD.id.value]) print(f"Total ticks in catalog: {len(all_ticks)}") if all_ticks: # Get timestamps from the data first_tick_time = pd.Timestamp(all_ticks[0].ts_init, unit="ns", tz="UTC") last_tick_time = pd.Timestamp(all_ticks[-1].ts_init, unit="ns", tz="UTC") print(f"Data range: {first_tick_time} to {last_tick_time}") # Set backtest range to first 2 weeks of data (as ISO strings for BacktestDataConfig) start_time = first_tick_time.isoformat() end_time = (first_tick_time + pd.Timedelta(days=14)).isoformat() print(f"Backtest range: {start_time} to {end_time}") # Preview selected data start_ns = all_ticks[0].ts_init end_ns = dt_to_unix_nanos(first_tick_time + pd.Timedelta(days=14)) selected_quote_ticks = catalog.quote_ticks( instrument_ids=[EURUSD.id.value], start=start_ns, end=end_ns, ) print(f"Selected ticks for backtest: {len(selected_quote_ticks)}") selected_quote_ticks[:2] else: raise ValueError("No ticks found in catalog") ``` ## Add venues ``` venue_configs = [ BacktestVenueConfig( name="SIM", oms_type="HEDGING", account_type="MARGIN", base_currency="USD", starting_balances=["1_000_000 USD"], ), ] ``` ## Add data ``` str(CATALOG_PATH) ``` ``` data_configs = [ BacktestDataConfig( catalog_path=str(CATALOG_PATH), data_cls=QuoteTick, instrument_id=instrument.id, start_time=start_time, end_time=end_time, ), ] ``` ## Add strategies ``` strategies = [ ImportableStrategyConfig( strategy_path="nautilus_trader.examples.strategies.ema_cross:EMACross", config_path="nautilus_trader.examples.strategies.ema_cross:EMACrossConfig", config={ "instrument_id": instrument.id, "bar_type": "EUR/USD.SIM-15-MINUTE-BID-INTERNAL", "fast_ema_period": 10, "slow_ema_period": 20, "trade_size": Decimal(1_000_000), }, ), ] ``` ## Configure the backtest `BacktestRunConfig` centralizes venue, data, and strategy configuration in one object. It is Partialable, so you can build it in stages. This reduces boilerplate when running parameter sweeps or grid searches. ``` config = BacktestRunConfig( engine=BacktestEngineConfig(strategies=strategies), data=data_configs, venues=venue_configs, ) ``` ## Run the backtest `BacktestNode` processes all data in timestamp order with deterministic execution semantics. The architectural patterns (strategies, actors, execution algorithms) carry forward to live trading with `TradingNode`. ``` node = BacktestNode(configs=[config]) results = node.run() results ``` # Backtest (Low-Level API) Source: https://nautilustrader.io/docs/latest/getting_started/backtest_low_level/ Use `BacktestEngine` for direct component access: load market data, wire up strategies and execution algorithms, and run backtests with full control over every step. This tutorial backtests an EMA cross strategy with a TWAP execution algorithm on a simulated Binance Spot exchange using historical trade tick data. [View source on GitHub](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/getting_started/backtest_low_level.py). ## Prerequisites - Python 3.12+ - [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install nautilus_trader`) ``` from decimal import Decimal from nautilus_trader.backtest.config import BacktestEngineConfig from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.examples.algorithms.twap import TWAPExecAlgorithm from nautilus_trader.examples.strategies.ema_cross_twap import EMACrossTWAP from nautilus_trader.examples.strategies.ema_cross_twap import EMACrossTWAPConfig from nautilus_trader.model import BarType from nautilus_trader.model import Money from nautilus_trader.model import TraderId from nautilus_trader.model import Venue from nautilus_trader.model.currencies import ETH from nautilus_trader.model.currencies import USDT from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.persistence.wranglers import TradeTickDataWrangler from nautilus_trader.test_kit.providers import TestDataProvider from nautilus_trader.test_kit.providers import TestInstrumentProvider ``` ## Load data Load bundled test data (ETHUSDT trades from Binance), initialize the matching instrument, and wrangle the raw CSV into Nautilus `TradeTick` objects. ``` # Load stub test data provider = TestDataProvider() trades_df = provider.read_csv_ticks("binance/ethusdt-trades.csv") # Initialize the instrument which matches the data ETHUSDT_BINANCE = TestInstrumentProvider.ethusdt_binance() # Process into Nautilus objects wrangler = TradeTickDataWrangler(instrument=ETHUSDT_BINANCE) ticks = wrangler.process(trades_df) ``` See the [Data](../concepts/data.md) concept guide for details on the data processing pipeline. ## Initialize the engine Pass a `BacktestEngineConfig` to configure the engine. Here we set a custom `trader_id` to show the pattern. ``` # Configure backtest engine config = BacktestEngineConfig(trader_id=TraderId("BACKTESTER-001")) # Build the backtest engine engine = BacktestEngine(config=config) ``` ## Add a venue Set up a simulated venue that matches the market data. Here we configure a Binance Spot exchange with a cash account. ``` # Add a trading venue (multiple venues possible) BINANCE = Venue("BINANCE") engine.add_venue( venue=BINANCE, oms_type=OmsType.NETTING, account_type=AccountType.CASH, # Spot CASH account (not for perpetuals or futures) base_currency=None, # Multi-currency account starting_balances=[Money(1_000_000.0, USDT), Money(10.0, ETH)], ) ``` ## Add data Add the instrument and trade ticks to the engine. ``` # Add instrument(s) engine.add_instrument(ETHUSDT_BINANCE) # Add data engine.add_data(ticks) ``` :::note You can add multiple data types (including custom types) and backtest across multiple venues. ::: ## Add strategies Configure and add an EMA cross strategy with TWAP execution parameters. ``` # Configure your strategy strategy_config = EMACrossTWAPConfig( instrument_id=ETHUSDT_BINANCE.id, bar_type=BarType.from_str("ETHUSDT.BINANCE-250-TICK-LAST-INTERNAL"), trade_size=Decimal("0.10"), fast_ema_period=10, slow_ema_period=20, twap_horizon_secs=10.0, twap_interval_secs=2.5, ) # Instantiate and add your strategy strategy = EMACrossTWAP(config=strategy_config) engine.add_strategy(strategy=strategy) ``` The strategy config references TWAP parameters, but the execution algorithm itself is a separate component. ## Add execution algorithms Add a TWAP execution algorithm to the engine. ``` # Instantiate and add your execution algorithm exec_algorithm = TWAPExecAlgorithm() # Using defaults engine.add_exec_algorithm(exec_algorithm) ``` ## Run the backtest Call `.run()` to process all available data. The engine replays events in timestamp order with deterministic execution semantics. ``` # Run the engine (from start to end of data) engine.run() ``` ## Post-run analysis The engine retains data and execution objects in memory for generating reports. It also logs a tearsheet with default statistics; see the [Portfolio statistics](../concepts/portfolio.md#portfolio-statistics) guide for custom statistics. ``` engine.trader.generate_account_report(BINANCE) ``` ``` engine.trader.generate_order_fills_report() ``` ``` engine.trader.generate_positions_report() ``` ## Repeated runs Reset the engine for repeated runs. Instruments, data, and loaded components persist across resets; loaded components have their internal state reset. ``` # For repeated backtest runs, reset the engine engine.reset() # Clear or remove loaded components before adding replacements. ``` Remove and add individual components (actors, strategies, execution algorithms) as required. See the [Trader](../api_reference/trading.md) API reference for a description of all methods available to achieve this. ``` # Once done, good practice to dispose of the object if the script continues engine.dispose() ``` # Getting Started Source: https://nautilustrader.io/docs/latest/getting_started/ To get started with NautilusTrader you need: - Python 3.12–3.14 with the `nautilus_trader` package installed - A way to run Python scripts or Jupyter notebooks ## Examples The docs cover a subset of examples. For the full set, see the [nautilus_trader repository](https://github.com/nautechsystems/nautilus_trader). | Directory | Description | | :------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------- | | [examples/](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples) | Self-contained runnable Python examples | | [docs/tutorials/](../tutorials/) | Jupyter notebook tutorials for common workflows | | [docs/concepts/](../concepts/) | Concept guides with code snippets | | [nautilus_trader/examples/](https://github.com/nautechsystems/nautilus_trader/tree/master/nautilus_trader/examples) | Basic strategies, indicators, and execution algorithms | | [tests/unit_tests/](https://github.com/nautechsystems/nautilus_trader/tree/master/tests/unit_tests) | Unit tests covering core functionality | ## Backtesting API levels NautilusTrader provides two API levels for backtesting: | API Level | Description | Characteristics | | :--------- | :----------------------------- | :--------------------------------------------------------------------------------------------- | | High-Level | `BacktestNode` / `TradingNode` | Recommended for production, easier transition to live trading; requires a Parquet data catalog | | Low-Level | `BacktestEngine` | For library development, direct component access; no live-trading path | :::warning[One node per process] Running multiple `BacktestNode` or `TradingNode` instances in the same process is not supported due to global singleton state. Sequential execution with proper disposal between runs is supported. See [Processes and threads](../concepts/architecture.md#processes-and-threads). ::: :::info See the [Backtesting](../concepts/backtesting.md) guide for help choosing an API level. ::: ## Running in Docker A self-contained Jupyter notebook server is available as a Docker image, no local setup required. ```bash docker pull ghcr.io/nautechsystems/jupyterlab:nightly --platform linux/amd64 docker run -p 8888:8888 ghcr.io/nautechsystems/jupyterlab:nightly ``` Then open `http://localhost:8888` in your browser. :::note Container data is ephemeral, deleting the container removes all data. ::: :::info NautilusTrader log output can exceed Jupyter's default rate limit, causing notebooks to hang. Set `log_level` to `"ERROR"` to avoid this. ::: # Installation Source: https://nautilustrader.io/docs/latest/getting_started/installation/ NautilusTrader is officially supported for Python 3.12-3.14 on the following 64-bit platforms: | Operating System | Supported Versions | CPU Architecture | |------------------------|--------------------|-------------------| | Linux (Ubuntu) | 22.04 and later | x86_64 | | Linux (Ubuntu) | 22.04 and later | ARM64 | | macOS | 15.0 and later | ARM64 | | Windows Server | 2022 and later | x86_64 | :::note NautilusTrader may work on other platforms, but only those listed above are regularly used by developers and tested in CI. ::: Continuous CI coverage comes from the GitHub Actions runners we build on: - `Linux (Ubuntu)` builds currently pin to `ubuntu-22.04` to keep glibc 2.35 compatibility even as `ubuntu-latest` moves ahead. - `macOS (ARM64)` builds run on `macos-latest`, so support tracks that runner image as it moves ahead. - `Windows (x86_64)` builds currently pin to `windows-2022` to keep the toolchain stable. On Linux, confirm your glibc version with `ldd --version` and ensure it reports 2.35 or newer before proceeding. We recommend using the latest supported version of Python and installing [nautilus_trader](https://pypi.org/project/nautilus_trader/) inside a virtual environment to isolate dependencies. **There are two supported ways to install**: 1. Pre-built binary wheel from PyPI *or* the Nautech Systems package index. 2. Build from source. :::tip We highly recommend installing using the [uv](https://docs.astral.sh/uv) package manager with a "vanilla" CPython. Conda and other Python distributions *may* work but aren’t officially supported. ::: ## From PyPI To install the latest [nautilus_trader](https://pypi.org/project/nautilus_trader/) binary wheel (or sdist package) from PyPI: ```bash uv pip install nautilus_trader ``` ## Extras Install optional dependencies as 'extras' for specific integrations: - `betfair`: Betfair adapter (integration) dependencies. - `docker`: Needed for Docker when using the IB gateway (with the Interactive Brokers adapter). - `ib`: Interactive Brokers adapter (integration) dependencies. - `polymarket`: Polymarket adapter (integration) dependencies. - `visualization`: Plotly-based interactive tearsheets and charts. To install with specific extras: ```bash uv pip install "nautilus_trader[docker,ib]" ``` ## From the Nautech Systems package index The Nautech Systems package index (`packages.nautechsystems.io`) complies with [PEP-503](https://peps.python.org/pep-0503/) and hosts both stable and development binary wheels for `nautilus_trader`. This enables users to install either the latest stable release or pre-release versions for testing. ### Stable wheels Stable wheels correspond to official releases of `nautilus_trader` on PyPI, and use standard versioning. To install the latest stable release: ```bash uv pip install nautilus_trader --index-url=https://packages.nautechsystems.io/simple ``` :::tip Use `--extra-index-url` instead of `--index-url` if you want uv to fall back to PyPI automatically: ::: ### Development wheels Development wheels are published from both the `nightly` and `develop` branches, allowing users to test features and fixes ahead of stable releases. This process also helps preserve compute resources and provides easy access to the exact binaries tested in CI pipelines, while adhering to [PEP-440](https://peps.python.org/pep-0440/) versioning standards: - `develop` wheels use the version format `dev{date}+{build_number}` (e.g., `1.208.0.dev20241212+7001`). - `nightly` wheels use the version format `a{date}` (alpha) (e.g., `1.208.0a20241212`). | Platform | Nightly | Develop | | :----------------- | :------ | :------ | | `Linux (x86_64)` | ✓ | ✓ | | `Linux (ARM64)` | ✓ | - | | `macOS (ARM64)` | ✓ | - | | `Windows (x86_64)` | ✓ | - | **Note**: Development wheels from the `develop` branch publish for Linux x86_64 only. Windows, macOS, and Linux ARM64 builds run on the nightly schedule to keep CI feedback fast. :::warning We do not recommend using development wheels in production environments, such as live trading controlling real capital. ::: ### Installation commands By default, uv will install the latest stable release. Adding the `--pre` flag ensures that pre-release versions, including development wheels, are considered. To install the latest available pre-release (including development wheels): ```bash uv pip install nautilus_trader --pre --index-url=https://packages.nautechsystems.io/simple ``` To install a specific development wheel (e.g., `1.221.0a20250912` for September 12, 2025): ```bash uv pip install nautilus_trader==1.221.0a20250912 --index-url=https://packages.nautechsystems.io/simple ``` ### Python v2 development wheels Python v2 is the PyO3 package under `python/`. It is published to a separate v2 index while the main package remains in transition. ```bash uv pip install --pre --index-url=https://packages.nautechsystems.io/v2/simple/ nautilus-trader ``` The `--pre` flag is required because these wheels are pre-release builds. The installed import name is still `nautilus_trader`. Run this command outside a NautilusTrader source checkout. The repository root uses an `exclude-newer` uv policy for reproducible development, which can filter out newly published v2 wheels. Inside a source checkout, use [Build Python v2 from source](#8-build-python-v2-from-source) instead. Current v2 development wheels target Python 3.12-3.14. Build from source when you need local Rust changes, a debug build, or a platform wheel that is not available on the v2 index. ### Available versions You can view all available versions of `nautilus_trader` on the [package index](https://packages.nautechsystems.io/simple/nautilus-trader/index.html). To programmatically request and list available versions: ```bash curl -s https://packages.nautechsystems.io/simple/nautilus-trader/index.html | grep -oP '(?<= C++ Clang tools for Windows (latest) > Modify # 2. Add to PATH: [System.Environment]::SetEnvironmentVariable('path', "C:\Program Files\Microsoft Visual Studio\2022\BuildTools\VC\Tools\Llvm\x64\bin\;" + $env:Path,"User") ``` Verify: `clang --version` ### 4. Install uv Install [uv](https://docs.astral.sh/uv/getting-started/installation): ```bash tab="Linux/macOS" curl -LsSf https://astral.sh/uv/install.sh | sh ``` ```powershell tab="Windows" irm https://astral.sh/uv/install.ps1 | iex ``` ### 5. Clone and install Clone the source with `git`, and install from the project's root directory: ```bash git clone --branch develop --depth 1 https://github.com/nautechsystems/nautilus_trader cd nautilus_trader uv sync --all-extras ``` For development hosts and CI runner images, see the [single source of truth for versions](../developer_guide/environment_setup.md#single-source-of-truth-for-versions) before installing pinned tools. :::note The `--depth 1` flag fetches just the latest commit for a faster, lightweight clone. ::: ### 6. Install Cap'n Proto for development Install [Cap'n Proto](https://capnproto.org/) if you plan to enable the `capnp` Rust feature, regenerate serialization schemas, or work on serialization code. Use the repository script on Linux or macOS to install the pinned version from `tools.toml`: ```bash ./scripts/install-capnp.sh ``` Verify: `capnp --version` :::note Cap'n Proto is a development dependency. It is not required when installing pre-built wheels. ::: ### 7. Set environment variables Set environment variables for PyO3 compilation (Linux and macOS only). Run these commands from the repository root after `uv sync`: ```bash # Set the Python executable path for PyO3 export PYO3_PYTHON="$PWD/.venv/bin/python" # Linux only: Set the library path for the uv-managed Python runtime PYTHON_LIB_DIR="$("$PYO3_PYTHON" -c 'import sysconfig; print(sysconfig.get_config_var("LIBDIR"))')" export LD_LIBRARY_PATH="$PYTHON_LIB_DIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" # Required for Rust tests when using uv-installed Python export PYTHONHOME="$("$PYO3_PYTHON" -c 'import sys; print(sys.base_prefix)')" ``` :::note The `LD_LIBRARY_PATH` export is Linux-specific and not needed on macOS. The `PYTHONHOME` variable is required when running `make cargo-test` with a `uv`-installed Python. Without it, tests that depend on PyO3 may fail to locate the Python runtime. ::: ### 8. Build Python v2 from source This path builds the PyO3 package from the `python/` directory and installs it into `python/.venv`. Use it from a NautilusTrader source checkout, when a v2 development wheel is not available for your platform, or when you need local Rust changes. From the repository root: ```bash make build-debug-v2 ``` This target syncs `python/.venv`, builds the Rust extension with maturin, and regenerates Python type stubs. It uses `target-v2/` for Cargo artifacts so it does not conflict with the legacy Python build in `target/`. Run a Python v2 example with the v2 environment: ```bash cd python .venv/bin/python examples/lighter/data_tester.py --lighter-environment testnet ``` For direct commands and test targets, see the [Python v2 README][python-v2-readme]. [python-v2-readme]: https://github.com/nautechsystems/nautilus_trader/blob/develop/python/README.md ## From GitHub release To install a binary wheel from GitHub, first navigate to the [latest release](https://github.com/nautechsystems/nautilus_trader/releases/latest). Download the appropriate `.whl` for your operating system and Python version, then run: ```bash uv pip install .whl ``` ## Versioning and releases NautilusTrader is still under active development. Some features may be incomplete, and while the API is becoming more stable, breaking changes can occur between releases. We strive to document these changes in the release notes on a **best-effort basis**. We aim to follow a **bi-weekly release schedule**, though experimental or larger features may cause delays. Use NautilusTrader only if you are prepared to adapt to these changes. ## Redis Using [Redis](https://redis.io) with NautilusTrader is **optional** and only required if configured as the backend for a cache database or [message bus](../concepts/message_bus.md). :::info The minimum supported Redis version is 6.2 (required for [streams](https://redis.io/docs/latest/develop/data-types/streams/) functionality). ::: For a quick setup, we recommend using a [Redis Docker container](https://hub.docker.com/_/redis/). You can find an example setup in the `.docker` directory, or run the following command to start a container: ```bash docker run -d --name redis -p 6379:6379 redis:latest ``` This command will: - Pull the latest version of Redis from Docker Hub if it's not already downloaded. - Run the container in detached mode (`-d`). - Name the container `redis` for easy reference. - Expose Redis on the default port 6379, making it accessible to NautilusTrader on your machine. To manage the Redis container: - Start it with `docker start redis` - Stop it with `docker stop redis` :::tip We recommend using [Redis Insight](https://redis.io/insight/) as a GUI to visualize and debug Redis data efficiently. ::: ## Precision mode NautilusTrader supports two precision modes for its core value types (`Price`, `Quantity`, `Money`), which differ in their internal bit-width and maximum decimal precision. - **High-precision**: 128-bit integers with up to 16 decimals of precision, and a larger value range. - **Standard-precision**: 64-bit integers with up to 9 decimals of precision, and a smaller value range. :::note By default, the official Python wheels ship in high-precision (128-bit) mode on Linux and macOS. On Windows, only standard-precision (64-bit) Python wheels are available because MSVC's C/C++ frontend does not support `__int128`, preventing the Cython/FFI layer from handling 128-bit integers. For pure Rust crates, high-precision works on all platforms (including Windows) since Rust handles `i128`/`u128` via software emulation. The default is standard-precision unless you explicitly enable the `high-precision` feature flag. ::: The performance tradeoff is that standard-precision is ~3–5% faster in typical backtests, but has lower decimal precision and a smaller representable value range. :::note Performance benchmarks comparing the modes are pending. ::: ### Build configuration The precision mode is determined by: - Setting the `HIGH_PRECISION` environment variable during compilation, **and/or** - Enabling the `high-precision` Rust feature flag explicitly. ```bash tab="High-precision (128-bit)" export HIGH_PRECISION=true make install-debug ``` ```bash tab="Standard-precision (64-bit)" export HIGH_PRECISION=false make install-debug ``` ### Rust feature flag To enable high-precision (128-bit) mode in Rust, add the `high-precision` feature to your `Cargo.toml`: ```toml [dependencies] nautilus_core = { version = "*", features = ["high-precision"] } ``` :::info See the [Value Types](../concepts/overview.md#value-types) specifications for more details. ::: # Quickstart Source: https://nautilustrader.io/docs/latest/getting_started/quickstart/ Run your first backtest in under five minutes. [View source on GitHub](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/getting_started/quickstart.py). ## Prerequisites - Python 3.12+ - `pip install nautilus_trader` ## Write a strategy A strategy extends the `Strategy` base class and overrides event handlers to react to market data. This one trades an EMA crossover: buy when a fast exponential moving average crosses above a slow one, sell when it crosses below. ``` from decimal import Decimal from nautilus_trader.config import StrategyConfig from nautilus_trader.indicators import ExponentialMovingAverage from nautilus_trader.model.data import Bar from nautilus_trader.model.data import BarType from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.identifiers import InstrumentId from nautilus_trader.trading.strategy import Strategy class EMACrossConfig(StrategyConfig, frozen=True): instrument_id: InstrumentId bar_type: BarType trade_size: Decimal fast_ema_period: int = 10 slow_ema_period: int = 20 class EMACross(Strategy): def __init__(self, config: EMACrossConfig): super().__init__(config) self.fast_ema = ExponentialMovingAverage(config.fast_ema_period) self.slow_ema = ExponentialMovingAverage(config.slow_ema_period) def on_start(self): self.register_indicator_for_bars(self.config.bar_type, self.fast_ema) self.register_indicator_for_bars(self.config.bar_type, self.slow_ema) self.subscribe_bars(self.config.bar_type) def on_bar(self, bar: Bar): if not self.indicators_initialized(): return if self.fast_ema.value >= self.slow_ema.value: if self.portfolio.is_flat(self.config.instrument_id): self.buy() elif self.portfolio.is_net_short(self.config.instrument_id): self.close_all_positions(self.config.instrument_id) self.buy() elif self.fast_ema.value < self.slow_ema.value: if self.portfolio.is_flat(self.config.instrument_id): self.sell() elif self.portfolio.is_net_long(self.config.instrument_id): self.close_all_positions(self.config.instrument_id) self.sell() def buy(self): instrument = self.cache.instrument(self.config.instrument_id) order = self.order_factory.market( self.config.instrument_id, OrderSide.BUY, instrument.make_qty(self.config.trade_size), ) self.submit_order(order) def sell(self): instrument = self.cache.instrument(self.config.instrument_id) order = self.order_factory.market( self.config.instrument_id, OrderSide.SELL, instrument.make_qty(self.config.trade_size), ) self.submit_order(order) def on_stop(self): self.close_all_positions(self.config.instrument_id) ``` `on_start` registers the two EMA indicators so the engine updates them automatically with each new bar. `on_bar` waits for the indicators to warm up, then enters or reverses a position based on the crossover signal. ## Generate synthetic data To keep the quickstart self-contained, we generate 10,000 synthetic EUR/USD 1-minute bars using a random walk. In practice you would load real market data from a vendor or the Parquet data catalog. ``` import numpy as np import pandas as pd from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.config import BacktestEngineConfig from nautilus_trader.config import LoggingConfig from nautilus_trader.model.currencies import USD from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.model.identifiers import Venue from nautilus_trader.model.objects import Money from nautilus_trader.persistence.wranglers import BarDataWrangler from nautilus_trader.test_kit.providers import TestInstrumentProvider # Create a EUR/USD instrument on the SIM venue EURUSD = TestInstrumentProvider.default_fx_ccy("EUR/USD") # Generate synthetic 1-minute bars (random walk around 1.10) rng = np.random.default_rng(42) n = 10_000 price = 1.10 + np.cumsum(rng.normal(0, 0.0002, n)) spread = np.abs(rng.normal(0, 0.0003, n)) bars_df = pd.DataFrame( { "open": price, "high": price + spread, "low": price - spread, "close": price + rng.normal(0, 0.00005, n), }, index=pd.date_range("2024-01-01", periods=n, freq="1min", tz="UTC"), ) bars_df["high"] = bars_df[["open", "high", "close"]].max(axis=1) bars_df["low"] = bars_df[["open", "low", "close"]].min(axis=1) bar_type = BarType.from_str("EUR/USD.SIM-1-MINUTE-LAST-EXTERNAL") bars = BarDataWrangler(bar_type, EURUSD).process(bars_df) ``` `BarDataWrangler` converts a pandas DataFrame with OHLCV columns into Nautilus `Bar` objects. The bar type string encodes the instrument, aggregation period, price source, and data origin. ## Configure and run the engine Create a `BacktestEngine`, add a simulated FX venue with a margin account, wire up the instrument, data, and strategy, then run. The engine processes all bars in timestamp order with deterministic execution semantics. ``` engine = BacktestEngine( config=BacktestEngineConfig( logging=LoggingConfig(log_level="ERROR"), ), ) # Add a simulated FX venue SIM = Venue("SIM") engine.add_venue( venue=SIM, oms_type=OmsType.NETTING, account_type=AccountType.MARGIN, starting_balances=[Money(1_000_000, USD)], base_currency=USD, default_leverage=Decimal(1), ) # Add instrument, data, and strategy engine.add_instrument(EURUSD) engine.add_data(bars) strategy = EMACross( EMACrossConfig( instrument_id=EURUSD.id, bar_type=bar_type, trade_size=Decimal(100000), ), ) engine.add_strategy(strategy) # Run the backtest engine.run() ``` The engine processes all 10,000 bars in timestamp order. Each bar updates the registered indicators, then triggers `on_bar`. The simulated exchange fills market orders at the current price. ## Review results The engine generates reports from the completed backtest. The account report shows balance changes over time, the positions report lists each round-trip trade with its realized PnL, and the order fills report shows every execution. ``` engine.trader.generate_account_report(SIM) ``` ``` engine.trader.generate_positions_report() ``` ``` engine.trader.generate_order_fills_report() ``` ## Next steps - [Backtest (low-level API)](backtest_low_level) for direct `BacktestEngine` usage with real market data and execution algorithms. - [Backtest (high-level API)](backtest_high_level) for config-driven backtesting with `BacktestNode` and the Parquet data catalog. - [Tutorials](../tutorials/) for strategy pattern walkthroughs covering market making, mean reversion, order book imbalance, and more. ``` engine.dispose() ``` # Configure a Live Trading Node Source: https://nautilustrader.io/docs/latest/how_to/configure_live_trading/ Set up a `TradingNode` for live market connectivity. For background on live trading architecture and reconciliation, see the [Live trading](../concepts/live.md) concept guide. :::danger[Jupyter notebooks not recommended for live trading] Do not run live trading nodes in Jupyter notebooks. Event loop conflicts and operational risks make them unsuitable: - Jupyter runs its own asyncio event loop, which conflicts with `TradingNode`'s event loop. - Workarounds like `nest_asyncio` are not production-grade. - Cells can run out of order, kernels can crash, and state can disappear. - Notebooks lack the logging, monitoring, and graceful shutdown needed for production trading. Use Jupyter for backtesting, analysis, and experimentation. For live trading, run nodes as standalone Python scripts or services. ::: :::warning[One TradingNode per process] Running multiple `TradingNode` instances concurrently in the same process is not supported due to global singleton state. Add multiple strategies to a single node, or run additional nodes in separate processes for parallel execution. See [Processes and threads](../concepts/architecture.md#processes-and-threads) for details. ::: :::warning[Do not block the event loop] User code on the event loop thread (strategy callbacks, actor handlers, `on_event` methods) must return quickly. This applies to both Python and Rust. Blocking operations like model inference, heavy calculations, or synchronous I/O cause missed fills, stale data, and delayed order submissions. Offload long-running work to an executor or a separate thread/process. ::: :::info[Platform differences] Windows signal handling differs from Unix-like systems. If you are running on Windows, please read the note on [Windows signal handling](#windows-signal-handling) for guidance on graceful shutdown behavior and Ctrl+C (SIGINT) support. ::: ## TradingNodeConfig `TradingNodeConfig` inherits from `NautilusKernelConfig` and adds live-specific options. For background on how config structs handle defaults and `Option` semantics, see the [Configuration](../concepts/configuration.md) concept guide. ```python from nautilus_trader.config import TradingNodeConfig config = TradingNodeConfig( trader_id="MyTrader-001", # Component configurations cache=CacheConfig(), message_bus=MessageBusConfig(), data_engine=LiveDataEngineConfig(), risk_engine=LiveRiskEngineConfig(), exec_engine=LiveExecEngineConfig(), portfolio=PortfolioConfig(), # Client configurations data_clients={ "BINANCE": BinanceDataClientConfig(), }, exec_clients={ "BINANCE": BinanceExecClientConfig(), }, ) ``` ### Core configuration parameters | Setting | Default | Description | |--------------------------|--------------|---------------------------------------------| | `trader_id` | "TRADER-001" | Unique trader identifier (name‑tag format). | | `instance_id` | `None` | Optional unique instance identifier. | | `timeout_connection` | 60.0 | Connection timeout in seconds. | | `timeout_reconciliation` | 30.0 | Reconciliation timeout in seconds. | | `timeout_portfolio` | 10.0 | Portfolio initialization timeout. | | `timeout_disconnection` | 10.0 | Disconnection timeout. | | `timeout_post_stop` | 10.0 | Post‑stop cleanup timeout. | ### Cache database configuration Rust-native live systems keep cache behavior in `CacheConfig` and Redis connection settings in `RedisCacheConfig`. ```rust use nautilus_common::{ cache::{CacheConfig, database::CacheDatabaseFactory}, enums::SerializationEncoding, }; use nautilus_infrastructure::redis::cache::RedisCacheConfig; let config = CacheConfig { encoding: SerializationEncoding::MsgPack, timestamps_as_iso8601: true, buffer_interval_ms: Some(100), flush_on_start: false, ..Default::default() }; let database = RedisCacheConfig { host: Some("localhost".to_string()), port: Some(6379), username: Some("nautilus".to_string()), password: Some("pass".to_string()), connection_timeout: 2, response_timeout: 2, ..Default::default() }; let cache_database = database .create(trader_id, instance_id, config.clone()) .await?; ``` ### MessageBus configuration Message bus behavior stays in `MessageBusConfig`. Redis connection settings live in `RedisMessageBusConfig`, which constructs the backing through `MessageBusBackingFactory`. ```rust use nautilus_common::{ enums::SerializationEncoding, msgbus::{backing::MessageBusBackingFactory, config::MessageBusConfig}, }; use nautilus_infrastructure::redis::msgbus::{RedisMessageBusConfig, RedisMessageBusFactory}; let config = MessageBusConfig { encoding: SerializationEncoding::Json, timestamps_as_iso8601: true, use_instance_id: false, types_filter: Some(vec!["QuoteTick".to_string(), "TradeTick".to_string()]), stream_per_topic: false, autotrim_mins: Some(30), heartbeat_interval_secs: Some(1), ..Default::default() }; let backing = RedisMessageBusConfig { connection_timeout: 2, response_timeout: 2, ..Default::default() }; let message_bus_backing = RedisMessageBusFactory::new(backing).create( trader_id, instance_id, config.clone(), )?; ``` ## Multi-venue configuration A node can connect to multiple venues. This example configures both spot and futures markets for Binance: ```python config = TradingNodeConfig( trader_id="MultiVenue-001", # Multiple data clients for different market types data_clients={ "BINANCE_SPOT": BinanceDataClientConfig( account_type=BinanceAccountType.SPOT, environment=BinanceEnvironment.LIVE, ), "BINANCE_FUTURES": BinanceDataClientConfig( account_type=BinanceAccountType.USDT_FUTURES, environment=BinanceEnvironment.LIVE, ), }, # Corresponding execution clients exec_clients={ "BINANCE_SPOT": BinanceExecClientConfig( account_type=BinanceAccountType.SPOT, environment=BinanceEnvironment.LIVE, ), "BINANCE_FUTURES": BinanceExecClientConfig( account_type=BinanceAccountType.USDT_FUTURES, environment=BinanceEnvironment.LIVE, ), }, ) ``` ## ExecutionEngine configuration `LiveExecEngineConfig` controls order processing, execution events, and venue reconciliation. For full details see the [API Reference](/docs/python-api-latest/config.html#nautilus_trader.live.config.LiveExecEngineConfig). ### Reconciliation Recovers missed order and position events to keep system state consistent with the venue. | Setting | Default | Description | |---------------------------------|---------|---------------------------------------------------------------------------------| | `reconciliation` | True | Activate reconciliation at startup to align internal state with the venue. | | `reconciliation_lookback_mins` | None | How far back (minutes) to request past events for reconciling uncached state. | | `reconciliation_instrument_ids` | None | Include list of instrument IDs to reconcile. | | `filtered_client_order_ids` | None | Client order IDs to skip during reconciliation (for venue‑side duplicates). | See [Execution reconciliation](../concepts/live.md#execution-reconciliation) for details. ### Order filtering Controls which order events and reports the system processes, preventing conflicts across trading nodes. | Setting | Default | Description | |------------------------------------|---------|-------------------------------------------------------------------------------| | `filter_unclaimed_external_orders` | False | Drop unclaimed external orders so they do not affect the strategy. | | `filter_position_reports` | False | Drop position status reports. Useful when multiple nodes trade one account. | :::note[Order tagging behavior] Reconciliation tags orders by origin: - **`VENUE` tag**: external orders discovered at the venue (placed outside this system). - **`RECONCILIATION` tag**: synthetic orders generated to align position discrepancies. When `filter_unclaimed_external_orders` is enabled, only `VENUE`-tagged orders are filtered. `RECONCILIATION`-tagged orders are never filtered, so position alignment always succeeds. ::: ### Continuous reconciliation Continuous reconciliation keeps runtime execution state aligned after startup by checking in-flight orders, polling open orders, checking position status, and auditing own order books. Configure the loop with these settings. For runtime state-transition rules, retry coordination, and caveats, see [Runtime checks](../concepts/live.md#runtime-checks). | Setting | Default | Description | |--------------------------------------|----------------|--------------------------------------------------------------------------------------------------| | `inflight_check_interval_ms` | 2,000 ms | How often to check in‑flight order status. Set to 0 to disable. | | `inflight_check_threshold_ms` | 5,000 ms | Time before an in‑flight order triggers a venue status check. Lower if colocated. | | `inflight_check_retries` | 5 retries | Retry attempts to verify an in‑flight order with the venue. | | `open_check_interval_secs` | None | How often (seconds) to check open orders at the venue. None or 0.0 disables. Recommended: 5-10s. | | `open_check_open_only` | True | When true, query only open orders; when false, fetch full history (resource‑intensive). | | `open_check_lookback_mins` | 60 min | Lookback window (minutes) for order status polling. Only orders modified within this window. | | `open_check_threshold_ms` | 5,000 ms | Minimum time since last cached event before acting on venue discrepancies. | | `open_check_missing_retries` | 5 retries | Max retries before targeted not‑found resolution for eligible orders. | | `max_single_order_queries_per_cycle` | 10 | Cap on single‑order queries per cycle. Prevents rate‑limit exhaustion. | | `single_order_query_delay_ms` | 100 ms | Delay (ms) between single‑order queries to avoid rate limits. | | `reconciliation_startup_delay_secs` | 10.0 s | Delay (seconds) *after* startup reconciliation before continuous checks begin. | | `own_books_audit_interval_secs` | None | Interval (seconds) between auditing own order books against public books. | | `position_check_interval_secs` | None | Interval (seconds) between position consistency checks. On discrepancy, queries for missing fills. None disables. Recommended: 30-60s. | | `position_check_lookback_mins` | 60 min | Lookback window (minutes) for querying fill reports on position discrepancy. | | `position_check_threshold_ms` | 5,000 ms | Minimum time since last local activity before acting on position discrepancies. | | `position_check_retries` | 3 retries | Max attempts per instrument/account before the engine stops retrying that discrepancy. Once exceeded, an error is logged and the discrepancy is no longer actively reconciled until it clears. | :::warning - **`open_check_lookback_mins`**: do not reduce below 60 minutes. A short window triggers false "missing order" resolutions because orders fall outside the query range. - **`open_check_threshold_ms`**: increase if venue timestamps lag the local clock, so recently updated orders are not marked missing prematurely. - **`reconciliation_startup_delay_secs`**: do not reduce below 10 seconds in production. The delay lets the system stabilize after startup reconciliation before continuous checks begin. ::: ### Additional options | Setting | Default | Description | |------------------------------------|---------|-------------------------------------------------------------------------------------------------| | `allow_overfills` | False | Allow fills exceeding order quantity (logs warning). Useful when reconciliation races fills. | | `generate_missing_orders` | True | Generate LIMIT orders during reconciliation to align position discrepancies (strategy `EXTERNAL`, tag `RECONCILIATION`). | | `snapshot_orders` | False | Take order snapshots on order events. | | `snapshot_positions` | False | Take position snapshots on position events. | | `snapshot_positions_interval_secs` | None | Interval (seconds) between position snapshots. | | `debug` | False | Enable debug logging for execution. | ### Memory management Periodically purges closed orders, closed positions, and account events from the in-memory cache, keeping memory bounded during long-running or HFT sessions. | Setting | Default | Description | |----------------------------------------|---------|------------------------------------------------------------------------------------| | `purge_closed_orders_interval_mins` | None | How often (minutes) to purge closed orders from memory. Recommended: 10-15 min. | | `purge_closed_orders_buffer_mins` | None | How long (minutes) an order must be closed before purging. Recommended: 60 min. | | `purge_closed_positions_interval_mins` | None | How often (minutes) to purge closed positions from memory. Recommended: 10-15 min. | | `purge_closed_positions_buffer_mins` | None | How long (minutes) a position must be closed before purging. Recommended: 60 min. | | `purge_account_events_interval_mins` | None | How often (minutes) to purge account events from memory. Recommended: 10-15 min. | | `purge_account_events_lookback_mins` | None | How old (minutes) an account event must be before purging. Recommended: 60 min. | | `purge_from_database` | False | Also delete from the backing database (Redis/PostgreSQL). **Use with caution**. | Setting an interval enables the purge loop; leaving it unset disables scheduling and deletion. Database records are unaffected unless `purge_from_database` is true. Each loop delegates to the cache APIs described in [Cache](../concepts/cache.md). ### Queue management | Setting | Default | Description | |----------------------------------|---------|---------------------------------------------------------------------------------| | `qsize` | 100,000 | Size of internal queue buffers. | | `graceful_shutdown_on_exception` | False | Gracefully shut down on unexpected queue processing exceptions (not user code). | ## Strategy configuration For a complete parameter list see the `StrategyConfig` [API Reference](/docs/python-api-latest/config.html#nautilus_trader.trading.config.StrategyConfig). ### Identification | Setting | Default | Description | |----------------|---------|---------------------------------------------------------------| | `strategy_id` | None | Unique strategy identifier. | | `order_id_tag` | None | Unique tag appended to this strategy's order IDs. | ### Order management | Setting | Default | Description | |-----------------------------|---------|-------------------------------------------------------------------------------------------| | `oms_type` | None | [OMS type](../concepts/execution#oms-configuration) for position ID and order processing. | | `use_uuid_client_order_ids` | False | Use UUID4 values for client order IDs. | | `external_order_claims` | None | Instrument IDs whose external orders and reconciliation activity this strategy claims. | | `manage_contingent_orders` | False | Automatically manage OTO, OCO, and OUO contingent orders. | | `manage_gtd_expiry` | False | Manage GTD expirations for orders. | ## Windows signal handling :::warning Windows: asyncio event loops do not implement `loop.add_signal_handler`. As a result, `TradingNode` does not receive OS signals via asyncio on Windows. Use Ctrl+C (SIGINT) handling or programmatic shutdown; SIGTERM parity is not expected on Windows. ::: On Windows, asyncio event loops do not implement `loop.add_signal_handler`, so Unix-style signal integration is unavailable. `TradingNode` does not receive OS signals via asyncio on Windows and will not stop gracefully unless you intervene. Recommended approaches: - Wrap `run` with `try/except KeyboardInterrupt` and call `node.stop()` then `node.dispose()`. Ctrl+C raises `KeyboardInterrupt` in the main thread, giving you a clean teardown path. - Publish a `ShutdownSystem` command programmatically (or call `shutdown_system(...)` from an actor/component) to trigger the same shutdown path. The "inflight check loop task still pending" message appears because the normal graceful shutdown path is not triggered. This is tracked as [#2785](https://github.com/nautechsystems/nautilus_trader/issues/2785). The v2 `LiveNode` handles Ctrl+C (SIGINT) and, on Unix, SIGTERM in its Rust run loop. The Python v2 bridge also routes SIGINT into the same shutdown path, so runner and tasks shut down cleanly. Example pattern for Windows: ```python try: node.run() except KeyboardInterrupt: pass finally: try: node.stop() finally: node.dispose() ``` # Get Started with Lighter Source: https://nautilustrader.io/docs/latest/how_to/get_started_lighter/ Lighter is available through the v2 Rust engine. You can use it from a pure Rust project, or from Python v2 through PyO3 bindings that expose the same Rust data and execution clients to a Python `LiveNode`. The shortest path is to start with public data. Once data subscriptions work, add execution credentials and then add a strategy that can submit orders. ## Choose a setup path | Path | Use when | First step | |:------------|:----------------------------------------------------|:-------------------------------------------| | Pure Rust | You want a compiled app with no Python runtime. | Copy the Rust quickstart. | | Python v2 | You want Python scripts on the Rust engine. | Run the Python v2 data tester. | | RWA example | You want Databento signal data and Lighter trading. | Read the composite market making tutorial. | Start from these files: - Rust quickstart: `examples/quickstarts/lighter-rust-data-client/`. - Python v2 data tester: `python/examples/lighter/data_tester.py`. - RWA tutorial: [Composite market making tutorial][lighter-rwa-composite-mm]. The Rust and Python v2 paths both use these pieces: - `LighterDataClientConfig` selects mainnet or testnet and optional transport settings. - `LighterExecClientConfig` adds trader/account IDs and resolves credentials. - `LighterDataClientFactory` and `LighterExecutionClientFactory` register clients with `LiveNode`. - `DataTester` and `ExecTester` provide smoke-test actors before you write a custom strategy. ## Pure Rust starter Copy the quickstart into your own workspace: ```bash cp -R examples/quickstarts/lighter-rust-data-client ~/lighter-rust-data-client cd ~/lighter-rust-data-client cargo run ``` This builds a `LiveNode`, registers the Lighter data client, adds a `DataTester`, and connects to testnet public streams. Stop it with Ctrl+C. The core setup uses builders, which fill in optional defaults for you: ```rust let data_config = LighterDataClientConfig::builder() .environment(LighterEnvironment::Testnet) .build(); let mut node = LiveNode::builder(trader_id, Environment::Live)? .with_name("LIGHTER-DATA-STARTER-001".to_string()) .add_data_client( None, Box::new(LighterDataClientFactory::new()), Box::new(data_config), )? .build()?; ``` After the data path works, add an execution client to the builder before calling `.build()`: ```rust let exec_config = LighterExecClientConfig::builder() .trader_id(trader_id) .account_id(account_id) .environment(LighterEnvironment::Testnet) .build(); let mut node = LiveNode::builder(trader_id, Environment::Live)? .with_name("LIGHTER-EXEC-STARTER-001".to_string()) .add_data_client( None, Box::new(LighterDataClientFactory::new()), Box::new(data_config), )? .add_exec_client( None, Box::new(LighterExecutionClientFactory::new()), Box::new(exec_config), )? .build()?; ``` For execution, set the matching environment variables before connecting: ```bash export LIGHTER_TESTNET_ACCOUNT_INDEX="123456" export LIGHTER_TESTNET_API_KEY_INDEX="0" export LIGHTER_TESTNET_API_SECRET="your-lighter-api-secret" ``` Use `LIGHTER_ACCOUNT_INDEX`, `LIGHTER_API_KEY_INDEX`, and `LIGHTER_API_SECRET` for mainnet. ## Python v2 starter Python v2 uses the Rust engine through PyO3. Install a Python v2 development wheel outside a source checkout, or build the v2 package from source before running these examples. See [Python v2 installation][python-v2-install]. From a source checkout with Python v2 installed: ```bash cd python .venv/bin/python examples/lighter/data_tester.py --lighter-environment testnet ``` That command builds the node and exits. Pass `--run` to connect: ```bash .venv/bin/python examples/lighter/data_tester.py \ --lighter-environment testnet \ --instrument BTC-PERP.LIGHTER \ --run ``` The Python script mirrors the Rust setup: ```python builder = LiveNode.builder( "LIGHTER-DATA-TESTER-001", TraderId.from_str("TESTER-001"), Environment.LIVE, ).add_data_client( None, LighterDataClientFactory(), LighterDataClientConfig(environment=LighterEnvironment.TESTNET), ) ``` Use the execution tester only after the data tester works: ```bash .venv/bin/python examples/lighter/exec_tester.py \ --lighter-environment testnet \ --instrument DOGE-PERP.LIGHTER ``` Like the data tester, that command builds the node and exits. Pass `--run` to connect in dry-run mode, then add `--live-orders` to submit real orders. ## Move to a strategy The starter paths prove client wiring, subscriptions, and credential lookup. The next step is to replace the tester with a strategy: - Use [Write a Strategy (Rust)](write_rust_strategy.md) for a pure Rust strategy. - Use `python/examples/lighter/nvda_composite_mm.py` for Python v2 node wiring with the built-in Rust `CompositeMarketMaker` strategy. - Use [Composite market making on Lighter RWA][lighter-rwa-composite-mm] when you need the full Databento signal setup. :::warning Rust execution examples can submit live orders when you set `DRY_RUN` to `false`. Python execution examples can submit live orders when you pass `--run --live-orders`. Start on testnet or use the smallest accepted size, and confirm the instrument, environment, account index, API key index, and private key before you run. ::: For emergency cleanup, `cargo run --bin lighter-flatten -p nautilus-lighter` cancels open orders and closes positions for the configured Lighter account. Review it before use because it scans the account, can take several minutes under the standard 60 req/min quota, and affects more than one strategy or market when the account has broader exposure. [lighter-rwa-composite-mm]: ../tutorials/lighter_rwa_composite_mm.md [python-v2-install]: ../getting_started/installation.md#python-v2-development-wheels # How-To Source: https://nautilustrader.io/docs/latest/how_to/ Goal-oriented recipes for common tasks. Each guide assumes familiarity with Nautilus concepts and focuses on achieving a specific outcome. New to Nautilus? Start with the [Getting Started](../getting_started/) path and [Tutorials](../tutorials/) first. } /> } /> Set up TradingNodeConfig, execution engine, and venues. } href="configure_live_trading" icon={} /> } /> } /> Use BacktestEngine or BacktestNode with a catalog in Rust. } href="run_rust_backtest" icon={} /> Connect to a venue with LiveNode in Rust. } href="run_rust_live_trading" icon={} /> # Run a Backtest (Rust) Source: https://nautilustrader.io/docs/latest/how_to/run_rust_backtest/ Nautilus provides two Rust APIs for backtesting: `BacktestEngine` (low-level) and `BacktestNode` (high-level with catalog streaming). This guide covers both. For background on backtesting concepts, fill models, and matching engine behavior, see the [Backtesting](../concepts/backtesting.md) concept guide. For project setup and feature flags, see the [Rust](../concepts/rust.md#project-setup) concept guide. ## Dependencies Add the following to your `Cargo.toml`. The `streaming` and `nautilus-persistence` entries are only needed for the high-level `BacktestNode` API. ```toml [dependencies] nautilus-backtest = { version = "0.59", features = ["streaming"] } nautilus-execution = "0.59" nautilus-model = { version = "0.59", features = ["stubs"] } nautilus-persistence = "0.59" nautilus-trading = { version = "0.59", features = ["examples"] } ahash = "0.8" anyhow = "1" tempfile = "3" ustr = "1" ``` If you only need the low-level `BacktestEngine`, drop `streaming`, `nautilus-persistence`, `tempfile`, and `ustr`. ## BacktestEngine (low-level API) The low-level API gives direct control: you build the engine, add venues and instruments, load data in memory, register strategies, and run. ### 1. Create the engine ```rust use nautilus_backtest::{config::BacktestEngineConfig, engine::BacktestEngine}; let mut engine = BacktestEngine::new(BacktestEngineConfig::default())?; ``` ### 2. Add a venue `SimulatedVenueConfig` uses a `bon::Builder`: only required fields must be set, every other setting falls back to a documented default. `build()` validates the configuration and returns a `ConfigResult`, so propagate or unwrap it. ```rust use nautilus_backtest::config::SimulatedVenueConfig; use nautilus_model::{ enums::{AccountType, BookType, OmsType}, identifiers::Venue, types::Money, }; engine.add_venue( SimulatedVenueConfig::builder() .venue(Venue::from("SIM")) .oms_type(OmsType::Hedging) .account_type(AccountType::Margin) .book_type(BookType::L1_MBP) .starting_balances(vec![Money::from("1_000_000 USD")]) .build()?, )?; ``` Override any default by chaining setters, e.g. `.reject_stop_orders(false)` or `.allow_cash_borrowing(true)`. ### 3. Add instruments and data ```rust use nautilus_model::instruments::{ Instrument, InstrumentAny, stubs::audusd_sim, }; let instrument = InstrumentAny::CurrencyPair(audusd_sim()); let instrument_id = instrument.id(); engine.add_instrument(&instrument)?; let quotes = generate_quotes(instrument_id); // Your data loading function engine.add_data(quotes, None, true, true)?; ``` ### 4. Register a strategy and run ```rust use nautilus_model::types::Quantity; use nautilus_trading::examples::strategies::EmaCross; let strategy = EmaCross::new( instrument_id, Quantity::from("100000"), 10, // fast EMA period 20, // slow EMA period ); engine.add_strategy(strategy)?; engine.run(None, None, None, false)?; ``` ### Run the full example ```bash cargo run -p nautilus-backtest --features examples --example engine-ema-cross ``` Source: [`crates/backtest/examples/engine_ema_cross.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/backtest/examples/engine_ema_cross.rs) ## BacktestNode (high-level API) The high-level API loads data from a `ParquetDataCatalog` and streams in configurable chunk sizes. Requires the `streaming` feature on `nautilus-backtest`. ### 1. Write data to a catalog ```rust use nautilus_model::instruments::{ Instrument, InstrumentAny, stubs::audusd_sim, }; use nautilus_persistence::backend::catalog::ParquetDataCatalog; use tempfile::TempDir; let instrument = InstrumentAny::CurrencyPair(audusd_sim()); let instrument_id = instrument.id(); let quotes = generate_quotes(instrument_id); let temp_dir = TempDir::new()?; let catalog_path = temp_dir.path().to_str() .context("temp dir path is not valid UTF-8")? .to_string(); let catalog = ParquetDataCatalog::new( temp_dir.path(), None, None, None, None, ); catalog.write_instruments(vec![instrument])?; catalog.write_to_parquet("es, None, None, None)?; ``` ### 2. Configure the run ```rust use nautilus_backtest::config::{ BacktestDataConfig, BacktestRunConfig, BacktestVenueConfig, NautilusDataType, }; use nautilus_model::enums::{AccountType, BookType, OmsType}; let venue_config = BacktestVenueConfig::builder() .name("SIM") .oms_type(OmsType::Hedging) .account_type(AccountType::Margin) .book_type(BookType::L1_MBP) .starting_balances(vec!["1_000_000 USD".to_string()]) .build()?; let data_config = BacktestDataConfig::builder() .data_type(NautilusDataType::QuoteTick) .catalog_path(catalog_path) .instrument_id(instrument_id) .build()?; let run_config = BacktestRunConfig::builder() .id("ema-cross-run".to_string()) .venues(vec![venue_config]) .data(vec![data_config]) .chunk_size(100) .build()?; ``` ### 3. Build, add strategies, and run ```rust use nautilus_backtest::node::BacktestNode; use nautilus_model::types::Quantity; use nautilus_trading::examples::strategies::EmaCross; let mut node = BacktestNode::new(vec![run_config])?; node.build()?; let engine = node.get_engine_mut("ema-cross-run") .context("engine not found for run config ID")?; let strategy = EmaCross::new( instrument_id, Quantity::from("100000"), 10, 20, ); engine.add_strategy(strategy)?; node.run()?; ``` ### Run the full example ```bash cargo run -p nautilus-backtest --features examples,streaming --example node-ema-cross ``` Source: [`crates/backtest/examples/node_ema_cross.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/backtest/examples/node_ema_cross.rs) # Run Live Trading (Rust) Source: https://nautilustrader.io/docs/latest/how_to/run_rust_live_trading/ The `LiveNode` connects to real venues through adapter clients. This guide walks through a complete live trading setup using OKX as an example. For background on live trading architecture and reconciliation, see the [Live trading](../concepts/live.md) concept guide. For project setup and feature flags, see the [Rust](../concepts/rust.md#project-setup) concept guide. ## Dependencies Add the live crate, your venue adapter, and supporting crates to `Cargo.toml`: ```toml [dependencies] nautilus-common = "0.59" nautilus-live = "0.59" nautilus-model = "0.59" nautilus-okx = "0.59" nautilus-trading = { version = "0.59", features = ["examples"] } anyhow = "1" dotenvy = "0.15" log = "0.4" tokio = { version = "1", features = ["full"] } ``` ## Build the node The `LiveNode` uses a builder pattern. Add data and execution client factories for your venue, configure logging, and build. ```rust use log::LevelFilter; use nautilus_common::{enums::Environment, logging::logger::LoggerConfig}; use nautilus_live::node::LiveNode; use nautilus_model::identifiers::{AccountId, TraderId}; use nautilus_okx::{ common::enums::OKXInstrumentType, config::{OKXDataClientConfig, OKXExecClientConfig}, factories::{OKXDataClientFactory, OKXExecutionClientFactory}, }; let trader_id = TraderId::from("TESTER-001"); let account_id = AccountId::from("OKX-001"); let data_config = OKXDataClientConfig::builder() .instrument_types(vec![OKXInstrumentType::Swap]) .build(); let exec_config = OKXExecClientConfig::builder() .trader_id(trader_id) .account_id(account_id) .instrument_types(vec![OKXInstrumentType::Swap]) .build(); let log_config = LoggerConfig { stdout_level: LevelFilter::Info, ..Default::default() }; let mut node = LiveNode::builder(trader_id, Environment::Live)? .with_name("MY-NODE-001".to_string()) .with_logging(log_config) .add_data_client( None, Box::new(OKXDataClientFactory::new()), Box::new(data_config), )? .add_exec_client( None, Box::new(OKXExecutionClientFactory::new()), Box::new(exec_config), )? .with_reconciliation(false) // Simplified; enable in production .with_delay_post_stop_secs(5) .build()?; ``` :::warning This example disables reconciliation for simplicity. In production, remove `.with_reconciliation(false)` so the engine aligns cached state with the venue on startup. See [Execution reconciliation](../concepts/live.md#execution-reconciliation). ::: ## Add strategies and run ```rust use nautilus_model::{identifiers::InstrumentId, types::Quantity}; use nautilus_trading::examples::strategies::{ GridMarketMaker, GridMarketMakerConfig, }; let mut config = GridMarketMakerConfig::builder() .instrument_id(InstrumentId::from("ETH-USDT-SWAP.OKX")) .max_position(Quantity::from("0.10")) .num_levels(3) .grid_step_bps(100) .skew_factor(0.5) .requote_threshold_bps(10) .expire_time_secs(8) .on_cancel_resubmit(true) .build(); // OKX rejects hyphens in client order IDs config.base.use_hyphens_in_client_order_ids = false; let strategy = GridMarketMaker::new(config); node.add_strategy(strategy)?; node.run().await?; ``` The node runs until interrupted (Ctrl+C) or shut down programmatically. ## Environment variables OKX reads API credentials from environment variables. Use a `.env` file with `dotenvy` or set them in your shell: ```bash export OKX_API_KEY="your_api_key" export OKX_API_SECRET="your_api_secret" export OKX_API_PASSPHRASE="your_passphrase" ``` For demo trading, set `.environment(OKXEnvironment::Demo)` on both config builders and use demo API credentials from OKX. Each adapter documents its required variables in the [integration guide](../integrations/) for that venue. ## Async runtime `LiveNode::run()` is async and requires a Tokio runtime. Use `#[tokio::main]` on your main function: ```rust #[tokio::main] async fn main() -> Result<(), Box> { dotenvy::dotenv().ok(); // ... node setup ... node.run().await?; Ok(()) } ``` ## Adapter examples Most adapters include runnable examples with data testers and execution testers: | Adapter | Example directory | |--------------|--------------------------------------------| | Architect AX | `crates/adapters/architect_ax/examples/` | | Betfair | `crates/adapters/betfair/examples/` | | Binance | `crates/adapters/binance/examples/` | | BitMEX | `crates/adapters/bitmex/examples/` | | Bybit | `crates/adapters/bybit/examples/` | | Databento | `crates/adapters/databento/examples/` | | Deribit | `crates/adapters/deribit/examples/` | | dYdX | `crates/adapters/dydx/examples/` | | Hyperliquid | `crates/adapters/hyperliquid/examples/` | | Kraken | `crates/adapters/kraken/examples/` | | OKX | `crates/adapters/okx/examples/` | | Polymarket | `crates/adapters/polymarket/examples/` | | Sandbox | `crates/adapters/sandbox/examples/` | | Tardis | `crates/adapters/tardis/examples/` | # Write an Actor (Rust) Source: https://nautilustrader.io/docs/latest/how_to/write_rust_actor/ An actor receives market data, custom data/signals, and system events but does not manage orders. This guide walks through building a `SpreadMonitor` that subscribes to quotes and logs the bid-ask spread. For background on actors, traits, and handler dispatch, see the [Actors](../concepts/actors.md) and [Rust](../concepts/rust.md) concept guides. ## Define the struct An actor owns a `DataActorCore` and any state it needs. The core stores runtime state for the actor. User code normally reaches that state through the `DataActor` facade methods such as: - `clock()` - `cache()` - `config()` - `actor_id()` - `trader_id()` - Subscription methods ```rust use nautilus_common::{nautilus_actor, actor::{DataActor, DataActorConfig, DataActorCore}}; use nautilus_model::{data::QuoteTick, identifiers::{ActorId, InstrumentId}}; pub struct SpreadMonitor { core: DataActorCore, instrument_id: InstrumentId, } ``` ## Implement the constructor Create a `DataActorConfig` with an actor ID, then pass it to `DataActorCore::new`. The config fields use `Option` with defaults, so `..Default::default()` covers everything except the actor ID. ```rust impl SpreadMonitor { pub fn new(instrument_id: InstrumentId) -> Self { let config = DataActorConfig { actor_id: Some(ActorId::from("SPREAD_MON-001")), ..Default::default() }; Self { core: DataActorCore::new(config), instrument_id, } } } ``` ## Wire up the core and implement Debug The `nautilus_actor!` macro connects the actor's `DataActorCore` field to the runtime contract. By default it delegates to a field named `core`; pass a second argument for a different field name. Normal callbacks do not call the generated native accessors; use the `DataActor` facade methods on `self`. Runtime registration uses blanket `Actor` and `Component` implementations. The macro supplies the native runtime wiring; implement `Debug` manually or derive it. ```rust nautilus_actor!(SpreadMonitor); impl std::fmt::Debug for SpreadMonitor { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { f.debug_struct("SpreadMonitor").finish() } } ``` ## Implement the DataActor trait Override handler methods to receive data. All handlers have default no-op implementations, so you only override what you need. Each handler returns `anyhow::Result<()>`. ```rust impl DataActor for SpreadMonitor { fn on_start(&mut self) -> anyhow::Result<()> { self.subscribe_quotes(self.instrument_id, None, None); Ok(()) } fn on_quote(&mut self, quote: &QuoteTick) -> anyhow::Result<()> { let spread = quote.ask_price.as_f64() - quote.bid_price.as_f64(); log::info!("Spread: {spread:.5}"); Ok(()) } } ``` `subscribe_quotes` is available directly on `self` through the `DataActor` trait. See the [handler table](../concepts/rust.md#handler-methods) for all available handlers. ## Native runtime access Use the public `DataActor` facade by default. Add `DataActorNative` only for an explicit native-only access path that the facade methods cannot serve. Read-only properties are available on the facade: - `config()` - `actor_id()` - `trader_id()` - `is_registered()` The [Rust native traits](../concepts/rust.md#native-traits) section covers the native-traits applicability matrix and this method table: - [`DataActorNative` methods](../concepts/rust.md#dataactornative-methods) Those types do not cross the Python boundary, so portable actors should use facade methods such as: - `clock()` - `cache()` ## Register the actor With a `BacktestEngine`: ```rust let actor = SpreadMonitor::new(instrument_id); engine.add_actor(actor)?; ``` With a `LiveNode`: ```rust let actor = SpreadMonitor::new(instrument_id); node.add_actor(actor)?; ``` ## Guard safety When the system dispatches messages to your actor, it obtains a short-lived `ActorRef` guard from the registry. You do not manage these guards directly. If you write code that accesses other actors in a callback, follow these rules: - Look up actors by ID each time; do not cache an `ActorRef`. - Drop the guard before the scope ends; never store it in a field. - Never hold a guard across an `.await` point. The subscription methods on `DataActorCore` handle this correctly by capturing the actor ID and performing the lookup inside the callback closure. See [Runtime invariants](../developer_guide/rust.md#runtime-invariants) for the full threading and registry model. ## Full example See [`BookImbalanceActor`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/actors/imbalance) for a more complete actor that tracks per-instrument state and prints a summary on stop. # Write a Strategy (Rust) Source: https://nautilustrader.io/docs/latest/how_to/write_rust_strategy/ A strategy extends an actor with order management. This guide walks through building a minimal strategy that subscribes to quotes and submits market orders. Read [Write an Actor (Rust)](write_rust_actor.md) first. For background on strategy concepts and order management, see the [Strategies](../concepts/strategies.md) and [Rust](../concepts/rust.md) concept guides. ## Define the struct A strategy stores a `StrategyCore` field for runtime wiring. Normal strategy logic does not use the field directly; use the facade methods on `self`. ```rust use nautilus_common::actor::DataActor; use nautilus_model::{ data::QuoteTick, enums::OrderSide, identifiers::{InstrumentId, StrategyId}, types::Quantity, }; use nautilus_trading::{nautilus_strategy, strategy::{Strategy, StrategyConfig, StrategyCore}}; pub struct MyStrategy { core: StrategyCore, instrument_id: InstrumentId, trade_size: Quantity, } ``` ## Implement the constructor `StrategyConfig` takes a `strategy_id` and an `order_id_tag`. The tag is appended to all client order IDs from this strategy, preventing collisions when multiple strategies trade the same instrument. ```rust impl MyStrategy { pub fn new(instrument_id: InstrumentId) -> Self { let config = StrategyConfig { strategy_id: Some(StrategyId::from("MY_STRAT-001")), order_id_tag: Some("001".to_string()), ..Default::default() }; Self { core: StrategyCore::new(config), instrument_id, trade_size: Quantity::from("1.0"), } } } ``` ## Wire up the core and implement Debug The `nautilus_strategy!` macro generates the native runtime wiring used by registration and the `Strategy` trait impl. By default it delegates to a field named `core`; pass a second argument for a different field name. The macro does not make your strategy or its `StrategyCore` deref to runtime internals. It also adds `config()`, which returns the `StrategyConfig` passed to `StrategyCore::new`. Runtime registration uses blanket `Actor` and `Component` implementations that require native wiring and `Debug`. The macro supplies the native wiring; implement `Debug` manually or derive it. ```rust nautilus_strategy!(MyStrategy); impl std::fmt::Debug for MyStrategy { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { f.debug_struct("MyStrategy").finish() } } ``` ## Implement the DataActor trait Data handling works the same as in an actor. Subscribe in `on_start`, respond in handlers. ```rust impl DataActor for MyStrategy { fn on_start(&mut self) -> anyhow::Result<()> { self.subscribe_quotes(self.instrument_id, None, None); Ok(()) } fn on_quote(&mut self, quote: &QuoteTick) -> anyhow::Result<()> { let order = self.order().market( self.instrument_id, OrderSide::Buy, self.trade_size, None, None, None, None, None, None, None, ); self.submit_order(order, None, None, None)?; Ok(()) } } ``` `self.order()` builds orders and order lists. Available methods: - `market` - `limit` - `stop_market` - `stop_limit` - `market_to_limit` - `market_if_touched` - `limit_if_touched` - `trailing_stop_market` - `trailing_stop_limit` - `bracket` - `create_list` - `generate_client_order_id` - `generate_order_list_id` `submit_order` is available on `self` through the `Strategy` trait impl generated by the macro. ## Native runtime access Use the public facade in strategy logic: - `clock()` - `cache()` - `order()` - `portfolio()` - `strategy_id()` - The order management methods on `Strategy` Normal strategy code does not import `DataActorNative` or `StrategyNative`, and does not call native handles such as: - `core()` - `core_mut()` - `strategy_core()` - `strategy_core_mut()` - `order_factory()` - `order_factory_rc()` - `portfolio_rc()` Those native handles expose borrowed runtime state and stay in engine, runtime, registration, PyO3, testkit, or explicit latency-sensitive native code. The [Rust native traits](../concepts/rust.md#native-traits) section covers the native-traits applicability matrix and these method tables: - [`DataActorNative` methods](../concepts/rust.md#dataactornative-methods) - [`StrategyNative` methods](../concepts/rust.md#strategynative-methods) ## Override Strategy hooks To override `Strategy` trait methods such as order or position event handlers, pass them in a block. The macro generates the internal plumbing automatically; keep `DataActor` handlers in the separate `impl DataActor` block. ```rust nautilus_strategy!(MyStrategy, { fn on_order_rejected(&mut self, event: OrderRejected) { log::warn!("Order rejected: {}", event.reason); } }); ``` ## Order management methods The `Strategy` trait provides these facade methods: | Method | Action | |-----------------------|-------------------------------------------------| | `submit_order` | Submit a new order to the venue. | | `submit_order_list` | Submit a list of contingent orders. | | `modify_order` | Modify price, quantity, or trigger price. | | `modify_orders` | Modify multiple orders for the same instrument. | | `cancel_order` | Cancel a specific order. | | `cancel_orders` | Cancel a filtered set of orders. | | `cancel_all_orders` | Cancel all orders for an instrument. | | `close_position` | Close a position with a market order. | | `close_all_positions` | Close all open positions. | ## Full examples - [`EmaCross`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/ema_cross): Dual-EMA crossover with indicator integration. - [`GridMarketMaker`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/grid_mm): Grid market making with configurable levels and requoting. # AX Exchange Source: https://nautilustrader.io/docs/latest/integrations/architect_ax/ [AX Exchange](https://architect.exchange) is the world's first centralized and regulated exchange for perpetual futures on traditional underlying asset classes. Operated by Architect Bermuda Ltd. and licensed by the [Bermuda Monetary Authority (BMA)](https://www.bma.bm/), AX brings crypto-style perpetual contracts to traditional financial markets including foreign exchange, metals, energy, equity indices, and interest rates. This integration supports live market data ingest and order execution with AX Exchange. ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/architect_ax/). ## Overview This guide assumes a trader is setting up for both live market data feeds, and trade execution. The AX Exchange adapter includes multiple components, which can be used together or separately depending on the use case. - `AxHttpClient`: Low-level HTTP API connectivity. - `AxMdWebSocketClient`: Market data WebSocket connectivity. - `AxOrdersWebSocketClient`: Orders WebSocket connectivity. - `AxInstrumentProvider`: Instrument parsing and loading functionality. - `AxDataClient`: A market data feed manager. - `AxExecutionClient`: An account management and trade execution gateway. - `AxLiveDataClientFactory`: Factory for AX data clients (used by the trading node builder). - `AxLiveExecClientFactory`: Factory for AX execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to necessarily work with these lower level components directly. ::: ## AX Exchange documentation AX Exchange provides documentation for users which can be found at the [Architect documentation site](https://docs.architect.exchange/). It's recommended you also refer to the AX Exchange documentation in conjunction with this NautilusTrader integration guide. ## Products AX Exchange specializes in perpetual futures contracts on traditional asset classes. Perpetual contracts never expire, eliminating rollover costs associated with standard futures. | Asset Class | Examples | Notes | |------------------|------------------------------------|------------------------------| | Foreign exchange | GBPUSD-PERP, EURUSD-PERP | Major and minor FX pairs. | | Stock indices | Equity index perpetuals | | | Metals | XAU-PERP (gold), XAG-PERP (silver) | Precious metals perpetuals. | | Energy | Crude oil, natural gas | Energy commodity perpetuals. | | Interest rates | SOFR, treasury yields | Rate perpetuals. | ### Perpetual contracts A perpetual contract (perpetual swap) is a derivative that tracks the price of an underlying asset without expiring. Unlike standard futures, there is no settlement date, which eliminates rollover costs and simplifies position management. A funding rate mechanism keeps the contract price aligned with the underlying index price through periodic payments between long and short holders. See the [Architect documentation](https://docs.architect.exchange/) for details on funding rate mechanics and contract specifications. Characteristics of AX perpetual contracts: - **Cash-settled in USD**: No physical delivery. All profit and loss is settled in USD. - **Funding rates**: Periodic payments keep the contract price aligned with the underlying. - **Multiplier of 1**: Each contract represents one unit of exposure to the underlying. - **Whole contracts only**: Fractional quantities are not supported. - **Margin**: Initial margin is required to open a position; maintenance margin to keep it open. In NautilusTrader, all AX instruments are represented as `PerpetualContract`, an asset-class agnostic perpetual swap type. The asset class (FX, commodity, equity, etc.) is inferred automatically from the underlying. The adapter uses `MARGIN` account type and `NETTING` order management. ## Symbology AX Exchange uses a straightforward naming convention. All instruments are perpetual futures identified by the `-PERP` suffix appended to the underlying asset symbol. **Format**: `{SYMBOL}-PERP` | Underlying | AX Symbol | Nautilus InstrumentId | |----------------|----------------|-----------------------| | GBP/USD | `GBPUSD-PERP` | `GBPUSD-PERP.AX` | | EUR/USD | `EURUSD-PERP` | `EURUSD-PERP.AX` | | Gold | `XAU-PERP` | `XAU-PERP.AX` | | Silver | `XAG-PERP` | `XAG-PERP.AX` | The venue identifier is `AX`. To construct a Nautilus `InstrumentId`: ```python from nautilus_trader.model.identifiers import InstrumentId instrument_id = InstrumentId.from_str("GBPUSD-PERP.AX") ``` ## Environments AX Exchange provides two trading environments. Configure the appropriate environment using the `environment` parameter in your client configuration. | Environment | Config | Description | |----------------|----------------------------------------|----------------------------------------| | **Sandbox** | `environment=AxEnvironment.SANDBOX` | Test environment with simulated funds. | | **Production** | `environment=AxEnvironment.PRODUCTION` | Live trading with real funds. | ### Sandbox The default environment for development and testing with simulated funds. All sandbox endpoints are resolved automatically when `environment=AxEnvironment.SANDBOX`. #### 1. Create a sandbox account Follow the [Architect documentation](https://docs.architect.exchange/) to create a sandbox account. An invite code is required during registration. #### 2. Create API keys and fund the account Use the AX sandbox UI to generate API keys and deposit simulated funds into your account. Store the `api_key` and `api_secret` securely. #### 3. Set environment variables ```bash export AX_API_KEY="your-sandbox-api-key" export AX_API_SECRET="your-sandbox-api-secret" ``` #### 4. Configure the trading node ```python config = TradingNodeConfig( ..., # Omitted data_clients={ AX: AxDataClientConfig( environment=AxEnvironment.SANDBOX, instrument_provider=InstrumentProviderConfig(load_all=True), ), }, exec_clients={ AX: AxExecClientConfig( environment=AxEnvironment.SANDBOX, instrument_provider=InstrumentProviderConfig(load_all=True), ), }, ) ``` ### Production For live trading with real funds. Requires a verified AX Exchange account. ```python config = AxExecClientConfig( environment=AxEnvironment.PRODUCTION, ) ``` :::warning Ensure you are using the correct environment before placing orders. Sandbox is the default to prevent accidental live trading. ::: ## Market data The adapter provides real-time market data via WebSocket subscriptions, with HTTP endpoints for historical data backfill. ### Data types | AX Data | Nautilus Data Type | Notes | |-------------------|----------------------|--------------------------------------------------------------------| | Order book (L1) | `QuoteTick` | Best bid/ask top‑of‑book from L1 book subscription. | | Order book (L2) | `OrderBookDelta` | Aggregated price levels. | | Order book (L3) | `OrderBookDelta` | Individual order quantities. | | Trades | `TradeTick` | Real‑time trade events from L1 subscription. | | Mark price | `MarkPriceUpdate` | Extracted from L1 ticker subscription. | | Bars/candles | `Bar` | OHLCV data (total volume only, no buy/sell breakdown). | | Funding rates | `FundingRateUpdate` | Polled via HTTP (not real‑time WebSocket); interval configurable. | | Instrument status | `InstrumentStatus` | State changes (open, halted, closed) from L1 ticker subscription. | :::note Historical quote tick requests are not supported by AX Exchange. Only real-time quote data is available via WebSocket L1 book subscriptions. ::: ### Bar intervals | Interval | Description | |----------|-------------| | `1s` | 1-second | | `5s` | 5-second | | `1m` | 1-minute | | `5m` | 5-minute | | `15m` | 15-minute | | `1h` | 1-hour | | `1d` | 1-day | ## Orders capability AX Exchange supports market and limit order types only. The venue has no native stop/conditional orders (the place-order payload has no trigger or order-type field). ### Order types | Order Type | Supported | Notes | |------------------------|-----------|----------------------------------------------------| | `MARKET` | ✓ | Execute immediately at best available price. | | `LIMIT` | ✓ | Execute at specified price or better. | | `STOP_LIMIT` | - | *Not supported by AX Exchange*. | | `LIMIT_IF_TOUCHED` | - | *Not supported by AX Exchange*. | | `STOP_MARKET` | - | *Not supported*. | | `MARKET_IF_TOUCHED` | - | *Not supported*. | | `TRAILING_STOP_MARKET` | - | *Not supported*. | ### Execution instructions | Instruction | Supported | Notes | |---------------|-----------|-----------------------------------------------------| | `post_only` | ✓ | Maker‑only; rejected if order would take liquidity. | | `reduce_only` | - | *Not supported*. | ### Time in force | Time in Force | Supported | Notes | |---------------|-----------|---------------------------------| | `GTC` | ✓ | Good Till Canceled. | | `GTD` | - | *Not supported by AX Exchange*. | | `DAY` | ✓ | Valid until end of trading day. | | `IOC` | ✓ | Immediate or Cancel. | | `FOK` | - | *Not supported by AX Exchange*. | | `AT_THE_OPEN` | - | *Not supported by AX Exchange*. | | `AT_THE_CLOSE`| - | *Not supported by AX Exchange*. | The venue deprecates `DAY` and recommends `GTC` instead. ### Advanced order features | Feature | Supported | Notes | |--------------------|-----------|--------------------------------------------------------------------| | Order modification | ✓ | Atomic replace via `POST /replace_order`. Returns a new order ID. | | Cancel order | ✓ | Single order cancellation. | | Cancel all orders | ✓ | Cancel all open orders for an instrument. | | Batch cancel | - | *Not supported by AX Exchange*. Individual cancels used instead. | | Order lists | ✓ | Sequential submission (orders submitted individually, non‑atomic). | ### Position management | Feature | Supported | Notes | |------------------|-----------|--------------------------------------| | Query positions | ✓ | Real‑time position updates. | | Position mode | - | Netting mode only. | | Cross margin | ✓ | Cross‑margin across all instruments. | ### Order querying | Feature | Supported | Notes | |----------------------|-----------|---------------------------------------------------------| | Query open orders | ✓ | List all active orders. | | Query single order | ✓ | By venue order ID or client order ID (any order state). | | Order status reports | ✓ | Reconciliation from open orders; see note below. | | Fill reports | ✓ | Execution and fill history. | :::note Order status reports for reconciliation are generated from the open orders endpoint. Filled or canceled orders are not included in the reconciliation snapshot. Single-order queries via `query_order` use the dedicated `/order-status` endpoint which works for any order state. ::: ## Authentication AX Exchange uses bearer token authentication: 1. API key and secret obtain a session token via `/authenticate`. 2. The session token is used as a bearer token for subsequent REST and WebSocket requests. 3. Session tokens expire after a configurable period (default: 86400 seconds). ## Configuration ### Environments and endpoints | Environment | HTTP API (market data) | HTTP API (orders) | Market Data WS | Orders WS | |-------------|--------------------------------------------------|-----------------------------------------------------|--------------------------------------------------|------------------------------------------------------| | Sandbox | `https://gateway.sandbox.architect.exchange/api` | `https://gateway.sandbox.architect.exchange/orders` | `wss://gateway.sandbox.architect.exchange/md/ws` | `wss://gateway.sandbox.architect.exchange/orders/ws` | | Production | `https://gateway.architect.exchange/api` | `https://gateway.architect.exchange/orders` | `wss://gateway.architect.exchange/md/ws` | `wss://gateway.architect.exchange/orders/ws` | :::info Order management HTTP endpoints (place, cancel, order status) use a separate base URL from market data endpoints. This is handled automatically by the adapter configuration. ::: ### Data client configuration options | Option | Default | Description | |------------------------------------|-----------|---------------------------------------------------------------------| | `api_key` | `None` | API key; loaded from `AX_API_KEY` env var when omitted. | | `api_secret` | `None` | API secret; loaded from `AX_API_SECRET` env var when omitted. | | `environment` | `SANDBOX` | Trading environment (`SANDBOX` or `PRODUCTION`). | | `base_url_http` | `None` | Override for the REST base URL. | | `base_url_ws_public` | `None` | Override for the market data WebSocket URL. | | `base_url_ws_private` | `None` | Override for the orders WebSocket URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `http_timeout_secs` | `60` | Timeout (seconds) for REST requests. | | `max_retries` | `3` | Maximum retry attempts for REST requests. | | `retry_delay_initial_ms` | `1000` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10000` | Maximum delay (milliseconds) between retries (exponential backoff). | | `heartbeat_interval_secs` | `20` | Heartbeat interval (seconds) for WebSocket connections. | | `recv_window_ms` | `5000` | Receive window (milliseconds) for signed requests. | | `update_instruments_interval_mins` | `60` | Interval (minutes) between instrument catalog refreshes. | | `funding_rate_poll_interval_mins` | `15` | Interval (minutes) between funding rate poll requests. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |---------------------------|-----------|---------------------------------------------------------------------| | `api_key` | `None` | API key; loaded from `AX_API_KEY` env var when omitted. | | `api_secret` | `None` | API secret; loaded from `AX_API_SECRET` env var when omitted. | | `environment` | `SANDBOX` | Trading environment (`SANDBOX` or `PRODUCTION`). | | `base_url_http` | `None` | Override for the REST base URL. | | `base_url_orders` | `None` | Override for the orders REST base URL. | | `base_url_ws_private` | `None` | Override for the orders WebSocket URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `http_timeout_secs` | `60` | Timeout (seconds) for REST requests. | | `max_retries` | `3` | Maximum retry attempts for REST requests. | | `retry_delay_initial_ms` | `1000` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10000` | Maximum delay (milliseconds) between retries (exponential backoff). | | `heartbeat_interval_secs` | `30` | Heartbeat interval (seconds) for WebSocket connections. | | `recv_window_ms` | `5000` | Receive window (milliseconds) for signed requests. | | `cancel_on_disconnect` | `false` | Cancel all open orders when the orders WebSocket disconnects. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | The most common use case is to configure a live `TradingNode` to include AX Exchange data and execution clients. To achieve this, add an `AX` section to your client configuration(s): ```python from nautilus_trader.adapters.architect_ax import AX from nautilus_trader.adapters.architect_ax import AxDataClientConfig from nautilus_trader.adapters.architect_ax import AxEnvironment from nautilus_trader.adapters.architect_ax import AxExecClientConfig from nautilus_trader.config import InstrumentProviderConfig from nautilus_trader.config import TradingNodeConfig config = TradingNodeConfig( ..., # Omitted data_clients={ AX: AxDataClientConfig( environment=AxEnvironment.SANDBOX, instrument_provider=InstrumentProviderConfig(load_all=True), ), }, exec_clients={ AX: AxExecClientConfig( environment=AxEnvironment.SANDBOX, instrument_provider=InstrumentProviderConfig(load_all=True), ), }, ) ``` Then, create a `TradingNode` and add the client factories: ```python from nautilus_trader.adapters.architect_ax import AX from nautilus_trader.adapters.architect_ax import AxLiveDataClientFactory from nautilus_trader.adapters.architect_ax import AxLiveExecClientFactory from nautilus_trader.live.node import TradingNode # Instantiate the live trading node with a configuration node = TradingNode(config=config) # Register the client factories with the node node.add_data_client_factory(AX, AxLiveDataClientFactory) node.add_exec_client_factory(AX, AxLiveExecClientFactory) # Finally build the node node.build() ``` ### API credentials There are two options for supplying your credentials to the AX Exchange clients. Either pass the corresponding `api_key` and `api_secret` values to the configuration objects, or set the following environment variables: - `AX_API_KEY` - `AX_API_SECRET` :::tip We recommend using environment variables to manage your credentials. ::: When starting the trading node, you'll receive immediate confirmation of whether your credentials are valid and have trading permissions. ## Implementation notes - **Whole contracts only**: AX Exchange uses integer contract quantities. Fractional quantities are not supported; the adapter generates `OrderDenied` locally. - **Rate limiting**: The adapter applies a conservative rate limit of 10 requests/second with automatic exponential backoff on rate limit responses. - **Market orders**: AX does not support native market orders. The adapter uses a preview endpoint to determine the take-through price and submits an aggressive IOC limit order. - **Order modification**: AX supports atomic order replacement via `POST /replace_order`. The adapter maps `modify_order` to this endpoint. The exchange cancels the original order and creates a new one with the updated fields, returning a new order ID. - **Cancel on disconnect**: Set `cancel_on_disconnect=True` in the execution client config to have the exchange cancel all open orders if the orders WebSocket disconnects. - **Fill commissions**: Real-time fill events from the WebSocket do not include fee data. Commission is reported as zero for streaming fills. During reconciliation, the REST `/fills` endpoint provides accurate fee information. - **Fill reconciliation window**: The `/fills` endpoint requires a bounded time range and caps the span at seven days. Reconciliation requests the most recent seven days of fills; fills older than that are not reconciled. - **Unfilled IOC/FOK**: AX reports an unfilled immediate order as an expiry; the adapter maps it to `OrderCanceled` to match NautilusTrader semantics. ## Contributing :::info For additional features or to contribute to the AX Exchange adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Betfair Source: https://nautilustrader.io/docs/latest/integrations/betfair/ Founded in 2000, Betfair operates the world’s largest online betting exchange, with its headquarters in London and satellite offices across the globe. NautilusTrader provides an adapter for integrating with the Betfair REST API and Exchange Streaming API. ## Installation Install NautilusTrader with Betfair support: ```bash uv pip install "nautilus_trader[betfair]" ``` To build from source with Betfair extras: ```bash uv sync --all-extras ``` ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/betfair/). ## Betfair documentation Betfair provides documentation for developers: - [Betfair Developer Portal](https://developer.betfair.com/): Main entry point for API access and documentation. - [Exchange API Guide](https://developer.betfair.com/exchange-api/): Overview of the Betting, Accounts, and Streaming APIs. ## Application keys Betfair requires an Application Key to authenticate API requests. After registering and funding your account, obtain your key using the [API-NG Developer AppKeys Tool](https://apps.betfair.com/visualisers/api-ng-account-operations/). Two App Keys are assigned per account: a **Live** key (requires a one-time activation fee) and a **Delayed** key for development and testing. :::info See the [Application Keys](https://betfair-developer-docs.atlassian.net/wiki/spaces/1smk3cen4v3lu3yomq5qye0ni/pages/2687105/Application+Keys) documentation for detailed setup instructions. ::: ## API credentials Supply your Betfair credentials via environment variables or client configuration: ```bash export BETFAIR_USERNAME= export BETFAIR_PASSWORD= export BETFAIR_APP_KEY= export BETFAIR_CERTS_DIR= ``` :::tip We recommend using environment variables to manage your credentials. ::: :::note Current Rust note: Rust currently reads `BETFAIR_USERNAME`, `BETFAIR_PASSWORD`, and `BETFAIR_APP_KEY`. It does not yet read `BETFAIR_CERTS_DIR`. ::: ## SSL Certificates Betfair recommends [non-interactive (bot) login](https://betfair-developer-docs.atlassian.net/wiki/spaces/1smk3cen4v3lu3yomq5qye0ni/pages/2687915/Non-Interactive+bot+login) with SSL certificates for automated trading systems. The `certs_dir` configuration is optional, but certificates are recommended for production deployments. ### Generating certificates Create a 2048-bit RSA certificate using OpenSSL: ```bash # Generate private key and certificate signing request openssl genrsa -out client-2048.key 2048 openssl req -new -key client-2048.key -out client-2048.csr # Self-sign the certificate (valid for 365 days) openssl x509 -req -days 365 -in client-2048.csr -signkey client-2048.key -out client-2048.crt ``` ### Uploading to Betfair Before using the certificate, attach it to your Betfair account: 1. Navigate to [My Betfair Account Security](https://myaccount.betfair.com/accountdetails/mysecurity?showAPI=1). 2. Scroll to **Automated Betting Program Access** and click **Edit**. 3. Upload your `client-2048.crt` file. ### Directory structure Place your certificate files in a directory and set `BETFAIR_CERTS_DIR` to that path: ``` /path/to/certs/ ├── client-2048.crt └── client-2048.key ``` :::info SSL certificates are used for the Exchange Streaming API connection. The REST API uses username/password authentication with your Application Key. ::: :::warning Enabling 2-Step Authentication on the Betfair website does not affect API access. Certificate-based login remains functional regardless of 2FA settings. ::: ## Overview The Betfair adapter provides three primary components: - `BetfairInstrumentProvider`: loads Betfair markets and converts them into Nautilus instruments. - `BetfairDataClient`: streams real-time market data from the Exchange Streaming API. - `BetfairExecutionClient`: submits orders (bets) and tracks execution status via the REST API. ## Implementation status NautilusTrader currently ships a stable Python Betfair adapter and an in-progress Rust parity path. This page remains the stable guide. It now calls out the main Rust differences inline. Use the [Betfair v2 transition guide](betfair_v2.md) for the current Rust-first behavior in `crates/adapters/betfair` and for the planned cutover path. ## Orders capability Betfair operates as a betting exchange with unique characteristics compared to traditional financial exchanges: ### Order types | Order Type | Supported | Notes | |------------------------|-----------|-------------------------------------| | `MARKET` | ✓* | Python maps regular market orders to aggressive `LIMIT`; Rust supports BSP `AT_THE_CLOSE` only. | | `LIMIT` | ✓ | Orders placed at specific odds. | | `STOP_MARKET` | - | *Not supported*. | | `STOP_LIMIT` | - | *Not supported*. | | `MARKET_IF_TOUCHED` | - | *Not supported*. | | `LIMIT_IF_TOUCHED` | - | *Not supported*. | | `TRAILING_STOP_MARKET` | - | *Not supported*. | ### Execution instructions | Instruction | Supported | Notes | |---------------|-----------|-------------------------------------| | `post_only` | - | Not applicable to betting exchange. | | `reduce_only` | - | Not applicable to betting exchange. | ### Time in force options | Time in force | Supported | Notes | |---------------|-----------|--------------------------------------------| | `GTC` | ✓ | Maps to Betfair `PERSIST` persistence. | | `GTD` | - | *Not supported*. | | `DAY` | ✓ | Maps to Betfair `LAPSE` persistence. | | `FOK` | ✓ | Maps to Betfair `FILL_OR_KILL`. | | `IOC` | ✓ | Maps to `FILL_OR_KILL` with partial fills. | :::note Betfair uses a persistence model rather than traditional time-in-force. The adapter maps `FOK` to Betfair's `FILL_OR_KILL`, while `IOC` uses `FILL_OR_KILL` with `min_fill_size=0` to allow partial fills. The current adapters also support BSP on-close order flows. Rust only accepts `MARKET` orders in `AT_THE_CLOSE` mode. Rust also maps `LIMIT` orders in `AT_THE_CLOSE` or `AT_THE_OPEN` mode to Betfair `LIMIT_ON_CLOSE` instructions. ::: ### Advanced order features | Feature | Supported | Notes | |--------------------|-----------|------------------------------------------| | Order Modification | ✓ | Limited to non‑exposure changing fields. | | Bracket/OCO Orders | - | *Not supported*. | | Iceberg Orders | - | *Not supported*. | ### Batch operations | Operation | Supported | Notes | |--------------------|-----------|----------------------| | Batch Submit | ✓ | Supports `SubmitOrderList` in Python and Rust. | | Batch Modify | - | *Not supported*. | | Batch Cancel | ✓ | Supports batched cancel requests in Python and Rust. | ### Position management | Feature | Supported | Notes | |---------------------|-----------|-----------------------------------------| | Query positions | - | Betting exchange model differs. | | Position mode | - | Not applicable to betting exchange. | | Leverage control | - | No leverage in betting exchange. | | Margin mode | - | No margin in betting exchange. | ### Order querying | Feature | Supported | Notes | |----------------------|-----------|----------------------------------------| | Query open orders | ✓ | List all active bets. | | Query order history | ✓ | Historical betting data. | | Order status updates | ✓ | Real‑time bet state changes. | | Trade history | ✓ | Bet matching and settlement reports. | ### Contingent orders | Feature | Supported | Notes | |---------------------|-----------|-----------------------------------------| | Order lists | - | *Not supported*. | | OCO orders | - | *Not supported*. | | Bracket orders | - | *Not supported*. | | Conditional orders | - | Basic bet conditions only. | ## Tick scheme and pricing Betfair uses a tiered tick scheme with varying increments across price ranges: | Price Range | Tick Size | |---------------|-----------| | 1.01 - 2.00 | 0.01 | | 2.00 - 3.00 | 0.02 | | 3.00 - 4.00 | 0.05 | | 4.00 - 6.00 | 0.10 | | 6.00 - 10.00 | 0.20 | | 10.00 - 20.00 | 0.50 | | 20.00 - 30.00 | 1.00 | | 30.00 - 50.00 | 2.00 | | 50.00 - 100.00 | 5.00 | | 100.00 - 1000.00 | 10.00 | The minimum price is 1.01 and the maximum is 1000.00. ## Order modification Order modification on Betfair has specific constraints: - **Price and size cannot be changed atomically** - these require separate operations. - **Price modification** uses `ReplaceOrders` (cancel + new order at new price). - **Size reduction** uses `CancelOrders` with a `size_reduction` parameter. - **Size increase** is not supported - submit a new order instead. :::warning A replace operation generates both a cancel event for the original order and an accepted event for the replacement order. The adapter tracks pending replacements to suppress synthetic cancel events. ::: ## Order stream fill handling The execution client processes order updates from the Betfair Exchange Streaming API. Two configuration options control how updates are filtered: - **`stream_market_ids_filter`**: Filters at the market level (early exit, silent skip). - **`ignore_external_orders`**: Filters at the order level. Python also uses it to control the log level for full-image cache checks. Rust currently only skips OCM updates with no `rfo`. The flowchart below matches the stable Python execution path. Python keeps `stream_market_ids_filter` separate from reconciliation scope (`reconcile_market_ids_only`). Rust currently falls back to `stream_market_ids_filter` during reconciliation when `reconcile_market_ids_only=False` and no explicit `reconcile_market_ids` are configured. ```mermaid flowchart TD A[Stream update arrives] --> B{Market in
stream_market_ids_filter?} B -->|No filter set| C{Instrument loaded?} B -->|Yes| C B -->|No| D[Skip silently] C -->|No| E[Warning: Instrument not loaded] C -->|Yes| F{Known order?
rfo or cache} F -->|Yes| G[Process order update] F -->|No| H{ignore_external_orders?} H -->|True| I[Debug log, skip] H -->|False| J[Warning log, skip] ``` Python also applies `stream_market_ids_filter` during full-image reconciliation in `check_cache_against_order_image`. Rust currently reconciles through `generate_mass_status()` and does not yet perform the same full-image cache check. When `ignore_external_orders=True`, the Python adapter skips orders and fills not found in cache: | Scenario | Description | |--------------------------------|-----------------------------------------------------| | Unknown order in stream update | No venue‑to‑client order ID mapping exists. | | Unknown order in full image | Order not found in cache during image sync. | | Unknown fill in full image | Fill does not match any known order during sync. | :::info For multi-node setups sharing a Betfair account, set both `stream_market_ids_filter` (your markets only) and `ignore_external_orders=True` to avoid warnings about orders managed by other nodes. ::: ### Fill handling The adapter handles several edge cases when processing fills from the stream: - **Incremental fills**: Betfair reports cumulative matched sizes. The adapter calculates incremental fills by tracking the last known filled quantity per order. - **Overfill protection**: Fills that would exceed the order quantity are rejected. - **Deduplication**: A cache of published trade IDs prevents duplicate fill events from late messages or stream reconnection replays. - **Race conditions**: When stream fills arrive before the HTTP order response, the adapter caches the venue order ID immediately to ensure correct order matching. - **Network error recovery**: When an HTTP order submission fails with a network error (timeout, connection reset), the order may still have been placed on the venue. The adapter leaves the order in SUBMITTED status and retains the customer order reference so the stream can confirm the order when it reconnects. API errors (where Betfair explicitly rejected) still reject immediately. ## Rate limiting The adapter uses separate rate limit buckets so that account state polling and reconciliation do not throttle order placement: | Bucket | Default | Endpoints | Configurable | |---------|---------|------------------------------------------------------|----------------------------------| | General | 5/s | Account state, reconciliation, keep‑alive. | | | Orders | 20/s | `placeOrders`, `replaceOrders`, `cancelOrders`. | `order_request_rate_per_second`. | Order status and fill report queries retry once on `TOO_MANY_REQUESTS` errors after a 1-second delay; order operations reject with the error message. Betfair's actual API limits are more nuanced: | Category | Limit | Notes | |--------------------------|----------------------|------------------------------------------------------| | Order operations | 1,000 transactions/s | Total instructions across `placeOrders`, `cancelOrders`, `replaceOrders`. | | Order projection queries | 3 concurrent | `listMarketBook` (with `OrderProjection`), `listCurrentOrders`, `listMarketProfitAndLoss`. | | Best practice | 5 requests/s | Recommended for `listMarketBook` per market. | :::info For details on rate limits, see [Why am I receiving the TOO_MANY_REQUESTS error?](https://support.developer.betfair.com/hc/en-us/articles/360000406111) and [Market Data Request Limits](https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/Market+Data+Request+Limits). ::: ## Custom data types The Betfair adapter provides several custom data types that flow through the market stream. All custom data is delivered automatically when subscribed to markets - no explicit subscription is required, though strategies can register handlers for specific data types. ### BetfairTicker Real-time ticker data for a betting selection. | Field | Type | Description | |-----------------------|---------|---------------------------------| | `instrument_id` | str | Nautilus instrument identifier. | | `last_traded_price` | float | Last matched price (odds). | | `traded_volume` | float | Total matched volume. | | `starting_price_near` | float | Near‑side BSP indicator. | | `starting_price_far` | float | Far‑side BSP indicator. | ### BetfairStartingPrice The realized Betfair Starting Price (BSP) after market close. | Field | Type | Description | |-----------------|-------|---------------------------------| | `instrument_id` | str | Nautilus instrument identifier. | | `bsp` | float | Final starting price (odds). | ### BetfairRaceRunnerData Live GPS tracking data for individual horses (Total Performance Data). Available for supported UK and Irish races. | Field | Type | Description | |--------------------|-------|-----------------------------------------| | `race_id` | str | Betfair race identifier. | | `market_id` | str | Betfair market identifier. | | `selection_id` | int | Betfair selection (runner) identifier. | | `latitude` | float | GPS latitude. | | `longitude` | float | GPS longitude. | | `speed` | float | Current speed in m/s (Doppler‑derived). | | `progress` | float | Distance to finish line in meters. | | `stride_frequency` | float | Stride frequency in Hz. | ### BetfairRaceProgress Race summary data with sectional times and running order. | Field | Type | Description | |------------------|------------|-----------------------------------------------| | `race_id` | str | Betfair race identifier. | | `market_id` | str | Betfair market identifier. | | `gate_name` | str | Timing gate (e.g., "1f", "2f", "Finish"). | | `sectional_time` | float | Time for this section in seconds. | | `running_time` | float | Total time since race start in seconds. | | `speed` | float | Lead horse speed in m/s. | | `progress` | float | Lead horse distance to finish in meters. | | `order` | list[int] | Selection IDs in current race position order. | | `jumps` | list[dict] | Jump obstacle data for National Hunt races. | ### Subscribing to custom data Custom data flows automatically through the Betfair market stream when you subscribe to markets. To receive custom data in your strategy or actor, register a handler with the Betfair client ID: ```python from nautilus_trader.adapters.betfair.constants import BETFAIR_CLIENT_ID from nautilus_trader.adapters.betfair.data_types import BetfairRaceRunnerData from nautilus_trader.adapters.betfair.data_types import BetfairRaceProgress from nautilus_trader.adapters.betfair.data_types import BetfairTicker from nautilus_trader.model.data import DataType class MyStrategy(Strategy): def on_start(self): # Subscribe to ticker data self.subscribe_data(DataType(BetfairTicker), client_id=BETFAIR_CLIENT_ID) # Subscribe to ALL race runner data (wildcard) self.subscribe_data(DataType(BetfairRaceRunnerData), client_id=BETFAIR_CLIENT_ID) # Or subscribe to a specific runner by selection_id self.subscribe_data( DataType(BetfairRaceRunnerData, metadata={"selection_id": 49411491}), client_id=BETFAIR_CLIENT_ID, ) # Subscribe to ALL race progress updates (wildcard) self.subscribe_data(DataType(BetfairRaceProgress), client_id=BETFAIR_CLIENT_ID) # Or subscribe to a specific race by race_id self.subscribe_data( DataType(BetfairRaceProgress, metadata={"race_id": "35278018.1617"}), client_id=BETFAIR_CLIENT_ID, ) def on_data(self, data): if isinstance(data, BetfairRaceRunnerData): self.log.info( f"Runner {data.selection_id}: speed={data.speed} m/s, " f"progress={data.progress}m to finish" ) elif isinstance(data, BetfairRaceProgress): self.log.info(f"Race order: {data.order}") elif isinstance(data, BetfairTicker): self.log.info(f"LTP: {data.last_traded_price}") ``` :::info Subscribing with `DataType(BetfairRaceRunnerData)` (no metadata) receives data for **all** runners. Adding `metadata={"selection_id": }` filters to a specific runner. Similarly, `DataType(BetfairRaceProgress)` receives progress for all races, while `metadata={"race_id": }` filters to a specific race. Race data (RCM messages) requires Total Performance Data (TPD) coverage and a Betfair API key with TPD access. Not all races have GPS tracking enabled. ::: ### Loading race data from files For backtesting with recorded race data, use the file parser: ```python from nautilus_trader.adapters.betfair.parsing.core import parse_betfair_rcm_file for data in parse_betfair_rcm_file("path/to/rcm_data.json"): if isinstance(data, BetfairRaceRunnerData): print(f"Runner {data.selection_id} at {data.latitude}, {data.longitude}") ``` ## Configuration ### Data client configuration options | Option | Default | Description | |---------------------------|-----------|-------------| | `account_currency` | Required | Betfair account currency for data and price feeds. | | `username` | `None` | Betfair account username; taken from environment when omitted. | | `password` | `None` | Betfair account password; taken from environment when omitted. | | `app_key` | `None` | Betfair application key used for API authentication. | | `certs_dir` | `None` | Directory containing Betfair SSL certificates for login. | | `instrument_config` | `None` | Optional `BetfairInstrumentProviderConfig` to scope available markets. | | `subscription_delay_secs` | `3` | Delay (seconds) before initial market subscription request is sent. | | `keep_alive_secs` | `36,000` | Keep‑alive interval (seconds) for the Betfair session. | | `subscribe_race_data` | `False` | When `True`, subscribe to Race Change Messages (RCM) for live GPS tracking data. | | `stream_conflate_ms` | `None` | Explicit stream conflation interval in milliseconds (`0` disables conflation). | | `stream_heartbeat_ms` | `5,000` | Stream heartbeat interval in milliseconds (500-5000). `None` to omit. | | `proxy_url` | `None` | Optional proxy URL for HTTP requests. | :::warning When `stream_conflate_ms` is `None`, Betfair applies its default conflation behavior (typically enabled). Set `stream_conflate_ms=0` explicitly to guarantee no conflation and receive every price update. ::: :::note Current Rust differences: - Rust does not yet expose `certs_dir`. - Rust does not use `instrument_config`; it scopes instruments with direct filter fields on `BetfairDataConfig`. - Rust uses a fixed 36,000 second keep-alive interval. - Rust currently requires `stream_heartbeat_ms`; it does not accept `None` to omit the heartbeat. ::: ### Execution client configuration options | Option | Default | Description | |------------------------------|----------|-------------| | `account_currency` | Required | Betfair account currency for order placement and balances. | | `username` | `None` | Betfair account username; taken from environment when omitted. | | `password` | `None` | Betfair account password; taken from environment when omitted. | | `app_key` | `None` | Betfair application key used for API authentication. | | `certs_dir` | `None` | Directory containing Betfair SSL certificates for login. | | `instrument_config` | `None` | Optional `BetfairInstrumentProviderConfig` to scope reconciliation. | | `calculate_account_state` | `True` | Calculate account state locally from events when `True`. | | `request_account_state_secs` | `300` | Interval (seconds) to poll Betfair for account state (`0` disables). | | `reconcile_market_ids_only` | `False` | When `True`, reconciliation only covers `instrument_config.market_ids` (no effect if unset). | | `reconcile_market_ids` | `None` | Rust only. Explicit market IDs to use for reconciliation when `reconcile_market_ids_only=True`. | | `stream_market_ids_filter` | `None` | List of market IDs to process from stream; others are silently skipped. | | `ignore_external_orders` | `False` | When `True`, ignore stream orders missing from the local cache. | | `use_market_version` | `False` | When `True`, attach the latest market version to order requests for price protection. | | `order_request_rate_per_second` | `20` | Rate limit (requests/second) for order endpoints, separate from general API endpoints. | | `stream_heartbeat_ms` | `5,000` | Order stream heartbeat interval in milliseconds (500-5000). `None` to omit. | | `proxy_url` | `None` | Optional proxy URL for HTTP requests. | :::warning If you set `stream_market_ids_filter`, ensure it includes all markets you trade. Orders placed on markets excluded from this filter will miss live fill and cancel updates from the stream. ::: :::note Current Rust differences: - Rust does not yet expose `certs_dir` or `instrument_config`. - Rust uses `calculate_account_state` as the gate for periodic account-state polling. - Rust uses `reconcile_market_ids` when `reconcile_market_ids_only=True`. - If `reconcile_market_ids_only=False`, Rust currently falls back to `stream_market_ids_filter` for startup reconciliation when `reconcile_market_ids` is unset. - Rust currently applies `ignore_external_orders` only to OCM updates with no `rfo`. - Rust currently requires `stream_heartbeat_ms`; it does not accept `None` to omit the heartbeat. ::: ## Session management Betfair sessions typically expire every 12-24 hours. The adapter automatically handles session reconnection when `NO_SESSION` or `INVALID_SESSION_INFORMATION` errors occur: - The HTTP client reconnects and obtains a new session token. - The streaming client re-authenticates and resubscribes to markets. - The keep-alive mechanism proactively extends sessions. Python exposes `keep_alive_secs`. Rust currently uses a fixed 10-hour interval. :::info Session errors during account state polling or keep-alive trigger automatic reconnection. No manual intervention is required for normal session expiry. ::: ## Market version price protection Betfair markets have a `version` number that increments whenever the market book changes (e.g., a new price level appears, a bet is matched). The adapter can attach this version to `placeOrders` and `replaceOrders` requests, providing price protection against stale orders. When `use_market_version=True`, each order request includes the market version last seen by the adapter. If the market has advanced beyond that version by the time Betfair processes the order, Betfair **lapses** the bet rather than matching it against a changed book. ```python from nautilus_trader.adapters.betfair.config import BetfairExecClientConfig exec_config = BetfairExecClientConfig( account_currency="GBP", use_market_version=True, ) ``` The adapter reads the market version from the instrument's `info` dictionary, which the Exchange Streaming API's `MarketDefinition` updates populate. This means: - The version reflects the most recent stream update, not the HTTP API snapshot. - There is inherent latency between a market change and the adapter receiving the updated version. - Orders submitted before the first stream `MarketDefinition` is received will not include a version. :::warning Market version protection is conservative. In fast-moving markets, the version may advance between your order signal and submission, causing the bet to lapse even though the price is still acceptable. Consider this trade-off between protection and fill rate. ::: ## Multi-node deployment When multiple trading nodes share a single Betfair account across different markets, configure each node to avoid interference: 1. Set `stream_market_ids_filter` to include only that node's markets. 2. Set `ignore_external_orders=True` to suppress warnings about orders from other nodes. 3. Set `reconcile_market_ids_only=True` to limit reconciliation scope. This prevents warning spam and ensures each node processes only its own orders and fills. Here is a minimal example showing how to configure a live `TradingNode` with Betfair clients: ```python from nautilus_trader.adapters.betfair import BETFAIR from nautilus_trader.adapters.betfair import BetfairLiveDataClientFactory from nautilus_trader.adapters.betfair import BetfairLiveExecClientFactory from nautilus_trader.config import TradingNodeConfig from nautilus_trader.live.node import TradingNode # Configure Betfair data and execution clients (using AUD account currency) config = TradingNodeConfig( data_clients={BETFAIR: {"account_currency": "AUD"}}, exec_clients={BETFAIR: {"account_currency": "AUD"}}, ) # Build the TradingNode with Betfair adapter factories node = TradingNode(config) node.add_data_client_factory(BETFAIR, BetfairLiveDataClientFactory) node.add_exec_client_factory(BETFAIR, BetfairLiveExecClientFactory) node.build() ``` ## Contributing :::info For additional features or to contribute to the Betfair adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Betfair v2 Source: https://nautilustrader.io/docs/latest/integrations/betfair_v2/ The Betfair Rust adapter is in active parity work. This page tracks the current Rust behavior and the planned cutover from the stable guide in [Betfair](betfair.md). This page mirrors the main section order from [Betfair](betfair.md). When the Rust adapter becomes the primary Betfair path, this file can replace `betfair.md` with small edits instead of a full rewrite. ## Scope - Source of truth for this page: `crates/adapters/betfair` - Stable guide today: [Betfair](betfair.md) - Purpose of this page: track the current Rust surface, the known gaps, and the cutover path ## Current Rust status | Area | Current Rust behavior | Difference from `betfair.md` today | Cutover work | |--------------------------|--------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|-----------------------------------------------------| | Order types | `MARKET` only supports `AT_THE_CLOSE`; `LIMIT` supports BSP on close flows. | Stable guide is still Python shaped in this area. | Decide final Betfair market order model. | | Batch operations | `SubmitOrderList` and `BatchCancelOrders` are implemented. | Stable guide used to mark these as unsupported. | Keep and promote. | | Reconciliation scope | `reconcile_market_ids_only` uses `reconcile_market_ids`; otherwise falls back to `stream_market_ids_filter`. | Stable guide says stream filtering and reconciliation are separate. | Decide if Rust keeps or removes this coupling. | | Full image cache checks | Rust uses `generate_mass_status()` at startup and on every stream reconnect; no `check_cache_against_order_image`. | Stable guide describes the Python full image cache check. | Add parity or document the Rust path as final. | | Post‑reconnect halt | `submit_order` and `submit_order_list` emit `OrderDenied STREAM_RECONCILING` while the reconcile is in flight. | Python keeps trading during reconnect. | Promote as the Rust default once `betfair.md` flips. | | External order filtering | `ignore_external_orders` only skips OCM updates with no `rfo`. | Python also uses it during full image cache checks. | Decide final filtering behavior. | | Config surface | No `certs_dir`, no `instrument_config`, fixed keep alive, required heartbeat value. | Stable guide still documents the Python config surface. | Decide whether to add parity or bless Rust surface. | | SSL certificates | Stream client currently hardcodes `certs_dir=None`. | Stable guide documents certificate configuration and `BETFAIR_CERTS_DIR`. | Add support or remove from the future guide. | ## Orders capability ### Order types | Order Type | Supported | Notes | |------------------------|-----------|-----------------------------------------------------------------------------| | `MARKET` | ✓* | Rust only supports `AT_THE_CLOSE`, which maps to Betfair `MARKET_ON_CLOSE`. | | `LIMIT` | ✓ | Rust supports regular limit orders and BSP on close limit orders. | | `STOP_MARKET` | - | Not supported. | | `STOP_LIMIT` | - | Not supported. | | `MARKET_IF_TOUCHED` | - | Not supported. | | `LIMIT_IF_TOUCHED` | - | Not supported. | | `TRAILING_STOP_MARKET` | - | Not supported. | ### Time in force | Time in force | Supported | Notes | |----------------|-----------|--------------------------------------------------------------| | `GTC` | ✓ | Maps to Betfair `PERSIST`. | | `DAY` | ✓ | Maps to Betfair `LAPSE`. | | `FOK` | ✓ | Maps to Betfair `FILL_OR_KILL`. | | `IOC` | ✓ | Maps to `FILL_OR_KILL` with `min_fill_size=0`. | | `AT_THE_CLOSE` | ✓ | Used for Betfair BSP `LIMIT_ON_CLOSE` and `MARKET_ON_CLOSE`. | Rust currently also accepts `LIMIT` orders in `AT_THE_OPEN` mode and routes them through Betfair `LIMIT_ON_CLOSE` instructions. Treat that as current behavior, not a settled public contract. ### Batch operations | Operation | Supported | Notes | |--------------|-----------|--------------------------------------------| | Batch Submit | ✓ | Implemented through `SubmitOrderList`. | | Batch Modify | - | Not supported. | | Batch Cancel | ✓ | Implemented through `BatchCancelOrders`. | ## Execution control flow Startup: 1. Connect the HTTP client and fetch initial account funds. 2. Seed OCM state from cached orders. 3. Connect the Betfair execution stream and subscribe to order updates. 4. Generate startup mass status from `listCurrentOrders`. 5. Reconcile order and fill reports into the execution engine. On every stream reconnect, the same mass-status reconciliation runs over a recent window and the adapter halts new exposure-increasing commands until it dispatches. See [Post-reconnect reconciliation](#post-reconnect-reconciliation). Current Rust notes: - `stream_market_ids_filter` filters live OCM updates. - `reconcile_market_ids_only=True` uses explicit `reconcile_market_ids`. - When `reconcile_market_ids_only=False` and `reconcile_market_ids` is unset, Rust currently falls back to `stream_market_ids_filter` for startup reconciliation. - Rust does not yet implement the Python `check_cache_against_order_image` full-image cache check. - `ignore_external_orders=True` currently skips only OCM updates with no `rfo`. ## Session management and reconnection Betfair sessions expire every 12-24 hours. The Rust adapter handles session recovery automatically through three mechanisms: | Mechanism | Trigger | Action | |---------------------|-----------------------------------|----------------------------------------------------------------------| | Periodic keep‑alive | Every 10 hours. | Renew session token, push to all stream watch channels. | | Keep‑alive fallback | Keep‑alive returns `LoginFailed`. | Full re‑login via `reconnect()`, push fresh token to streams. | | Stream reconnect | `Connection` message after drop. | Try keep‑alive, fall back to re‑login on `LoginFailed`, update auth. | Transient errors (network timeouts, 5xx responses) during keep-alive are logged and skipped. The existing session token is preserved and the next keep-alive interval retries. Only `LoginFailed` errors (session expiry) trigger a full re-login. Both the data and execution clients run identical reconnection logic. Each spawns: - A **keep-alive task** that periodically refreshes the session and pushes updated auth bytes to the stream watch channels. - A **reconnect handler** that listens for `Connection` messages after a stream reconnect, refreshes the session, and pushes the new token. The stream client stores auth bytes in a `tokio::sync::watch` channel. The `post_reconnection` closure reads from this channel on each TCP reconnect, so a token refreshed by either the keep-alive task or reconnect handler is picked up on the next connection attempt. The data client reconnect handler also updates the race stream auth when a race stream is active. ## Post-reconnect reconciliation When the Betfair execution stream reconnects, the adapter assumes the cache may have diverged from venue state during the gap (in particular, fills can complete and roll off the unmatched book before the post-reconnect stream image arrives). It therefore runs a mass-status reconciliation over a recent window before allowing strategies to add new exposure. | Step | Trigger | Action | |------|------------------------------------------------|-----------------------------------------------------------------------------------------------------------------| | 1 | Second `Connection` message after stream drop. | OCM handler raises `pending_resync` and `is_reconciling`, sends a reconnect signal to the background task. | | 2 | Reconnect task receives signal. | Re‑asserts `is_reconciling` so a queued second reconnect halts during its own iteration too. | | 3 | Reconnect task body. | Refreshes session, updates stream auth, fetches `getAccountFunds`, and calls `listCurrentOrders` for orders+fills. | | 4 | Mass status built. | Dispatched as `ExecutionReport::MassStatus` so the engine reconciles into the cache. | | 5 | Iteration ends. | `is_reconciling` cleared. A failed iteration also clears it (fail‑open, consistent with the rest of Nautilus). | While `is_reconciling` is set: - `submit_order` and `submit_order_list` emit `OrderDenied` with reason `STREAM_RECONCILING: post-reconnect reconciliation in progress, retry once it completes`. - `cancel_order`, `batch_cancel_orders`, and `modify_order` pass through unchanged so a strategy can always reduce exposure during the window. - Buffered OCMs that arrived during the gap are drained on the next strategy command via `process_pending_resync` (separate `pending_resync` flag). If the client disconnects while a reconciliation is still in flight, `clear_resync_state` clears `is_reconciling` so a subsequent connect/submit cycle starts clean. The lookback window for the mass-status fetch is `stream_gap_recovery_lookback_mins` (default `10`). It should comfortably exceed the longest expected reconnect duration so a fill that completed mid-gap is still captured. ## Tick scheme and pricing Betfair uses a tiered tick scheme with varying increments across price ranges: | Price range | Tick size | |----------------|-----------| | 1.01 - 2.00 | 0.01 | | 2.00 - 3.00 | 0.02 | | 3.00 - 4.00 | 0.05 | | 4.00 - 6.00 | 0.10 | | 6.00 - 10.00 | 0.20 | | 10.00 - 20.00 | 0.50 | | 20.00 - 30.00 | 1.00 | | 30.00 - 50.00 | 2.00 | | 50.00 - 100.00 | 5.00 | | 100.00 - 1000 | 10.00 | Minimum price is 1.01, maximum is 1000.00. ## Order modification - Price and size cannot change atomically; these require separate operations. - Price modification uses `ReplaceOrders` (cancel + new order at new price). - Size reduction uses `CancelOrders` with a `size_reduction` parameter. - Size increase is not supported; submit a new order instead. A replace operation generates both a cancel event for the original order and an accepted event for the replacement. The adapter tracks pending replacements to suppress synthetic cancel events. ## Order stream fill handling The execution client processes order updates from the Betfair Exchange Streaming API. Two configuration options control how updates are filtered: - `stream_market_ids_filter`: filters at the market level (early exit, silent skip). - `ignore_external_orders`: filters at the order level (skips OCM updates with no `rfo`). ### Fill handling The adapter handles several edge cases when processing fills from the stream: - **Incremental fills**: Betfair reports cumulative matched sizes. The adapter calculates incremental fills by tracking the last known filled quantity per order. - **Overfill protection**: fills that would exceed the order quantity are rejected. - **Race conditions**: when stream fills arrive before the HTTP order response, the adapter caches the venue order ID immediately to ensure correct order matching. - **Network error recovery**: when an HTTP order submission fails with a network error (timeout, connection reset), the order may still have been placed on the venue. The adapter leaves the order in SUBMITTED status and retains the customer order reference so the stream can confirm the order when it reconnects. API errors (where Betfair explicitly rejected) reject immediately. - **Gap-window fills**: a fill that completes and rolls off the unmatched book during a stream disconnect is recovered by the post-reconnect mass-status reconciliation; see [Post-reconnect reconciliation](#post-reconnect-reconciliation). ## Rate limiting The adapter uses separate rate limit buckets so that account state polling and reconciliation do not throttle order placement: | Bucket | Default | Endpoints | |---------|---------|-------------------------------------------------| | General | 5/s | Account state, reconciliation, keep‑alive. | | Orders | 20/s | `placeOrders`, `replaceOrders`, `cancelOrders`. | Order status and fill report queries retry once on session errors after refreshing the session. `TOO_MANY_REQUESTS` errors retry after a 5-second delay. ## Market version price protection When `use_market_version=True`, each order request includes the market version last seen by the adapter. If the market has advanced beyond that version by the time Betfair processes the order, Betfair lapses the bet rather than matching it against a changed book. The adapter reads the market version from the instrument's `info` dictionary, which the Exchange Streaming API's `MarketDefinition` updates populate. Orders submitted before the first `MarketDefinition` is received do not include a version. ## Custom data types The Rust adapter emits the same custom data types as the Python adapter through the market and race streams. All custom data flows automatically when subscribed to markets. | Type | Stream | Description | |----------------------------|--------|---------------------------------------------------| | `BetfairTicker` | Market | Last traded price, traded volume, BSP indicators. | | `BetfairStartingPrice` | Market | Realized BSP after market close. | | `BetfairSequenceCompleted` | Market | Marks end of a market change sequence. | | `BetfairOrderVoided` | Order | Voided order details (size voided, price, side). | | `BetfairRaceRunnerData` | Race | Live GPS tracking per runner (TPD). | | `BetfairRaceProgress` | Race | Sectional times, running order, jump data. | Race data requires Total Performance Data (TPD) coverage and a Betfair API key with TPD access. Enable with `subscribe_race_data=True`. ## Multi-node deployment When multiple trading nodes share a single Betfair account across different markets: 1. Set `stream_market_ids_filter` to include only that node's markets. 2. Set `ignore_external_orders=True` to suppress warnings about orders from other nodes. 3. Set `reconcile_market_ids_only=True` to limit reconciliation scope. ## Current Rust configuration ### Data client configuration | Option | Default | Notes | |-------------------------------------|----------|-----------------------------------------------| | `account_currency` | Required | Betfair account currency. | | `username` | `None` | Falls back to `BETFAIR_USERNAME`. | | `password` | `None` | Falls back to `BETFAIR_PASSWORD`. | | `app_key` | `None` | Falls back to `BETFAIR_APP_KEY`. | | `proxy_url` | `None` | Optional proxy URL for HTTP requests. | | `request_rate_per_second` | `5` | General HTTP rate limit. | | `default_min_notional` | `None` | Optional minimum notional override. | | `event_type_ids` | `None` | Optional navigation filter. | | `event_type_names` | `None` | Optional navigation filter. | | `event_ids` | `None` | Optional navigation filter. | | `country_codes` | `None` | Optional navigation filter. | | `market_types` | `None` | Optional navigation filter. | | `market_ids` | `None` | Optional navigation filter. | | `min_market_start_time` | `None` | Optional navigation filter. | | `max_market_start_time` | `None` | Optional navigation filter. | | `stream_host` | `None` | Optional stream host override. | | `stream_port` | `None` | Optional stream port override. | | `stream_heartbeat_ms` | `5,000` | Required in Rust today. | | `stream_idle_timeout_ms` | `60,000` | Idle timeout before reconnect. | | `stream_reconnect_delay_initial_ms` | `2,000` | Initial reconnect delay. | | `stream_reconnect_delay_max_ms` | `30,000` | Maximum reconnect delay. | | `stream_use_tls` | `True` | Use TLS for the stream connection. | | `stream_conflate_ms` | `None` | Explicit conflation setting. | | `subscription_delay_secs` | `3` | Delay before the first market subscription. | | `subscribe_race_data` | `False` | Subscribe to RCM updates. | Rust does not yet expose `certs_dir` or `instrument_config`. Rust also uses a fixed 36,000 second keep-alive interval. ### Execution client configuration | Option | Default | Notes | |-------------------------------------|---------------|--------------------------------------------------------| | `trader_id` | `TRADER-001` | Trader ID for the client core. | | `account_id` | `BETFAIR-001` | Account ID for the client core. | | `account_currency` | `GBP` | Betfair account currency. | | `username` | `None` | Falls back to `BETFAIR_USERNAME`. | | `password` | `None` | Falls back to `BETFAIR_PASSWORD`. | | `app_key` | `None` | Falls back to `BETFAIR_APP_KEY`. | | `proxy_url` | `None` | Optional proxy URL for HTTP requests. | | `request_rate_per_second` | `5` | General HTTP rate limit. | | `order_request_rate_per_second` | `20` | Order endpoint rate limit. | | `stream_host` | `None` | Optional stream host override. | | `stream_port` | `None` | Optional stream port override. | | `stream_heartbeat_ms` | `5,000` | Required in Rust today. | | `stream_idle_timeout_ms` | `60,000` | Idle timeout before reconnect. | | `stream_reconnect_delay_initial_ms` | `2,000` | Initial reconnect delay. | | `stream_reconnect_delay_max_ms` | `30,000` | Maximum reconnect delay. | | `stream_use_tls` | `True` | Use TLS for the stream connection. | | `stream_market_ids_filter` | `None` | Optional live OCM market filter. | | `ignore_external_orders` | `False` | Only skips OCM updates with no `rfo`. | | `calculate_account_state` | `True` | Gates periodic account state polling in Rust today. | | `request_account_state_secs` | `300` | Poll interval for account funds. | | `reconcile_market_ids_only` | `False` | When `True`, use `reconcile_market_ids`. | | `reconcile_market_ids` | `None` | Explicit startup reconciliation market IDs. | | `use_market_version` | `False` | Attach market version to place and replace requests. | | `stream_gap_recovery_lookback_mins` | `10` | Lookback window for the post‑reconnect mass‑status reconciliation. | Rust does not yet expose `certs_dir` or `instrument_config`. ## Cutover plan Use this page as the transition tracker until the Rust adapter becomes the primary Betfair path. At cutover: 1. Decide whether Rust keeps its current reconciliation filter behavior or matches the Python split. 2. Decide whether Rust adds certificate configuration and other Python config fields. 3. Decide whether Rust keeps BSP-only `MARKET` orders or adds the Python aggressive-limit path. 4. Promote this file to `betfair.md`. 5. Move any remaining Python-only notes into a short legacy note or release note. # Binance Source: https://nautilustrader.io/docs/latest/integrations/binance/ Founded in 2017, Binance is one of the largest cryptocurrency exchanges in terms of daily trading volume, and open interest of crypto assets and crypto derivative products. NautilusTrader provides Binance integration in both Python and Rust. The Rust adapter supports all product types listed below and includes additional features (noted inline). The Python adapter supports the same product types. Supported products: - **Binance Spot** (including Binance US) - **Binance USDT-Margined Futures** (perpetuals and delivery contracts) - **Binance Coin-Margined Futures** (perpetuals and delivery contracts) ## Examples - [Python live examples](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/binance/) - [Rust spot examples](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/binance/examples/spot/) - [Rust futures examples](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/binance/examples/futures/) ## Overview The Binance adapter includes multiple components that can be used together or separately: - `BinanceHttpClient`: Low-level HTTP API connectivity. - `BinanceWebSocketClient`: Low-level WebSocket API connectivity. - `BinanceInstrumentProvider`: Instrument parsing and loading. - `BinanceSpotDataClient` / `BinanceFuturesDataClient`: Market data feed manager. - `BinanceSpotExecutionClient` / `BinanceFuturesExecutionClient`: Account management and trade execution gateway. - `BinanceLiveDataClientFactory`: Factory for Binance data clients (used by the trading node builder). - `BinanceLiveExecClientFactory`: Factory for Binance execution clients (used by the trading node builder). :::note Most users configure a live trading node (as below) and do not interact with these lower-level components directly. ::: ### Product support | Product Type | Supported | Notes | |-----------------------------------------|-----------|------------------------------------| | Spot Markets (incl. Binance US) | ✓ | | | Margin Accounts (Cross & Isolated) | - | *Not implemented.* Planned for v2. | | USDT-Margined Futures (PERP & Delivery) | ✓ | | | Coin‑Margined Futures | ✓ | | :::note Margin account features (borrow, repay, isolated margin management) are not implemented. The Python adapter will not add margin support. Full margin trading support is planned for v2. ::: :::info Each Binance client instance handles one product type. The Rust configs use a singular `product_type` field, and the live factories create one data or execution client from one config. To run Spot and Futures in the same node, configure separate clients with distinct IDs such as `BINANCE_SPOT` and `BINANCE_FUTURES`, then pass the matching `client_id` when a strategy subscribes or submits orders. The Python adapter uses different config field names, but `examples/live/binance/binance_spot_and_futures_market_maker.py` shows the same multi-client ID routing pattern. ::: ## Data types The integration includes several custom data types: - `BinanceFuturesTicker`: Futures 24-hour ticker data including price and statistics. - `BinanceBar`: Bar data with additional volume metrics for historical and real-time use. - `BinanceFuturesMarkPriceUpdate`: Mark price updates for Binance Futures. - `BinanceFuturesLiquidation`: Futures liquidation events from the `forceOrder` stream. See the Binance [API Reference](/docs/python-api-latest/adapters/binance.html) for full definitions. ## Symbology Native Binance symbols are used where possible for spot and futures contracts. Because NautilusTrader supports multi-venue trading, it must distinguish between `BTCUSDT` the spot pair and `BTCUSDT` the perpetual futures contract (Binance uses the same symbol for both). Nautilus appends the `-PERP` suffix to all perpetual symbols. For example, the Binance Futures `BTCUSDT` perpetual contract becomes `BTCUSDT-PERP` within Nautilus. ## Order capability The following tables detail order types, execution instructions, and time-in-force options across Binance account types. ### Order types | Order Type | Spot | Margin | USDT Futures | Coin Futures | Notes | |------------------------|------|--------|--------------|--------------|------------------------------------| | `MARKET` | ✓ | - | ✓ | ✓ | Quote quantity support: Spot only. | | `LIMIT` | ✓ | - | ✓ | ✓ | | | `STOP_MARKET` | - | - | ✓ | ✓ | Futures only. | | `STOP_LIMIT` | ✓ | - | ✓ | ✓ | | | `MARKET_IF_TOUCHED` | - | - | ✓ | ✓ | Futures only. | | `LIMIT_IF_TOUCHED` | ✓ | - | ✓ | ✓ | | | `TRAILING_STOP_MARKET` | - | - | ✓ | ✓ | Futures only. | ### Execution instructions | Instruction | Spot | Margin | USDT Futures | Coin Futures | Notes | |---------------|------|--------|--------------|--------------|---------------------------------------| | `post_only` | ✓ | - | ✓ | ✓ | See restrictions below. | | `reduce_only` | - | - | ✓ | ✓ | Futures only; disabled in Hedge Mode. | #### Post-only restrictions Only *limit* order types support `post_only`. | Order Type | Spot | Margin | USDT Futures | Coin Futures | Notes | |--------------------------|------|--------|--------------|--------------|-----------------------------------------------------| | `LIMIT` | ✓ | - | ✓ | ✓ | Uses `LIMIT_MAKER` for Spot, `GTX` TIF for Futures. | | `STOP_LIMIT` | - | - | ✓ | ✓ | Futures only. | ### Time in force | Time in force | Spot | Margin | USDT Futures | Coin Futures | Notes | |---------------|------|--------|--------------|--------------|--------------------------------------------| | `GTC` | ✓ | - | ✓ | ✓ | Good Till Canceled. | | `GTD` | ✓* | - | ✓ | ✓ | *Converted to GTC for Spot with warning. | | `FOK` | ✓ | - | ✓ | ✓ | Fill or Kill. | | `IOC` | ✓ | - | ✓ | ✓ | Immediate or Cancel. | ### Advanced order features | Feature | Spot | Margin | USDT Futures | Coin Futures | Notes | |--------------------|------|--------|--------------|--------------|----------------------------------------------| | Order Modification | ✓ | - | ✓ | ✓ | Price and quantity for `LIMIT` orders only. | | OCO Orders | ✓ | - | - | - | Spot OCO submitted via `orderList/oco`. | | Bracket Orders | - | - | - | - | *Planned*. Currently denied at submission. | | Iceberg Orders | ✓ | - | ✓ | ✓ | Large orders split into visible portions. | ### Batch operations | Operation | Spot | Margin | USDT Futures | Coin Futures | Notes | |--------------------|------|--------|--------------|--------------|----------------------------------------------| | Batch Submit | ✓ | - | ✓ | ✓ | Orders submitted individually (no batch API call). | | Batch Modify | - | - | - | - | Not implemented. | | Batch Cancel | -* | - | ✓ | ✓ | *Spot falls back to individual cancels. | #### Cancel all orders behavior When calling `cancel_all_orders()` from a strategy, the adapter includes orders in both open and inflight (SUBMITTED) states so that the adapter also cancels orders not yet acknowledged by Binance. **Multi-strategy safety**: When multiple strategies trade the same instrument, the adapter compares orders owned by the requesting strategy against all orders for that instrument. If the strategy owns all orders, a single cancel-all API call is used. Otherwise, per-strategy cancels are sent (batch for regular orders, individual for algo orders) to avoid affecting other strategies. **Futures algo orders**: Conditional order types (`STOP_MARKET`, `STOP_LIMIT`, `TAKE_PROFIT`, `TAKE_PROFIT_MARKET`, `TRAILING_STOP_MARKET`) require a different cancel endpoint. The adapter routes these through the correct endpoint automatically. Once an algo order triggers and becomes a regular order, it uses the standard cancel endpoint. **Endpoints used**: | Account Type | Regular Orders | Algo Orders (batch) | Algo Orders (individual) | |--------------|---------------------------------|----------------------------------|-----------------------------| | Spot/Margin | `DELETE /api/v3/openOrders` | N/A | N/A | | USDT Futures | `DELETE /fapi/v1/allOpenOrders` | `DELETE /fapi/v1/algoOpenOrders` | `DELETE /fapi/v1/algoOrder` | | Coin Futures | `DELETE /dapi/v1/allOpenOrders` | `DELETE /dapi/v1/algoOpenOrders` | `DELETE /dapi/v1/algoOrder` | ### Position management | Feature | Spot | Margin | USDT Futures | Coin Futures | Notes | |---------------------|------|--------|--------------|--------------|---------------------------------------------| | Query positions | - | - | ✓ | ✓ | Real‑time position updates. | | Position mode | - | - | ✓ | ✓ | One‑Way vs Hedge mode (position IDs). | | Leverage control | - | - | ✓ | ✓ | Dynamic leverage adjustment per symbol. | | Margin mode | - | - | ✓ | ✓ | Cross vs Isolated margin per symbol. | ### Risk events | Feature | Spot | Margin | USDT Futures | Coin Futures | Notes | |----------------------|------|--------|--------------|--------------|---------------------------------------------| | Liquidation handling | - | - | ✓ | ✓ | Exchange‑forced position closures. | | ADL handling | - | - | ✓ | ✓ | Auto‑Deleveraging events. | Binance Futures can trigger exchange-generated orders in response to risk events: - **Liquidations**: When insufficient margin exists to maintain a position, Binance forcibly closes it at the bankruptcy price. These orders have client IDs starting with `autoclose-`. - **ADL (Auto-Deleveraging)**: When the insurance fund is depleted, Binance closes profitable positions to cover losses. These orders use client ID prefix `adl_autoclose`. - **Settlements (USDT-M)**: Funding/margin settlement orders use client IDs starting with `settlement_autoclose-`. - **Deliveries (COIN-M)**: Expiring delivery contracts auto-close with client IDs starting with `delivery_autoclose-`. - **Insurance fund**: Takeover by the insurance fund uses status `NEW_INSURANCE` (deprecated on the public changelog but still observed on the wire). The adapter detects these special order types via their client ID patterns (checked before the execution type), then: 1. Logs a warning with order details for monitoring. 2. Generates a `FillReport` with correct fill details and TAKER liquidity side. 3. Generates an `OrderStatusReport` for reconciliation. Upstream references: - [USDT-M `ORDER_TRADE_UPDATE`](https://developers.binance.com/docs/derivatives/usds-margined-futures/user-data-streams/Event-Order-Update) - [COIN-M `ORDER_TRADE_UPDATE`](https://developers.binance.com/docs/derivatives/coin-margined-futures/user-data-streams/Event-Order-Update) The execution engine creates external orders from runtime status reports when the order is not already in cache. This covers first-seen exchange-generated orders (the typical case for a live liquidation or ADL event). The engine assigns the order to any strategy that has claimed the instrument via `external_order_claims`, or to the `EXTERNAL` strategy by default. #### Commission estimation When Binance omits the commission fields (`N`/`n`) from the fill event, the Rust adapter estimates commission as `default_taker_fee * qty * price` using the quote currency. This applies to USD-M linear contracts only. COIN-M inverse contracts use zero commission as a fallback because the linear formula does not account for contract size. Configure `default_taker_fee` on `BinanceExecClientConfig` to match your fee tier (default: 0.0004 / 0.04%). #### Hedge-mode position IDs When `use_position_ids` is enabled (default), exchange-generated fill reports include a `venue_position_id` derived from the instrument and position side (e.g. `ETHUSDT-PERP.BINANCE-LONG`). Set `use_position_ids` to false on `BinanceExecClientConfig` for virtual positions with `OmsType.HEDGING`. :::note The status report and fill report are emitted bundled as a single `OrderWithFills` execution report. The engine creates the external order from the status report and then applies the real fill, preserving the venue's `trade_id` and `commission`. Any residual quantity not covered by the bundled fills is closed with an inferred fill from the status report's `avg_px`. ::: ### Order querying | Feature | Spot | Margin | USDT Futures | Coin Futures | Notes | |---------------------|------|--------|--------------|--------------|---------------------------------------------| | Query open orders | ✓ | ✓ | ✓ | ✓ | List all active orders. | | Query order history | ✓ | ✓ | ✓ | ✓ | Historical order data. | | Order status updates| ✓ | ✓ | ✓ | ✓ | Real‑time order state changes. | | Trade history | ✓ | ✓ | ✓ | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Spot | Margin | USDT Futures | Coin Futures | Notes | |---------------------|------|--------|--------------|--------------|----------------------------------------------| | Order lists | ✓ | - | ✓ | ✓ | Spot OCO lists; Futures independent batches. | | OCO orders | ✓ | - | - | - | Spot only, via `orderList/oco`. | | Bracket orders | - | - | - | - | *Planned*. Currently denied at submission. | | Conditional orders | ✓ | ✓ | ✓ | ✓ | Stop and market‑if‑touched orders. | ### Order parameters Customize individual orders by supplying a `params` dictionary when calling `Strategy.submit_order` (Python) or setting `Params` on a `SubmitOrder` command (Rust). The Binance execution clients recognize: | Parameter | Type | Account types | Description | |------------------|--------|-------------------|-------------| | `price_match` | `str` | USDT/COIN Futures | Set one of Binance's `priceMatch` modes (see Price match section below) to delegate price selection to the exchange. Cannot be combined with `post_only` or iceberg (`display_qty`) instructions. | | `close_position` | `bool` | USDT/COIN Futures | Close the entire position when the trigger fires (see Close position section below). Only valid for `StopMarket` and `MarketIfTouched` orders. Cannot be combined with `reduce_only`. | ### Price match Binance Futures supports BBO (Best Bid/Offer) price matching via the `priceMatch` parameter, which delegates price selection to the exchange. Limit orders dynamically join the order book at optimal prices without specifying an exact price level. When using `price_match`, you submit a limit order with a reference price (for local risk checks), and Binance determines the actual working price based on the current market state and price match mode. #### Valid price match values | Value | Behavior | |---------------|----------------------------------------------------------------| | `OPPONENT` | Join the best price on the opposing side of the book. | | `OPPONENT_5` | Join the opposing side price but allow up to a 5-tick offset. | | `OPPONENT_10` | Join the opposing side price but allow up to a 10-tick offset. | | `OPPONENT_20` | Join the opposing side price but allow up to a 20-tick offset. | | `QUEUE` | Join the best price on the same side (stay maker). | | `QUEUE_5` | Join the same‑side queue but offset up to 5 ticks. | | `QUEUE_10` | Join the same‑side queue but offset up to 10 ticks. | | `QUEUE_20` | Join the same‑side queue but offset up to 20 ticks. | :::info For more details, see the [official documentation](https://developers.binance.com/docs/derivatives/usds-margined-futures/trade/rest-api). ::: #### Event sequence When an order is submitted with `price_match`: 1. Nautilus sends the order to Binance with the `priceMatch` parameter but omits the limit price from the API request. 2. Binance accepts the order and determines the actual working price. 3. Nautilus generates an `OrderAccepted` event. 4. If the Binance-accepted price differs from the reference price, Nautilus generates an `OrderUpdated` event with the actual working price. 5. The order price in the Nautilus cache now matches the Binance-accepted price. #### Example ```python order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTCUSDT-PERP.BINANCE"), order_side=OrderSide.BUY, quantity=Quantity.from_int(1), price=Price.from_str("65000"), # Reference price for local risk checks ) strategy.submit_order( order, params={"price_match": "QUEUE"}, ) ``` :::note If Binance accepts the order at a different price (e.g. 64,995.50), you receive an `OrderAccepted` event followed by an `OrderUpdated` event with the new price. ::: ### Close position Binance Futures conditional orders support `closePosition`, which closes the entire position when the trigger fires. Binance resolves the quantity server-side from the current position size at trigger time. Unlike `reduce_only`, `closePosition` adapts to position size changes, and Binance auto-cancels the order when the position is closed by other means. Pass `close_position` via the `params` dictionary on `StopMarket` or `MarketIfTouched` orders. Cannot be combined with `reduce_only`. ```rust tab="Rust" let params = Params::from([("close_position", true.into())]); let cmd = SubmitOrder::new(order).with_params(params); ``` ```python tab="Python" strategy.submit_order(order, params={"close_position": True}) ``` :::info Nautilus omits `quantity` and `reduceOnly` from the API request when `close_position` is set. The order quantity is used only for local risk checks. ::: ### Trailing stops For trailing stop market orders on Binance: - Use `activation_price` (optional) to specify when the trailing mechanism activates. - When omitted, Binance uses the current market price at submission time. - Use `trailing_offset` for the callback rate (in basis points). :::warning Do not use `trigger_price` for trailing stop orders: it will fail with an error. Use `activation_price` instead. ::: ## Link & Trade The NautilusTrader integration ID is automatically prefixed to all system-generated client order IDs for every order placed through the Binance Rust adapter. This provides transparent order attribution through Binance's [Link and Trade](https://developers.binance.com/docs/binance_link/link-and-trade) program without requiring any user configuration. The adapter uses a deterministic two-way encoding to compress outgoing `ClientOrderId` values into a compact format that fits within Binance's 36-character `newClientOrderId` limit, and decodes incoming order events back to the original ID before they reach strategies. This transformation is fully transparent: strategies see only their original `ClientOrderId` values at all times. :::note The integration ID prefix applies to all order operations including submissions, modifications, cancellations, and status queries. Orders placed before this support was added are handled gracefully through passthrough decoding. ::: :::info This feature is currently available in the Rust adapter only. Users can opt out by passing a custom `client_order_id` on their orders, or by removing the encoding calls and recompiling. There is no technical limitation preventing either approach. ::: ### Decoding client order IDs When querying Binance directly (REST API, web UI, or your own HTTP code), the `clientOrderId` field contains the encoded form. Two utility functions recover the original Nautilus `ClientOrderId`: ```python from nautilus_trader.adapters.binance import ( decode_binance_futures_client_order_id, decode_binance_spot_client_order_id, ) # Encoded ID from Binance REST response or web UI encoded = "x-TD67BGP9-T0A4b1H2vj50H" original = decode_binance_spot_client_order_id(encoded) # -> "O-20260305-120000-001-001-100" # Futures equivalent encoded_futures = "x-aHRE4BCj-U2xK9mPqR7sT1vW3y" original_futures = decode_binance_futures_client_order_id(encoded_futures) ``` Strings without the broker prefix pass through unchanged, so these are safe to call on any `clientOrderId` value. :::note The domain-level HTTP clients (`BinanceSpotHttpClient`, `BinanceFuturesHttpClient`) decode automatically when returning Nautilus types such as `OrderStatusReport`. Manual decoding is only needed when working outside the adapter: direct REST queries, the Binance web UI, or raw venue models. ::: ## Order books Order books can be maintained at full or partial depths. WebSocket stream update rates differ between Spot and Futures, with Nautilus using the highest available rate: - **Spot SBE diff depth**: 25ms - **Spot JSON diff depth**: 100ms - **Futures**: 0ms (unthrottled) Only one order book per instrument per trader instance is supported. When stream subscriptions vary, the Binance data client uses the latest order book data subscription (deltas or snapshots). Order book snapshot rebuilds will be triggered on: - Initial subscription of the order book data. - Data websocket reconnects. The sequence of events is as follows: - Deltas will start buffered. - Snapshot is requested and awaited. - Snapshot response is parsed to `OrderBookDeltas`. - Snapshot deltas are sent to the `DataEngine`. - Buffered deltas are iterated, dropping those where the sequence number is not greater than the last delta in the snapshot. - Deltas will stop buffering. - Remaining deltas are sent to the `DataEngine`. :::note This snapshot-and-buffer sequence applies to Futures and Spot `BookDeltas` subscriptions without an explicit depth. Spot partial-depth subscriptions deliver self-contained top-N snapshots. See [Spot market data mode](#spot-market-data-mode). ::: ## Binance data differences The `ts_event` field on `QuoteTick` differs between Spot and Futures. Spot does not provide an event timestamp, so the adapter uses `ts_init` (meaning `ts_event` and `ts_init` are identical). ## Binance specific data You can subscribe to Binance-specific data streams as they become available. :::note Bars, mark prices, index prices, and funding rates can be subscribed to in the normal way via the Rust adapter. The custom data subscriptions below are for the Python adapter. ::: Binance USD-M mark-price payloads may include an `ap` moving-average field. The Rust adapter parses this raw venue field but does not emit it as domain data or Binance custom data; Nautilus mark-price subscriptions emit mark, index, and funding-rate updates from the same stream. ### `BinanceFuturesTicker` Subscribe to 24-hour ticker statistics for a specific Futures instrument: ```python from nautilus_trader.core import nautilus_pyo3 as pyo3 client_id = pyo3.ClientId.from_str("BINANCE") self.subscribe_data( data_type=pyo3.DataType( "BinanceFuturesTicker", {"instrument_id": "BTCUSDT-PERP.BINANCE"}, ), client_id=client_id, ) ``` The adapter subscribes to the instrument `@ticker` stream and emits `BinanceFuturesTicker` custom data with `metadata={"instrument_id": ""}`. Ticker custom data requires `instrument_id`; all-market ticker subscriptions are not supported. ### `BinanceFuturesMarkPriceUpdate` Subscribe to `BinanceFuturesMarkPriceUpdate` (including funding rate info) from your actor or strategy: ```python from nautilus_trader.adapters.binance import BinanceFuturesMarkPriceUpdate from nautilus_trader.model import DataType from nautilus_trader.model import ClientId # In your `on_start` method self.subscribe_data( data_type=DataType(BinanceFuturesMarkPriceUpdate, metadata={"instrument_id": self.instrument.id}), client_id=ClientId("BINANCE"), ) ``` Received `BinanceFuturesMarkPriceUpdate` objects are passed to your `on_data` method. Check the type, as this method handles all custom/generic data. ```python from nautilus_trader.core import Data def on_data(self, data: Data): # First check the type of data if isinstance(data, BinanceFuturesMarkPriceUpdate): # Do something with the data ``` ### `BinanceFuturesLiquidation` Subscribe to liquidation updates for either: - a specific instrument (`@forceOrder`), or - all symbols (`!forceOrder@arr`) by omitting `instrument_id`. ```python from nautilus_trader.core import nautilus_pyo3 as pyo3 client_id = pyo3.ClientId.from_str("BINANCE") # Instrument-specific self.subscribe_data( data_type=pyo3.DataType( "BinanceFuturesLiquidation", {"instrument_id": "BTCUSDT-PERP.BINANCE"}, ), client_id=client_id, ) # All-market (no instrument_id metadata) self.subscribe_data( data_type=pyo3.DataType("BinanceFuturesLiquidation"), client_id=client_id, ) ``` For instrument-specific subscriptions, `CustomData.data_type` includes `metadata={"instrument_id": ""}`. For all-market subscriptions, the data type has no metadata. When both modes are subscribed concurrently, all-market takes precedence. The adapter suspends per-symbol liquidation streams while all-market is active, and restores active per-symbol streams after all-market is unsubscribed. ## Funding rates The Rust adapter emits `FundingRateUpdate` as a first-class data type through `subscribe_funding_rates`. The data comes from the [Mark Price Stream](https://developers.binance.com/docs/derivatives/usds-margined-futures/websocket-market-streams/Mark-Price-Stream) WebSocket endpoint, which provides the current funding rate and next funding time alongside mark and index prices. All three subscriptions (`subscribe_mark_prices`, `subscribe_index_prices`, `subscribe_funding_rates`) share a single `@markPrice@1s` stream with ref-counted subscription management. Historical funding rates are available through `request_funding_rates`, which queries the [Get Funding Rate History](https://developers.binance.com/docs/derivatives/usds-margined-futures/market-data/rest-api/Get-Funding-Rate-History) REST endpoint (`GET /fapi/v1/fundingRate` for USD-M, `GET /dapi/v1/fundingRate` for COIN-M). Each history row maps to a `FundingRateUpdate` with `ts_event` set to the funding time. The `next_funding_ns` field is `None` for historical rows because the endpoint does not provide it. The Python adapter exposes funding rate data through `BinanceFuturesMarkPriceUpdate` custom data subscriptions (see [Binance specific data](#binance-specific-data) below). The `interval` field on `FundingRateUpdate` is `None` for Binance because the Mark Price Stream and the funding rate history endpoint do not include a funding interval field. Binance exposes `fundingIntervalHours` through the [Get Funding Rate Info](https://developers.binance.com/docs/derivatives/usds-margined-futures/market-data/rest-api/Get-Funding-Rate-Info) REST endpoint, but the adapter does not consume it. ## Instrument status polling :::info[Rust adapter only] This feature is available in the Rust data clients (`LiveNode`). The Python data clients do not poll for status changes. ::: The adapter periodically polls Binance `exchangeInfo` to detect changes in instrument trading status. When a symbol transitions between states (e.g. Trading to Halt, or Trading to Delivering for a futures contract approaching expiry), the adapter emits an `InstrumentStatus` event. The polling interval defaults to 3600 seconds (60 minutes) and is configurable via `instrument_status_poll_secs` in the data client config. Set to `0` to disable polling entirely. On initial connect, the adapter seeds its status cache from the exchange info response without emitting events. Only subsequent polls that detect a status change emit `InstrumentStatus` events. If a symbol disappears from exchange info (e.g. after delisting or contract expiry), the adapter emits `NotAvailableForTrading`. ### Status mapping #### Spot | Binance status | MarketStatusAction | |--------------------|----------------------------| | Trading | Trading | | EndOfDay | Close | | Halt | Halt | | Break | Pause | | NonRepresentable | NotAvailableForTrading | #### Futures (USD-M) | Binance status | MarketStatusAction | |--------------------|----------------------------| | Trading | Trading | | PendingTrading | PreOpen | | PreTrading | PreOpen | | PostTrading | PostClose | | EndOfDay | Close | | Halt | Halt | | AuctionMatch | Cross | | Break | Pause | #### Futures (COIN-M) | Binance status | MarketStatusAction | |--------------------|----------------------------| | Trading | Trading | | PendingTrading | PreOpen | | PreDelivering | PreClose | | Delivering | Close | | Delivered | Close | | PreSettle | PreClose | | Settling | Close | | Close | Close | | PreDelisting | PreClose | | Delisting | Suspend | | Down | NotAvailableForTrading | :::note Only instruments that are in a tradable state at connect time are tracked. Symbols that start in a non-trading state (e.g. halted at connect) do not appear in the instruments cache, so status transitions for them are not monitored. ::: ## Rate limiting Binance uses an interval-based rate limiting system where request weight is tracked per fixed time window (every minute, resetting at :00 seconds). Each API endpoint has an assigned weight cost, and total weight usage is tracked per IP address. ### Global weight limits These are the primary limits shared across all endpoints: | Account Type | Weight Limit | Interval | |--------------|--------------|----------| | Spot/Margin | 6,000 | 1 minute | | Futures | 2,400 | 1 minute | ### Endpoint weight costs Some endpoints have higher weight costs per request: | Endpoint | Weight | Notes | |---------------------------|--------|----------------------------------------| | `/api/v3/order` | 1 | Spot order placement. | | `/api/v3/allOrders` | 20 | Spot historical orders (expensive). | | `/api/v3/klines` | 2+ | Scales with `limit` parameter. | | `/fapi/v1/order` | 1 | Futures order placement. | | `/fapi/v1/allOrders` | 20 | Futures historical orders (expensive). | | `/fapi/v1/commissionRate` | 20 | Futures commission rate query. | | `/fapi/v1/klines` | 5+ | Scales with `limit` parameter. | ### WebSocket API limits The WebSocket API (used for user data streams) shares the same weight quota as the REST API: | Limit Type | Value | Notes | |------------------|--------|---------------------------------------| | Request weight | Shared | Counts against REST API weight quota. | | Handshake | 5 | Weight cost per connection attempt. | | Ping/pong frames | 5/sec | Maximum ping/pong rate. | ### Adapter behavior The adapter uses token bucket rate limiters to approximate Binance's interval-based limits. This reduces the risk of quota violations while maintaining throughput for normal operations. For endpoints with dynamic weight (e.g. `/klines` scales with the `limit` parameter), the adapter draws a single token per call. Large history requests may need manual pacing. Monitor the `X-MBX-USED-WEIGHT-*` response headers to track actual usage. :::warning Binance returns HTTP 429 when you exceed the allowed weight. Repeated violations trigger temporary IP bans (escalating from 2 minutes to 3 days for repeat offenders). ::: :::info For the latest rate limits, query `/api/v3/exchangeInfo` (Spot) or `/fapi/v1/exchangeInfo` (Futures), or see: - [Spot API Limits](https://developers.binance.com/docs/binance-spot-api-docs/rest-api/limits) - [Futures API Limits](https://developers.binance.com/docs/derivatives/usds-margined-futures/general-info) ::: ## Configuration :::note The configuration tables below describe the **Python adapter**. The Rust adapter uses `BinanceDataClientConfig` and `BinanceExecClientConfig` with different field names. See the Rust source at `crates/adapters/binance/src/config.rs` for the definitive list of Rust config options. ::: ### Data client configuration options | Option | Default | Description | |------------------------------------|-----------|-------------| | `venue` | `BINANCE` | Venue identifier used when registering the client. | | `api_key` | `None` | Binance API key; loaded from environment variables when omitted. | | `api_secret` | `None` | Binance API secret; loaded from environment variables when omitted. | | `key_type` | `HMAC` | **Deprecated**: key type is now auto‑detected from the API secret format. Only needed to force `RSA`. | | `account_type` | `SPOT` | Account type for data endpoints (spot, margin, USDT futures, coin futures). | | `base_url_http` | `None` | Override for the HTTP REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `us` | `False` | Route requests to Binance US endpoints when `True`. | | `environment` | `None` | Binance environment: `LIVE`, `TESTNET`, or `DEMO`. Defaults to `LIVE` when `None`. | | `update_instruments_interval_mins` | `60` | Interval (minutes) between instrument catalogue refreshes. | | `use_agg_trade_ticks` | `False` | When `True`, subscribe to aggregated trade ticks instead of raw trades. Futures WebSocket subscriptions always use `@aggTrade` regardless of this flag. | | `spot_market_data_mode` | `Sbe` | *Rust only.* Spot market data transport (`Sbe` or `Json`). See [Spot market data mode](#spot-market-data-mode). | | `instrument_status_poll_secs` | `3600` | *Rust only.* Interval (seconds) between exchange info polls to detect instrument status changes. Set to `0` to disable. | | `transport_backend` | `Sockudo` | *Rust only.* WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |-----------------------------------------|-----------|-------------| | `venue` | `BINANCE` | Venue identifier used when registering the client. | | `api_key` | `None` | Binance API key; loaded from environment variables when omitted. | | `api_secret` | `None` | Binance API secret; loaded from environment variables when omitted. | | `key_type` | `HMAC` | **Deprecated**: key type is now auto‑detected from the API secret format. Only needed to force `RSA` (data clients only, RSA is not supported for execution). | | `account_type` | `SPOT` | Account type for order placement (spot, margin, USDT futures, coin futures). | | `base_url_http` | `None` | Override for the HTTP REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket API base URL. | | `base_url_ws_stream` | `None` | Override for the WebSocket stream URL (futures user data event delivery). | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `us` | `False` | Route requests to Binance US endpoints when `True`. | | `environment` | `None` | Binance environment: `LIVE`, `TESTNET`, or `DEMO`. Defaults to `LIVE` when `None`. | | `use_gtd` | `True` | When `False`, remaps GTD orders to GTC for local expiry management. | | `use_reduce_only` | `True` | When `True`, passes through `reduce_only` instructions to Binance. | | `use_position_ids` | `True` | Enable Binance hedging position IDs; set `False` for virtual hedging. | | `use_trade_lite` | `False` | Use TRADE_LITE execution events that include derived fees. | | `treat_expired_as_canceled` | `False` | Treat `EXPIRED` execution types as `CANCELED` when `True`. | | `recv_window_ms` | `5,000` | Receive window (milliseconds) for signed REST requests. | | `max_retries` | `None` | Maximum retry attempts for order submission/cancel/modify calls. | | `retry_delay_initial_ms` | `None` | Initial delay (milliseconds) between retry attempts. | | `retry_delay_max_ms` | `None` | Maximum delay (milliseconds) between retry attempts. | | `futures_leverages` | `None` | Mapping of `BinanceSymbol` to initial leverage for futures accounts. | | `futures_margin_types` | `None` | Mapping of `BinanceSymbol` to futures margin type (isolated/cross). | | `use_ws_trading` | `True` | Use the WebSocket trading API for order operations (Spot and USD-M Futures). When `False`, HTTP is used. | | `default_taker_fee` | `0.0004` | Default taker fee rate for commission estimation on exchange‑generated fills (liquidation, ADL, settlement). | | `bnfcr_currency` | `USDT` | USD-M Futures Credits Trading Mode: currency that `BNFCR` balances and fees resolve to. See [Futures Credits Trading Mode (BNFCR)](#futures-credits-trading-mode-bnfcr). | | `log_rejected_due_post_only_as_warning` | `True` | Log post‑only rejections as warnings when `True`; otherwise as errors. | | `transport_backend` | `Sockudo` | *Rust only.* WebSocket transport backend. | The most common use case is to configure a live `TradingNode` with Binance data and execution clients. Add a `BINANCE` section to your client configuration: ```python from nautilus_trader.adapters.binance import BINANCE from nautilus_trader.live.node import TradingNode config = TradingNodeConfig( ..., # Omitted data_clients={ BINANCE: { "api_key": "YOUR_BINANCE_API_KEY", "api_secret": "YOUR_BINANCE_API_SECRET", "account_type": "spot", # {spot, usdt_futures, coin_futures} "base_url_http": None, # Override with custom endpoint "base_url_ws": None, # Override with custom endpoint "us": False, # If client is for Binance US }, }, exec_clients={ BINANCE: { "api_key": "YOUR_BINANCE_API_KEY", "api_secret": "YOUR_BINANCE_API_SECRET", "account_type": "spot", # {spot, usdt_futures, coin_futures} "base_url_http": None, # Override with custom endpoint "base_url_ws": None, # Override with custom endpoint "us": False, # If client is for Binance US }, }, ) ``` Then, create a `TradingNode` and add the client factories: ```python from nautilus_trader.adapters.binance import BINANCE from nautilus_trader.adapters.binance import BinanceLiveDataClientFactory from nautilus_trader.adapters.binance import BinanceLiveExecClientFactory from nautilus_trader.live.node import TradingNode # Instantiate the live trading node with a configuration node = TradingNode(config=config) # Register the client factories with the node node.add_data_client_factory(BINANCE, BinanceLiveDataClientFactory) node.add_exec_client_factory(BINANCE, BinanceLiveExecClientFactory) # Finally build the node node.build() ``` ### Futures Credits Trading Mode (BNFCR) Binance Futures Credits Trading Mode is an EU regulatory mode in which the USD-M futures wallet, margin, PnL, and fees are denominated in `BNFCR`: an internal credit unit pegged 1:1 to USD that replaces stablecoin balances. Because `BNFCR` is not a tradable asset, the adapter maps it to the `bnfcr_currency` execution config option (default `USDT`) so account balances and commissions reconcile against the stablecoin the traded contracts settle in. Set `bnfcr_currency` to `USDC` when trading USDC-margined perpetuals. Any other unrecognized futures asset is registered as a generic crypto currency rather than failing. ### Spot market data mode `spot_market_data_mode` (Rust `BinanceDataClientConfig`) selects the Spot data transport. It affects Spot only; Futures is unchanged. | Mode | Credentials | Quotes | |--------|--------------------|--------------| | `Sbe` | Ed25519 (required) | `bestBidAsk` | | `Json` | None (public) | `bookTicker` | `Sbe` (default) uses Binance Simple Binary Encoding streams and requires Ed25519 keys (see [Key types](#key-types)); the client refuses to connect without them. `Json` uses public streams with no credentials. Full Spot `BookDeltas` subscriptions use SBE diff-depth streams at 25ms in `Sbe` mode, or public JSON diff-depth streams at 100ms in `Json` mode, with REST snapshot synchronization. Explicit depth subscriptions use partial-book snapshots (see [Order books](#order-books)). :::note Exposed to Python as `BinanceSpotMarketDataMode` on `nautilus_trader.core.nautilus_pyo3.binance`; not on the legacy Python adapter config. ::: ### Key types Binance supports three API key types: **Ed25519**, **HMAC-SHA256**, and **RSA**. The adapter auto-detects the key type from your API secret format, so no configuration is needed. **Ed25519 is strongly recommended.** Binance recommends Ed25519 for its superior performance and security. A future version of NautilusTrader will require Ed25519 exclusively. | Key Type | Data Clients | Execution Clients | Status | |----------|--------------|-------------------|--------| | Ed25519 | ✓ | ✓ | **Recommended** | | HMAC | ✓ | ✓ | Deprecated, will be removed in a future version. | | RSA | ✓ | - | Deprecated, not supported for execution. | :::tip Switch to Ed25519 keys now. Generate an Ed25519 keypair and register it with Binance. See [Generating Ed25519 keys](#generating-ed25519-keys) below. ::: :::note Ed25519 keys must be provided in unencrypted PEM format (base64-encoded ASN.1/DER). The implementation automatically extracts the 32-byte seed from the DER structure. Encrypted (password-protected) PEM keys are not supported. If your key is encrypted, decrypt it first: `openssl pkey -in encrypted.pem -out decrypted.pem` ::: #### Generating Ed25519 keys **Option 1: OpenSSL (recommended)** ```bash # Generate private key (PKCS#8 PEM format) openssl genpkey -algorithm ed25519 -out binance_ed25519_private.pem # Extract public key openssl pkey -in binance_ed25519_private.pem -pubout -out binance_ed25519_public.pem ``` **Option 2: Binance Key Generator** Download the [Binance Asymmetric Key Generator](https://github.com/binance/asymmetric-key-generator) from the releases page and run it to generate a keypair. **Registering with Binance** 1. Log in to Binance and go to **Profile** -> **API Management** 2. Click **Create API** and select **Self-generated** 3. Paste the contents of your public key file (including the `-----BEGIN PUBLIC KEY-----` header/footer) 4. Configure permissions (Enable Spot & Margin Trading, etc.) **Using with NautilusTrader** Set the private key as your API secret: ```bash export BINANCE_API_KEY="your-api-key-from-binance" export BINANCE_API_SECRET="$(cat binance_ed25519_private.pem)" ``` Or pass the PEM content directly in your configuration. :::warning Keep your private key secure. Never share it or commit it to version control. ::: ### API credentials Pass credentials directly to the configuration objects, or set the appropriate environment variables (see [Environments](#environments) for per-environment variables). :::tip Use Ed25519 keys for all clients. HMAC keys still work for both data and execution clients, but Ed25519 offers better performance and will become the only supported key type in a future version. See [Key types](#key-types). ::: :::warning The `BINANCE_ED25519_*` and `BINANCE_*_ED25519_*` environment variables have been removed for Spot/Margin. For Futures, they are deprecated and will be removed in a future version. Rename them to `BINANCE_API_KEY` / `BINANCE_API_SECRET` (Ed25519 keys are now auto-detected). ::: When the trading node starts, you receive confirmation of whether your credentials are valid and have trading permissions. ### Account type Set `account_type` using the `BinanceAccountType` enum: - `SPOT` - `USDT_FUTURES` (USDT or BUSD stablecoins as collateral) - `COIN_FUTURES` (other cryptocurrency as collateral) :::note `MARGIN` and `ISOLATED_MARGIN` account types exist in the enum but margin trading is not implemented. See [Product support](#product-support). ::: ### Base URL overrides Override the default base URLs for both HTTP REST and WebSocket APIs. This is useful for configuring API clusters or when Binance has provided specialized endpoints. ### Binance US Set `us=True` in the config to use Binance US endpoints (`False` by default). All functionality available to US accounts behaves identically to standard Binance. ### Environments Binance provides three trading environments, each with separate API credentials and endpoints. The `environment` config option selects which to use. | Environment | Config | Description | |-------------|-------------------------|------------------------------------------------------------------------| | **Live** | `environment="LIVE"` | Production trading with real funds (default). | | **Demo** | `environment="DEMO"` | Demo Trading with simulated Spot and Futures funds. | | **Testnet** | `environment="TESTNET"` | Legacy Spot and Futures test network. | #### Live (production) The default environment for live trading with real funds. Uses your main Binance account credentials. ```python config = BinanceExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", account_type=BinanceAccountType.SPOT, # environment=BinanceEnvironment.LIVE (default) ) ``` | Variable | Description | |----------------------|---------------------| | `BINANCE_API_KEY` | Live API key. | | `BINANCE_API_SECRET` | Live API secret. | #### Demo trading Practice trading with simulated funds on production infrastructure. Demo accounts use the same Binance login as your live account but trade with virtual balances. **How to get demo credentials:** 1. Log in at [binance.com/en/demo-trading](https://www.binance.com/en/demo-trading). 2. Go to **API Management** and create a demo API key. 3. Demo keys work for Spot and Futures demo endpoints. | Endpoint | URL | |----------------|-------------------------------| | Spot HTTP | `demo-api.binance.com` | | Spot WS | `demo-stream.binance.com` | | USD-M HTTP | `demo-fapi.binance.com` | | USD-M WS | `demo-fstream.binance.com` | | COIN-M HTTP | `demo-dapi.binance.com` | | COIN-M WS | `demo-dstream.binance.com` | ```python config = BinanceExecClientConfig( api_key="YOUR_DEMO_API_KEY", api_secret="YOUR_DEMO_API_SECRET", account_type=BinanceAccountType.SPOT, environment=BinanceEnvironment.DEMO, ) ``` | Variable | Description | |---------------------------|------------------| | `BINANCE_DEMO_API_KEY` | Demo API key. | | `BINANCE_DEMO_API_SECRET` | Demo API secret. | #### Testnet A legacy test network with its own user accounts, balances, and order books. Prefer `environment=BinanceEnvironment.DEMO` for new simulated trading setups. Spot testnet remains at `testnet.binance.vision`; futures testnet endpoints may route through the Demo Trading infrastructure. **How to get Spot testnet credentials:** 1. Go to [testnet.binance.vision](https://testnet.binance.vision/). 2. Log in with GitHub. 3. Generate an API key (HMAC, RSA, or Ed25519). **Futures testnet:** Existing configs with `BinanceEnvironment.TESTNET` continue to work, but new Futures testing should use `BinanceEnvironment.DEMO`. ```python config = BinanceExecClientConfig( api_key="YOUR_TESTNET_API_KEY", api_secret="YOUR_TESTNET_API_SECRET", account_type=BinanceAccountType.SPOT, environment=BinanceEnvironment.TESTNET, ) ``` | Variable | Description | |--------------------------------------|----------------------------------------------------| | `BINANCE_TESTNET_API_KEY` | Spot testnet API key. | | `BINANCE_TESTNET_API_SECRET` | Spot testnet API secret. | | `BINANCE_FUTURES_TESTNET_API_KEY` | Futures testnet API key. | | `BINANCE_FUTURES_TESTNET_API_SECRET` | Futures testnet API secret. | :::note Testnet credentials are completely separate from your live account. Market data and liquidity differ from production. ::: ### Aggregated trades Binance provides aggregated trade data endpoints as an alternative source of trades. Unlike the default trade endpoints, aggregated trade endpoints can return all ticks between a `start_time` and `end_time`. Set `use_agg_trade_ticks=True` to use aggregated trades (`False` by default). :::note For Futures (USD-M and COIN-M), the WebSocket trade subscription always uses `@aggTrade`. Binance only publishes aggregated trades on the Futures WebSocket; the legacy `@trade` stream was undocumented and has been silenced. The HTTP `request_trade_ticks` path continues to honour `use_agg_trade_ticks`. ::: ### Commission rate queries By default, Binance Futures instruments use fee tier tables based on your VIP level. For market maker accounts with negative maker fees or when precise rates are required, enable per-symbol commission rate queries: ```python from nautilus_trader.adapters.binance import BinanceInstrumentProviderConfig instrument_provider=BinanceInstrumentProviderConfig( load_all=True, query_commission_rates=True, # Query accurate rates per symbol ) ``` When enabled, the adapter queries Binance's `/fapi/v1/commissionRate` endpoint for each symbol in parallel during instrument loading. Useful for: - Market maker accounts with negative maker fees. - Accounts with custom fee arrangements. - Exact commission rates for PnL calculations. The adapter uses parallel requests with rate limiting (120 requests/minute, accounting for the endpoint's weight of 20). If a query fails, it falls back to the fee tier table. ### Parser warnings Some Binance instruments cannot be parsed into Nautilus objects if they contain field values beyond what the platform handles. These instruments are skipped with a warning. To suppress these warnings: ```python from nautilus_trader.config import InstrumentProviderConfig instrument_provider=InstrumentProviderConfig( load_all=True, log_warnings=False, ) ``` ### Futures hedge mode Binance Futures Hedge mode allows holding both long and short positions on the same instrument simultaneously. To use hedge mode: 1. Configure hedge mode on Binance before starting the strategy. 2. Set `use_reduce_only=False` in `BinanceExecClientConfig` (`True` by default). ```python from nautilus_trader.adapters.binance import BINANCE config = TradingNodeConfig( ..., # Omitted data_clients={ BINANCE: BinanceDataClientConfig( api_key=None, # 'BINANCE_API_KEY' env var api_secret=None, # 'BINANCE_API_SECRET' env var account_type=BinanceAccountType.USDT_FUTURES, base_url_http=None, # Override with custom endpoint base_url_ws=None, # Override with custom endpoint ), }, exec_clients={ BINANCE: BinanceExecClientConfig( api_key=None, # 'BINANCE_API_KEY' env var api_secret=None, # 'BINANCE_API_SECRET' env var account_type=BinanceAccountType.USDT_FUTURES, base_url_http=None, # Override with custom endpoint base_url_ws=None, # Override with custom endpoint use_reduce_only=False, # Must be disabled for Hedge mode ), } ) ``` 3. When submitting an order, use the `LONG` or `SHORT` suffix in `position_id` to indicate position direction. ```python class EMACrossHedgeMode(Strategy): ..., # Omitted def buy(self) -> None: order: MarketOrder = self.order_factory.market( instrument_id=self.instrument_id, order_side=OrderSide.BUY, quantity=self.instrument.make_qty(self.trade_size), # time_in_force=TimeInForce.FOK, ) # LONG suffix is recognized as a long position by Binance adapter. position_id = PositionId(f"{self.instrument_id}-LONG") self.submit_order(order, position_id) def sell(self) -> None: order: MarketOrder = self.order_factory.market( instrument_id=self.instrument_id, order_side=OrderSide.SELL, quantity=self.instrument.make_qty(self.trade_size), # time_in_force=TimeInForce.FOK, ) # SHORT suffix is recognized as a short position by Binance adapter. position_id = PositionId(f"{self.instrument_id}-SHORT") self.submit_order(order, position_id) ``` ### COIN-M / USD-M architecture Binance COIN-M Futures (CM / DAPI) and USD-M Futures (UM / FAPI) share a unified architecture. This section covers the implications for the adapter. See the [Important CM-UM Integration Notice](https://developers.binance.com/docs/derivatives/coin-margined-futures/Important-CM-UM-Integration-Notice) for the full details. #### WebSocket streams Market-data stream payloads include `st` (symbol type: `1` = UM, `2` = CM) on `@aggTrade`, `@ticker`, `@bookTicker`, `@depth`, `@miniTicker`, and all `!*@arr` streams. UM-side single-symbol streams also include `ps` (pair symbol) on `@bookTicker`, `@depth`, `@miniTicker`, and `@rpiDepth`. The adapter uses `msgspec` (Python) and `serde` (Rust) for JSON decoding, both of which ignore unknown fields by default. These fields are silently dropped. All-market array streams (`!ticker@arr`, `!miniTicker@arr`, `!bookTicker`, `!forceOrder@arr`, `!contractInfo`) deliver merged UM + CM content on both `fstream` and `dstream`. #### REST and WebSocket API - Order placement and modification acknowledgement responses do not include `avgPrice` / `cumQuote` / `cumBase`. The adapter sources fills from the user data stream. Query endpoints (`GET /{f,d}api/v1/order`, `userTrades`) still return these fields. - `PUT /dapi/v1/order` (COIN-M modify) requires both `price` and `quantity`. The adapter's `_modify_order` sends both fields, falling back to the cached order's values. - COIN-M conditional orders (STOP, TAKE_PROFIT, etc.) use the `/dapi/v1/algoOrder` endpoint. The adapter routes all futures conditional orders through the algo order API. - `GET /dapi/v1/openOrders` with an invalid symbol returns error `-1121`. #### Rate-limit pools UM and CM share a single rate-limit pool per IP (2400 weight/min, 1200 orders/min, 300 orders/10s). The adapter creates separate HTTP client instances for UM and CM, each with its own rate limiter. If a node drives both UM and CM clients simultaneously, the combined traffic may exceed the shared server-side budget. #### dualSidePosition UM and CM share the same `dualSidePosition` setting. Changing it on either side affects both. Ensure both UM and CM have no open orders or positions before flipping the setting. ## Contributing :::info To contribute to the Binance adapter, see the [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # BitMEX Source: https://nautilustrader.io/docs/latest/integrations/bitmex/ Founded in 2014, BitMEX (Bitcoin Mercantile Exchange) is a cryptocurrency derivatives trading platform offering spot, perpetual contracts, traditional futures, prediction markets, and other advanced trading products. This integration supports live market data ingest and order execution with BitMEX. ## Overview This adapter is implemented in Rust, with optional Python bindings for ease of use in Python-based workflows. It does not require external BitMEX client libraries; the core components are compiled as a static library and linked automatically during the build. ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/bitmex/). ## Components This guide assumes a trader is setting up for both live market data feeds, and trade execution. The BitMEX adapter includes multiple components, which can be used together or separately depending on the use case. - `BitmexHttpClient`: Low-level HTTP API connectivity. - `BitmexWebSocketClient`: Low-level WebSocket API connectivity. - `BitmexInstrumentProvider`: Instrument parsing and loading functionality. - `BitmexDataClient`: A market data feed manager. - `BitmexExecutionClient`: An account management and trade execution gateway. - `BitmexDataClientFactory`: Factory for BitMEX data clients (used by the trading node builder). - `BitmexExecutionClientFactory`: Factory for BitMEX execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to necessarily work with these lower level components directly. ::: ## BitMEX documentation BitMEX provides extensive documentation for users: - [BitMEX API Explorer](https://www.bitmex.com/app/restAPI) - Interactive API documentation. - [BitMEX API Documentation](https://www.bitmex.com/app/apiOverview) - Complete API reference. - [BitMEX Exchange Rules](https://www.bitmex.com/exchange-rules) - Official exchange rules and regulations. - [Contract Guides](https://www.bitmex.com/app/contract) - Detailed contract specifications. - [Spot Trading Guide](https://www.bitmex.com/app/spotGuide) - Spot trading overview. - [Perpetual Contracts Guide](https://www.bitmex.com/app/perpetualContractsGuide) - Perpetual swaps explained. - [Futures Contracts Guide](https://www.bitmex.com/app/futuresGuide) - Traditional futures information. It's recommended you refer to the BitMEX documentation in conjunction with this NautilusTrader integration guide. ## Product support | Product Type | Data Feed | Trading | Notes | |-------------------|-----------|---------|-----------------------------------------------------| | Spot | ✓ | ✓ | Limited pairs, unified wallet with derivatives. | | Perpetual Swaps | ✓ | ✓ | Inverse and linear contracts available. | | Stock Perpetuals | - | - | *Not yet supported*. Currently on testnet only. | | Futures | ✓ | ✓ | Traditional fixed expiration contracts. | | Quanto Futures | ✓ | ✓ | Settled in different currency than underlying. | | Prediction Markets| ✓ | ✓ | Event‑based contracts, 0-100 pricing, USDT settled. | | Options | - | - | *Not provided by BitMEX*. | :::note BitMEX has discontinued their options products to focus on their core derivatives and spot offerings. ::: ### Spot trading - Direct token/coin trading with immediate settlement. - Major pairs including XBT/USDT, ETH/USDT, ETH/XBT. - Additional altcoin pairs (LINK, SOL, UNI, APE, AXS, BMEX against USDT). ### Derivatives - **Perpetual contracts**: Inverse (e.g., XBTUSD) and linear (e.g., ETHUSDT). - **Traditional futures**: Fixed expiration date contracts. - **Quanto futures**: Contracts settled in a different currency than the underlying. - **Prediction markets**: Event-based derivatives (e.g., P_FTXZ26, P_SBFJAILZ26) allowing traders to speculate on outcomes across crypto, finance, and other events. No leverage, priced 0-100, settled in USDT. - **Stock perpetuals**: Equity-based perpetuals (e.g., SPYUSDT, CRCLUSDT). *Currently on testnet only; not yet supported by this adapter.* ### Instrument type codes (CFI) BitMEX uses CFI (Classification of Financial Instruments) codes following the ISO 10962 standard. The adapter recognizes the following instrument type codes: | Code | Type | Status | Description | |----------|----------------------|-------------|-------------------------------------------------| | `FFWCSX` | Perpetual Contract | Supported | Crypto‑based perpetual swaps (e.g., XBTUSD). | | `FFWCSF` | Perpetual FX | Supported | FX‑based perpetual contracts. | | `FFCCSX` | Futures | Supported | Calendar futures with fixed expiration. | | `FFICSX` | Prediction Market | Supported | Event‑based prediction contracts. | | `IFXXXP` | Spot | Supported | Spot trading pairs. | | `FFSCSX` | Stock Perpetual | Unsupported | Stock/equity‑based perpetuals. Testnet only. | | `SRMCSX` | Swap Rate | Unsupported | Yield‑based swap products (historical). | | `MR****` | Index | Reference | BitMEX indices (non‑tradeable, for price ref). | See [BitMEX Typ Values](https://support.bitmex.com/hc/en-gb/articles/6299296145565-What-are-the-Typ-Values-for-Instrument-endpoint) for more details. ## Symbology BitMEX uses a specific naming convention for its trading symbols. Understanding this convention is crucial for correctly identifying and trading instruments. ### Symbol format BitMEX symbols typically follow these patterns: - **Spot pairs**: Base currency + Quote currency (e.g., `XBT/USDT`, `ETH/USDT`). - **Perpetual contracts**: Base currency + Quote currency (e.g., `XBTUSD`, `ETHUSD`). - **Futures contracts**: Base currency + Expiry code (e.g., `XBTM24`, `ETHH25`). - **Quanto contracts**: Special naming for non-USD settled contracts. - **Prediction markets**: `P_` prefix + Event identifier + Expiry code (e.g., `P_POWELLK26`, `P_FTXZ26`). :::info BitMEX uses `XBT` as the symbol for Bitcoin instead of `BTC`. This follows the ISO 4217 currency code standard where "X" denotes non-national currencies. XBT and BTC refer to the same asset - Bitcoin. ::: ### Expiry codes Futures contracts use standard futures month codes: - `F` = January - `G` = February - `H` = March - `J` = April - `K` = May - `M` = June - `N` = July - `Q` = August - `U` = September - `V` = October - `X` = November - `Z` = December Followed by the year (e.g., `24` for 2024, `25` for 2025). ### NautilusTrader instrument IDs Within NautilusTrader, BitMEX instruments are identified using the native BitMEX symbol directly, combined with the venue identifier: ```python from nautilus_trader.model.identifiers import InstrumentId # Spot pairs (note: no slash in the symbol) spot_id = InstrumentId.from_str("XBTUSDT.BITMEX") # XBT/USDT spot eth_spot_id = InstrumentId.from_str("ETHUSDT.BITMEX") # ETH/USDT spot # Perpetual contracts perp_id = InstrumentId.from_str("XBTUSD.BITMEX") # Bitcoin perpetual (inverse) linear_perp_id = InstrumentId.from_str("ETHUSDT.BITMEX") # Ethereum perpetual (linear) # Futures contract (June 2024) futures_id = InstrumentId.from_str("XBTM24.BITMEX") # Bitcoin futures expiring June 2024 # Prediction market contracts prediction_id = InstrumentId.from_str("P_XBTETFV23.BITMEX") # Bitcoin ETF SEC approval prediction expiring October 2023 ``` :::note BitMEX spot symbols in NautilusTrader don't include the slash (/) that appears in the BitMEX UI. Use `XBTUSDT` instead of `XBT/USDT`. ::: ### Quantity scaling BitMEX reports spot and derivative quantities in *contract* units. The actual asset size per contract is exchange-specific and published on the instrument definition: - `lotSize` – minimum number of contracts you can trade. - `underlyingToPositionMultiplier` – number of contracts per unit of the underlying asset. For example, the SOL/USDT spot instrument (`SOLUSDT`) exposes `lotSize = 1000` and `underlyingToPositionMultiplier = 10000`, meaning one contract represents `1 / 10000 = 0.0001` SOL, and the minimum order (`lotSize * contract_size`) is `0.1` SOL. The adapter now derives the contract size directly from these fields and scales both inbound market data and outbound orders accordingly, so quantities in Nautilus are always expressed in base units (SOL, ETH, etc.). See the BitMEX API documentation for details on these fields: . ## Orders capability The BitMEX integration supports the following order types and execution features. ### Order types | Order Type | Supported | Notes | |------------------------|-----------|-----------------------------------------------| | `MARKET` | ✓ | Executed immediately at current market price. Quote quantity not supported. | | `LIMIT` | ✓ | Executed only at specified price or better. | | `STOP_MARKET` | ✓ | Supported (set `trigger_price`). | | `STOP_LIMIT` | ✓ | Supported (set `price` and `trigger_price`). | | `MARKET_IF_TOUCHED` | ✓ | Supported (set `trigger_price`). | | `LIMIT_IF_TOUCHED` | ✓ | Supported (set `price` and `trigger_price`). | | `TRAILING_STOP_MARKET` | ✓ | Supported (set `trailing_offset`). Price offset type only. | | `TRAILING_STOP_LIMIT` | ✓ | Supported (set `price` and `trailing_offset`). Price offset type only. | ### Execution instructions | Instruction | Supported | Notes | |---------------|-----------|-----------------------------------------------------------------------------------| | `post_only` | ✓ | Supported via `ParticipateDoNotInitiate` execution instruction on `LIMIT` orders. | | `reduce_only` | ✓ | Supported via `ReduceOnly` execution instruction. | :::note Post-only orders that would cross the spread are canceled by BitMEX rather than rejected. The integration surfaces these as rejections with `due_post_only=True` so strategies can handle them consistently. ::: ### Trigger types BitMEX supports multiple reference prices to evaluate stop/conditional order triggers for: - `STOP_MARKET` - `STOP_LIMIT` - `MARKET_IF_TOUCHED` - `LIMIT_IF_TOUCHED` Choose the trigger type that matches your strategy and/or risk preferences. | Reference price | Nautilus `TriggerType` | BitMEX value | Notes | |-----------------|------------------------|---------------|---------------------------------------------------------------------------------| | Last trade | `LAST_PRICE` | `LastPrice` | BitMEX default; triggers on the last traded price. | | Mark price | `MARK_PRICE` | `MarkPrice` | Recommended for many stop‑loss use cases to reduce stop‑outs from price spikes. | | Index price | `INDEX_PRICE` | `IndexPrice` | Tracks the external index; useful for some contracts. | - If no `trigger_type` is provided, BitMEX uses its venue default (`LastPrice`). - These trigger references are exchange-evaluated; the order remains resting at the venue until triggered. **Example**: ```python from nautilus_trader.model.enums import TriggerType order = self.order_factory.stop_market( instrument_id=instrument_id, order_side=order_side, quantity=qty, trigger_price=trigger, trigger_type=TriggerType.MARK_PRICE, # Use BitMEX Mark Price as reference ) ``` `ExecTester` example configuration also demonstrates setting `stop_trigger_type=TriggerType.MARK_PRICE` in `examples/live/bitmex/bitmex_exec_tester.py`. ### Trailing stops BitMEX supports trailing stop orders that automatically adjust the stop price as the market moves favorably. The adapter maps `TRAILING_STOP_MARKET` and `TRAILING_STOP_LIMIT` orders to BitMEX's pegged orders with the `TrailingStopPeg` price type; the limit variant additionally carries the limit `price`. **Limitations:** - Only `PRICE` trailing offset type is supported (absolute price offset, not basis points or ticks). - The offset sign is handled automatically: sell stops use negative offset, buy stops use positive. - Trigger type can be combined with trailing stops for additional control. **Example**: ```python from nautilus_trader.model.enums import TrailingOffsetType order = self.order_factory.trailing_stop_market( instrument_id=instrument_id, order_side=OrderSide.SELL, quantity=qty, trailing_offset=Decimal("100"), # $100 trailing offset trailing_offset_type=TrailingOffsetType.PRICE, trigger_type=TriggerType.LAST_PRICE, # Optional ) ``` :::note BitMEX updates trailing stop prices periodically as the market moves. The stop price freezes when the market moves toward the trigger level. See the [BitMEX API documentation](https://www.bitmex.com/app/perpetualContractsGuide) for current update cadence details. ::: ### Pegged orders BitMEX supports pegged orders (BBO) that automatically track a reference price. The adapter supports pegged orders via the `params` dict on `submit_order`, which overrides the order type to `Pegged` on the exchange side. | Peg price type | Description | |----------------|------------------------------------------------------------------| | `PrimaryPeg` | Pegs to the best bid (buy) or best ask (sell). | | `MarketPeg` | Pegs to the opposite side (best ask for buy, best bid for sell). | | `MidPricePeg` | Pegs to the mid‑price between bid and ask. | | `LastPeg` | Pegs to the last traded price. | **Requirements**: - The underlying order must be a `LIMIT` order. Other order types are rejected. - `peg_price_type` is required; `peg_offset_value` is optional (defaults to 0). - `peg_offset_value` can be negative (e.g., sell-side offsets) or fractional. **Example**: ```python # Pegged to best bid with zero offset (BBO) order = self.order_factory.limit( instrument_id=instrument_id, order_side=OrderSide.BUY, quantity=qty, price=price, # Required for LIMIT order, but overridden by peg ) self.submit_order(order, params={"peg_price_type": "PrimaryPeg", "peg_offset_value": "0"}) # Pegged to mid-price with a -0.5 offset self.submit_order(order, params={"peg_price_type": "MidPricePeg", "peg_offset_value": "-0.5"}) ``` :::note The `price` field is still required when constructing the `LimitOrder`, but BitMEX ignores it for pegged orders and instead continuously tracks the reference price plus offset. ::: ### Time in force | Time in force | Supported | Notes | |----------------|-----------|-----------------------------------------------------| | `GTC` | ✓ | Good Till Canceled (default). | | `GTD` | - | *Not supported by BitMEX*. | | `FOK` | ✓ | Fill or Kill - fills entire order or cancels. | | `IOC` | ✓ | Immediate or Cancel - partial fill allowed. | | `DAY` | ✓ | Expires at 00:00 UTC (BitMEX trading day boundary). | :::note `DAY` orders expire at 12:00am UTC, which marks the BitMEX trading day boundary (end of trading hours for that day). See the [BitMEX Exchange Rules](https://www.bitmex.com/exchange-rules) and [API documentation](https://www.bitmex.com/api/explorer/) for complete details. ::: ### Advanced order features | Feature | Supported | Notes | |--------------------|-----------|--------------------------------------------------------------------------| | Order Modification | ✓ | Modify price, quantity, and trigger price. | | Bracket Orders | ✓ | Use `contingency_type` and `linked_order_ids`. | | Iceberg Orders | ✓ | Use `display_qty`. | | Trailing Stops | ✓ | Use `trailing_offset`. Price offset type only. | | Pegged Orders | ✓ | Use `params` with `peg_price_type`. See [Pegged orders](#pegged-orders). | ### Batch operations | Operation | Supported | Notes | |--------------------|-----------|---------------------------------------------| | Batch Submit | - | *Not supported by BitMEX*. | | Batch Modify | - | *Not supported by BitMEX*. | | Batch Cancel | ✓ | Cancel multiple orders in a single request. | ### Position management | Feature | Supported | Notes | |---------------------|-----------|----------------------------------------------------| | Query positions | ✓ | REST and real‑time position updates via WebSocket. | | Cross margin | ✓ | Default margin mode. | | Isolated margin | ✓ | | ### Order querying | Feature | Supported | Notes | |----------------------|-----------|----------------------------------------------| | Query open orders | ✓ | List all active orders. | | Query order history | ✓ | Historical order data. | | Order status updates | ✓ | Real‑time order state changes via WebSocket. | | Trade history | ✓ | Execution and fill reports. | ### Liquidation and ADL handling BitMEX surfaces forced-close fills through the `execType` field on the `execution` channel: | `execType` | Meaning | |---------------|--------------------------------------------------------------| | `Trade` | Normal execution (user or taker‑initiated). | | `Liquidation` | Position was force‑closed by the liquidation engine. BitMEX uses this code for both auto‑deleveraging and counterparty liquidation fills. | | `Bankruptcy` | Account bankruptcy; position closed against the insurance fund. | | `Settlement` | Scheduled contract settlement. | | `Funding` | Funding settlement on open positions. | The adapter routes `Liquidation` and `Bankruptcy` through the standard `FillReport` path and logs a warning on bankruptcy executions. BitMEX's public API does **not** distinguish auto-deleveraging from counterparty liquidation in `execType`; both appear as `Liquidation`. An ADL-closed position can usually be identified by zero commission and the absence of a matching order in the local cache (the engine creates an external order for it). Upstream references: - [`/execution` field definitions](https://support.bitmex.com/hc/en-gb/articles/6205689858077--execution-field-definitions) - [Auto-Deleveraging overview](https://support.bitmex.com/hc/en-gb/articles/18589621443357-What-is-Auto-Deleveraging) - [Liquidation overview](https://support.bitmex.com/hc/en-gb/articles/360003188434-Liquidations) ## Market data - Order book deltas: `L2_MBP` only; `depth` 0 (full book) or 25. - Order book depth10 snapshots: fixed 10 levels via `orderBook10` channel. - Quotes, trades, and instrument updates are supported via WebSocket. - Funding rates, mark prices, and index prices are supported where applicable. - REST requests: - Current L2 order book snapshots with optional depth. - Trade ticks with optional `start`, `end`, and `limit` filters (up to 1,000 results per call). - Time bars (`1m`, `5m`, `1h`, `1d`) for externally aggregated LAST prices, including optional partial bins. - Funding rates with optional `start`, `end`, and `limit` filters. :::note BitMEX table REST page sizes vary by endpoint. Funding rate requests paginate in 500-row pages until the requested limit or time range is exhausted; trade tick and bar requests currently return one venue page per request and allow up to 1,000 rows. ::: ### Trade ID derivation Trade ticks and fills use the venue-provided `trdMatchID` (UUID) as the `TradeId`. When the venue omits `trdMatchID` (bucketed trades or certain execution types), the execution path falls back to the venue's `execID`; market data parsers fall back to a deterministic FNV-1a hash of the symbol, `ts_event`, price, size, and side. The same venue event yields the same trade ID across replays, keeping downstream dedup intact. ## Connection management ### HTTP Keep-Alive The BitMEX adapter uses HTTP keep-alive for optimal performance: - **Connection pooling**: Connections are automatically pooled and reused. - **Keep-alive timeout**: 90 seconds (matches BitMEX server-side timeout). - **Automatic reconnection**: Failed connections are automatically re-established. - **SSL session caching**: Reduces handshake overhead for subsequent requests. This configuration ensures low-latency communication with BitMEX servers by maintaining persistent connections and avoiding the overhead of establishing new connections for each request. ### Request authentication and expiration BitMEX uses an `api-expires` header for request authentication to prevent replay attacks: - Signed requests include an `api-expires` Unix timestamp set `recv_window_ms / 1000` seconds ahead (10 seconds by default). - BitMEX rejects any request once that timestamp has passed, so keep latency within your configured window. ## Funding rates The adapter receives funding rate data from the [Funding](https://www.bitmex.com/app/wsAPI#Funding) WebSocket stream. BitMEX returns a `fundingInterval` datetime field in each message, and the adapter reads the hours and minutes to compute the `interval` field on `FundingRateUpdate`. ## Rate limiting BitMEX implements a dual-layer rate limiting system: ### REST limits - **Burst limit**: 10 requests per second for authenticated users (applies to order placement, modification, and cancel endpoints). - **Rolling minute limit**: 120 requests per minute for authenticated users (30 requests per minute for unauthenticated users). - **Order caps**: 200 open orders and 10 stop orders per symbol; exceeding these caps triggers exchange-side rejections. The adapter enforces these quotas locally using the configured `max_requests_per_second` and `max_requests_per_minute` values. ### WebSocket limits - Connection requests: follow the exchange guidance (currently 3 connections per second per IP). - Private streams require authentication; the adapter reconnects automatically if a limit is exceeded. :::warning Exceeding BitMEX rate limits returns HTTP 429 and may trigger temporary IP bans; persistent 4xx/5xx errors can extend the lockout period. ::: ### Configurable rate limits The rate limits can be configured if your account has different limits than the defaults: | Parameter | Default (authenticated) | Default (unauthenticated) | Description | |----------------------------|-------------------------|---------------------------|-----------------------------------------------------| | `max_requests_per_second` | 10 | 10 | Maximum requests per second (burst limit). | | `max_requests_per_minute` | 120 | 30 | Maximum requests per minute (rolling window). | :::info For more details on rate limiting, see the [BitMEX API documentation on rate limits](https://www.bitmex.com/app/restAPI#Limits). ::: :::warning **Cancel Broadcaster Rate Limit Considerations** The cancel broadcaster (when `canceller_pool_size > 1`) fans out each cancel request to multiple independent HTTP clients in parallel. Each client maintains its own rate limiter, which means the effective request rate is multiplied by the pool size. **Example**: With `canceller_pool_size=3` and `max_requests_per_second=10`, a single cancel operation consumes **3 requests** (one per client), potentially reaching **30 requests/second** if canceling rapidly. Since BitMEX enforces rate limits **at the account level** (not per connection), the broadcaster can push you over the exchange's default limits of 10 req/s burst and 120 req/min rolling window. **Mitigations**: Reduce `max_requests_per_second` and `max_requests_per_minute` proportionally (divide by `canceller_pool_size`), or adjust the pool size itself (see [Cancel broadcaster configuration](#cancel-broadcaster)). Future versions may support shared rate limiters across the pool. ::: ### Rate-limit headers BitMEX exposes the current allowance via response headers: - `x-ratelimit-limit`: total requests permitted in the current window. - `x-ratelimit-remaining`: remaining requests before throttling occurs. - `x-ratelimit-reset`: UNIX timestamp when the allowance resets. - `retry-after`: seconds to wait after a 429 response. ## Submit broadcaster The BitMEX execution client includes a submit broadcaster that provides higher assurance of market and limit orders being accepted at target prices through parallel request fanout, trading lower minimum latency for the risk of duplicate submissions. ### Concepts Order submissions are time-critical operations - when a strategy decides to enter a position, any delay can result in missed opportunities or adverse pricing. The submit broadcaster addresses this by: - **Parallel fanout**: Submit requests are simultaneously broadcast to multiple independent HTTP client instances. - **First-success short-circuiting**: The first successful response wins, minimizing latency to acceptance. - **Shared client_order_id**: All transports use the same `client_order_id`. BitMEX rejects duplicate submissions with "duplicate clOrdID" (tracked as expected rejections). - **Latency vs. duplicates tradeoff**: Accepts the risk of potential duplicate fills (if multiple transports succeed before rejection) in exchange for lower minimum latency and higher assurance of acceptance. This architecture reduces the minimum latency to order acceptance by parallelizing across multiple network paths. ### Usage The submit broadcaster is opt-in and controlled via the `submit_tries` parameter when submitting orders. By default, orders are submitted through a single HTTP client. To enable broadcasting: ```python # Single submission (default behavior) self.submit_order(order) # Broadcast to 2 parallel HTTP clients for redundancy self.submit_order(order, params={"submit_tries": 2}) # Broadcast to 3 parallel HTTP clients (maximum recommended) self.submit_order(order, params={"submit_tries": 3}) ``` **Key points**: - `submit_tries` must be a positive integer. - Broadcasting only occurs when `submit_tries > 1`. Default submits go through a single HTTP client. - If `submit_tries` exceeds `submitter_pool_size`, it will be capped at the pool size with a warning. - All transports use the same `client_order_id`; BitMEX rejects duplicates as expected rejections. ### Health monitoring Each HTTP client in the broadcaster pool maintains health metrics: - Successful submissions mark a client as healthy. - Failed requests increment error counters. - Background health checks periodically verify client connectivity. - Degraded clients are tracked but remain in the pool to maintain fault tolerance. The broadcaster exposes metrics including total submits, successful submits, failed submits, and expected rejects for operational monitoring and debugging. #### Tracked metrics | Metric | Type | Description | |--------------------------|--------|-----------------------------------------------------------------------------------------------------------------------| | `total_submits` | `u64` | Total number of submit operations initiated. | | `successful_submits` | `u64` | Number of submit operations that successfully received acknowledgement from BitMEX. | | `failed_submits` | `u64` | Number of submit operations where all HTTP clients in the pool failed (no healthy clients or all requests failed). | | `expected_rejects` | `u64` | Number of expected rejection patterns detected (e.g., duplicate clOrdID from parallel submissions). | | `healthy_clients` | `usize`| Current number of healthy HTTP clients in the pool (clients that passed recent health checks). | | `total_clients` | `usize`| Total number of HTTP clients configured in the pool (`submitter_pool_size`). | These metrics can be accessed programmatically via the `get_metrics()` method on the `SubmitBroadcaster` instance. ### Configuration The submit broadcaster is configured via the execution client configuration: | Option | Default | Description | |------------------------|---------|-------------------------------------------------------------------------------------------| | `submitter_pool_size` | `None` | Size of the HTTP client pool. `None` resolves to 1 (single client, no redundancy). | | `submitter_proxy_urls` | `None` | Optional list of proxy URLs for submit broadcaster path diversity. | **Example configuration**: ```python from nautilus_trader.adapters.bitmex.config import BitmexExecClientConfig exec_config = BitmexExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", submitter_pool_size=3, # Recommended pool size for redundancy ) ``` :::tip For HFT strategies without higher rate limits, consider the advantages of using the submit broadcaster against potentially hitting rate limits, as each client has an independent rate limit budget. The default `submitter_pool_size=None` disables the broadcaster. The recommended setting of `submitter_pool_size=3` broadcasts each submit request to 3 parallel HTTP clients for fault tolerance, which consumes 3× the rate limit quota per submit operation but provides higher assurance against network or exchange issues. ::: The broadcaster is automatically started when the execution client connects and stopped when it disconnects. Submit operations are routed through the broadcaster only when `submit_tries > 1`; default submits use a single HTTP client directly. ## Cancel broadcaster The BitMEX execution client includes a cancel broadcaster that provides fault-tolerant order cancellation through parallel request fanout. ### Concepts Order cancellations are time-critical operations - when a strategy decides to cancel an order, any delay or failure can result in unintended fills, slippage, or unwanted position exposure. The cancel broadcaster addresses this by: - **Parallel fanout**: Cancel requests are simultaneously broadcast to multiple independent HTTP client instances. - **First-success short-circuiting**: The first successful response wins, and remaining in-flight requests are immediately aborted. - **Fault tolerance**: If one HTTP client experiences network issues, DNS failures, or connection timeouts, other clients in the pool continue processing. - **Idempotent success handling**: Responses indicating the order was already canceled (such as "orderID not found" or similar idempotent states) are treated as success rather than failure, preventing unnecessary error propagation. This architecture ensures that a single network path failure or slow connection doesn't block cancel operations, improving the reliability of risk management and position control in live trading. ### Health monitoring Each HTTP client in the broadcaster pool maintains health metrics: - Successful cancellations mark a client as healthy. - Failed requests increment error counters. - Background health checks periodically verify client connectivity. - Degraded clients are tracked but remain in the pool to maintain fault tolerance. The broadcaster exposes metrics including total cancels, successful cancels, failed cancels, expected rejects (already canceled orders), and idempotent successes for operational monitoring and debugging. #### Tracked metrics | Metric | Type | Description | |--------------------------|--------|-----------------------------------------------------------------------------------------------------------------------| | `total_cancels` | `u64` | Total number of cancel operations initiated (includes single, batch, and cancel‑all requests). | | `successful_cancels` | `u64` | Number of cancel operations that successfully received acknowledgement from BitMEX. | | `failed_cancels` | `u64` | Number of cancel operations where all HTTP clients in the pool failed (no healthy clients or all requests failed). | | `expected_rejects` | `u64` | Number of expected rejection patterns detected (e.g., post‑only order rejections). | | `idempotent_successes` | `u64` | Number of idempotent success responses (order already cancelled, order not found, unable to cancel due to state). | | `healthy_clients` | `usize`| Current number of healthy HTTP clients in the pool (clients that passed recent health checks). | | `total_clients` | `usize`| Total number of HTTP clients configured in the pool (`canceller_pool_size`). | These metrics can be accessed programmatically via the `get_metrics()` method on the `CancelBroadcaster` instance. ### Configuration The cancel broadcaster is configured via the execution client configuration: | Option | Default | Description | |------------------------|---------|-------------------------------------------------------------------------------------------| | `canceller_pool_size` | `None` | Size of the HTTP client pool. `None` resolves to 1 (single client, no redundancy). | | `canceller_proxy_urls` | `None` | Optional list of proxy URLs for cancel broadcaster path diversity. | **Example configuration**: ```python from nautilus_trader.adapters.bitmex.config import BitmexExecClientConfig exec_config = BitmexExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", canceller_pool_size=3, # Recommended pool size for redundancy ) ``` :::tip For HFT strategies without higher rate limits, consider the advantages of using the cancel broadcaster against potentially hitting rate limits, as each client has an independent rate limit budget. The default `canceller_pool_size=None` disables the broadcaster. The recommended setting of `canceller_pool_size=3` broadcasts each cancel request to 3 parallel HTTP clients for fault tolerance, which consumes 3× the rate limit quota per cancel operation but provides higher assurance against network or exchange issues. ::: The broadcaster is automatically started when the execution client connects and stopped when it disconnects. All cancel operations (`cancel_order`, `cancel_all_orders`, `batch_cancel_orders`) are automatically routed through the broadcaster without requiring any changes to strategy code. ## Dead man's switch The adapter supports BitMEX's [dead man's switch](https://www.bitmex.com/app/restAPI#OrdercancelAllAfter) (`cancelAllAfter`), which provides automatic order cancellation as a safety net against connectivity failures. ### How it works When enabled, a server-side timer is set on BitMEX. If the timer expires without being refreshed, BitMEX cancels **all** open orders on the account. The adapter keeps the timer alive by sending periodic heartbeat requests. If the adapter loses connectivity (network failure, process crash, etc.), the heartbeats stop and BitMEX cancels the orders after the configured timeout. The flow: 1. On **connect**, the adapter calls `POST /api/v1/order/cancelAllAfter` with the configured timeout (in milliseconds) to arm the server-side timer. 2. A background task sends the same request at a **refresh interval** of `timeout / 4` (minimum 1 second) to keep resetting the timer before it expires. 3. On **disconnect**, the adapter waits for the background heartbeat task to fully shut down, then calls `cancelAllAfter` with `timeout=0` to **disarm** the server-side timer. For example, with a 60-second timeout the adapter sends a heartbeat every 15 seconds. If four consecutive heartbeats fail (60 seconds of lost connectivity), BitMEX cancels all open orders. ### Disconnect ordering Disarming the dead man's switch during disconnect requires careful ordering. The disarm request (`timeout=0`) should be the last `cancelAllAfter` call to reach BitMEX. If an in-flight heartbeat were processed after the disarm, it would re-arm the server-side timer and orders could be unexpectedly cancelled after the timeout expires, even though the adapter disconnected gracefully. The adapter mitigates this in both implementations: - **Rust**: The heartbeat task is immediately stopped (abort + await) so disconnect does not stall waiting for a sleep or HTTP timeout to elapse. The disarm request is then sent after the task has exited. - **Python**: The heartbeat task is cancelled and awaited, ensuring the coroutine fully unwinds before the disarm request is sent. In a force-stop scenario (e.g., process shutdown via `stop()`), the heartbeat task is aborted without disarming. This is intentional, as the server-side timer provides the desired safety behavior when the process exits unexpectedly. :::note Each heartbeat consumes one REST rate limit token. A 60-second timeout uses approximately 4 requests per minute from the 120/min budget. ::: ### Configuration Enable the dead man's switch by setting `deadmans_switch_timeout_secs` on the execution client config: ```python from nautilus_trader.adapters.bitmex.config import BitmexExecClientConfig exec_config = BitmexExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", deadmans_switch_timeout_secs=60, # Cancel all orders after 60s of lost connectivity ) ``` When enabled, the adapter logs: ``` Starting dead man's switch: timeout=60s, refresh_interval=15s ``` on connect, and: ``` Disarming dead man's switch ``` on disconnect. :::tip A timeout of **60 seconds** is the recommended starting point. Shorter timeouts provide faster protection but are more sensitive to transient network blips. Longer timeouts are more tolerant of brief outages but leave orders exposed longer during a real failure. ::: :::warning The dead man's switch applies to **all** open orders on the account, not just orders placed by the adapter. If other systems place orders on the same account, enabling the dead man's switch will affect those orders too. ::: ## Configuration ### API credentials BitMEX API credentials can be provided either directly in the configuration or via environment variables: - `BITMEX_API_KEY`: Your BitMEX API key for production. - `BITMEX_API_SECRET`: Your BitMEX API secret for production. - `BITMEX_TESTNET_API_KEY`: Your BitMEX API key for testnet. - `BITMEX_TESTNET_API_SECRET`: Your BitMEX API secret for testnet. To generate API keys: 1. Log in to your BitMEX account. 2. Navigate to Account & Security -> API Keys. 3. Create a new API key with appropriate permissions. 4. For testnet, use [testnet.bitmex.com](https://testnet.bitmex.com). :::note **Testnet API endpoints**: - REST API: `https://testnet.bitmex.com/api/v1` - WebSocket: `wss://ws.testnet.bitmex.com/realtime` The adapter automatically routes requests to the correct endpoints when `environment=BitmexEnvironment.TESTNET` is configured. ::: ### Data client configuration options The BitMEX data client provides the following configuration options: | Option | Default | Description | |------------------------------------|-----------|-------------| | `api_key` | `None` | Optional API key; if `None`, loaded from the environment selected by `environment`. | | `api_secret` | `None` | Optional API secret; if `None`, loaded from the environment selected by `environment`. | | `environment` | `None` | Environment enum (`MAINNET` or `TESTNET`). | | `base_url_http` | `None` | Override for the REST base URL (defaults to production). | | `base_url_ws` | `None` | Override for the WebSocket base URL (defaults to production). | | `http_timeout_secs` | `60` | Request timeout applied to HTTP calls. | | `max_retries` | `3` | Maximum retry attempts for HTTP calls. | | `retry_delay_initial_ms` | `1,000` | Initial backoff delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10,000` | Maximum backoff delay (milliseconds) between retries. | | `recv_window_ms` | `10,000` | Expiration window (milliseconds) for signed requests. See [Request authentication](#request-authentication-and-expiration). | | `update_instruments_interval_mins` | `None` | Interval (minutes) between instrument catalogue refreshes. `None` disables periodic refresh. | | `max_requests_per_second` | `10` | Burst rate limit enforced by the adapter for REST calls. | | `max_requests_per_minute` | `120` | Rolling minute rate limit enforced by the adapter for REST calls. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options The BitMEX execution client provides the following configuration options: | Option | Default | Description | |--------------------------------|-----------|-------------| | `api_key` | `None` | Optional API key; if `None`, loaded from the environment selected by `environment`. | | `api_secret` | `None` | Optional API secret; if `None`, loaded from the environment selected by `environment`. | | `environment` | `None` | Environment enum (`MAINNET` or `TESTNET`). | | `base_url_http` | `None` | Override for the REST base URL (defaults to production). | | `base_url_ws` | `None` | Override for the WebSocket base URL (defaults to production). | | `http_timeout_secs` | `60` | Request timeout applied to HTTP calls. | | `max_retries` | `3` | Maximum retry attempts for HTTP calls. | | `retry_delay_initial_ms` | `1,000` | Initial backoff delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10,000` | Maximum backoff delay (milliseconds) between retries. | | `recv_window_ms` | `10,000` | Expiration window (milliseconds) for signed requests. See [Request authentication](#request-authentication-and-expiration). | | `max_requests_per_second` | `10` | Burst rate limit enforced by the adapter for REST calls. | | `max_requests_per_minute` | `120` | Rolling minute rate limit enforced by the adapter for REST calls. | | `deadmans_switch_timeout_secs` | `None` | Timeout in seconds for the dead man's switch. `None` disables. See [Dead man's switch](#dead-mans-switch). | | `canceller_pool_size` | `None` | Number of HTTP clients in the cancel broadcaster pool. `None` resolves to 1. See [Cancel broadcaster](#cancel-broadcaster). | | `submitter_pool_size` | `None` | Number of HTTP clients in the submit broadcaster pool. `None` resolves to 1. See [Submit broadcaster](#submit-broadcaster). | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `submitter_proxy_urls` | `None` | Optional list of proxy URLs for submit broadcaster path diversity. | | `canceller_proxy_urls` | `None` | Optional list of proxy URLs for cancel broadcaster path diversity. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Configuration examples A typical BitMEX configuration for live trading includes both testnet and mainnet options: ```python from nautilus_trader.adapters.bitmex.config import BitmexDataClientConfig from nautilus_trader.adapters.bitmex.config import BitmexExecClientConfig from nautilus_trader.core.nautilus_pyo3 import BitmexEnvironment # Using environment variables (recommended) testnet_data_config = BitmexDataClientConfig( environment=BitmexEnvironment.TESTNET, ) # Using explicit credentials mainnet_data_config = BitmexDataClientConfig( api_key="YOUR_API_KEY", # Or use os.getenv("BITMEX_API_KEY") api_secret="YOUR_API_SECRET", # Or use os.getenv("BITMEX_API_SECRET") environment=BitmexEnvironment.MAINNET, ) mainnet_exec_config = BitmexExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", environment=BitmexEnvironment.MAINNET, ) ``` ## Trading considerations ### Contingent orders The BitMEX execution adapter maps Nautilus contingent order lists to the exchange's native `clOrdLinkID`/`contingencyType` mechanics. When the engine submits `ContingencyType::Oco` or `ContingencyType::Oto` orders, the adapter: - Creates/maintains the linked order group on BitMEX so child stops and targets inherit the parent order status. - Propagates order list updates and cancellations so that contingent peers stay aligned with the current position state. - Surfaces execution reports with the appropriate contingency metadata, enabling strategy-level tracking without additional manual wiring. The adapter does not map Nautilus `ContingencyType::Ouo` to BitMEX. For bracket flows with an entry, stop, and take-profit, BitMEX can natively link the OTO entry-to-contingent activation step, but the mutual cancel or update behavior between the stop-loss and take-profit legs requires strategy-level emulation. When defining strategies, continue to use Nautilus `OrderList`/`ContingencyType` abstractions, but do not rely on the adapter to provide OUO pairing for the contingent exit legs. ### Contract specifications - **Inverse contracts**: Settled in cryptocurrency (e.g., XBTUSD settled in XBT). - **Linear contracts**: Settled in stablecoin (e.g., ETHUSDT settled in USDT). - **Contract size**: Varies by instrument, check specifications carefully. - **Tick size**: Minimum price increment varies by contract. ### Margin requirements - Initial margin requirements vary by contract and market conditions. - Maintenance margin is typically lower than initial margin. - Liquidation occurs when maintenance margin requirement is not satisfied. - BitMEX supports both isolated margin and cross margin modes. - Risk limits can be adjusted based on position size per the [Exchange Rules](https://www.bitmex.com/exchange-rules). ### Fees - **Maker fees**: Typically negative (rebate) for providing liquidity. - **Taker fees**: Positive fee for taking liquidity. - **Funding rates**: Apply to perpetual contracts every 8 hours. - **Prediction market fees**: Maker 0.00%, Taker 0.25% (no leverage allowed). ## Contributing :::info For additional features or to contribute to the BitMEX adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Blockchain Source: https://nautilustrader.io/docs/latest/integrations/blockchain/ ## Overview The blockchain adapter ingests DeFi data from EVM chains and exposes it through the NautilusTrader data model. It combines three services: - HyperSync for high-throughput historical blocks and contract logs. - HTTP RPC for contract calls, Multicall reads, and final on-chain state hydration. - Postgres for optional durable cache state, pool metadata, decoded events, and snapshots. HyperSync and RPC serve different roles. HyperSync is the fast event source. HTTP RPC remains the source of truth for current contract state, including Uniswap V3 slot state, active ticks, and positions. ## Core primitives The DeFi domain model lives in `nautilus_model::defi`. ### Chain `Chain` defines the target blockchain and its default service endpoints. | Field | Type | Description | |-----------------------------|--------------|--------------------------------------------------------------------| | `name` | `Blockchain` | Chain enum value, such as `Ethereum` or `Arbitrum`. | | `chain_id` | `u32` | EVM chain ID, such as `1` for Ethereum. | | `hypersync_url` | `String` | HyperSync endpoint, by default `https://{chain_id}.hypersync.xyz`. | | `rpc_url` | `Option` | Optional direct RPC endpoint stored on the chain model. | | `native_currency_decimals` | `u8` | Native gas token decimal precision, usually `18`. | Chains can be loaded by numeric ID with `Chain::from_chain_id` or by name with `Chain::from_chain_name`. | Chain family | Code | Name | Decimals | |-----------------------------|------|--------------|----------| | Ethereum and L2s | ETH | Ethereum | 18 | | Polygon | POL | Polygon | 18 | | Avalanche | AVAX | Avalanche | 18 | | BSC | BNB | Binance Coin | 18 | ### DEX and pools DEX integrations register factory addresses, event signatures, parser functions, and AMM type. Pool definitions bind a chain, DEX, pool contract, token pair, fee tier, tick spacing, and creation block into a stable Nautilus instrument ID. Uniswap V3 and compatible concentrated-liquidity pools also use: - `Initialize(uint160,int24)` for initial price state. - `Mint` and `Burn` events for position and tick state replay. - `Swap` events for live pool price movement. - HTTP RPC final-state reads for `slot0`, liquidity, active ticks, and position data. ## Configuration | Option | Default | Description | |-----------------------------------|--------------------|--------------------------------------------------------| | `chain` | Required | Target `Chain`, such as Ethereum or Arbitrum. | | `dex_ids` | `[]` | DEX integrations to register and sync. | | `http_rpc_url` | Required | HTTP RPC endpoint for contract reads and Multicall. | | `wss_rpc_url` | `None` | Optional WSS RPC endpoint for RPC live streams. | | `rpc_requests_per_second` | `None` | Optional RPC request throttle. | | `multicall_calls_per_rpc_request` | `200` | Requested maximum Multicall targets per RPC request. | | `use_hypersync_for_live_data` | `false` in Rust | When true, live block and event streams use HyperSync. | | `from_block` | `None` | Optional start block for historical sync. | | `pool_filters` | `DexPoolFilters()` | Pool universe filtering rules. | | `postgres_cache_database_config` | `None` | Optional Postgres cache configuration. | | `proxy_url` | `None` | Optional HTTP and WebSocket proxy URL. | | `transport_backend` | `Tungstenite` | WebSocket transport backend. | :::note Pool snapshot requests currently require a Postgres cache database. The in-memory cache can hold tokens and pools, but latest pool profiler bootstrap reads snapshot and event state through the cache database path. ::: ## Environment Set the HyperSync token and RPC URLs outside the repository. Do not commit `.env` files containing secrets. ```bash export ENVIO_API_TOKEN="" export RPC_HTTP_URL="https://your-rpc.example" export RPC_WSS_URL="wss://your-rpc.example" ``` For local `.env` usage: ```dotenv ENVIO_API_TOKEN= RPC_HTTP_URL=https://your-rpc.example RPC_WSS_URL=wss://your-rpc.example ``` `ENVIO_API_TOKEN` is required by the Rust HyperSync client. Missing or malformed tokens fail client construction before any query is sent. ### RPC endpoints `RPC_HTTP_URL` (or `--rpc-url`) must point at an EVM JSON-RPC endpoint for the target chain. It is required, not optional: the data client resolves it at construction, and a first-time pool sync reads on-chain state through it. The HyperSync endpoint is derived per chain (`https://{chain_id}.hypersync.xyz`) and needs no separate URL. Verified free public HTTP endpoints (June 2026, no API key): | Chain | HTTP endpoint | Archive | | ------------ | -------------------------------------- | ------- | | Arbitrum One | `https://arb1.arbitrum.io/rpc` | No | | Arbitrum One | `https://arbitrum.gateway.tenderly.co` | Yes | | Ethereum | `https://ethereum-rpc.publicnode.com` | No | Free archive endpoints exist (for example Tenderly above, Blast `https://arbitrum-one.public.blastapi.io`, and dRPC `https://arbitrum.drpc.org`). They are rate-limited, but snapshot validation hydrates only a handful of `eth_call`s per pool, so a free archive endpoint is enough to get `validation_state = on_chain`. Archive vs non-archive controls snapshot validation, not whether the sync runs: - On an archive node a historical-block snapshot validates against on-chain state and is stored with `validation_state = on_chain`. - On a non-archive node the historical read fails and the snapshot is kept `validation_state = replay`, which is still usable as a replay start point. - A first-time sync on a non-archive node must run to a recent `--to-block`, because the bootstrap reads on-chain state at the target block; only recent state is served. For other chains or archive access, use a directory such as [chainlist.org](https://chainlist.org) or [comparenodes.com](https://www.comparenodes.com), or a keyed provider (Infura, Alchemy, dRPC). ## Local services The development compose file starts Postgres, Redis, and pgAdmin. ```bash make start-services make init-db ``` The default Postgres service listens on `127.0.0.1:5432` with database `nautilus`, user `nautilus`, and password `pass`. Check that the schema exists: ```bash docker exec nautilus-database psql -U nautilus -d nautilus -Atc \ "select count(*) from information_schema.tables where table_schema='public'" ``` For destructive DeFi test runs, use a separate database or resettable Docker volume. Pool discovery and snapshot tests can write many rows to `token`, `pool`, `pool_*_event`, `pool_snapshot`, `pool_position`, and `pool_tick`. ## Data flow ### Architecture The adapter draws on three backends: HyperSync (Envio) for high-throughput logs and events, HTTP RPC with Multicall3 for on-chain reads, and Postgres for the durable cache. `sync-dex` discovers and registers pools once; `analyze-pool(s)` then generates `pool_snapshot` rows, each carrying a `validation_state`. ```mermaid flowchart TD HS["HyperSync (Envio): logs and events"] RPC["HTTP RPC + Multicall3: on-chain reads"] PG[("Postgres cache")] subgraph discovery["sync-dex (one-time discovery)"] direction TB D1["Stream factory PoolCreated logs"] D2["Fetch ERC-20 token metadata"] D3["Write pool and token rows"] D1 --> D2 --> D3 end subgraph analyze["analyze-pool(s) (snapshot generation, one task per pool)"] direction TB A1["sync_pool_events: Mint / Burn"] A2["Bootstrap profiler: seed from latest usable snapshot, replay events forward"] A3["extract_snapshot per --checkpoint-blocks"] A4["Persist snapshot + ticks + positions"] A5{"check_snapshot_validity"} A1 --> A2 --> A3 --> A4 --> A5 A5 -->|"matches chain"| V1["validation_state = on_chain"] A5 -->|"RPC cannot reach block, or --skip-validation"| V2["validation_state = replay"] A5 -->|"structural mismatch"| V3["validation_state = invalid"] end R["Backtest replay: load latest usable snapshot (not invalid), replay forward"] HS --> D1 RPC --> D2 D3 --> PG HS --> A1 PG --> A2 A4 --> PG RPC --> A5 PG --> R ``` `analyze-pools` runs each pool through the analyze pipeline concurrently, bounded by `--concurrency`; each pool uses its own data client and shares no state. A snapshot is usable as a replay start point unless its `validation_state` is `invalid`. ### Pool discovery Pool discovery streams DEX factory events from HyperSync, fetches ERC-20 metadata through RPC, and stores valid tokens and pools in the cache. Pools with invalid or empty token metadata can be filtered out through `DexPoolFilters`. ### Live data When `use_hypersync_for_live_data` is true, the adapter subscribes to blocks through HyperSync and then fetches matching DEX contract events for subscribed pools. When false, WSS RPC is used where a streaming implementation exists. ### Snapshot bootstrap For Uniswap V3 snapshots, bootstrap uses a two-stage process: - Replay historical Initialize, Mint, and Burn events from HyperSync to rebuild ticks and positions. - Fetch the final on-chain state through HTTP RPC and Multicall, then restore the profiler from that snapshot. If final RPC hydration fails, the adapter must fail closed. It must not emit a snapshot built from replayed events with stale price state. ### Snapshot validation Before marking a snapshot valid, the bootstrap compares the replayed profiler against the on-chain state. Structural state must match exactly: the current tick, active liquidity, per-tick net and gross liquidity, and position liquidity. A mismatch in any of these fails closed, and the snapshot is not marked valid. Three kinds of mismatch are tolerated as non-blocking and logged as a warning rather than an error: - Sqrt price, which differs when replay is event-scoped but the RPC snapshot is block-scoped. - Fee protocol, retained as a non-blocking safety net. Uniswap V3 `SetFeeProtocol` events are indexed and applied during replay, so the replayed `fee_protocol` matches the on-chain value for Uniswap V3 pools. The tolerance covers residual differences, such as an event not yet synced or a fork's non-Uniswap-V3 fee-protocol semantics. - Protocol-fee balances (`protocol_fees_token0` and `protocol_fees_token1`), which can diverge when per-step rounding during replay accrual differs from the on-chain accumulator. The on-chain snapshot reads `protocolFees()` directly, and Uniswap V3 `CollectProtocol` withdrawals are indexed and applied during replay (each withdrawal decrements the tracked balances), so the replayed balances track the on-chain ones. A non-structural-only mismatch still accepts the snapshot, matching backtest replay behavior. Because `SetFeeProtocol` is applied during both the analyze-pool bootstrap and backtest replay-forward, the accepted snapshot carries the replayed `fee_protocol` consistent with the events that produced it, so a profiler restored from it splits protocol and LP fees with that setting. ### Snapshot bootstrap guard Use `--require-existing-snapshot` when a pool analysis job should prepare a bounded replay only from the local snapshot cache. The command checks for the latest valid `pool_snapshot` at or before the target block before syncing pool events. If no usable snapshot exists, or the only match is the empty creation-block snapshot with no positions or ticks, it returns `needs_bootstrap` and skips the creation-to-target bootstrap for that pool. ```bash nautilus blockchain analyze-pools \ --chain ethereum \ --dex UniswapV3 \ --addresses-file pools.txt \ --to-block 25218797 \ --require-existing-snapshot \ --rpc-url "$RPC_HTTP_URL" ``` Both `analyze-pool` and `analyze-pools` print one JSON result per requested `--checkpoint-blocks` entry, or a single result at `--to-block` when none are given. A pool that needs a first-time bootstrap (under `--require-existing-snapshot`) has this shape: ```json { "chain": "Ethereum", "dex": "UniswapV3", "pool_address": "0x1111111111111111111111111111111111111111", "target_block": 25218797, "status": "needs_bootstrap" } ``` A successful analysis reports `validation_state`, one of `on_chain` (hydrated and matched against chain), `replay` (replay-derived, not checked, still usable as a replay start point), or `invalid` (hydrated and mismatched, not usable): ```json { "chain": "Ethereum", "dex": "UniswapV3", "pool_address": "0x1111111111111111111111111111111111111111", "target_block": 25218797, "status": "success", "snapshot_block": 25218790, "positions": 2, "ticks": 7, "validation_state": "replay", "already_valid": false, "liquidity_utilization_rate": 0.25 } ``` ### Checkpoints and concurrency `--checkpoint-blocks b1,b2,...` produces a `pool_snapshot` at each block in a single bootstrap pass (sorted, deduped, clamped to `--to-block`), instead of one run per block. `analyze-pools` analyzes pools concurrently up to `--concurrency` (default 4), each with its own data client. `--skip-validation` skips the on-chain compare and keeps snapshots `replay`. Each snapshot is keyed to the last liquidity event at or before its checkpoint, so checkpoints with no events between them resolve to the same snapshot: every requested checkpoint still prints a result line, but they share one stored row (deduped on insert). ### Backtest replay In backtest mode the adapter does not service live snapshot requests, so the pool profiler must initialize from a snapshot supplied in the replay data. `load_pool_snapshot` reads a pool snapshot from the Postgres cache, reconstructed with its full position and tick state, as of a chosen block: ```python from nautilus_trader.adapters.blockchain import load_pool_snapshot snapshot = load_pool_snapshot( pg_config=postgres_config, chain_id=chain_id, pool_address=pool_address, before_block=replay_start_block, # latest snapshot at or before this block ) ``` By default only snapshots validated against on-chain state are returned; pass `require_valid=False` to accept unvalidated snapshots. The function returns `None` when the cache holds no matching snapshot, which should be treated as a setup error rather than replayed without profiler state. Wrap the snapshot as `DefiData.PoolSnapshot(snapshot)` and pass it to `BacktestEngine.add_defi_data` alongside the events to replay. The data engine restores the profiler from the snapshot, buffering any pool events that precede it in the stream and applying them once the profiler is ready. Replay every pool event from the snapshot's block forward: a snapshot earlier than the first replayed event leaves the profiler stale. Cached block timestamps load into Nautilus data objects as UNIX nanoseconds. Cache rows written with second-resolution block timestamps are normalized to nanoseconds when snapshots and pool events are loaded, while nanosecond rows preserve their stored precision. ## Contracts ### Base contract and Multicall3 `BaseContract` batches contract calls through Multicall3 at `0xcA11bde05977b3631167028862bE2a173976CA11`. - Calls use `allow_failure: true` so individual contract call failures can be reported. - Reads execute against a single block context. - Transport and provider failures surface as RPC errors. ### ERC-20 metadata `Erc20Contract` reads `name`, `symbol`, and `decimals` through Multicall. Non-standard token contracts may return malformed strings, raw bytes, or empty fields. The adapter can skip pools with tokens that fail metadata validation. ### Uniswap V3 pools `UniswapV3PoolContract` reads global pool state, active ticks, and positions. Large pools can exceed provider limits if too many ticks or positions are packed into a single RPC call. The current safety behavior is fail-closed on hydration failure; successful delivery for very large pools depends on provider limits or future chunked/minimal hydration work. PancakeSwap V3 reuses `UniswapV3PoolContract` for on-chain hydration because its pool read functions (`slot0`, `ticks`, `positions`, `liquidity`, `feeGrowthGlobal0X128`, `feeGrowthGlobal1X128`) share the Uniswap V3 ABI. The one difference is `slot0.feeProtocol`: PancakeSwap V3 returns a `uint32` (it packs both token protocol fees), which the Uniswap V3 `uint8` binding decodes to its low byte, so the recorded fee protocol differs from the on-chain value. A fee-protocol-only mismatch is non-blocking for validation, so the structural state (tick, liquidity, ticks, positions) still validates to `on_chain`. ## Smoke tests ### HyperSync authentication ```bash curl -fsS --max-time 15 \ -H "Authorization: Bearer $ENVIO_API_TOKEN" \ https://1.hypersync.xyz/height ``` Expected result: JSON with a numeric `height`. ### Small HyperSync query ```bash query='{"from_block":25170900,"to_block":25170901,"include_all_blocks":true,"field_selection":{"block":["number","timestamp","hash"]}}' curl -sS --max-time 30 \ -H "Authorization: Bearer $ENVIO_API_TOKEN" \ -H "Content-Type: application/json" \ --data "$query" \ https://1.hypersync.xyz/query/arrow-ipc \ -o /dev/null \ -w "http_code=%{http_code} size_download=%{size_download}\n" ``` Expected result: HTTP `200` with a non-zero response size. ### Adapter compile check ```bash cargo check -p nautilus-blockchain --features hypersync ``` ### Live fail-closed regression This ignored test uses real HyperSync replay for the Ethereum WETH/USDT Uniswap V3 pool and a deliberately invalid local HTTP RPC URL. It verifies that final RPC hydration failure returns an error instead of allowing a stale snapshot through the construction path. ```bash cargo test -p nautilus-blockchain --features hypersync \ live_hypersync_bootstrap_fails_closed_when_rpc_hydration_fails \ -- --ignored --nocapture ``` Expected result: one ignored test passes. On a live network this can take several minutes. ## Operational notes - Use HyperSync for high-volume historical log scans. - Use HTTP RPC for final contract state and validation. - Use a paid or high-limit RPC provider for large Uniswap V3 pools. - Keep `ENVIO_API_TOKEN`, RPC keys, and Postgres credentials outside version control. - Use a separate Postgres database for repeatable DeFi test runs that write pool snapshots. - Treat failed final-state hydration as a hard failure for emitted snapshots. ### Pool analysis prerequisites and gotchas These surface as `analyze-pool(s)` failures with a clear cause and fix. #### Discover pools before analysis `analyze-pool(s)` reads pool metadata from the cache and fails with `Pool
is not registered` if the pool was never discovered. Run `sync-dex` for the chain/DEX once to populate the `pool` table first. #### Unsupported DEX combinations fail before sync A DEX can be registered for a chain yet lack the event parsers a command needs. `analyze-pool(s)` rejects such a DEX up front with `missing pool-event parser(s) for ...`, listing the absent families (analysis needs Initialize, Swap, Mint, Burn, and Collect parsers). `sync-dex` likewise rejects a DEX that cannot parse `PoolCreated` logs for discovery. This fails fast instead of syncing and erroring deep in profiling. The two commands need different parsers, so a DEX can support one and not the other: - `sync-dex` (discovery) needs a `PoolCreated` parser. - `analyze-pool(s)` (snapshots) need the Initialize, Swap, Mint, Burn, and Collect parsers, seeding the starting price from Initialize. - Replay-ready DEXes additionally parse `SetFeeProtocol`, so replay keeps `fee_protocol` correct. Uniswap V3 is replay-ready on Ethereum, Base, Arbitrum, and BSC. PancakeSwap V3 is snapshot-capable on the same four chains, but has no `SetFeeProtocol` parser, so it is not replay-ready. Aerodrome Slipstream is snapshot-capable on Base, but has no `PoolCreated` parser, so `sync-dex` cannot discover its pools; register an Aerodrome Slipstream pool another way before `analyze-pool(s)`. Other registered DEXes (for example Uniswap V2/V4, Camelot, Fluid) support discovery only. Polygon is a valid chain for `sync-blocks`, but has no DEX registrations, so the DEX commands reject it. `blockchain analyze-pool --help` and `blockchain sync-dex --help` print the current supported chain and DEX combinations, derived from the registered parsers. #### Use checksummed pool addresses Addresses must be EIP-55 checksummed; a lowercase address fails with `Blockchain address '
' has incorrect checksum`. Resolving a pool from `UniswapV3Factory.getPool` returns lowercase, so checksum it before passing `--address`. #### Lower the multicall batch on capped RPCs Public nodes enforce a per-call gas limit, so a large multicall returns `out of gas` and the adapter falls back to slow per-item fetches. Pass a smaller `--multicall-calls-per-rpc-request` (for example `50` on `https://arb1.arbitrum.io/rpc`) to keep batches under the cap. #### Use a recent target block on non-archive RPCs A first-time sync reads on-chain state at `--to-block`, and a non-archive node only serves recent state, so historical targets fail the on-chain read. See [RPC endpoints](#rpc-endpoints). #### HyperSync rate limits are shared per token A free Envio token caps requests per window (for example 40), and `--concurrency` makes all pools draw from that one budget at once, so high concurrency on a free token spends most of its time backing off (`rate limited by server (remaining=0/40 ...)`). Keep `--concurrency` low (or `1`) on a free token, or raise the limit with a paid plan. A full first-time sync of a large, long-lived pool needs many thousands of requests, so it is impractical on a free token regardless of concurrency. #### Pools with no liquidity events fail cleanly A pool with no processed Mint/Burn events up to the target block has no state to snapshot, so snapshot extraction returns an error instead of a snapshot. Under `analyze-pools` the pool is reported as a JSON line with `"status": "failure"` while the other pools still complete; under single-pool `analyze-pool` the command returns the error. Choose pools with liquidity activity to avoid the per-pool failure. #### Exit code reflects per-pool failures `analyze-pool(s)` exits non-zero when any pool fails, and each failed pool is also reported as a JSON line with `"status": "failure"`. Rely on the exit code for an overall pass/fail signal, and parse each result line's `status` for per-pool detail. ## Runbook: live pool-sync smoke test A reproducible end-to-end check that pool discovery, event parsing, and snapshot generation work for a DEX on a chain. The example uses PancakeSwap V3 on Arbitrum (the smallest PancakeSwap V3 deployment, about 935 pools). ### Prerequisites - `ENVIO_API_TOKEN` exported. The HyperSync client panics at construction without it. - An RPC HTTP URL for the chain (`--rpc-url` or `RPC_HTTP_URL`). Arbitrum: `https://arb1.arbitrum.io/rpc`. - Postgres up with the schema (`make start-services && make init-db`). Defaults: `127.0.0.1:5432`, database `nautilus`, user `nautilus`, password `pass`. - A built CLI: `cargo build -p nautilus-cli --features defi --bin nautilus`. The `defi` feature pulls in `nautilus-blockchain/hypersync`, which gates the `exchanges` parsers. ### Steps Discover pools first (cheap: `PoolCreated` is sparse, token metadata batches through Multicall3), then analyze specific pools: ```bash ./target/debug/nautilus blockchain sync-dex --chain arbitrum --dex PancakeSwapV3 \ --rpc-url https://arb1.arbitrum.io/rpc \ --host 127.0.0.1 --port 5432 --username nautilus --password pass --database nautilus ./target/debug/nautilus blockchain analyze-pools --chain arbitrum --dex PancakeSwapV3 \ --address --address \ --rpc-url https://arb1.arbitrum.io/rpc \ --host 127.0.0.1 --port 5432 --username nautilus --password pass --database nautilus \ --concurrency 1 ``` Verify by counting rows in these tables: - `pool_swap_event` - `pool_liquidity_event` - `pool_collect_event` - `pool_flash_event` - `pool_fee_protocol_update_event` - `pool_fee_protocol_collect_event` - `pool_snapshot` - `pool_position` - `pool_tick` The `pool_fee_protocol_update_event` and `pool_fee_protocol_collect_event` tables stay small, since `SetFeeProtocol` and `CollectProtocol` fire rarely (often zero to a handful of times per pool). ### Gotchas found running this - A free Envio token caps requests per window (for example 40). Discovery is cheap, but analyzing a high-activity pool from its creation block spends most of its time backing off (`rate limited by server (remaining=0/40 ...)`). Pick short-history or low-event pools, or bound the range (see below). Probe a pool's size with a HyperSync log query before a full run. - The development Postgres can be emptied out-of-band mid-session (the schema stays, the data goes), which surfaces later as `Pool
is not registered` because `load_pool` finds nothing. Run `sync-dex` immediately before `analyze-pool(s)`; do not assume pools persist across a session. - On-chain snapshot validation covers Uniswap V3 and PancakeSwap V3 (both share the V3 pool read ABI). Other forks with a different pool ABI (for example Algebra-based DEXes) still log `Could not validate snapshot against on-chain state ... Fetching on-chain snapshot for Dex protocol is not supported yet` and keep `validation_state = replay`. The compare reads pool state at the snapshot block, so an archive RPC is only needed when that block predates a non-archive node's retention; recent targets reach `on_chain` on a non-archive node. - `--from-block` at a mid-life block skips the creation-block `Initialize` event, so the profiler bootstrap fails with `Pool is not initialized and it doesn't contain initial price, cannot bootstrap profiler`. Sync from creation when you need a snapshot. A bounded window still parses and persists the events it covers (this is how a single `Flash` event was captured), only the snapshot step fails. - Addresses must be EIP-55 checksummed; the `pool.address` column is a custom domain, so plain text equality from an external SQL client is unreliable. Use the CLI or `count(*)` to inspect. - The capability guard fails an unregistered combination before any sync: `sync-dex` needs a `PoolCreated` parser, and `analyze-pool(s)` need `Initialize`, `Swap`, `Mint`, `Burn`, and `Collect` parsers (see [Unsupported DEX combinations fail before sync](#unsupported-dex-combinations-fail-before-sync)). ## Extending the adapter The event model targets Uniswap V3 concentrated-liquidity pools. `DexPoolData` and its structs encode V3 semantics directly: `PoolSwap` carries `sqrt_price_x96` and `tick`, `PoolLiquidityUpdate` carries `tick_lower` and `tick_upper`. The `DexType` and `AmmType` enums name other families (Uniswap V2, Uniswap V4, Curve, Balancer, Maverick), but only Uniswap V2 (pool discovery) and Uniswap V4 (`Initialize`) are wired at all. ### Adding an event or protocol family Design the taxonomy before writing a parser. Most families do not fit the V3 structs: Uniswap V2 emits `Sync`, Uniswap V4 replaces mint and burn with `ModifyLiquidity` plus `Donate`, and Curve and Balancer pools hold more than two tokens. Adding events piecemeal forces optional fields, duplicate variants, and renames. The design pass should: - Map the protocol's events and decide, per event, whether each reuses, extends, or adds a `DexPoolData` variant. - Decide whether the family needs a new taxonomy axis. Singleton or `poolId` protocols (Uniswap V4, Balancer) and multi-token pools (Curve) break the per-pool-address, token-pair assumptions. - Name events with the `_` convention, such as `fee_protocol_update`. Reserve the literal on-chain event name for signatures and error labels. Then wire each event through the full path, mirroring an existing one such as `fee_protocol_collect`: - Event struct - HyperSync and RPC parsers - `DexExtended` parser slot - `DexPoolData` and `DefiData` variants - Profiler apply method - Event table and its insert - `stream_pool_events` UNION arm and row mapper - PyO3 binding Cover it with a parser round-trip test, a profiler apply test, and the parser-parity test. Incremental sync resumes from each pool's last-synced block, so adding an event type does not backfill pools already synced past those blocks: the new event tables stay empty for historical ranges. Re-sync a pool from its creation block (a reset sync) to populate them. ### Adding a chain A new chain is registration only, provided its DEXes reuse modeled events: add the `Chain`, its RPC client, and the per-DEX registrations. A chain that brings a new family needs the design pass above. ## Current limitations - Very large Uniswap V3 pools can still hit provider payload, timeout, or rate limits during final-state Multicall hydration. - `multicall_calls_per_rpc_request` documents the intended batching limit, but some final snapshot paths still need chunking hardening. - A full successful WETH/USDT or WETH/USDC delivery test needs a real HTTP RPC provider that can serve the final-state reads, or the adapter needs minimal/chunked hydration first. - On-chain snapshot validation covers Uniswap V3 and PancakeSwap V3 (shared V3 pool read ABI). Forks with a different pool ABI sync events and produce replay snapshots, but cannot reach `validation_state = on_chain` until the final-state hydration covers their pool contracts. # Bybit Source: https://nautilustrader.io/docs/latest/integrations/bybit/ Founded in 2018, Bybit is one of the largest cryptocurrency exchanges in terms of daily trading volume and open interest of crypto assets and crypto derivative products. This integration supports live market data ingest and order execution with Bybit. ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/bybit/). ## Overview This guide assumes a trader is setting up for both live market data feeds, and trade execution. The Bybit adapter includes multiple components, which can be used together or separately depending on the use case. - `BybitHttpClient`: Low-level HTTP API connectivity. - `BybitWebSocketClient`: Low-level WebSocket API connectivity. - `BybitInstrumentProvider`: Instrument parsing and loading functionality. - `BybitDataClient`: A market data feed manager. - `BybitExecutionClient`: An account management and trade execution gateway. - `BybitLiveDataClientFactory`: Factory for Bybit data clients (used by the trading node builder). - `BybitLiveExecClientFactory`: Factory for Bybit execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to necessarily work with these lower level components directly. ::: ## Bybit documentation Bybit provides extensive documentation for users which can be found in the [Bybit help center](https://www.bybit.com/en/help-center). It’s recommended you also refer to the Bybit documentation in conjunction with this NautilusTrader integration guide. ## Products A product is an umbrella term for a group of related instrument types. :::note Product is also referred to as `category` in the Bybit v5 API. ::: The following product types are supported on Bybit: | Product Type | Supported | Notes | |-----------------------------|-----------|------------------------------------------| | Spot cryptocurrencies | ✓ | Native spot markets with margin support. | | Linear perpetual contracts | ✓ | USDT/USDC margined perpetual swaps. | | Linear futures contracts | ✓ | Delivery‑settled linear futures. | | Inverse perpetual contracts | ✓ | Coin‑margined perpetual swaps. | | Inverse futures contracts | ✓ | Coin‑margined delivery futures. | | Option contracts | ✓ | USDT‑settled European options. | ## Symbology To distinguish between different product types on Bybit, Nautilus uses specific product category suffixes for symbols: - `-SPOT`: Spot cryptocurrencies - `-LINEAR`: Perpetual and futures contracts - `-INVERSE`: Inverse perpetual and inverse futures contracts - `-OPTION`: Option contracts These suffixes must be appended to the Bybit raw symbol string to identify the specific product type for the instrument ID. For example: - The Ether/Tether spot currency pair is identified with `-SPOT`, such as `ETHUSDT-SPOT`. - The BTCUSDT perpetual futures contract is identified with `-LINEAR`, such as `BTCUSDT-LINEAR`. - The BTCUSD inverse perpetual futures contract is identified with `-INVERSE`, such as `BTCUSD-INVERSE`. - A BTC USDT-settled put option: `BTC-27MAR26-70000-P-USDT-OPTION`. - A ETH USDC-settled call option: `ETH-28FEB25-2800-C-OPTION`. Bybit's option symbols include the settlement currency for USDT-settled contracts (e.g. `BTC-27MAR26-70000-P-USDT`) but omit it for USDC-settled contracts (e.g. `ETH-28FEB25-2800-C`). The adapter appends `-OPTION` to whatever symbol the API returns. ## Instrument loading Bybit data and execution clients use the common `instrument_provider` config. Configure it to load instruments before strategies subscribe to market data or submit orders. Subscriptions do not request missing instrument definitions. ```python from nautilus_trader.adapters.bybit import BybitProductType from nautilus_trader.adapters.bybit.config import BybitDataClientConfig from nautilus_trader.config import InstrumentProviderConfig BybitDataClientConfig( instrument_provider=InstrumentProviderConfig(load_all=True), product_types=(BybitProductType.SPOT,), ) ``` Use `load_ids` when you only need a known set of instruments: ```python from nautilus_trader.adapters.bybit import BybitProductType from nautilus_trader.adapters.bybit.config import BybitDataClientConfig from nautilus_trader.config import InstrumentProviderConfig from nautilus_trader.model.identifiers import InstrumentId BybitDataClientConfig( instrument_provider=InstrumentProviderConfig( load_ids=frozenset([InstrumentId.from_str("BTCUSDT-SPOT.BYBIT")]), ), product_types=(BybitProductType.SPOT,), ) ``` The configured `product_types` must include the product suffix in each instrument ID. ## Environments Bybit provides three trading environments. Configure the appropriate environment with the `environment` enum on your client configuration. | Environment | Config | Description | |--------------|---------------------------------------|------------------------------------------------------------------| | **Mainnet** | `BybitEnvironment.MAINNET` | Production trading with real funds. | | **Demo** | `BybitEnvironment.DEMO` | Practice trading with simulated funds on mainnet infrastructure. | | **Testnet** | `BybitEnvironment.TESTNET` | Separate test network for development and integration testing. | ### Mainnet (Production) The default environment for live trading with real funds. ```python from nautilus_trader.adapters.bybit import BybitEnvironment config = BybitExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", environment=BybitEnvironment.MAINNET, ) ``` Environment variables: `BYBIT_API_KEY`, `BYBIT_API_SECRET` ### Demo trading Demo trading uses Bybit's mainnet infrastructure with simulated funds. Create demo API keys from the [Bybit demo trading page](https://www.bybit.com/en/demo-trading). ```python from nautilus_trader.adapters.bybit import BybitEnvironment config = BybitExecClientConfig( api_key="YOUR_DEMO_API_KEY", api_secret="YOUR_DEMO_API_SECRET", environment=BybitEnvironment.DEMO, ) ``` Environment variables: `BYBIT_DEMO_API_KEY`, `BYBIT_DEMO_API_SECRET` :::warning **Demo environment limitations:** - The WebSocket Trade API is **not supported** for demo trading. NautilusTrader automatically uses the HTTP REST API for order operations in demo mode. - Native TP/SL and option params (`order_iv`, `mmp`) on new orders work in demo via the HTTP create-order endpoint. - The custom TP/SL trigger prices `tp_trigger_price` and `sl_trigger_price` are not supported in demo (orders setting them are denied); the create-order endpoint cannot carry them. - Demo private streams use `wss://stream-demo.bybit.com`, but public market data uses Bybit's mainnet public stream `wss://stream.bybit.com`. ::: ### Testnet A separate test network for development and integration testing. ```python from nautilus_trader.adapters.bybit import BybitEnvironment config = BybitExecClientConfig( api_key="YOUR_TESTNET_API_KEY", api_secret="YOUR_TESTNET_API_SECRET", environment=BybitEnvironment.TESTNET, ) ``` Environment variables: `BYBIT_TESTNET_API_KEY`, `BYBIT_TESTNET_API_SECRET` :::note Testnet supports all trading features including the WebSocket Trade API. It uses completely separate infrastructure from mainnet, so market data and liquidity differ significantly from production. ::: When `environment=BybitEnvironment.TESTNET`, the adapter resolves Bybit's documented testnet endpoints automatically: - REST API: `https://api-testnet.bybit.com` - Public WebSocket: `wss://stream-testnet.bybit.com/v5/public/{spot|linear|inverse|option}` - Private WebSocket: `wss://stream-testnet.bybit.com/v5/private` - Trade WebSocket: `wss://stream-testnet.bybit.com/v5/trade` ### Testnet setup To set up a Bybit testnet account and credentials: 1. Open [testnet.bybit.com](https://testnet.bybit.com) in a desktop browser. 2. Create a separate testnet account or sign in to your existing testnet account. 3. Request test coins from **Assets -> Assets Overview -> Request Test Coins** so the account has balances for testing. 4. Open **API Management** at [testnet.bybit.com/app/user/api-management](https://testnet.bybit.com/app/user/api-management). 5. Click **Create New Key**. 6. Select the required permissions for your use case. 7. Complete the 2FA prompt and copy the API key and secret. 8. Export the credentials in your shell: ```bash export BYBIT_TESTNET_API_KEY="YOUR_TESTNET_API_KEY" export BYBIT_TESTNET_API_SECRET="YOUR_TESTNET_API_SECRET" ``` Bybit's current testnet guidance also notes: - API keys are created on the website, not in the mobile app. - New users may be unable to create API keys for the first 48 hours after registration. - Testnet is separate from mainnet. Do not deposit real funds into a testnet account. - Bybit currently documents testnet account setup through a desktop browser. ## Orders capability Bybit offers a flexible combination of trigger types, enabling a broader range of Nautilus orders. All the order types listed below can be used as *either* entries or exits, except for trailing stops (which use a position-related API). ### Order types | Order Type | Spot | Linear | Inverse | Option | Notes | |------------------------|------|--------|---------|--------|-----------------------------------| | `MARKET` | ✓ | ✓ | ✓ | ✓ | Supports quote quantity. | | `LIMIT` | ✓ | ✓ | ✓ | ✓ | | | `STOP_MARKET` | ✓ | ✓ | ✓ | - | *Not supported for Options*. | | `STOP_LIMIT` | ✓ | ✓ | ✓ | - | *Not supported for Options*. | | `MARKET_IF_TOUCHED` | ✓ | ✓ | ✓ | - | *Not supported for Options*. | | `LIMIT_IF_TOUCHED` | ✓ | ✓ | ✓ | - | *Not supported for Options*. | | `TRAILING_STOP_MARKET` | - | ✓ | ✓ | - | *Not supported for Spot/Options*. | ### Execution instructions | Instruction | Spot | Linear | Inverse | Option | Notes | |---------------|------|--------|---------|--------|-----------------------------------| | `post_only` | ✓ | ✓ | ✓ | ✓ | Only supported on `LIMIT` orders. | | `reduce_only` | - | ✓ | ✓ | ✓ | *Not supported for Spot*. | ### Time in force | Time in force | Spot | Linear | Inverse | Option | Notes | |---------------|------|--------|---------|--------|------------------------------| | `GTC` | ✓ | ✓ | ✓ | ✓ | Good Till Canceled. | | `GTD` | - | - | - | - | *Not supported*. | | `FOK` | ✓ | ✓ | ✓ | ✓ | Fill or Kill. | | `IOC` | ✓ | ✓ | ✓ | ✓ | Immediate or Cancel. | ### Advanced order features | Feature | Spot | Linear | Inverse | Option | Notes | |--------------------|------|--------|---------|--------|----------------------------------------| | Order Modification | ✓ | ✓ | ✓ | ✓ | Price and quantity modification. | | Bracket/OCO Orders | ✓ | ✓ | ✓ | - | UI only; API users implement manually. | | Iceberg Orders | ✓ | ✓ | ✓ | - | Max 10 per account, 1 per symbol. | ### Batch operations | Operation | Spot | Linear | Inverse | Option | Notes | |--------------------|------|--------|---------|--------|-------------------------------------------| | Batch Submit | ✓ | ✓ | ✓ | ✓ | Submit multiple orders in single request. | | Batch Modify | ✓ | ✓ | ✓ | ✓ | Modify multiple orders in single request. | | Batch Cancel | ✓ | ✓ | ✓ | ✓ | Cancel multiple orders in single request. | ### Position management | Feature | Spot | Linear | Inverse | Option | Notes | |---------------------|------|--------|---------|--------|------------------------------------------| | Query positions | - | ✓ | ✓ | ✓ | Real‑time position updates. | | Position mode | - | ✓ | ✓ | - | One‑Way only for Options. | | Leverage control | - | ✓ | ✓ | - | Not applicable for Options. | | Margin mode | - | ✓ | ✓ | ✓ | Cross, Isolated, or Portfolio Margin. | #### Hedge mode (BothSides) Bybit only accepts `BOTH_SIDES` on USDT linear perpetuals. For other product types configure `MERGED_SINGLE` or omit them from `position_mode`. Configure per symbol: ```python from nautilus_trader.adapters.bybit import BybitPositionMode config = BybitExecClientConfig( ..., position_mode={"ETHUSDT-LINEAR": BybitPositionMode.BOTH_SIDES}, ) ``` On connect the adapter calls `/v5/position/switch-mode` for each entry, then derives `positionIdx` for every order: opening BUY -> `1` (long), opening SELL -> `2` (short), reduce-only SELL -> `1`, reduce-only BUY -> `2`. Bybit documents this in the V5 [switch position mode](https://bybit-exchange.github.io/docs/v5/position/position-mode) and [place order](https://bybit-exchange.github.io/docs/v5/order/create-order#request-parameters) APIs: `mode=3` enables Both Sides, and hedge-mode orders require `positionIdx`. Orders and reports with `positionIdx=0` (one-way / Merged Single mode) carry no venue position ID. For hedge-mode indexes `1` and `2`, the adapter maps reports to venue position IDs ending in `-LONG` and `-SHORT`, and carries the same ID onto fills when Bybit execution messages do not include `positionIdx`. To override, pass `position_idx` via `params`: ```python params={"position_idx": 1} # 0 one-way, 1 long, 2 short ``` ### Risk events | Feature | Spot | Linear | Inverse | Option | Notes | |---------------------------|------|--------|---------|--------|-------------------------------------------| | Liquidation handling | - | ✓ | ✓ | ✓ | Takeover fills flagged as exchange‑generated. | | ADL handling | - | ✓ | ✓ | ✓ | Auto‑deleveraging fills flagged and logged. | | ADL rank warnings | - | ✓ | ✓ | ✓ | Position reports logged when `adlRankIndicator >= 4`. | Bybit emits venue-initiated fills with `execType` set to: - `AdlTrade`: Auto-deleveraging execution. An opposing profitable position was selected to close the undercollateralised counterparty after the insurance fund could not cover the loss. - `BustTrade`: Liquidation takeover. The liquidation engine seized the position after margin was exhausted. - `Delivery`: USDC futures delivery. - `Settle`: Inverse futures settlement. The adapter flags each as exchange-generated and logs a warning containing the execution ID, symbol, side, quantity, and price. Fills flow through the normal `FillReport` path; because these orders carry an empty `orderLinkId`, the execution engine treats them as external and assigns them via `external_order_claims` (or the `EXTERNAL` strategy by default). Bybit also publishes an ADL ranking on position updates via the `adlRankIndicator` field. The range is 0 (flat / no position) to 5 (next to deleverage). The adapter logs a warning whenever an open position carries a rank of 4 or higher so you can react before the venue force-closes. Upstream references: - [V5 `execType` values](https://bybit-exchange.github.io/docs/v5/enum#exectype) - [V5 `createType` values](https://bybit-exchange.github.io/docs/v5/enum#createtype) - [Liquidation mechanism](https://www.bybit.com/en/help-center/article/Liquidation-Process-Derivatives-Trading) - [Auto-Deleveraging mechanism](https://www.bybit.com/en/help-center/article/Auto-Deleveraging-ADL-Derivatives-Trading) ### Order querying | Feature | Spot | Linear | Inverse | Option | Notes | |---------------------|------|--------|---------|--------|-----------------------------------------| | Query open orders | ✓ | ✓ | ✓ | ✓ | List all active orders. | | Query order history | ✓ | ✓ | ✓ | ✓ | Historical order data. | | Order status updates| ✓ | ✓ | ✓ | ✓ | Real‑time order state changes. | | Trade history | ✓ | ✓ | ✓ | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Spot | Linear | Inverse | Option | Notes | |---------------------|------|--------|---------|--------|-----------------------------------------| | Order lists | ✓ | ✓ | ✓ | ✓ | Submitted as a batch via WebSocket. | | OCO orders | ✓ | ✓ | ✓ | - | UI only; API users implement manually. | | Bracket orders | ✓ | ✓ | ✓ | - | UI only; API users implement manually. | | Conditional orders | ✓ | ✓ | ✓ | - | Stop and limit‑if‑touched orders. | ### Order parameters Individual orders can be customized using the `params` dictionary when submitting orders: | Parameter | Type | Description | |--------------------|------------------------|-------------------------------------------------------------------------| | `is_leverage` | `bool` | Spot only. Enables margin trading (borrowing). Default: `False`. | | `take_profit` | `str` or `float` | TP trigger price. Attaches a native TP to the order. | | `stop_loss` | `str` or `float` | SL trigger price. Attaches a native SL to the order. | | `tp_trigger_by` | `str` | TP trigger type: `"LastPrice"`, `"IndexPrice"`, or `"MarkPrice"`. | | `sl_trigger_by` | `str` | SL trigger type: `"LastPrice"`, `"IndexPrice"`, or `"MarkPrice"`. | | `tp_order_type` | `str` | TP execution type: `"Market"` or `"Limit"`. Default: `"Market"`. | | `sl_order_type` | `str` | SL execution type: `"Market"` or `"Limit"`. Default: `"Market"`. | | `tp_limit_price` | `str` or `float` | Limit price for TP when `tp_order_type` is `"Limit"`. | | `sl_limit_price` | `str` or `float` | Limit price for SL when `sl_order_type` is `"Limit"`. | | `tp_trigger_price` | `str` or `float` | Custom TP trigger price (overrides `take_profit`). | | `sl_trigger_price` | `str` or `float` | Custom SL trigger price (overrides `stop_loss`). | | `close_on_trigger` | `bool` | Close the position when TP/SL triggers. Default: `False`. | | `position_idx` | `int` | Hedge‑mode position index. See [Hedge mode](#hedge-mode-bothsides). | | `bbo_side_type` | `str` | Linear/inverse BBO side: `"Queue"` or `"Counterparty"`. | | `bbo_level` | `str` or `int` | Linear/inverse BBO book level: `"1"` through `"5"`. | :::note On demo, native TP/SL params route through the HTTP create-order endpoint, with one exception: the custom trigger prices `tp_trigger_price` and `sl_trigger_price` are not supported because that endpoint cannot carry them, and orders that set either are denied. The `is_leverage` param applies to Spot products only. See [Bybit's isLeverage documentation](https://bybit-exchange.github.io/docs/v5/order/create-order#request-parameters). ::: When `bbo_side_type` and `bbo_level` are set, Nautilus sends Bybit's `bboSideType` and `bboLevel` fields and omits the order price from the API request. BBO orders are supported for linear and inverse limit, stop-limit, and limit-if-touched orders. #### Example: Order with native TP/SL ```python order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTCUSDT-LINEAR.BYBIT"), order_side=OrderSide.BUY, quantity=Quantity.from_str("0.01"), price=Price.from_str("60000.0"), params={ "take_profit": "65000.0", "stop_loss": "58000.0", "tp_trigger_by": "LastPrice", "sl_trigger_by": "LastPrice", }, ) strategy.submit_order(order) ``` #### Example: BBO order ```python order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTCUSDT-LINEAR.BYBIT"), order_side=OrderSide.BUY, quantity=Quantity.from_str("0.01"), price=Price.from_str("60000.0"), params={"bbo_side_type": "Queue", "bbo_level": 1}, ) strategy.submit_order(order) ``` #### Example: Spot margin trading ```python # Submit a Spot order with margin enabled order = strategy.order_factory.market( instrument_id=InstrumentId.from_str("BTCUSDT-SPOT.BYBIT"), order_side=OrderSide.BUY, quantity=Quantity.from_str("0.1"), params={"is_leverage": True} # Enable margin for this order ) strategy.submit_order(order) ``` :::note Without `is_leverage=True` in the params, Spot orders use your available balance and do not borrow funds, even if you have auto-borrow enabled on your Bybit account. ::: For a complete example of using order parameters including `is_leverage`, see the [bybit_exec_tester.py](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/bybit/bybit_exec_tester.py) example. ## Spot margin borrowing and repayment NautilusTrader provides automated spot margin borrow repayment functionality to prevent interest accrual after closing short positions on Bybit. ### Background When trading Spot with margin enabled (`is_leverage=True`), Bybit automatically borrows coins when you execute short positions. However, after you close the short position (BUY order fills), the borrowed coins are **NOT automatically repaid** - they continue accruing hourly interest charges until manually repaid. This can result in significant interest costs if left unattended. ### Automatic repayment (recommended) NautilusTrader automatically repays spot margin borrows immediately after BUY orders fill on Spot instruments. This feature is **enabled by default** via the `auto_repay_spot_borrows` configuration flag. **How it works:** 1. When a Spot BUY order fills, the execution client automatically attempts to repay any outstanding borrows for that coin. 2. The repayment uses Bybit's `no-convert-repay` endpoint, which repays the full outstanding borrow amount. 3. If the repayment fails (e.g., API error), it logs the error but does not crash the execution client. 4. Repayments are automatically skipped during Bybit's UTC blackout window (see below). **Example:** ```python from nautilus_trader.adapters.bybit import BybitExecClientConfig config = BybitExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", product_types=[BybitProductType.SPOT], auto_repay_spot_borrows=True, # Default is True ) ``` ### Manual margin operations Strategies can control margin borrowing and repayment directly via `query_account` with the `BybitMarginAction` enum: | Action | Description | |---------------------------------------|----------------------------------| | `BybitMarginAction.BORROW` | Borrow funds for margin trading. | | `BybitMarginAction.REPAY` | Repay borrowed funds. | | `BybitMarginAction.GET_BORROW_AMOUNT` | Query current borrowed amount. | #### Borrow ```python self.query_account( account_id=self.account_id, params={"action": BybitMarginAction.BORROW, "coin": "USDT", "amount": 1000}, ) ``` #### Repay ```python # Repay specific amount self.query_account( account_id=self.account_id, params={"action": BybitMarginAction.REPAY, "coin": "USDT", "amount": 500}, ) # Repay all (omit amount) self.query_account( account_id=self.account_id, params={"action": BybitMarginAction.REPAY, "coin": "USDT"}, ) ``` #### Query borrow amount ```python self.query_account( account_id=self.account_id, params={"action": BybitMarginAction.GET_BORROW_AMOUNT, "coin": "USDT"}, ) ``` :::note The `account_id` can be obtained from `self.portfolio.account(BYBIT_VENUE).id` or stored during strategy initialization via the config. ::: #### Receiving results Results are published as custom data on the message bus. Subscribe in your strategy to receive them: ```python from nautilus_trader.adapters.bybit import BybitMarginAction from nautilus_trader.adapters.bybit import BybitMarginBorrowResult from nautilus_trader.adapters.bybit import BybitMarginRepayResult from nautilus_trader.adapters.bybit import BybitMarginStatusResult from nautilus_trader.model.data import DataType class MyStrategy(Strategy): def on_start(self): self.subscribe_data(DataType(BybitMarginBorrowResult)) self.subscribe_data(DataType(BybitMarginRepayResult)) self.subscribe_data(DataType(BybitMarginStatusResult)) def on_data(self, data): if isinstance(data, BybitMarginBorrowResult): if data.success: self.log.info(f"Borrowed {data.amount} {data.coin}") else: self.log.error(f"Borrow failed: {data.message}") elif isinstance(data, BybitMarginRepayResult): if data.success: self.log.info(f"Repaid {data.amount or 'all'} {data.coin}") else: self.log.error(f"Repay failed: {data.message}") elif isinstance(data, BybitMarginStatusResult): self.log.info(f"Borrow amount for {data.coin}: {data.borrow_amount}") ``` ### UTC blackout window Bybit blocks `no-convert-repay` operations daily during **04:00-05:30 UTC** for interest calculation processing. NautilusTrader automatically detects this window and skips repayment attempts, logging a warning instead. During the blackout window, any BUY order fills will trigger a warning like: ``` Skipping borrow repayment for BTC due to Bybit blackout window (04:00-05:30 UTC daily). Will need manual repayment. ``` **Important:** If your BUY orders fill during the blackout window, you'll need to manually repay the borrows after 05:30 UTC to stop interest accrual, or wait for the next BUY order fill outside the blackout window. ### Configuration options | Option | Type | Default | Description | |---------------------------|--------|---------|-----------------------------------------------------------------------------| | `auto_repay_spot_borrows` | `bool` | `True` | If `True`, automatically repay spot margin borrows after BUY orders fill. Prevents interest accrual on borrowed coins. Repayment is skipped during blackout window. | ### Important notes - Auto-repayment only triggers on **Spot BUY orders**, not derivatives. - Repayment uses the `no-convert-repay` endpoint which repays the full outstanding borrow by default. - The feature gracefully handles API errors and logs failures without crashing. - Manual borrowing is still required before opening short positions unless auto-borrow is enabled on your Bybit account. ### Spot trading limitations The following limitations apply to Spot products, as positions are not tracked on the venue side: - `reduce_only` orders are *not supported*. - Trailing stop orders are *not supported*. ### Options trading Bybit lists European-style options on BTC and ETH, settled in USDT or USDC. The adapter uses the `CryptoOption` instrument type and the `-OPTION` symbol suffix. See the [symbology section](#symbology) for the full symbol format. #### Options data The adapter supports real-time options market data through the WebSocket ticker channel: | Data type | Description | |----------------------------|----------------------------------------------------------| | Quotes (bid/ask) | Top‑of‑book prices and sizes for each option contract. | | Greeks | Delta, gamma, vega, theta, plus bid/ask/mark IV. | | Mark price | Exchange mark price for each option contract. | | Index price | Underlying index price. | | Underlying (forward) price | Per‑expiry forward price, used for ATM determination. | | Open interest | Per‑contract open interest. | | Order book deltas | L2 MBP updates from the option orderbook stream. | Subscribe to per-instrument Greeks or aggregate them into option chain snapshots with ATM-relative strike filtering. See the [options concept guide](../concepts/options.md) for subscription patterns and the [options data tutorial](../tutorials/options_data_bybit.md) for a step-by-step walkthrough. NautilusTrader builds the option chain view locally from Bybit's per-contract option market data. Bar (kline) data is not available for options. Bybit does not provide kline streams for this product type. #### Options order parameters In addition to the standard order parameters, option orders accept: | Parameter | Type | Description | |------------|------------------|----------------------------------------------------------| | `order_iv` | `str` or `float` | Place or amend the order by implied volatility instead of price. | | `mmp` | `bool` | Enable Market Maker Protection for the order. | These parameters are passed through `params` on `SubmitOrder`. On mainnet they flow through the WebSocket trade channel; on demo they route through the HTTP create-order endpoint. Amending an existing order by `order_iv` is not supported in demo mode. #### Options trading limitations - Amending an order by implied volatility (`order_iv`) and other WS-trade-only features are not supported in demo mode. - Leverage is not configurable. Option buyers pay premium; sellers post margin. - Position mode is one-way only. Hedge mode is not supported. - Conditional order types (`STOP_MARKET`, `STOP_LIMIT`, `MARKET_IF_TOUCHED`, `LIMIT_IF_TOUCHED`) are not supported. - Trading stops (TP/SL on positions) are not supported. - Funding rates do not apply to options. - Options require a Unified Trading Account (UTA). ### Trailing stops Trailing stops on Bybit do not have a client order ID on the venue side (though there is a `venue_order_id`). This is because trailing stops are associated with a netted position for an instrument. Consider the following points when using trailing stops on Bybit: - `reduce_only` instruction is available - When the position associated with a trailing stop is closed, the trailing stop is automatically "deactivated" (closed) on the venue side. - You cannot query trailing stop orders that are not already open (the `venue_order_id` is unknown until then). - You can manually adjust the trigger price in the GUI, which will update the Nautilus order. ## Funding rates The adapter receives funding rate data from the [Linear Ticker](https://bybit-exchange.github.io/docs/v5/websocket/public/ticker#linear-inverse-perpetual-response) WebSocket stream. Bybit provides the `fundingIntervalHour` field in ticker updates, which the adapter uses to populate the `interval` field on `FundingRateUpdate`. The adapter caches the last known `fundingIntervalHour` per symbol so that partial ticker updates (which may omit the field) still carry the correct interval. For historical funding rate requests, the adapter computes the interval from consecutive funding timestamps. ## Rate limiting Every HTTP call consumes the global token bucket as well as any keyed quota(s). When usage exceeds a bucket, requests are queued automatically, so manual throttling is rarely required. | Key / Endpoint | Limit (requests/sec) | Notes | |---------------------------|----------------------|----------------------------------------------------| | `bybit:global` | 120 | Exchange‑wide 600 req / 5 s ceiling. | | `/v5/market/kline` | 20 | Historical sweeps throttled slightly below global. | | `/v5/market/trades` | 24 | Matches the global quota. | | `/v5/order/create` | 10 | Standard order placement. | | `/v5/order/cancel` | 10 | Single‑order cancellation. | | `/v5/order/create-batch` | 5 | Batch placement endpoints. | | `/v5/order/cancel-batch` | 5 | Batch cancellation endpoints. | | `/v5/order/cancel-all` | 2 | Full book cancel to mirror Bybit guidance. | :::warning Bybit responds with error code `10016` when the rate limit is exceeded and may temporarily block the IP if requests continue without back-off. ::: :::info For more details on rate limiting, see the official documentation: . ::: ### Data clients If no product types are specified then all product types will be loaded and available. ### Execution clients The adapter automatically determines the account type based on configured product types: - **Spot only**: Uses `CASH` account type with borrowing support enabled - **Derivatives or mixed products**: Uses `MARGIN` account type (UTA - Unified Trading Account) This allows you to trade Spot alongside derivatives in a single Unified Trading Account, which is the standard account type for most Bybit users. :::info **Unified Trading Accounts (UTA) and Spot margin trading** Most Bybit users now have Unified Trading Accounts (UTA) as Bybit steers new users to this account type. Classic accounts are considered legacy. For Spot margin trading on UTA accounts: - Borrowing is **NOT automatically enabled** - it requires explicit API configuration - To use Spot margin via API, you must submit orders with `is_leverage=True` in the parameters (see [Bybit docs](https://bybit-exchange.github.io/docs/v5/order/create-order#request-parameters)) - If auto-borrow/auto-repay is enabled on your Bybit account, the venue will automatically borrow/repay funds for those margin orders - Without auto-borrow enabled, you'll need to manually manage borrowing through Bybit's interface **Important**: The Nautilus Bybit adapter defaults to `is_leverage=False` for Spot orders, meaning they won't use margin unless you explicitly enable it. ::: ## Fee currency logic Understanding how Bybit determines the currency for trading fees is important for accurate accounting and position tracking. The fee currency rules vary between Spot and derivatives products. ### Spot trading fees For Spot trading, the fee currency depends on the order side and whether the fee is a rebate (negative fee for maker orders): #### Normal fees (positive) - **BUY orders**: Fee is charged in the **base currency** (e.g., BTC for BTCUSDT) - **SELL orders**: Fee is charged in the **quote currency** (e.g., USDT for BTCUSDT) #### Maker rebates (negative fees) When maker fees are negative (rebates), the currency logic is **inverted**: - **BUY orders with maker rebate**: Rebate is paid in the **quote currency** (e.g., USDT for BTCUSDT) - **SELL orders with maker rebate**: Rebate is paid in the **base currency** (e.g., BTC for BTCUSDT) :::note **Taker orders never have inverted logic**, even if the maker fee rate is negative. Taker fees always follow the normal fee currency rules. ::: #### Example: BTCUSDT Spot - **Buy 1 BTC as taker (0.1% fee)**: Pay 0.001 BTC in fees - **Sell 1 BTC as taker (0.1% fee)**: Pay equivalent USDT in fees - **Buy 1 BTC as maker (-0.01% rebate)**: Receive USDT rebate (inverted) - **Sell 1 BTC as maker (-0.01% rebate)**: Receive BTC rebate (inverted) ### Derivatives trading fees For all derivatives products (LINEAR, INVERSE, OPTION), fees are always charged in the **settlement currency**: | Product Type | Settlement Currency | Fee Currency | |--------------|---------------------------------------|--------------| | LINEAR | USDT (typically) | USDT | | INVERSE | Base coin (e.g., BTC for BTCUSD) | Base coin | | OPTION | USDT | USDT | ### Fee calculation When the WebSocket execution message doesn't provide the exact fee amount (`execFee`), the adapter calculates fees as follows: #### Spot products - **BUY orders**: `fee = base_quantity × fee_rate` - **SELL orders**: `fee = notional_value × fee_rate` (where `notional_value = quantity × price`) #### Derivatives - All derivatives: `fee = notional_value × fee_rate` ### Official documentation For complete details on Bybit's fee structure and currency rules, refer to: - [Bybit WebSocket Private Execution](https://bybit-exchange.github.io/docs/v5/websocket/private/execution) - [Bybit Spot Fee Currency Instruction](https://bybit-exchange.github.io/docs/v5/enum#spot-fee-currency-instruction) ## Configuration The product types for each client must be specified in the configurations. ### Data client configuration options | Option | Default | Description | |------------------------------------|-----------|-------------| | `api_key` | `None` | API key; loaded from the matching environment variable when omitted. | | `api_secret` | `None` | API secret; loaded from the matching environment variable when omitted. | | `product_types` | `None` | Sequence of `BybitProductType` values to enable; loads all products when `None`. | | `instrument_provider` | default | Instrument loading config. Use `load_all=True` or `load_ids` before subscribing. | | `environment` | `None` | Bybit environment enum. Use `BybitEnvironment.MAINNET`, `BybitEnvironment.DEMO`, or `BybitEnvironment.TESTNET`. | | `base_url_http` | `None` | Override for the REST base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `update_instruments_interval_mins` | `60` | Interval (minutes) between instrument catalogue refreshes. | | `recv_window_ms` | `5,000` | Receive window (milliseconds) for signed REST requests. | | `bars_timestamp_on_close` | `True` | Timestamp bars on the close (`True`) or open (`False`) of the interval. | | `max_retries` | `None` | Maximum retry attempts for REST/WebSocket recovery. | | `retry_delay_initial_ms` | `None` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `None` | Maximum delay (milliseconds) between retries. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |-----------------------------------------|-----------|-------------| | `api_key` | `None` | API key; loaded from the matching environment variable when omitted. | | `api_secret` | `None` | API secret; loaded from the matching environment variable when omitted. | | `product_types` | `None` | Sequence of `BybitProductType` values to enable (Spot cannot be mixed with derivatives for execution). | | `instrument_provider` | default | Instrument loading config. Use `load_all=True` or `load_ids` before submitting orders. | | `environment` | `None` | Bybit environment enum. Use `BybitEnvironment.MAINNET`, `BybitEnvironment.DEMO`, or `BybitEnvironment.TESTNET`. | | `base_url_http` | `None` | Override for the REST base URL. | | `base_url_ws_private` | `None` | Override for the private WebSocket base URL. | | `base_url_ws_trade` | `None` | Override for the trade WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `use_gtd` | `False` | Remap GTD orders to GTC when `True` (Bybit lacks native GTD support). | | `use_ws_execution_fast` | `False` | Subscribe to the low‑latency execution stream. | | `use_http_batch_api` | `False` | Use Bybit's HTTP batch trading API (deprecated). | | `use_spot_position_reports` | `False` | Report Spot wallet balances as positions when `True`. | | `auto_repay_spot_borrows` | `True` | Automatically repay Spot margin borrows after BUY orders fully fill (Spot only). | | `repay_queue_interval_secs` | `1.0` | Interval (seconds) between processing repayment queues for spot borrows. | | `ignore_uncached_instrument_executions` | `False` | Ignore execution messages for instruments not yet cached. | | `max_retries` | `None` | Maximum retry attempts for order submission/cancel/modify calls. | | `retry_delay_initial_ms` | `None` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `None` | Maximum delay (milliseconds) between retries. | | `recv_window_ms` | `5,000` | Receive window (milliseconds) for signed REST requests. | | `ws_trade_timeout_secs` | `5.0` | Timeout (seconds) waiting for trade WebSocket acknowledgements. | | `ws_auth_timeout_secs` | `5.0` | Timeout (seconds) waiting for auth WebSocket acknowledgements. | | `futures_leverages` | `None` | Mapping of `BybitSymbol` to leverage settings. | | `position_mode` | `None` | Mapping of `BybitSymbol` to position mode. See [Hedge mode](#hedge-mode-bothsides). | | `margin_mode` | `None` | Margin mode setting for the account. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | The most common use case is to configure a live `TradingNode` to include Bybit data and execution clients. To achieve this, add a `BYBIT` section to your client configuration(s): ```python from nautilus_trader.adapters.bybit import BYBIT from nautilus_trader.adapters.bybit import BybitEnvironment from nautilus_trader.adapters.bybit import BybitProductType from nautilus_trader.live.node import TradingNode from nautilus_trader.live.node import TradingNodeConfig config = TradingNodeConfig( ..., # Omitted data_clients={ BYBIT: { "api_key": "YOUR_BYBIT_API_KEY", "api_secret": "YOUR_BYBIT_API_SECRET", "base_url_http": None, # Override with custom endpoint "environment": BybitEnvironment.MAINNET, "product_types": [BybitProductType.LINEAR], }, }, exec_clients={ BYBIT: { "api_key": "YOUR_BYBIT_API_KEY", "api_secret": "YOUR_BYBIT_API_SECRET", "base_url_http": None, # Override with custom endpoint "environment": BybitEnvironment.MAINNET, "product_types": [BybitProductType.LINEAR], }, }, ) ``` Then, create a `TradingNode` and add the client factories: ```python from nautilus_trader.adapters.bybit import BYBIT from nautilus_trader.adapters.bybit import BybitLiveDataClientFactory from nautilus_trader.adapters.bybit import BybitLiveExecClientFactory from nautilus_trader.live.node import TradingNode # Instantiate the live trading node with a configuration node = TradingNode(config=config) # Register the client factories with the node node.add_data_client_factory(BYBIT, BybitLiveDataClientFactory) node.add_exec_client_factory(BYBIT, BybitLiveExecClientFactory) # Finally build the node node.build() ``` ### API credentials There are two options for supplying your credentials to the Bybit clients. Either pass the corresponding `api_key` and `api_secret` values to the configuration objects, or set the following environment variables: For Bybit live clients, you can set: - `BYBIT_API_KEY` - `BYBIT_API_SECRET` For Bybit demo clients, you can set: - `BYBIT_DEMO_API_KEY` - `BYBIT_DEMO_API_SECRET` For Bybit testnet clients, you can set: - `BYBIT_TESTNET_API_KEY` - `BYBIT_TESTNET_API_SECRET` :::tip We recommend using environment variables to manage your credentials. ::: When starting the trading node, you'll receive immediate confirmation of whether your credentials are valid and have trading permissions. ## Contributing :::info For additional features or to contribute to the Bybit adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Coinbase Source: https://nautilustrader.io/docs/latest/integrations/coinbase/ Founded in 2012, Coinbase is one of the largest US-regulated cryptocurrency exchanges, offering trading across spot, perpetual swaps, and dated futures via the Advanced Trade API. This adapter supports live market data ingest and order execution on both spot (Cash) and CFM derivatives (Margin) accounts through a shared execution client, with the account type selected by the factory (see [Execution scope](#execution-scope)). :::note This adapter is Rust-only and is consumed by the v2 system (and the Rust `LiveNode`). It does not ship a legacy Python `TradingNode` integration; only configuration and enum types are exported through PyO3 so v2 Python entry points can construct them. ::: ## Overview The Coinbase adapter is implemented in Rust and consumed by the v2 system. The adapter does not ship a legacy Python `TradingNode` integration; only configuration and enum types are exported through PyO3 so v2 entry points can construct them from Python. Components: - `CoinbaseHttpClient`: Two‑layer REST client (raw endpoint methods + domain wrapper). - `CoinbaseWebSocketClient`: Low‑level WebSocket connectivity with JWT subscribe auth. - `CoinbaseInstrumentProvider`: Instrument parsing and loading. - `CoinbaseDataClient`: Market data feed manager. - `CoinbaseDataClientFactory`: Data client factory. - `CoinbaseExecutionClient`: Execution client (spot or CFM derivatives; REST orders + WS streams). - `CoinbaseExecutionClientFactory`: Execution client factory; spot vs CFM derivatives is selected by `account_type` on the config. PyO3 surface available from `nautilus_trader.core.nautilus_pyo3.coinbase`: - `CoinbaseDataClientConfig`, `CoinbaseExecClientConfig` - `CoinbaseEnvironment`, `CoinbaseMarginType` - `COINBASE` venue constant ## Coinbase documentation Coinbase provides documentation for the Advanced Trade API: - [REST API reference](https://docs.cdp.coinbase.com/advanced-trade/reference) - [WebSocket channels](https://docs.cdp.coinbase.com/advanced-trade/docs/ws-channels) - [API key authentication](https://docs.cdp.coinbase.com/coinbase-app/authentication-authorization/api-key-authentication) - [Rate limits](https://docs.cdp.coinbase.com/advanced-trade/docs/rate-limits) It's recommended you also refer to the Coinbase documentation in conjunction with this NautilusTrader integration guide. :::info This adapter targets the Coinbase Advanced Trade API. The separate [Coinbase International Exchange (INTX)](https://international.coinbase.com) venue is supported by the dedicated `coinbase_intx` adapter. ::: ## Products A product is an umbrella term for a group of related instrument types. The following product types are supported: | Product Type | Supported | Notes | |---------------------|-----------|----------------------------------------------------------| | Spot | ✓ | USD, USDC, and USDT-quoted spot pairs. | | Perpetual contracts | ✓ | USD-margined perpetual swaps on the FCM venue. | | Futures contracts | ✓ | Dated delivery futures (nano BTC, nano ETH, etc). | ## Symbology Coinbase uses the venue's native `product_id` field directly as the Nautilus symbol. The instrument ID is `{product_id}.COINBASE`. | Product | Format | Examples | |------------------|------------------------------------|------------------------------------| | Spot | `{base}-{quote}` | `BTC-USD`, `ETH-USDC`, `SOL-USDT`. | | Perpetual | `{contract_code}-{ddMMMyy}-CDE` | `BIP-20DEC30-CDE` (BTC PERP). | | Dated future | `{contract_code}-{ddMMMyy}-CDE` | `BIT-24APR26-CDE` (BTC Apr 2026). | The `-CDE` suffix denotes the Coinbase Derivatives Exchange (FCM venue). Perpetuals carry an exchange-assigned far-future expiry (e.g. `20DEC30`) but are classified as `CryptoPerpetual` based on the presence of an ongoing funding rate. Dated futures are classified as `CryptoFuture`. The adapter resolves the product type structurally from API metadata (`future_product_details.contract_expiry_type` and, when that is `EXPIRING`, the presence of a non-empty `future_product_details.funding_rate` as a perpetual-only structural signal); the fallback heuristic checks `display_name` for `PERP` or `Perpetual` substrings. Examples of full Nautilus instrument IDs: - `BTC-USD.COINBASE` (spot Bitcoin/USD). - `ETH-USDC.COINBASE` (spot Ether/USDC). - `BIP-20DEC30-CDE.COINBASE` (BTC perpetual swap). - `BIT-24APR26-CDE.COINBASE` (BTC dated future, Apr 2026). ### Aliased products (USDC and USD) Coinbase consolidates USDC- and USD-quoted versions of the same pair into a single matching-engine book and exposes the relationship in `GET /products` via the `alias` and `alias_to` fields: ```text BTC-USD : alias="" alias_to=["BTC-USDC"] # canonical BTC-USDC: alias="BTC-USD" alias_to=[] # alias of BTC-USD ``` When a caller subscribes or submits using the alias side, the venue rewrites the request to the canonical id on the wire. The adapter handles this transparently: it records the `product_id -> alias` map at bootstrap, sends the canonical id on subscribe and order submit, registers a reverse mapping on the WebSocket clients, and re-keys inbound messages back to the caller-supplied id before parsing. A strategy holding only USDC can therefore trade `BTC-USDC.COINBASE` end to end without referencing the canonical `BTC-USD`. Settlement currency is determined by the submitted `product_id`, so an order placed on `BTC-USDC.COINBASE` always debits or credits the USDC wallet. ## Environments Coinbase provides two trading environments. Configure the appropriate environment using the `environment` field in your client configuration. | Environment | `environment` value | REST base URL | |-------------|---------------------------------|------------------------------------| | Live | `CoinbaseEnvironment.LIVE` | `https://api.coinbase.com` | | Sandbox | `CoinbaseEnvironment.SANDBOX` | `https://api-sandbox.coinbase.com` | ### Live (production) The default environment for live trading with real funds. ```python config = CoinbaseExecClientConfig( api_key="YOUR_API_KEY", api_secret="YOUR_API_SECRET", # environment=CoinbaseEnvironment.LIVE (default) ) ``` Environment variables: `COINBASE_API_KEY`, `COINBASE_API_SECRET`. ### Sandbox A static-mock test environment for integration plumbing, per the [Sandbox docs](https://docs.cdp.coinbase.com/coinbase-app/advanced-trade-apis/sandbox). ```python config = CoinbaseExecClientConfig( api_key="ANY_NON_EMPTY_STRING", # required by the adapter constructor api_secret="ANY_NON_EMPTY_STRING", environment=CoinbaseEnvironment.SANDBOX, ) ``` The sandbox venue does not enforce authentication, but `CoinbaseExecutionClient::new` still requires both fields (or the matching environment variables) to be present in order to construct. :::warning **Sandbox is not a parallel trading venue:** - All responses are static and pre-defined; there is no live market or dynamic pricing. - Only Accounts and Orders endpoints are available; other resources are not. - Authentication is not required (and not enforced). - A custom `X-Sandbox` request header can trigger predefined error scenarios. Use sandbox to wire up your client and verify request/response shape; use production (with real funds and care) for any realistic behaviour testing. ::: ## Authentication Coinbase Advanced Trade uses ES256 JWT authentication. Each REST request and each WebSocket subscription generates a short-lived JWT signed with your EC private key. The adapter resolves credentials from environment variables or from the config fields. ### Creating an API key Coinbase has several key types. The adapter requires a **Coinbase App Secret API key** with the **ECDSA** signature algorithm (not Ed25519). Go to the CDP portal API keys page: [portal.cdp.coinbase.com/projects/api-keys](https://portal.cdp.coinbase.com/projects/api-keys). Select the **Secret API Keys** tab and click **Create API key**. Enter a nickname (e.g. `nautilus-trading`). Expand **API restrictions** and set permissions to **View** and **Trade**. Expand **Advanced Settings** and change the signature algorithm from Ed25519 to **ECDSA**. This step is required: Ed25519 keys do not work with the Advanced Trade API. Click **Create API key**. Save the key name and private key from the modal. The key name looks like `organizations/{org_id}/apiKeys/{key_id}`. The private key is a PEM-encoded EC key (SEC1 format). :::warning Coinbase no longer auto-downloads the key file. Copy the values from the creation modal or click the download button before closing it. You cannot retrieve the private key afterward. ::: :::info Do not use legacy API keys from coinbase.com/settings/api (UUID format with HMAC-SHA256 signing). Those use a different auth scheme (`CB-ACCESS-*` headers) that the adapter does not support. ::: For full details see the Coinbase [API key authentication guide](https://docs.cdp.coinbase.com/coinbase-app/authentication-authorization/api-key-authentication). ### Environment variables | Variable | Description | |-----------------------|-----------------------------------------------------------| | `COINBASE_API_KEY` | Key name (`organizations/{org_id}/apiKeys/{key_id}`). | | `COINBASE_API_SECRET` | PEM‑encoded EC private key (full multi‑line string). | Example: ```bash export COINBASE_API_KEY="organizations/abc-123/apiKeys/def-456" export COINBASE_API_SECRET="$(cat ~/path/to/cdp_api_key.pem)" ``` :::tip We recommend using environment variables to manage your credentials. ::: ### JWT lifetime Coinbase JWTs expire after 120 seconds. Per the [WebSocket overview](https://docs.cdp.coinbase.com/coinbase-app/advanced-trade-apis/websocket/websocket-overview), a different JWT must be generated for each authenticated WebSocket message (i.e. for each subscribe). The adapter regenerates a fresh JWT for every signed REST request and for every authenticated subscribe message; no manual rotation is required. ## Portfolios A Coinbase account holds one or more **portfolios**. Each portfolio has its own wallets (USD, USDC, BTC, etc.), balances, and order scope. Every account has a `DEFAULT` portfolio; users can create additional `CONSUMER` portfolios to segregate strategies, risk, or tax lots. A CDP API key is **bound to a single portfolio at creation time**. Every authenticated request (account lookup, order submission, cancel) operates against that portfolio unless a different one is explicitly specified. ### Finding your portfolio UUIDs Run the adapter's authenticated probe binary; it prints the portfolios visible to your CDP key, the account balances in the bound portfolio, and a few reference REST calls: ```bash cargo run --bin coinbase-http-private --package nautilus-coinbase ``` Sample output: ``` Found 1 portfolio(s) name=Default type=DEFAULT uuid=ca7244bc-21d1-5e4c-bfe5-80f208ac5723 deleted=false Account has 3 balance(s) USDC total=100.00000000 USDC free=100.00000000 USDC locked=0.00000000 USDC AUD total=0.00 AUD free=0.00 AUD locked=0.00 AUD BTC total=0.00000000 BTC free=0.00000000 BTC locked=0.00000000 BTC ``` Equivalent curl (you have to sign your own ES256 JWT with your CDP PEM key first): ```bash curl -H "Authorization: Bearer $JWT" \ https://api.coinbase.com/api/v3/brokerage/portfolios ``` ### When `retail_portfolio_id` is required Coinbase's `POST /orders` endpoint routes to the key's bound portfolio by default, so a single-portfolio account does not need to set this field. Set it on [`CoinbaseExecClientConfig`](#execution-client-configuration-options) when either is true: - The account holds multiple portfolios and you want to trade against one that is not the key's default. - The venue rejects orders with `account is not available` and the wallet diagnosis below has been ruled out. ### Creating a new portfolio Most users will not need to create a new portfolio; the account's default works out of the box. Create one on [coinbase.com/portfolios](https://www.coinbase.com/portfolios) only if you want to: - Segregate API‑driven trading from manual retail activity. - Isolate risk or P&L between strategies. - Work around a restricted default (e.g. a Vault). After creating a portfolio, fund it (transfer from the default portfolio's wallet on coinbase.com) before sending any orders, otherwise the venue returns `account is not available` for the quote currency. ### Troubleshooting `account is not available` The venue returns this error for several distinct reasons; diagnose by running the probe binary above and inspecting the portfolio wallet list. | Symptom | Likely cause | Fix | |----------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------| | Rejected only for a specific product (e.g. `BTC-USD` with only USDC) | Portfolio is missing a wallet for the product's quote currency. USD and USDC are separate on Coinbase, and the venue routes orders by the submitted `product_id`, not by the canonical alias. | Submit against the product whose quote currency you hold (e.g. `BTC-USDC` for USDC wallets). The adapter resolves the data‑side alias internally; no config change needed. Funding the missing wallet via coinbase.com is also an option but unnecessary when only one currency is held. | | Every order rejected across all products | Key is bound to a non‑default portfolio and `retail_portfolio_id` is unset. | Set `retail_portfolio_id` on `CoinbaseExecClientConfig` to the target portfolio UUID. | | Rejected for `*-USD` products on a non‑US account | Jurisdictional restriction (e.g. AU accounts cannot trade USD‑quoted pairs). | Use locally‑available quotes (USDC, AUD, EUR, etc.) instead of USD. | | Rejected right after key rotation | New key was created in a different portfolio than the previous one. | Update `retail_portfolio_id` to match the new key's portfolio, or move funds. | ## Orders capability The tables below describe the Coinbase **venue** order surface. The shipped [`CoinbaseExecutionClient`](#execution-scope) handles spot or CFM derivatives based on the configured `account_type`. Coinbase order capabilities differ between Spot and Derivatives (perpetuals and dated futures share the same FCM order surface). ### Execution scope `CoinbaseExecutionClientFactory` produces a single `CoinbaseExecutionClient` type. The product family is selected by the `account_type` field on `CoinbaseExecClientConfig`: | `account_type` | Bootstrap instruments | Account state source | |-----------------------|-----------------------------------------------|-----------------------------------------------------------| | `AccountType::Cash` | `CoinbaseProductType::Spot` only. | `/accounts` REST endpoint. | | `AccountType::Margin` | `CoinbaseProductType::Future` (perp + dated). | CFM `balance_summary` REST + `futures_balance_summary` WS, plus position reports from `cfm/positions`. | Other account types are rejected at factory creation. OMS is always `Netting` because the venue does not expose hedge mode. To prevent cross-account bleed-through: 1. Connect-time instrument bootstrap is limited to the configured product family; the other family's products never enter the in-process cache. 2. `submit_order` denies any order whose instrument is outside that cache. 3. `generate_order_status_report(s)` and `generate_fill_reports` post-filter their output through the same cache, so a Coinbase account that holds both spot and derivative activity will not surface the other scope's reports through a single client. Run one execution client per scope; if you need both spot and CFM activity on the same trader, instantiate two clients with distinct `account_type` values (and distinct `account_id`s). ### Order types The matrix lists order types as exposed through the Nautilus model. The right column shows the corresponding `order_configuration` keys the adapter emits. Coinbase order types not in this table (TWAP, Bracket, Scaled, SOR LIMIT IOC) are documented under [Advanced order features](#advanced-order-features) and noted there as *Not yet supported* by the adapter. | Order Type | Spot | Perpetual | Future | Wire shape | |------------------------|------|-----------|--------|-------------------------------------------------------------| | `MARKET` | ✓ | ✓ | ✓ | `market_market_ioc` (spot + CFM); `market_market_fok` (CFM only) | | `LIMIT` | ✓ | ✓ | ✓ | `limit_limit_gtc` / `limit_limit_gtd` / `limit_limit_fok` | | `STOP_LIMIT` | - | ✓ | ✓ | `stop_limit_stop_limit_gtc` / `stop_limit_stop_limit_gtd` | | `STOP_MARKET` | - | - | - | *Not exposed by the venue.* | | `MARKET_IF_TOUCHED` | - | - | - | *Not exposed by the venue.* | | `LIMIT_IF_TOUCHED` | - | - | - | *Not exposed by the venue.* | | `TRAILING_STOP_MARKET` | - | - | - | *Not exposed by the venue.* | ### Execution instructions | Instruction | Spot | Perpetual | Future | Notes | |---------------|------|-----------|--------|--------------------------------------------------------------------| | `post_only` | ✓ | ✓ | ✓ | LIMIT GTC and LIMIT GTD only. | | `reduce_only` | - | ✓ | ✓ | Derivatives only. | ### Time in force The adapter accepts the values in this matrix; combinations not listed are rejected at submit time with `"Unsupported TIF {tif} for {order_type}"`. | Order type | GTC | GTD | IOC | FOK | Notes | |--------------|-----|-----|-----|-----|----------------------------------------------------------------| | `MARKET` | ✓ | - | ✓ | (✓) | GTC is mapped to IOC; explicit IOC is honoured. FOK builds the venue's `market_market_fok` shape, but the matching engine currently rejects it on spot with `UNSUPPORTED_ORDER_CONFIGURATION`; usable on CFM derivatives only. | | `LIMIT` | ✓ | ✓ | - | ✓ | GTD requires `expire_time`. LIMIT IOC *not yet supported* (see [SOR LIMIT IOC](#advanced-order-features)). | | `STOP_LIMIT` | ✓ | ✓ | - | - | Requires `trigger_price`. Derivatives only. | ### Advanced order features | Feature | Spot | Perpetual | Future | Notes | |--------------------|------|-----------|--------|------------------------------------------------------------------------------------| | Order Modification | ✓ | ✓ | ✓ | GTC variants only (LIMIT, STOP_LIMIT, Bracket); other types use cancel‑replace. | | Bracket Orders | - | - | - | *Not yet supported.* Venue exposes `trigger_bracket_gtc` / `trigger_bracket_gtd`. | | OCO Orders | - | - | - | *Not exposed by the venue* as a distinct order type. | | Iceberg Orders | - | - | - | *Not exposed by the venue.* | | TWAP Orders | - | - | - | *Not yet supported.* Venue exposes `twap_limit_gtd`. | | Scaled Orders | - | - | - | *Not yet supported.* Venue exposes `scaled_limit_gtc`. | | SOR LIMIT IOC | - | - | - | *Not yet supported.* Venue exposes `sor_limit_ioc` for smart‑order‑routed LIMIT IOC. | See the [Create Order reference](https://docs.cdp.coinbase.com/api-reference/advanced-trade-api/rest-api/orders/create-order) and [Edit Order reference](https://docs.cdp.coinbase.com/api-reference/advanced-trade-api/rest-api/orders/edit-order) for the underlying venue specification. ### Position controls (derivatives) | Control | Notes | |---------------|----------------------------------------------------------------------| | Leverage | Set per order; default `1.0`. | | Margin type | Set per order: cross (default) or isolated. | | Position mode | One‑way only; hedge mode is not exposed. | ### Batch operations | Operation | Notes | |---------------|----------------------------------------------------------------------------------------------------| | Batch Submit | Not supported. Each order is one `Create Order` request. | | Batch Modify | Not supported. Each edit is one `Edit Order` request. | | Batch Cancel | `POST /api/v3/brokerage/orders/batch_cancel` accepts an `order_ids` array. No documented max size; per‑order success/failure in the response. | ### Order querying | Feature | Spot | Perpetual | Future | Notes | |----------------------|------|-----------|--------|---------------------------------------------| | Query open orders | ✓ | ✓ | ✓ | List all active orders. | | Query order history | ✓ | ✓ | ✓ | Historical order data with cursor paging. | | Order status updates | ✓ | ✓ | ✓ | Real‑time state changes via `user` channel. | | Trade history | ✓ | ✓ | ✓ | Execution and fill reports. | ### Spot trading limitations - `reduce_only` is not supported on spot orders (the instruction applies to derivatives). - Trailing stop orders are not supported. - Native stop‑limit and bracket orders are not available on Spot. - Quote‑denominated MARKET orders are supported; LIMIT orders are sized in base units. ### Derivatives trading Coinbase derivatives trade through the FCM (Futures Commission Merchant) venue. The exec client submits orders through the same `POST /orders` endpoint used for spot; per-order `leverage` and `margin_type` (`CROSS` or `ISOLATED`) defaults come from `CoinbaseExecClientConfig.default_leverage` and `default_margin_type`. Margin balances update from both the REST `cfm/balance_summary` endpoint (connect-time snapshot, `query_account`, and on WebSocket reconnect) and the authenticated `futures_balance_summary` WebSocket channel. Position reports come from the REST `cfm/positions` endpoints. Coinbase's Advanced Trade API does not document a `reduce_only` field on the create-order schema, even though the venue's failure-reason enum acknowledges the concept. The client threads `reduce_only` through its `submit_order` signature for API parity and includes the flag on the wire only when set to `true`; if the venue later accepts it, no client changes are required. #### Funding rates The adapter polls the REST `/products/{id}` endpoint at `derivatives_poll_interval_secs` (default 15 s) and emits a `FundingRateUpdate` from the FCM `future_product_details` payload when `funding_rate` is present. The funding interval is parsed from the `funding_interval` field (typically `"3600s"`, hourly funding) and the next funding timestamp from `funding_time`. Coinbase Advanced Trade does not publish `funding_rate` on the WebSocket `ticker` channel, so REST polling is the only live source. Historical funding rate requests (`DataTester` TC-D53) are not yet implemented; the same REST products endpoint can serve them in a future revision, deriving the interval from consecutive funding timestamps. #### Instrument status `subscribe_instrument_status` joins the Coinbase WebSocket `status` channel on first subscription (the venue publishes one status feed for all products), filters incoming events to the subscribed instruments, and emits `InstrumentStatus` events with `MarketStatusAction::Trading` for `online`, `Halt` for `offline`, and `Close` for `delisted`. Futures products that report an empty `status` string carry no information for the data engine and are skipped. The channel subscription is dropped when the last instrument unsubscribes. #### Position reconciliation For Cash (spot) accounts the client returns no position reports because Coinbase spot has no positions. For Margin accounts position reports come from the REST `cfm/positions` (list) and `cfm/positions/{product_id}` (single) endpoints and are post-filtered to the bootstrap instrument cache. Open orders and historical fills are reconciled from REST via `generate_order_status_report(s)` and `generate_fill_reports` on connect and on the standard reconciliation interval set by `LiveExecEngineConfig`. #### Fill deduplication The user-channel WebSocket can replay events on reconnect. The execution client maintains a 10,000-entry FIFO dedup keyed on `(venue_order_id, trade_id)` and drops any fill whose synthesized trade ID matches a recently-seen one. The cumulative-state map is bounded with the same capacity to protect against orders that never receive a terminal event in this client's lifetime. After very long disconnections (beyond the in-memory dedup window) replayed fills may emit duplicate `OrderFilled` events; strategies should rely on REST reconciliation to recover canonical state in that case. ## Execution client behaviour This section documents how `CoinbaseExecutionClient` translates Nautilus order commands and Coinbase venue events into Nautilus execution events. ### Order submission `submit_order` builds the Coinbase `order_configuration` shape directly from Nautilus order fields: - `MARKET` -> `market_market_ioc`. Only `TimeInForce::Ioc` and `Gtc` (the Nautilus default) are accepted; any explicit `Fok`, `Day`, or `Gtd` on a market order is rejected before the HTTP call so callers do not silently receive IOC semantics. A `MARKET` order built with `Gtc` executes as IOC at the venue; strategies that require strict backtest/live parity should construct `MarketOrder` with `Ioc` explicitly. - `LIMIT` GTC -> `limit_limit_gtc`, GTD -> `limit_limit_gtd` (requires `expire_time`), FOK -> `limit_limit_fok`. - `STOP_LIMIT` GTC -> `stop_limit_stop_limit_gtc`, GTD -> `stop_limit_stop_limit_gtd`. Stop direction is derived from the order side (`Buy` -> `STOP_DIRECTION_STOP_UP`, `Sell` -> `STOP_DIRECTION_STOP_DOWN`). - `STOP_MARKET`, `MARKET_IF_TOUCHED`, `LIMIT_IF_TOUCHED`, and trailing-stop variants are not exposed by the venue. They surface as `OrderRejected` carrying the `build_order_configuration` error from the spawned submit task (the order is emitted as `OrderSubmitted` first). On a successful HTTP create, an `OrderAccepted` is emitted carrying the venue order ID returned in `success_response.order_id`. On a `success=false` response, `OrderRejected` is emitted with the formatted venue failure reason. HTTP failures with unknown venue outcome leave the order in flight for WebSocket updates, open-order polling, or reconciliation. ### Order modification `modify_order` posts to `/orders/edit` with the typed `EditOrderRequest`. Coinbase restricts edits to GTC variants (LIMIT, STOP_LIMIT, Bracket); other order types must use cancel-replace. Coinbase's `/orders/edit` requires both `price` and `size` even when only one is changing; an omitted `size` is read as 0 and rejected with `INVALID_EDITED_SIZE` or `CANNOT_EDIT_TO_BELOW_FILLED_SIZE`. The exec client auto-fills missing fields from the cached order, so strategies can call `modify_order(price=X)` without repeating the current quantity. Values from the `ModifyOrder` command win; otherwise the cached order's current `price` and `quantity` are used. Venue edit failures emit `OrderModifyRejected` with the typed `EditOrderResponse` reason (preferring `edit_failure_reason`, falling back to `preview_failure_reason`). HTTP failures with unknown venue outcome leave the order in `PENDING_UPDATE` until an update, query result, or reconciliation resolves it. ### Cancellation - `cancel_order` posts a single-id `batch_cancel`. An explicit per-order venue failure surfaces as `OrderCancelRejected`; a whole-request transport failure with unknown venue outcome leaves the order in `PENDING_CANCEL` for reconciliation. - `cancel_all_orders` lists open orders via REST without the `OPEN`-only filter (because Coinbase's `OPEN` filter excludes `PENDING` and `QUEUED` orders that are still cancelable), filters locally to `{Submitted, Accepted, Triggered, PendingUpdate, PartiallyFilled}` and the requested side, then chunks `batch_cancel` calls in groups of 100. Per-order venue failures emit `OrderCancelRejected`; whole-request failures with unknown venue outcome leave affected orders pending reconciliation. - `batch_cancel_orders` chunks the same way and surfaces explicit per-order venue failures as `OrderCancelRejected`. Transport failures with unknown venue outcome leave affected orders pending reconciliation. ### User WebSocket channel `CoinbaseExecutionClient` subscribes to the `user` channel with no `product_ids` filter and a fresh JWT, parses each event into an `OrderStatusReport`, and feeds it to the execution event stream. Coinbase reports cumulative state per order rather than per-trade fills, so the exec client synthesizes a `FillReport` from the cumulative delta. The per-fill price is derived as `(avg_now * qty_now - avg_prev * qty_prev) / delta_qty` so multi-fill orders carry the correct trade price, not the cumulative weighted average. The original quantity is restored on terminal updates (`CANCELLED`, `EXPIRED`, `FAILED`) where the venue zeroes `leaves_quantity`. The user channel does not echo `price`, `stop_price`, `trigger_type`, or maker/taker classification. The exec client caches these at submit time under the `client_order_id` and patches reports before emit, so the reconciler does not observe a `Some(price) -> None` divergence and `post_only` fills are correctly stamped `liquidity_side = Maker`. Order status `PENDING`, `QUEUED`, and `OPEN` all map to `OrderStatus::Accepted` to avoid spurious backwards-transition warnings when user-channel updates race the REST `OrderAccepted` event. A `submit_order` rejection carrying `INVALID_LIMIT_PRICE_POST_ONLY` (or the preview/new-order equivalent) is emitted with `due_post_only = true` so strategies can react to post-only crossings (typically by re-quoting against the new TOB). On reconnect, account state is re-fetched via REST so balance changes during the disconnect window are recovered. Cumulative per-order tracking persists across reconnects so synthesized fill deltas remain correct. ## Rate limiting Coinbase publishes the following limits for the Advanced Trade APIs: | Surface | Limit | Source | |-----------------------------------|------------------------------------------------------|-------------------------------------------------------| | WebSocket connections | 8 per second per IP address | Advanced Trade WebSocket Rate Limits | | WebSocket unauthenticated msgs | 8 per second per IP address | Advanced Trade WebSocket Rate Limits | | WebSocket subscribe deadline | First subscribe message must arrive within 5 s of connect or the server disconnects | Advanced Trade WebSocket Overview | | Authenticated WebSocket JWT | 120 s; a fresh JWT must be generated for every authenticated subscribe message | Advanced Trade WebSocket Overview | | REST per‑key quota | 10,000 requests per hour per API key (Coinbase App general policy) | Coinbase App Rate Limiting | When the REST limit is exceeded, Coinbase returns HTTP `429` with this body: ```json { "errors": [ { "id": "rate_limit_exceeded", "message": "Too many requests" } ] } ``` :::info The Advanced Trade-specific REST quota (per-second ceilings, per-portfolio limits) is not separately published in the Advanced Trade docs at the time of writing; the Coinbase App per-hour quota above is the most specific documented value. References: [REST rate limits](https://docs.cdp.coinbase.com/advanced-trade/docs/rest-api-rate-limits/), [WebSocket rate limits](https://docs.cdp.coinbase.com/advanced-trade/docs/ws-rate-limits), [Coinbase App rate limiting](https://docs.cdp.coinbase.com/coinbase-app/api-architecture/rate-limiting). ::: ## Reconnect and resubscribe The WebSocket client uses exponential backoff with a base of 250ms and a cap of 30s on reconnect. After reconnect, subscriptions are restored automatically in the order they were created. Coinbase requires a subscribe message within 5 seconds of connection or the server disconnects; the adapter sends queued subscriptions immediately after the WebSocket handshake completes. For authenticated channels (`user`, and `futures_balance_summary` on Margin clients), the adapter generates a fresh JWT for every subscribe message; per the Coinbase docs, "you must generate a different JWT for each websocket message sent, since the JWTs will expire after 120 seconds." Once a subscription is accepted the data flow continues for the lifetime of the WebSocket connection without further authentication. When the exec client's WebSocket reconnects, the inner client is rebuilt from scratch (rather than relying on the existing connection's state machine) to guarantee a fresh `cmd_tx`/`out_rx`/signal trio even if the prior session's `Disconnect` command lost a race with the shutdown signal. Cumulative per-order tracking persists across reconnects so synthesized fill deltas remain correct. ## Configuration ### Data client configuration options | Option | Default | Description | |------------------------------------|-------------|-----------------------------------------------------------------------------------| | `api_key` | `None` | Falls back to `COINBASE_API_KEY` env var. | | `api_secret` | `None` | Falls back to `COINBASE_API_SECRET` env var. | | `base_url_rest` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket market data URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `environment` | `Live` | `Live` or `Sandbox`. | | `http_timeout_secs` | `10` | HTTP request timeout (seconds). | | `ws_timeout_secs` | `30` | WebSocket timeout (seconds). | | `update_instruments_interval_mins` | `60` | Interval between instrument catalogue refreshes. | | `derivatives_poll_interval_secs` | `15` | Interval between REST polls that emit `IndexPriceUpdate` and `FundingRateUpdate`. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |--------------------------|-----------|----------------------------------------------------------------------------------------------------------| | `api_key` | `None` | Falls back to `COINBASE_API_KEY` env var. | | `api_secret` | `None` | Falls back to `COINBASE_API_SECRET` env var. | | `base_url_rest` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the user data WebSocket URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `environment` | `Live` | `Live` or `Sandbox`. | | `http_timeout_secs` | `10` | HTTP request timeout (seconds). | | `max_retries` | `3` | Maximum retry attempts for HTTP requests. | | `retry_delay_initial_ms` | `100` | Initial retry delay (milliseconds). | | `retry_delay_max_ms` | `5000` | Maximum retry delay (milliseconds). | | `account_type` | `Cash` | `Cash` for spot or `Margin` for CFM derivatives. See [Execution scope](#execution-scope). | | `default_margin_type` | `None` | Default `CoinbaseMarginType` (`Cross` or `Isolated`) applied to derivatives orders. Ignored on Cash. | | `default_leverage` | `None` | Default leverage applied to derivatives orders. Ignored on Cash. | | `retail_portfolio_id` | `None` | CDP retail portfolio UUID. Required when the API key is bound to a non‑default portfolio (the venue rejects orders with `account is not available` otherwise). See [Portfolios](#portfolios). | | `transport_backend` | `Sockudo` | WebSocket transport backend. | Configurations are constructed from Python via the PyO3-exported types: ```python from nautilus_trader.core.nautilus_pyo3 import CoinbaseDataClientConfig from nautilus_trader.core.nautilus_pyo3 import CoinbaseExecClientConfig from nautilus_trader.core.nautilus_pyo3 import CoinbaseEnvironment data_config = CoinbaseDataClientConfig( api_key="YOUR_COINBASE_API_KEY", api_secret="YOUR_COINBASE_API_SECRET", environment=CoinbaseEnvironment.LIVE, ) exec_config = CoinbaseExecClientConfig( api_key="YOUR_COINBASE_API_KEY", api_secret="YOUR_COINBASE_API_SECRET", environment=CoinbaseEnvironment.LIVE, ) ``` The v2 system instantiates the Rust factories directly from these configs; no Python factory wiring is required. ## Known limitations ### Venue-side - Order modification is restricted to GTC orders (LIMIT, STOP_LIMIT, Bracket); other types must use cancel-replace. - OCO orders are not exposed as a distinct order type. - Trailing stop, MARKET_IF_TOUCHED, LIMIT_IF_TOUCHED, and iceberg orders are not exposed by the venue. - Batch submit and batch modify are not available; only batch cancel is. - Sandbox is a static-mock environment (Accounts and Orders endpoints only, pre-defined responses, no real market data). - The user-channel WebSocket reports cumulative per-order state, not per-trade fills. The exec client derives per-fill quantity, price, and commission from the cumulative delta; per-trade `trade_id`s are synthesized from `(venue_order_id, cumulative_quantity)`. ### Adapter-side - **One product family per client.** Submission, modification, cancellation, and report generation are filtered to the configured product family (spot under `AccountType::Cash`; perp + dated futures under `AccountType::Margin`). Orders whose instrument falls outside the bootstrapped cache are denied. See [Execution scope](#execution-scope). - **Position reports are always empty for Cash accounts.** Coinbase spot has no positions. Derivatives (CFM) position reports come from `cfm/positions` and appear only on Margin clients. - **User-channel updates omit `price`, `stop_price`, and `trigger_type`.** For orders this client submitted, the missing fields are patched from a cache populated at `submit_order` time. For external orders (submitted by another process or via the Coinbase UI), the user-channel handler enriches the report on first sight by fetching `/orders/historical/{venue_order_id}` and caching the result. The REST call adds latency to the first user-channel update for an external order; subsequent updates use the cached enrichment. - **Cancel-all and batch-cancel REST list failures are logged only.** If the list-open-orders REST call fails, no per-order `OrderCancelRejected` is emitted; orders remain in `PendingCancel` until the next reconciliation recovers them. Mirrors the Bybit adapter pattern. - **Newly listed products require a reconnect to be tradeable.** The instrument cache is populated on connect; products listed after that are not in the cache and `submit_order` will deny them. - **MARKET orders default to IOC.** A `MarketOrder` constructed with the Nautilus default `TimeInForce::Gtc` is mapped to `market_market_ioc` at the venue. Explicit `TimeInForce::Ioc` is honoured; `TimeInForce::Fok` routes to `market_market_fok` but is rejected at runtime by the matching engine on spot with `UNSUPPORTED_ORDER_CONFIGURATION` (the wire shape is documented in the API spec but only accepted on CFM derivatives). `Day` and `Gtd` are rejected at submit time. ## Authenticated binaries Two binaries assist with live verification and account hygiene: - `coinbase-http-private` lists portfolios, prints wallet balances, runs `/orders/preview` for `BTC-USD` and `BTC-USDC`, and surfaces per-product gating flags. Recommended first stop when bringing a new account online. - `coinbase-cancel-all-open` cancels every open order on the authenticated CDP key. Useful between test runs to clear resting orders. Both read `COINBASE_API_KEY` and `COINBASE_API_SECRET` from the environment. ## Contributing :::info For additional features or to contribute to the Coinbase adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Databento Source: https://nautilustrader.io/docs/latest/integrations/databento/ NautilusTrader includes an adapter for the [Databento](https://databento.com/) API and for data in [Databento Binary Encoding (DBN)](https://databento.com/docs/standards-and-conventions/databento-binary-encoding). Databento is a market data provider only. The adapter does not include an execution client, but you can pair it with a sandbox for simulated execution. You can also match Databento data with Interactive Brokers execution, or calculate traditional asset class signals for crypto trading. The adapter supports: - Loading historical data from DBN files and decoding to Nautilus objects for backtesting or catalog storage. - Requesting historical data decoded to Nautilus objects for live trading and backtesting. - Subscribing to real-time data feeds decoded to Nautilus objects for live trading and sandbox environments. :::tip [Databento](https://databento.com/signup) offers $125 in free data credits for new sign-ups. Databento currently allows those credits for historical data or toward the first month of a subscription plan. With careful requests, this covers testing and evaluation. Check the [/metadata.get_cost](https://databento.com/docs/api-reference-historical/metadata/metadata-get-cost) endpoint before requesting data. ::: ## Overview The adapter uses the [databento-rs](https://crates.io/crates/databento) crate, Databento's official Rust client library. :::info You do not need to install `databento` separately. The adapter compiles as a static library and links automatically during the build. ::: The following adapter classes are available: - `DatabentoDataLoader`: Loads DBN data from files. - `DatabentoInstrumentProvider`: Fetches latest or historical instrument definitions via the Databento HTTP API. - `DatabentoHistoricalClient`: Fetches historical market data via the Databento HTTP API. - `DatabentoLiveClient`: Subscribes to real-time data feeds via Databento's raw TCP API. - `DatabentoDataClient`: `LiveMarketDataClient` implementation for live trading nodes. :::info Most users configure a live trading node (covered below) and do not work with these components directly. ::: ## Examples See the [live examples](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/databento/). Rust examples live under [`crates/adapters/databento/examples/`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/databento/examples/). The data tester subscribes to live quotes and trades for the configured instrument when run: ```bash cargo run --example databento-data-tester --package nautilus-databento ``` ## Databento documentation See the [Databento new users guide](https://databento.com/docs/quickstart/new-user-guides). Refer to it alongside this integration guide. ## Databento Binary Encoding (DBN) Databento Binary Encoding (DBN) is a fast message encoding and storage format for normalized market data. The [DBN spec](https://databento.com/docs/standards-and-conventions/databento-binary-encoding) includes a self-describing metadata header and a fixed set of struct definitions that standardize how market data is normalized. The adapter decodes DBN data to Nautilus objects. The same Rust decoder handles: - Loading and decoding DBN files from disk. - Decoding historical and live data in real time. ## Supported schemas The following Databento schemas are supported by NautilusTrader: | Databento schema | Nautilus data type | Description | |:------------------------------------------------------------------------------|:----------------------------------|:--------------------------------| | [MBO](https://databento.com/docs/schemas-and-data-formats/mbo) | `OrderBookDelta` | Market by order (L3). | | [MBP_1](https://databento.com/docs/schemas-and-data-formats/mbp-1) | `(QuoteTick, TradeTick \| None)` | Market by price (L1). | | [MBP_10](https://databento.com/docs/schemas-and-data-formats/mbp-10) | `OrderBookDepth10` | Market depth (L2). | | [BBO_1S](https://databento.com/docs/schemas-and-data-formats/bbo-1s) | `QuoteTick` | 1-second best bid/offer. | | [BBO_1M](https://databento.com/docs/schemas-and-data-formats/bbo-1m) | `QuoteTick` | 1-minute best bid/offer. | | [CMBP_1](https://databento.com/docs/schemas-and-data-formats/cmbp-1) | `(QuoteTick, TradeTick \| None)` | Consolidated MBP across venues. | | [CBBO_1S](https://databento.com/docs/schemas-and-data-formats/cbbo-1s) | `QuoteTick` | Consolidated 1-second BBO. | | [CBBO_1M](https://databento.com/docs/schemas-and-data-formats/cbbo-1m) | `QuoteTick` | Consolidated 1-minute BBO. | | [TCBBO](https://databento.com/docs/schemas-and-data-formats/tcbbo) | `(QuoteTick, TradeTick)` | Trade‑sampled consolidated BBO. | | [TBBO](https://databento.com/docs/schemas-and-data-formats/tbbo) | `(QuoteTick, TradeTick)` | Trade‑sampled best bid/offer. | | [TRADES](https://databento.com/docs/schemas-and-data-formats/trades) | `TradeTick` | Trade ticks. | | [OHLCV_1S](https://databento.com/docs/schemas-and-data-formats/ohlcv-1s) | `Bar` | 1-second bars. | | [OHLCV_1M](https://databento.com/docs/schemas-and-data-formats/ohlcv-1m) | `Bar` | 1-minute bars. | | [OHLCV_1H](https://databento.com/docs/schemas-and-data-formats/ohlcv-1h) | `Bar` | 1-hour bars. | | [OHLCV_1D](https://databento.com/docs/schemas-and-data-formats/ohlcv-1d) | `Bar` | Daily bars. | | [DEFINITION](https://databento.com/docs/schemas-and-data-formats/definition) | `Instrument` (various types) | Instrument definitions. | | [IMBALANCE](https://databento.com/docs/schemas-and-data-formats/imbalance) | `DatabentoImbalance` | Auction imbalance data. | | [STATISTICS](https://databento.com/docs/schemas-and-data-formats/statistics) | `DatabentoStatistics` | Market statistics. | | [STATUS](https://databento.com/docs/schemas-and-data-formats/status) | `InstrumentStatus` | Market status updates. | :::note Databento also documents reference schemas, including corporate actions, adjustment factors, and security master data. This adapter currently maps only the schemas listed above to Nautilus data types. Daily Databento OHLCV uses `ohlcv-1d`. Official settlement prices and open interest come from the `statistics` schema, not OHLCV bars. ::: :::info Instrument definitions for unsupported `instrument_class` values (`'I'` Index, `'B'` Bond) are skipped with a warning rather than aborting the batch. FX spot definitions with currencies that Nautilus cannot map are also skipped. Index-emitting publishers include CGIF.TITANIUM (110), IEX Options (108), and MEMX MX2 (109). Open an issue if you need Nautilus modeling for these. Statistics messages with `stat_type` values outside the modeled range (currently 1-20) are also skipped with a warning. This includes venue-specific values `VenueSpecificVolume1` (10001) and `VenueSpecificPrice1` (10002), which exceed the `u8` Arrow column width used for persistence. ::: ### Schema considerations - **TBBO and TCBBO**: Trade-sampled feeds that pair every trade with the BBO immediately *before* the trade's effect. Use them for trades aligned with contemporaneous quotes without managing two streams. - **MBP-1 and CMBP-1 (L1)**: Event-level updates that emit trades only on trade events. Choose them for a complete top-of-book event tape. For quote and trade alignment, prefer TBBO or TCBBO. - **MBP-10 (L2)**: Top 10 levels with trades. Use it for depth-aware strategies that do not need full MBO data. Includes orders per level. Databento order book depth subscriptions support only `depth=10`. - **MBO (L3)**: Per-order events for queue position modeling and exact book reconstruction. Start at node initialization for proper replay context. - **BBO_1S/BBO_1M and CBBO_1S/CBBO_1M**: Sampled top-of-book updates at fixed intervals (1s or 1m). The adapter emits `QuoteTick` only for these schemas. Use them for monitoring, spreads, and low-cost signals. They are not suited for microstructure work. - **TRADES**: Trades only. Pair with MBP-1 (`include_trades=True`) or use TBBO or TCBBO for quote context with trades. - **OHLCV**: Aggregated bars from trades. Use them for higher-timeframe analytics. Set `bars_timestamp_on_close=True` for close timestamps. Daily bars use `ohlcv-1d`; use `statistics` for official settlements and open interest. - **Imbalance and statistics**: Venue operational data. Subscribe via `subscribe_data` with a `DataType` carrying `instrument_id` metadata. - **Status**: Venue trading-state updates. Subscribe via `subscribe_instrument_status`. :::tip Consolidated schemas (CMBP_1, CBBO_1S, CBBO_1M, TCBBO) aggregate data across multiple venues. Useful for cross-venue analysis. ::: :::info See also the Databento [Schemas and data formats](https://databento.com/docs/schemas-and-data-formats) guide. ::: ## Dataset availability and selection Databento dataset IDs are separate from Nautilus venue identifiers. The adapter supports the schemas listed above, but each Databento dataset exposes its own subset. Check the metadata endpoints before adding a new dataset or schema to a live configuration: ```bash databento_auth="$(printf '%s:' "$DATABENTO_API_KEY" | base64 | tr -d '\n')" curl --header "Authorization: Basic ${databento_auth}" \ "https://hist.databento.com/v0/metadata.list_schemas?dataset=EQUS.MINI" curl --header "Authorization: Basic ${databento_auth}" \ "https://hist.databento.com/v0/metadata.list_unit_prices?dataset=EQUS.MINI" curl --header "Authorization: Basic ${databento_auth}" \ "https://hist.databento.com/v0/metadata.get_cost" \ --data-urlencode "dataset=EQUS.MINI" \ --data-urlencode "symbols=AAPL" \ --data-urlencode "stype_in=raw_symbol" \ --data-urlencode "schema=bbo-1s" \ --data-urlencode "start=2026-06-24T14:30:00Z" \ --data-urlencode "end=2026-06-24T14:31:00Z" ``` For the two common evaluation datasets: - `GLBX.MDP3` is the CME Globex MDP 3.0 dataset for CME, CBOT, NYMEX, and COMEX futures, options on futures, and spreads. It supports MBO, MBP-1, MBP-10, TBBO, trades, BBO intervals, OHLCV, definitions, statistics, and status. It does not expose the consolidated equity schemas (`cmbp-1`, `cbbo-*`, or `tcbbo`). - `EQUS.MINI` is Databento US Equities Mini. It is a derived aggregated top-of-book dataset with anonymized component venues. It supports MBP-1, TBBO, trades, BBO intervals, OHLCV, and definitions. It does not support MBO, MBP-10, imbalance, statistics, status, or consolidated schemas. Use `EQUS` as the Nautilus venue for US Equities Mini instruments: `AAPL.EQUS`, `MSFT.EQUS`, and so on. The built-in venue-to-dataset map routes `EQUS` to `EQUS.MINI`. Venue codes such as `XNAS` and `XNYS` refer to venue-specific datasets unless you override them with `venue_dataset_map`. :::warning If you override a venue such as `XNAS` to `EQUS.MINI`, keep downstream instrument IDs consistent. Mini records carry the consolidated `EQUS` publisher, and file or historical decoding without an explicit `instrument_id` emits `*.EQUS` identifiers. ::: Cost depends on the schema, symbols, and time range. For exploration, start with tight ranges, `definition`, `bbo-1s`, `bbo-1m`, or `trades`, and call `metadata.get_cost` before pulling historical time series data. Avoid duplicate quote and trade subscriptions when a combined schema such as `mbp-1` or `tbbo` already carries the data needed by the strategy. ## Schema selection for live subscriptions Nautilus subscription methods map to Databento schemas as follows: | Nautilus subscription method | Default schema | Available Databento schemas | Nautilus data type | |:--------------------------------|:---------------|:-----------------------------------------------------------------------------|:-------------------| | `subscribe_quote_ticks()` | `mbp-1` | `mbp-1`, `bbo-1s`, `bbo-1m`, `cmbp-1`, `cbbo-1s`, `cbbo-1m`, `tbbo`, `tcbbo` | `QuoteTick` | | `subscribe_trade_ticks()` | `trades` | `trades`, `tbbo`, `tcbbo`, `mbp-1`, `cmbp-1` | `TradeTick` | | `subscribe_order_book_depth()` | `mbp-10` | `mbp-10` | `OrderBookDepth10` | | `subscribe_order_book_deltas()` | `mbo` | `mbo` | `OrderBookDeltas` | | `subscribe_bars()` | varies | `ohlcv-1s`, `ohlcv-1m`, `ohlcv-1h`, `ohlcv-1d` | `Bar` | :::warning The "Available Databento schemas" column lists adapter-supported choices for that Nautilus subscription method. The selected dataset must also support the schema. For example, `EQUS.MINI` cannot serve `mbo`, `mbp-10`, `statistics`, or `status`. ::: :::note The examples below assume a `Strategy` or `Actor` context where `self` has subscription methods. Import the required types: ```python from nautilus_trader.adapters.databento import DATABENTO_CLIENT_ID from nautilus_trader.model import BarType from nautilus_trader.model.enums import BookType from nautilus_trader.model.identifiers import InstrumentId ``` ::: ### Quote subscriptions (MBP and L1) ```python # Default MBP-1 quotes (may include trades) self.subscribe_quote_ticks(instrument_id, client_id=DATABENTO_CLIENT_ID) # Explicit MBP-1 schema self.subscribe_quote_ticks( instrument_id=instrument_id, params={"schema": "mbp-1"}, client_id=DATABENTO_CLIENT_ID, ) # 1-second BBO snapshots (adapter emits QuoteTick only) self.subscribe_quote_ticks( instrument_id=instrument_id, params={"schema": "bbo-1s"}, client_id=DATABENTO_CLIENT_ID, ) # Consolidated quotes across venues self.subscribe_quote_ticks( instrument_id=instrument_id, params={"schema": "cbbo-1s"}, # or "cmbp-1" for consolidated MBP client_id=DATABENTO_CLIENT_ID, ) # Trade-sampled BBO (includes quotes and trades) self.subscribe_quote_ticks( instrument_id=instrument_id, params={"schema": "tbbo"}, # Receives QuoteTick and TradeTick on the message bus client_id=DATABENTO_CLIENT_ID, ) ``` ### Trade subscriptions ```python # Trade ticks only self.subscribe_trade_ticks(instrument_id, client_id=DATABENTO_CLIENT_ID) # Trades from MBP-1 feed (only when trade events occur) self.subscribe_trade_ticks( instrument_id=instrument_id, params={"schema": "mbp-1"}, client_id=DATABENTO_CLIENT_ID, ) # Trade-sampled data (includes quotes at trade time) self.subscribe_trade_ticks( instrument_id=instrument_id, params={"schema": "tbbo"}, # Also provides quotes at trade events client_id=DATABENTO_CLIENT_ID, ) ``` ### Order book depth subscriptions (MBP and L2) ```python # Subscribe to top 10 levels of market depth self.subscribe_order_book_depth( instrument_id=instrument_id, depth=10 # MBP-10 schema is automatically selected ) # The depth parameter must be 10 for Databento # Receives OrderBookDepth10 updates ``` ### Order book deltas subscriptions (MBO and L3) ```python # Subscribe to full order book updates (market by order) self.subscribe_order_book_deltas( instrument_id=instrument_id, book_type=BookType.L3_MBO # Uses MBO schema ) # Make MBO subscriptions at node startup so Databento can replay from session start ``` ### Bar subscriptions ```python # Subscribe to 1-minute bars (automatically uses ohlcv-1m schema) self.subscribe_bars( bar_type=BarType.from_str(f"{instrument_id}-1-MINUTE-LAST-EXTERNAL") ) # Subscribe to 1-second bars (automatically uses ohlcv-1s schema) self.subscribe_bars( bar_type=BarType.from_str(f"{instrument_id}-1-SECOND-LAST-EXTERNAL") ) # Subscribe to hourly bars (automatically uses ohlcv-1h schema) self.subscribe_bars( bar_type=BarType.from_str(f"{instrument_id}-1-HOUR-LAST-EXTERNAL") ) # Subscribe to daily bars (automatically uses ohlcv-1d schema) self.subscribe_bars( bar_type=BarType.from_str(f"{instrument_id}-1-DAY-LAST-EXTERNAL") ) ``` ### Custom data type subscriptions Imbalance and statistics data require the generic `subscribe_data` method: ```python from nautilus_trader.adapters.databento import DATABENTO_CLIENT_ID from nautilus_trader.adapters.databento import DatabentoImbalance from nautilus_trader.adapters.databento import DatabentoStatistics from nautilus_trader.model import DataType # Subscribe to imbalance data self.subscribe_data( data_type=DataType(DatabentoImbalance, metadata={"instrument_id": instrument_id}), client_id=DATABENTO_CLIENT_ID, ) # Subscribe to statistics data self.subscribe_data( data_type=DataType(DatabentoStatistics, metadata={"instrument_id": instrument_id}), client_id=DATABENTO_CLIENT_ID, ) ``` Instrument status uses the dedicated status subscription API: ```python # Subscribe to instrument status updates self.subscribe_instrument_status( instrument_id=instrument_id, client_id=DATABENTO_CLIENT_ID, ) ``` ## Instrument IDs and symbology Databento market data includes an `instrument_id` field: a numeric ID assigned by the publisher in most cases, or synthesized by Databento when the publisher does not provide one. Databento only guarantees this ID is unique within a given day. This differs from the Nautilus `InstrumentId`, a string of symbol + venue separated by a period: `"{symbol}.{venue}"`. The decoder maps the Databento `raw_symbol` to the Nautilus `symbol`. Publisher IDs map to the default Nautilus venue through `publishers.json`. Subscription `InstrumentId` metadata can also seed the symbol-to-venue map before market data arrives. Databento identifies datasets with a *dataset ID*, separate from venue identifiers. See [Databento dataset naming conventions](https://databento.com/docs/api-reference-historical/basics/datasets) for details. For historical requests and live subscriptions, the adapter sends the Nautilus symbol portion of each `InstrumentId` as the Databento symbol and infers `stype_in` from that string: - Symbols ending in `.FUT` or `.OPT` use Databento parent symbology, for example `ES.FUT.XCME`. - Three-part symbols whose last part is numeric use continuous symbology, for example `ES.c.0.GLBX`. - All-numeric symbols use Databento `instrument_id` symbology. - All other symbols use raw symbol symbology, for example `ESZ6.XCME` or `AAPL.EQUS`. All symbols in one request or subscription must use the same symbology type. Batch `AAPL.EQUS` with `MSFT.EQUS`, or `ES.FUT.XCME` with `NQ.FUT.XCME`, but do not mix raw and parent symbols in one Databento request. For CME Globex MDP 3.0 (`GLBX.MDP3`), publisher defaults map to the `GLBX` venue. When `use_exchange_as_venue=True`, definition messages can override `GLBX` with the instrument's exchange MIC: - `CBCM`: XCME-XCBT inter-exchange spread - `NYUM`: XNYM-DUMX inter-exchange spread - `XCBT`: Chicago Board of Trade (CBOT) - `XCEC`: Commodities Exchange Center (COMEX) - `XCME`: Chicago Mercantile Exchange (CME) - `XFXS`: CME FX Link spread - `XNYM`: New York Mercantile Exchange (NYMEX) :::info Other venue MICs are in the `venue` field of responses from the [metadata.list_publishers](https://databento.com/docs/api-reference-historical/metadata/metadata-list-publishers) endpoint. ::: ## Timestamps Databento data includes these timestamp fields: - `ts_event`: Matching-engine-received timestamp in nanoseconds since the UNIX epoch. - `ts_in_delta`: Matching-engine-sending timestamp in nanoseconds before `ts_recv`. - `ts_recv`: Capture-server-received timestamp in nanoseconds since the UNIX epoch. - `ts_out`: Databento sending timestamp (live only). Nautilus data requires at least two timestamps (per the `Data` contract): - `ts_event`: UNIX timestamp (nanoseconds) when the data event occurred. - `ts_init`: UNIX timestamp (nanoseconds) when the data instance was created. Quote and trade-like schemas map Databento `ts_recv` to Nautilus `ts_event` because it is more reliable and monotonically increases per Databento symbol. Bars use the DBN bar interval timestamp; `bars_timestamp_on_close` controls whether Nautilus bars use the interval open or close timestamp. `InstrumentStatus` uses the status event timestamp from the decoded status message. `DatabentoImbalance` and `DatabentoStatistics` preserve Databento timestamp fields because they are adapter-specific types. :::info See these Databento docs for details: - [Databento standards and conventions - timestamps](https://databento.com/docs/standards-and-conventions/common-fields-enums-types#timestamps) - [Databento timestamping guide](https://databento.com/docs/architecture/timestamping-guide) ::: ## Data types This section maps Databento schemas to Nautilus data types. :::info See Databento [schemas and data formats](https://databento.com/docs/schemas-and-data-formats). ::: ### Instrument definitions Databento uses a single schema for all instrument classes. The decoder maps each to the appropriate Nautilus `Instrument` type. | Databento instrument class | Code | Nautilus instrument type | |----------------------------|------|--------------------------| | Stock | `K` | `Equity` | | Future | `F` | `FuturesContract` | | Call | `C` | `OptionContract` | | Put | `P` | `OptionContract` | | Future spread | `S` | `FuturesSpread` | | Option spread | `T` | `OptionSpread` | | Mixed spread | `M` | `OptionSpread` | | FX spot | `X` | `CurrencyPair` | | Index | `I` | Not yet available | | Bond | `B` | Not yet available | ### Option expiration correction OPRA option definitions (dataset `OPRA.PILLAR`) carry the expiration with date-level precision: the time-of-day is zeroed to midnight UTC. An option expiring at 16:00 New York time therefore arrives stamped on the prior evening in New York, which makes the matching engine treat the contract as expired before its final trading session. The loader corrects such midnight-UTC OPRA expirations to 16:00 New York time by default, leaving every other dataset (and any expiration that already carries an intraday time, such as CME Globex) untouched. Override the default, or set per-underlying times, with `expiration_overrides`. It maps a dataset to a mapping of underlying symbol to time, where the reserved key `default` sets the dataset-wide time: ```python loader.from_dbn_file( path, expiration_overrides={ "OPRA.PILLAR": {"default": "16:00", "SPX": "09:30"}, }, ) ``` Times use `HH:MM` or `HH:MM:SS` in the exchange-local timezone (New York for OPRA). Only datasets with a built-in correction rule (currently `OPRA.PILLAR`) can be tuned; an unknown or rule-less dataset such as `GLBX.MDP3` raises a `ValueError`. The correction keys on the option's underlying, so it cannot distinguish series that share an underlying but settle at different times (for example AM-settled SPX versus PM-settled SPXW); set the time that matches the contracts you are loading. ### Price precision Databento raw prices are fixed-point integers scaled by 1e-9. The adapter derives price precision from the instrument's tick size in the definition message. For live feeds, the feed handler maintains a per-instrument precision map populated from `InstrumentDefMsg` records as they arrive. Market data handlers resolve precision in this order: 1. `InstrumentDefMsg` metadata for the Databento record `instrument_id`. 2. Cached instrument precision passed by the Python subscription path. 3. Explicit `price_precisions` passed to the direct live client. 4. The USD default precision of 2. The fallback maps are keyed by Databento record `instrument_id` after symbol mapping, so parent, continuous, and other non-raw symbology requests can still use cached or explicit precision until definition metadata arrives. **Instrument definitions must arrive before market data** for correct precision on instruments with non-standard tick sizes (e.g., treasury futures with fractional ticks like 1/256). Subscribe to `DEFINITION` schema for your instruments before or alongside market data subscriptions. For historical requests and file-based loading, precision is resolved per record in this order: 1. An explicit `price_precision` argument on the call. 2. A per-symbol cache populated by loading definitions (`load_instruments` on the file loader, `get_range_instruments` on the historical client) or by an explicit `set_price_precision(symbol, precision)` call. The Python data client seeds the historical-client cache from the instrument provider before every request, so already-loaded instruments need no extra configuration. When precision cannot be resolved, loading fails with an explicit error rather than silently defaulting to USD precision. :::tip The Python adapter automatically subscribes to instrument definitions before market data and passes cached instrument precision as a fallback, so the precision map populates without extra configuration. For direct Rust client usage, subscribe to `DEFINITION` schema before market data or pass explicit precision fallbacks. ::: ### MBO (market by order) MBO is the highest granularity data from Databento, representing full order book depth. Some messages include trade data. The decoder produces an `OrderBookDelta` and optionally a `TradeTick`. The live client buffers MBO messages until it sees an `F_LAST` flag, then passes an `OrderBookDeltas` container to the handler. The client also buffers order book snapshots into `OrderBookDeltas` during the replay startup sequence. ### MBP-1 (market by price, top-of-book) MBP-1 represents top-of-book quotes and trades. Some messages carry trade data. The decoder produces a `QuoteTick` and also a `TradeTick` when the message is a trade. ### TBBO and TCBBO (top-of-book with trades) TBBO and TCBBO provide both quote and trade data in each message. Both schemas emit `QuoteTick` and `TradeTick` per message, more efficient than separate quote and trade subscriptions. TCBBO provides consolidated data across venues. #### Trade ID derivation (CMBP-1 and TCBBO) The CMBP-1 and TCBBO schemas do not publish a native trade identifier. The decoder derives a deterministic `TradeId` by FNV-1a hashing the instrument ID, `ts_event`, `ts_recv`, price, size, and aggressor side of the trade. The same venue event yields the same trade ID across replays, so downstream dedup stays intact. Two logically distinct trades with identical fields collide; this matches the venue's inability to distinguish them. ### OHLCV (bar aggregates) Databento timestamps bar messages at the **open** of the interval. By default, the decoder normalizes bar `ts_event` to the bar **close**: the original `ts_event` plus the interval. `ts_init` uses the live receipt time, or the close time for historical and file-based loads when no explicit init timestamp is supplied. Set `bars_timestamp_on_close=False` to timestamp bar `ts_event` on the interval open. ### Imbalance and statistics The `imbalance` and `statistics` schemas have no built-in Nautilus equivalents. The adapter defines `DatabentoImbalance` and `DatabentoStatistics` in Rust. PyO3 bindings expose these types in Python. Their attributes are PyO3 objects and may not work with methods expecting Cython types. See the API reference for PyO3 to Cython conversion methods. Convert a PyO3 `Price` to a Cython `Price`: ```python price = Price.from_raw(pyo3_price.raw, pyo3_price.precision) ``` Requesting and subscribing to these types requires the generic `subscribe_data` method. Subscribe to `imbalance` for `AAPL.XNAS`: ```python from nautilus_trader.adapters.databento import DATABENTO_CLIENT_ID from nautilus_trader.adapters.databento import DatabentoImbalance from nautilus_trader.model import DataType instrument_id = InstrumentId.from_str("AAPL.XNAS") self.subscribe_data( data_type=DataType(DatabentoImbalance, metadata={"instrument_id": instrument_id}), client_id=DATABENTO_CLIENT_ID, ) ``` Request a bounded range of `statistics` for the `ES.FUT` parent symbol (all active E-mini S&P 500 futures). Use Databento's Historical [`metadata.get_cost`](https://databento.com/docs/api-reference-historical/metadata/metadata-get-cost) endpoint before real historical pulls: ```python from nautilus_trader.adapters.databento import DATABENTO_CLIENT_ID from nautilus_trader.adapters.databento import DatabentoStatistics from nautilus_trader.model import DataType instrument_id = InstrumentId.from_str("ES.FUT.GLBX") metadata = { "instrument_id": instrument_id, "start": "2024-03-06", "end": "2024-03-07", } self.request_data( data_type=DataType(DatabentoStatistics, metadata=metadata), client_id=DATABENTO_CLIENT_ID, ) ``` ### Catalog persistence Both types support Arrow serialization for catalog storage. The Arrow serializers register automatically when you import the adapter package. #### Writing to the catalog ```python from nautilus_trader.adapters.databento import DatabentoDataLoader from nautilus_trader.model.identifiers import InstrumentId from nautilus_trader.persistence.catalog import ParquetDataCatalog catalog = ParquetDataCatalog.from_env() loader = DatabentoDataLoader() imbalances = loader.from_dbn_file( path="aapl-imbalance.dbn.zst", instrument_id=InstrumentId.from_str("AAPL.XNAS"), as_legacy_cython=False, # Required for Databento-specific types ) catalog.write_data(imbalances) ``` #### Reading from the catalog ```python from nautilus_trader.adapters.databento import DatabentoImbalance results = catalog.query(DatabentoImbalance, identifiers=["AAPL.XNAS"]) for imbalance in results: print(imbalance.ref_price) # DatabentoImbalance fields ``` :::warning Catalog persistence supports writing and querying these types, but streaming them through `BacktestNode` or `BacktestEngine` is not yet supported. For backtesting with imbalance or statistics data, query the catalog directly and process the results in your strategy or analysis code. ::: #### Encoding and decoding in Rust The `nautilus_databento::arrow` module provides Arrow record batch encoding and decoding. Enable the `arrow` feature flag. ```rust use nautilus_databento::arrow::imbalance::{ decode_imbalance_batch, imbalance_to_arrow_record_batch, }; let batch = imbalance_to_arrow_record_batch(imbalances)?; let metadata = batch.schema().metadata().clone(); let decoded = decode_imbalance_batch(&metadata, batch)?; ``` The `statistics` module follows the same pattern with `decode_statistics_batch` and `statistics_to_arrow_record_batch`. ## Performance considerations Two options for backtesting with DBN data: - Store data as DBN (`.dbn.zst`) files and decode to Nautilus objects every run. - Convert DBN files to Nautilus objects once and write to the data catalog (Nautilus Parquet format). The DBN decoder is optimized Rust, but writing to the catalog once gives the best backtest performance. [DataFusion](https://arrow.apache.org/datafusion/) streams Nautilus Parquet data from disk at high throughput, at least an order of magnitude faster than decoding DBN per run. :::note Performance benchmarks are under development. ::: For live data, decoded delivery from the feed handler to Nautilus is intentionally unbounded. This prevents slow consumers from stalling the feed path; a process under memory pressure should fail rather than block live decoding. ## Loading DBN data The `DatabentoDataLoader` class loads DBN files and converts records to Nautilus objects. Two primary uses: - Pass data to `BacktestEngine.add_data` for backtesting. - Write data to `ParquetDataCatalog` for streaming with a `BacktestNode`. ### DBN data to a BacktestEngine Load DBN data and pass to a `BacktestEngine`. The engine requires an instrument. This example uses `TestInstrumentProvider` (an instrument parsed from a DBN file also works). The data covers one month of TSLA trades on Nasdaq: ```python # Add instrument TSLA_NASDAQ = TestInstrumentProvider.equity(symbol="TSLA") engine.add_instrument(TSLA_NASDAQ) # Decode data to Cython objects loader = DatabentoDataLoader() trades = loader.from_dbn_file( path=TEST_DATA_DIR / "databento" / "temp" / "tsla-xnas-20240107-20240206.trades.dbn.zst", instrument_id=TSLA_NASDAQ.id, ) # Add data engine.add_data(trades) ``` ### DBN data to a ParquetDataCatalog Load DBN data and write to a `ParquetDataCatalog`. Set `as_legacy_cython=False` to decode as PyO3 objects. ### Loading instruments **Important**: Load instrument definitions from DEFINITION schema files before loading market data into a catalog. The catalog requires instruments before it can store market data. Market data files do not contain instrument definitions. ```python # Initialize the catalog interface # (will use the `NAUTILUS_PATH` env var as the path) catalog = ParquetDataCatalog.from_env() loader = DatabentoDataLoader() # Step 1: Load instrument definitions first # Obtain DEFINITION schema files from Databento for your instruments instruments = loader.from_dbn_file( path=TEST_DATA_DIR / "databento" / "temp" / "tsla-xnas-definition.dbn.zst", as_legacy_cython=False, # Use PyO3 for optimal performance ) # Write instruments to catalog catalog.write_data(instruments) # Step 2: Now load and write market data instrument_id = InstrumentId.from_str("TSLA.XNAS") # Decode trades to PyO3 objects trades = loader.from_dbn_file( path=TEST_DATA_DIR / "databento" / "temp" / "tsla-xnas-20240107-20240206.trades.dbn.zst", instrument_id=instrument_id, as_legacy_cython=False, # This is an optimization for writing to the catalog ) # Write market data catalog.write_data(trades) ``` #### Loading multiple data types for backtesting Always load instruments before market data: ```python from nautilus_trader.adapters.databento.loaders import DatabentoDataLoader from nautilus_trader.model.identifiers import InstrumentId from nautilus_trader.persistence.catalog import ParquetDataCatalog catalog = ParquetDataCatalog.from_env() loader = DatabentoDataLoader() # Step 1: Load instrument definitions from DEFINITION files instruments = loader.from_dbn_file( path="equity-definitions.dbn.zst", as_legacy_cython=False, ) catalog.write_data(instruments) # Step 2: Load market data (MBO, trades, quotes, etc.) instrument_id = InstrumentId.from_str("AAPL.XNAS") # Load MBO order book deltas deltas = loader.from_dbn_file( path="aapl-mbo.dbn.zst", instrument_id=instrument_id, # Optional but improves performance as_legacy_cython=False, ) catalog.write_data(deltas) # Load trades trades = loader.from_dbn_file( path="aapl-trades.dbn.zst", instrument_id=instrument_id, as_legacy_cython=False, ) catalog.write_data(trades) # Verify instruments are in the catalog print(catalog.instruments()) # Shows your loaded instruments ``` :::tip Call `catalog.instruments()` to verify. An empty list means you need to load DEFINITION files first. ::: :::info Download DEFINITION schema files through the Databento API or CLI for your symbols and date ranges. See the [Databento documentation](https://databento.com/docs/api-reference-historical/timeseries/timeseries-get-range) for details. ::: :::info See also the [Data concepts guide](../concepts/data.md). ::: ### Historical loader options Parameters for `from_dbn_file`: - `instrument_id`: Speeds up decoding by skipping symbology lookup. - `price_precision`: Override applied to every record read. When omitted, the loader resolves precision per symbol from its cache (populated by `load_instruments` or `set_price_precision`); loading fails if unresolved. - `include_trades`: For MBP-1/CMBP-1 schemas, `True` emits both `QuoteTick` and `TradeTick` when trade data is present. - `as_legacy_cython`: Set to `False` for IMBALANCE/STATISTICS schemas (required) or for better catalog write performance. :::warning IMBALANCE and STATISTICS schemas require `as_legacy_cython=False` (PyO3-only types). `True` raises a `ValueError`. ::: ### Loading consolidated data Consolidated schemas aggregate data across multiple venues: ```python # Load consolidated MBP-1 quotes loader = DatabentoDataLoader() cmbp_quotes = loader.from_dbn_file( path="consolidated.cmbp-1.dbn.zst", instrument_id=InstrumentId.from_str("AAPL.XNAS"), include_trades=True, # Includes both quotes and trades if available as_legacy_cython=True, ) # Load consolidated BBO quotes cbbo_quotes = loader.from_dbn_file( path="consolidated.cbbo-1s.dbn.zst", instrument_id=InstrumentId.from_str("AAPL.XNAS"), as_legacy_cython=False, # Use PyO3 for better performance ) # Load TCBBO (trade-sampled consolidated BBO) with quotes and trades # include_trades=True loads quotes, include_trades=False loads trades tcbbo_quotes = loader.from_dbn_file( path="consolidated.tcbbo.dbn.zst", instrument_id=InstrumentId.from_str("AAPL.XNAS"), include_trades=True, # Loads quotes as_legacy_cython=True, ) tcbbo_trades = loader.from_dbn_file( path="consolidated.tcbbo.dbn.zst", instrument_id=InstrumentId.from_str("AAPL.XNAS"), include_trades=False, # Loads trades as_legacy_cython=True, ) ``` :::tip Avoid subscribing to both TBBO/TCBBO and separate trade feeds for the same instrument. These schemas already include trades. Duplicating wastes cost and creates duplicate data. ::: ## Real-time client architecture The `DatabentoDataClient` wraps the other Databento adapter classes. Each dataset uses two `DatabentoLiveClient` instances: - One for MBO (order book deltas) real-time feeds - One for all other real-time feeds :::warning Make all MBO subscriptions for a dataset at node startup to replay from session start. The client logs subscriptions after start as errors and ignores them. This limitation does not apply to other schemas. ::: A single `DatabentoHistoricalClient` serves both `DatabentoInstrumentProvider` and `DatabentoDataClient` for historical requests. ## Configuration Add a `DATABENTO` section to your `TradingNode` client configuration. Load specific instruments; the adapter does not support `load_all=True` for Databento datasets because a dataset can contain millions of definitions. ```python from nautilus_trader.adapters.databento import DATABENTO from nautilus_trader.config import InstrumentProviderConfig from nautilus_trader.config import TradingNodeConfig from nautilus_trader.model.identifiers import InstrumentId instrument_ids = [ InstrumentId.from_str("ESZ6.XCME"), # GLBX.MDP3 # InstrumentId.from_str("AAPL.EQUS"), # EQUS.MINI ] config = TradingNodeConfig( data_clients={ DATABENTO: { "api_key": None, # 'DATABENTO_API_KEY' env var "http_gateway": None, # Override for the default HTTP historical gateway "live_gateway": None, # Override for the default raw TCP real-time gateway "instrument_provider": InstrumentProviderConfig( load_ids=frozenset(instrument_ids), ), "instrument_ids": instrument_ids, # Definitions to load on start "parent_symbols": {"GLBX.MDP3": {"ES.FUT"}}, # Optional definition trees }, }, ) ``` Create the `TradingNode` and register the factory: ```python from nautilus_trader.adapters.databento.factories import DatabentoLiveDataClientFactory from nautilus_trader.live.node import TradingNode # Create the live trading node with the configuration node = TradingNode(config=config) # Register the client factory with the node node.add_data_client_factory(DATABENTO, DatabentoLiveDataClientFactory) # Build the node node.build() ``` ### Configuration parameters | Option | Default | Description | |---------------------------|---------|-----------------------------------------------------------------------| | `api_key` | `None` | Databento API secret; falls back to `DATABENTO_API_KEY`. | | `http_gateway` | `None` | Historical HTTP endpoint override, mainly for tests. | | `live_gateway` | `None` | Live TCP endpoint override, mainly for tests. | | `instrument_provider` | default | Provider settings; use `load_ids`, not `load_all=True`. | | `use_exchange_as_venue` | `True` | Use exchange MIC venues for GLBX definitions. | | `timeout_initial_load` | `15.0` | Definition load timeout per dataset, in seconds. | | `mbo_subscriptions_delay` | `3.0` | Delay before starting MBO/L3 streams, in seconds. | | `bars_timestamp_on_close` | `True` | Use bar close time for `ts_event`; `False` uses open. | | `reconnect_timeout_mins` | `10` | Retry window in minutes; `None` retries indefinitely. | | `venue_dataset_map` | `None` | Override venue‑to‑dataset mappings. | | `parent_symbols` | `None` | Preload parent definition trees by dataset. | | `instrument_ids` | `None` | Definitions to preload at startup. | :::tip Use environment variables for credentials. ::: ### Connection stability The live client reconnects automatically on: - **Network interruptions**: Temporary connectivity issues. - **Gateway restarts**: Databento scheduled live gateway restarts. See the [maintenance schedule](https://databento.com/docs/api-reference-live/basics#maintenance-schedule). - **Market closures**: Sessions ending during off-hours. #### Reconnection strategy Backoff strategy depends on the timeout configuration: **With timeout** (default 10 minutes): - Exponential backoff capped at **60 seconds**. - Pattern: 1s, 2s, 4s, 8s, 16s, 32s, 60s, 60s, and so on (with jitter). - Reconnects quickly within the timeout window. **Without timeout** (`reconnect_timeout_mins=None`): - Exponential backoff capped at **10 minutes**. - Pattern: 1s, 2s, 4s, 8s, 16s, 32s, 64s, 128s, 256s, 512s, 600s, 600s, and so on (with jitter). - Suited for unattended systems through overnight closures and scheduled maintenance. All reconnections include: - **Jitter**: Random delay (up to 1 second) to prevent simultaneous reconnection storms. - **Automatic resubscription**: Restores all active subscriptions after reconnecting. - **Cycle reset**: Each successful session (>60s) resets the timeout clock. Individual unsubscribe requests log a warning and are ignored because Databento live sessions do not support granular unsubscribe. Stop the session to remove a subscription from the live gateway. #### Timeout configuration The `reconnect_timeout_mins` parameter controls how long the client attempts reconnection: **Default (10 minutes)**: Suitable for most use cases. - Handles transient network issues. - Survives scheduled gateway restarts. - Stops retrying overnight when markets close. - Requires manual intervention for longer outages. :::warning Setting `reconnect_timeout_mins=None` retries indefinitely. Use only for unattended systems that must survive overnight market closures. This can mask persistent configuration or authentication issues. ::: #### Scheduled maintenance Databento restarts live gateways on this schedule (all clients disconnect): | Dataset | Restart time | |--------------------|-------------------| | CME Globex | Saturday 02:15 CT | | All ICE venues | Sunday 09:45 UTC | | All other datasets | Sunday 10:30 UTC | The default 10-minute timeout covers typical restarts. For unattended systems, use `reconnect_timeout_mins=None` or a longer value. See the [Databento Maintenance Schedule](https://databento.com/docs/api-reference-live/basics/maintenance-schedule) for details. ## Contributing :::info To contribute, see the [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Deribit Source: https://nautilustrader.io/docs/latest/integrations/deribit/ Founded in 2016, Deribit is a cryptocurrency derivatives exchange for options, futures, perpetuals, spot, and combo instruments. It is one of the largest crypto options exchanges by volume, and a leading platform for crypto derivatives trading. This integration supports live market data ingest and order execution with Deribit. ## Overview This adapter is implemented in Rust, with optional Python bindings for use in Python-based workflows. Deribit uses JSON-RPC 2.0 over both HTTP and WebSocket transports. WebSocket is preferred for subscriptions and real-time data. The official Deribit API reference can be found at [docs.deribit.com](https://docs.deribit.com/). The Deribit adapter includes multiple components, which can be used together or separately depending on your use case: - `DeribitHttpClient`: Low-level HTTP API connectivity (JSON-RPC over HTTP). - `DeribitWebSocketClient`: Low-level WebSocket API connectivity (JSON-RPC over WebSocket). - `DeribitInstrumentProvider`: Instrument parsing and loading functionality. - `DeribitDataClient`: Market data feed manager. - `DeribitExecutionClient`: Account management and trade execution gateway. - `DeribitDataClientFactory`: Factory for Deribit data clients (used by the live node builder). - `DeribitExecutionClientFactory`: Factory for Deribit execution clients (used by the live node builder). :::note Most users will define a configuration for a live trading node (as shown below), and won't need to work directly with these lower-level components. ::: ### Product support | Product type | Data feed | Trading | Notes | |-------------------|-----------|---------|------------------------------------------------| | Perpetual futures | ✓ | ✓ | Loaded with `DeribitProductType.FUTURE`. | | Dated futures | ✓ | ✓ | Loaded with `DeribitProductType.FUTURE`. | | Options | ✓ | ✓ | Loaded with `DeribitProductType.OPTION`. | | Spot | ✓ | ✓ | Loaded with `DeribitProductType.SPOT`. | | Future combos | ✓ | ✓ | Loaded with `DeribitProductType.FUTURE_COMBO`. | | Option combos | ✓ | ✓ | Loaded with `DeribitProductType.OPTION_COMBO`. | ## Symbology Deribit uses specific symbol conventions for different instrument types. All instrument IDs should include the `.DERIBIT` suffix when referencing them (e.g., `BTC-PERPETUAL.DERIBIT` for BTC perpetual). ### Quantity units Nautilus quantities map to Deribit's `amount` field, not the optional `contracts` field. Deribit reports perpetual and inverse futures amounts in USD units, and reports options and linear futures amounts in the underlying base currency. The Deribit `contract_size` field converts between `amount` and contract count; the adapter does not apply it again as the Nautilus multiplier. ### Perpetual futures Format: `{Currency}-PERPETUAL` Examples: - `BTC-PERPETUAL` - Bitcoin perpetual swap. - `ETH-PERPETUAL` - Ethereum perpetual swap. To subscribe to BTC perpetual in your strategy: ```python InstrumentId.from_str("BTC-PERPETUAL.DERIBIT") ``` ### Dated futures Format: `{Currency}-{DDMMMYY}` Examples: - `BTC-25DEC26` - Bitcoin future expiring December 25, 2026. - `ETH-26MAR27` - Ethereum future expiring March 26, 2027. ```python InstrumentId.from_str("BTC-25DEC26.DERIBIT") ``` ### Options Format: `{Currency}-{DDMMMYY}-{Strike}-{Type}` Examples: - `BTC-25DEC26-100000-C` - Bitcoin call option, $100,000 strike, expiring December 25, 2026. - `BTC-25DEC26-80000-P` - Bitcoin put option, $80,000 strike, expiring December 25, 2026. - `ETH-26MAR27-4000-C` - Ethereum call option, $4,000 strike, expiring March 26, 2027. Where: - `C` = Call option. - `P` = Put option. ```python InstrumentId.from_str("BTC-25DEC26-100000-C.DERIBIT") ``` ### Spot Format: `{BaseCurrency}_{QuoteCurrency}` Examples: - `BTC_USDC` - Bitcoin against USDC. - `ETH_USDC` - Ethereum against USDC. ```python InstrumentId.from_str("BTC_USDC.DERIBIT") ``` ### Future combos Format: `{Currency}-FS-{LegA}_{LegB}` Legs are dated futures or the perpetual (denoted `PERP` inside combo names, even though the standalone instrument is `BTC-PERPETUAL`). The combo expires with its earliest leg. Examples: - `BTC-FS-25DEC26_PERP` - calendar spread between the December 2026 future and the perpetual. - `BTC-FS-26MAR27_25DEC26` - inter-month spread between two dated futures. ```python InstrumentId.from_str("BTC-FS-25DEC26_PERP.DERIBIT") ``` The adapter models future combos as `CryptoFuturesSpread`, priced in USD as the spread between legs, with crypto settlement currency and `is_inverse` set per the upstream `instrument_type`. ### Option combos Format: `{Currency}-{Strategy}-{DDMMMYY}-{Strikes}` Strategy codes include CS (call spread), PS (put spread), STRG (strangle), STRD (straddle), BOX (box), and RR (risk reversal). The strikes segment separates multiple strikes with `_`. Examples: - `BTC-CS-25DEC26-70000_75000` - 70k / 75k call spread expiring December 25, 2026. - `BTC-STRG-26MAR27-72000_80000` - 72k / 80k strangle expiring March 26, 2027. - `BTC-STRD-26MAR27-77000` - 77k straddle expiring March 26, 2027. - `BTC-BOX-26MAR27-58000_60000` - 58k / 60k box expiring March 26, 2027. ```python InstrumentId.from_str("BTC-STRG-26MAR27-72000_80000.DERIBIT") ``` The adapter models option combos as `CryptoOptionSpread`, priced in the base currency under Deribit's inverse-option convention; fractional `size_increment` (e.g. `0.1`) is preserved end-to-end. ## Traded expirations Deribit exposes active traded expirations through the `public/get_expirations` HTTP endpoint. Option-chain loaders can use the high-level HTTP client to refresh active option series without scanning every instrument. ```rust tab="Rust" use nautilus_deribit::http::models::DeribitCurrency; let expirations = client .request_option_expirations(DeribitCurrency::BTC) .await?; ``` ```python tab="Python" from nautilus_trader.adapters.deribit import DeribitCurrency from nautilus_trader.adapters.deribit import DeribitHttpClient client = DeribitHttpClient() expirations = await client.request_option_expirations(DeribitCurrency.BTC) ``` The high-level method returns option expirations only. For lower-level Rust requests, call `client.inner().get_expirations(...)` with `GetExpirationsParams`. Deribit returns a currency-keyed result for concrete currencies such as `BTC`, and a direct kind-keyed result for `currency=any`; the adapter handles both shapes. ## Combo instruments The instrument provider loads combos when `product_types` includes the future-combo or option-combo enum variant. In Python, use `DeribitProductType.FUTURE_COMBO` or `DeribitProductType.OPTION_COMBO`. Deribit exposes the leg makeup of every active combo on `/public/get_combos`, and the combo's trading metadata (tick size, contract size, expiration, min trade amount) on the standard `/public/get_instruments?kind=option_combo|future_combo` response. ### Trade publishing Deribit publishes each combo trade twice: - On the combo's trade channel (`trades.{combo_name}.{interval}`): the parent trade plus a `legs[]` array describing each leg fill. - On each leg's trade channel (`trades.{leg_instrument}.{interval}`): a standalone trade for the leg, tagged with `combo_id` and `combo_trade_id` pointing back to the parent. A subscriber to a plain option or future therefore sees combo-origin fills on its existing trade stream, and a subscriber to the combo itself sees the combo-level trade. The adapter does not fan out combo parent messages into extra leg ticks; it forwards the upstream parent and per-leg messages as separate `TradeTick`s against their respective `InstrumentId`s, so a subscriber to both the combo and an underlying leg sees one combo tick plus one leg tick for that combo trade, not duplicate ticks against the same instrument. To have the Deribit data client open the real leg trade channels alongside a combo trade subscription, pass `params={"subscribe_combo_legs": True}` to `subscribe_trade_ticks`. When unsubscribing that combo trade stream, Nautilus also closes the leg subscriptions opened by this opt-in. Deribit already publishes block trades and Block RFQs per leg, so the adapter forwards them through the standard 1:1 trade path. See [Trade ID provenance](#trade-id-provenance) for how block- and RFQ-origin trades are tagged on the resulting `TradeTick`. ### Historical combo trades The standard per-instrument trades endpoint accepts combo instrument names. To sweep all combos of a given product kind in one call, use `get_last_trades_by_currency` via `DeribitHttpClient::inner()`: ```rust use nautilus_deribit::http::{ models::{DeribitCurrency, DeribitProductType}, query::GetLastTradesByCurrencyParams, }; let params = GetLastTradesByCurrencyParams::builder() .currency(DeribitCurrency::BTC) .kind(DeribitProductType::FutureCombo) .count(50_u32) .include_old(true) .build()?; let resp = client.inner().get_last_trades_by_currency(params).await?; ``` Each returned `DeribitPublicTrade` carries `legs: Option>` plus the `combo_id` and `combo_trade_id` fields used to correlate per-leg trades. ## Trade ID provenance Public `TradeTick`s emitted by the adapter prefix the venue trade ID when the trade originated from a Block RFQ, a block trade, or a combo. Strategies that need to distinguish these from plain trades can pattern-match the prefix on `TradeTick.trade_id`. The raw Deribit `trade_id` is preserved after the prefix, so reconciliation against Deribit's own IDs is a prefix strip. | Prefix | Source field | Meaning | |--------------|------------------|----------------------------------------------------------------| | `RFQ-` | `block_rfq_id` | Trade originated from a Block RFQ. | | `BLK-` | `block_trade_id` | Trade is a non‑RFQ block trade. | | `COMBO-` | `combo_id` | Per‑leg trade whose parent originated from a combo instrument. | | *unprefixed* | (none of above) | Standard trade. | Precedence when multiple tags are present: `RFQ-` > `BLK-` > `COMBO-`. Block RFQs are themselves block trades on Deribit, so the RFQ tag wins; combos executed as block trades are tagged `BLK-` because the block flow is the more important reconciliation signal. This applies only to public trades (`TradeTick`). `FillReport.trade_id` is unchanged so reconciliation against `get_user_trades_*` keeps working. :::note This is a one-way convention. Replay data captured before this version lacks prefixes. Strategies that store and compare `trade_id` strings across versions should strip the prefix on the new-data side, or filter by prefix only on data they know was captured post-upgrade. ::: ## Order book subscriptions Deribit provides two types of order book feeds, each suited for different use cases. ### Raw feeds (tick-by-tick) Raw channels deliver every single update as an individual message. Subscribing to a raw order book gives you a notification for every order insertion, update, or deletion in the book. - Requires authenticated connection (safeguard against abuse). - Use when you need every price level change for HFT or market making. - Higher message volume. ### Aggregated feeds (batched) Aggregated channels deliver updates in batches at a fixed interval (e.g., every 100ms). This groups multiple order book changes into single messages. - Available without authentication. - Recommended for most use cases. - Lower message volume, easier to process. - Default unauthenticated interval: 100ms. ### Subscription parameters The Nautilus adapter supports both feed types via subscription parameters: | Parameter | Values | Notes | |------------|------------------------|---------------------------------------------------------------------------| | `interval` | `raw`, `100ms`, `agg2` | `agg2` batches at about 1 second intervals. `raw` requires auth. | | `group` | `none`, price group | Default: `none`. Applies only to grouped non‑raw book channels. | | `depth` | `1`, `10`, `20` | Default: `10`. Number of price levels per side for grouped book channels. | The data client chooses the order book interval as follows: 1. Uses `params["interval"]` when supplied. 2. Uses `raw` when the WebSocket connection is authenticated and no interval is supplied. 3. Uses Deribit's public `100ms` grouped feed when the connection is not authenticated. ```python from nautilus_trader.model.identifiers import InstrumentId instrument_id = InstrumentId.from_str("BTC-PERPETUAL.DERIBIT") # Public 100ms aggregated feed when no API credentials are configured. strategy.subscribe_order_book_deltas(instrument_id) # Raw feed. This is also the authenticated default when no interval is supplied. strategy.subscribe_order_book_deltas( instrument_id, params={"interval": "raw"}, ) # Force an aggregated feed on an authenticated connection. strategy.subscribe_order_book_deltas( instrument_id, params={"interval": "100ms", "depth": 10}, ) ``` :::note Raw order book feeds require an authenticated WebSocket connection. Ensure API credentials are configured before subscribing to raw feeds. ::: :::tip For most strategies, the 100ms aggregated feed provides sufficient granularity with lower message overhead. Set `params={"interval": "100ms"}` when you provide credentials but do not need raw tick-by-tick book updates. ::: ### Sequence gap recovery The adapter tracks `change_id` / `prev_change_id` sequence numbers on every book update. When a gap is detected (missed message), the adapter automatically: 1. Drops all incoming deltas for the affected instrument. 2. Unsubscribes from the book channel. 3. Resubscribes to obtain a fresh snapshot. 4. Resumes normal processing once the snapshot arrives. During resync, the strategy will not receive stale or incomplete book updates. ## Orders capability Below are the order types, execution instructions, and time-in-force options supported on Deribit. ### Order types | Nautilus order type | Deribit order type | Supported | Notes | |------------------------|--------------------|-----------|-----------------------------------------| | `MARKET` | `market` | ✓ | Immediate execution at market price. | | `LIMIT` | `limit` | ✓ | Execution at specified price or better. | | `STOP_MARKET` | `stop_market` | ✓ | Conditional market order on trigger. | | `STOP_LIMIT` | `stop_limit` | ✓ | Conditional limit order on trigger. | | `MARKET_IF_TOUCHED` | `take_market` | ✓ | Take‑profit style market order. | | `LIMIT_IF_TOUCHED` | `take_limit` | ✓ | Take‑profit style limit order. | | `TRAILING_STOP_MARKET` | `trailing_stop` | - | *Not currently implemented*. | | `TRAILING_STOP_LIMIT` | N/A | - | *Not supported by Deribit*. | | `MARKET_TO_LIMIT` | `market_limit` | - | *Not currently implemented*. | ### Execution instructions | Instruction | Supported | Notes | |---------------|-----------|----------------------------------------------------------------------------------| | `post_only` | ✓ | Order will be rejected if it would take liquidity. Uses `reject_post_only=true`. | | `reduce_only` | ✓ | Order can only reduce an existing position. | ### Time in force | Time in force | Supported | Notes | |---------------|-----------|------------------------------------------------------| | `GTC` | ✓ | Good till canceled (`good_til_cancelled`). | | `GTD` | ✓ | Good till day. Expires at 8:00 UTC (`good_til_day`). | | `IOC` | ✓ | Immediate or cancel (`immediate_or_cancel`). | | `FOK` | ✓ | Fill or kill (`fill_or_kill`). | Deribit applies time in force to limit-style orders. The adapter omits `time_in_force` for `MARKET`, `STOP_MARKET`, and `MARKET_IF_TOUCHED` orders because Deribit rejects that parameter on market-style order types. :::note **GTD on Deribit**: Unlike other exchanges where GTD accepts an arbitrary expiry time, Deribit's `good_til_day` always expires at 8:00 UTC the same or next day. Custom expiry times will be logged as warnings and the order will use the exchange's fixed expiry behavior. ::: ### Trigger types Conditional orders (stop orders) support different trigger price sources: | Trigger type | Supported | Notes | |---------------|-----------|---------------------------------------| | `last_price` | ✓ | Uses the last traded price (default). | | `mark_price` | ✓ | Uses the mark price. | | `index_price` | ✓ | Uses the underlying index price. | ```python # Example: Stop loss using mark price trigger stop_order = order_factory.stop_market( instrument_id=instrument_id, order_side=OrderSide.SELL, quantity=Quantity.from_str("0.1"), trigger_price=Price.from_str("45000.0"), trigger_type=TriggerType.MARK_PRICE, # Use mark price for trigger ) strategy.submit_order(stop_order) ``` ### Batch operations | Operation | Supported | Notes | |--------------------------|-----------|--------------------------------------------------------------------------| | Submit order list | ✓ | Sends each order as an individual Deribit order. No atomic venue batch. | | Batch cancel by order ID | ✓ | Sends individual `private/cancel` requests for each venue order ID. | | Cancel all by instrument | ✓ | Uses `private/cancel_all_by_instrument` when no side filter is supplied. | | Side‑filtered cancel all | ✓ | Filters cached open orders locally, then cancels each matching order. | | Batch modify | - | *Not currently implemented*: single order modify is supported. | ### Post-only behavior Deribit offers two post-only modes: 1. **Price adjustment (Deribit default)**: If a post-only order would cross the spread and execute, Deribit automatically adjusts the price to one tick inside the spread. 2. **Reject mode**: Order is immediately rejected if it would cross the spread. The Nautilus adapter uses **reject mode** (`reject_post_only=true`) for deterministic behavior. If a post-only order would take liquidity, it is rejected with error code `11054`, and an `OrderRejected` event is emitted with the `due_post_only` flag set to `true`. This allows strategies to differentiate between: - Orders rejected due to post-only violation (attempted to take liquidity). - Orders rejected for other reasons (insufficient margin, invalid price, etc.). ### Order modification The adapter uses Deribit's native `private/edit` endpoint rather than cancel-and-replace. This provides several advantages: | Benefit | Description | |----------------------------|--------------------------------------------------------------------| | Single request | Faster execution, lower latency than cancel + new order. | | Queue priority preservation | Keeps position when only reducing quantity or keeping same price. | | Fill history maintained | Partial fills remain linked to the same order ID. | **Queue priority rules:** - **Decreasing quantity only**: Keeps queue position. - **Same price**: Keeps queue position. - **Increasing quantity or changing price**: Loses queue position (treated as new order). ### Position management | Feature | Supported | Notes | |------------------|-----------|-------------------------------------------------------------------| | Query positions | ✓ | Real‑time position updates. | | Position mode | - | *Not supported by Deribit*: net position mode only. | | Leverage control | - | *Not supported by Deribit*: no direct leverage setting. | | Margin mode | - | *Not currently implemented*: Deribit exposes account margin modes. | ### Order querying | Feature | Supported | Notes | |----------------------|-----------|-----------------------------------| | Query open orders | ✓ | List all active orders. | | Query order history | ✓ | Historical order data. | | Order status updates | ✓ | Real‑time order state changes. | | Trade history | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Supported | Notes | |--------------------------------|-----------|--------------------------------------------------------------------| | Order lists | ✓* | Submitted sequentially. The adapter does not provide atomic lists. | | Native linked orders | - | *Not currently implemented*: Deribit supports linked orders. | | OCO orders | - | *Not currently implemented*: Deribit supports OCO links. | | Bracket orders | - | *Not currently implemented*: Deribit supports OTOCO links. | | Conditional stop orders | ✓ | Stop market and stop limit orders. | | Conditional take‑profit orders | ✓ | Market‑if‑touched and limit‑if‑touched orders. | ### Liquidation handling Deribit tags any trade that was triggered by a liquidation. On the `user.trades` stream and `private/get_user_trades_*` endpoints, the optional `liquidation` field indicates which side was being liquidated: | Value | Meaning | |--------|-------------------------------| | `"M"` | Maker side was liquidated. | | `"T"` | Taker side was liquidated. | | `"MT"` | Both sides were liquidated. | | absent | Normal non‑liquidation trade. | The adapter logs a warning for each liquidation-tagged fill with the instrument, trade ID, order ID, and liquidation side, and then emits the `FillReport` through the normal pipeline. Deribit does not operate an ADL mechanism distinct from the liquidation + insurance-fund / portfolio margin process, so there is no separate ADL signal to surface. Upstream references: - [`user.trades.{instrument_name}.{interval}` channel](https://docs.deribit.com/#user-trades-instrument_name-interval) - [Liquidation documentation](https://support.deribit.com/hc/en-us/articles/25944769313309-Liquidations) ## Funding rates Deribit exchanges funding continuously (every few seconds) rather than at fixed intervals like most other exchanges. The `interval` field on `FundingRateUpdate` is `None` for Deribit because this continuous model does not map to a discrete period. ## Deribit specific data The adapter emits `DeribitVolatilityIndex` custom data from Deribit's `deribit_volatility_index.{index_name}` WebSocket channel. Deribit provides volatility index streams such as `btc_usd` and `eth_usd`. | Field | Type | Description | |--------------|---------|----------------------------------------------------------| | `index_name` | `str` | Deribit volatility index name, for example `btc_usd`. | | `volatility` | `float` | Current volatility index value. | | `ts_event` | `int` | UNIX timestamp in nanoseconds when the update occurred. | | `ts_init` | `int` | UNIX timestamp in nanoseconds when the object was built. | Subscribe from an actor or strategy with `DataType(DeribitVolatilityIndex)`. The `index_name` metadata key is required: ```python from nautilus_trader.adapters.deribit import DeribitVolatilityIndex from nautilus_trader.model import ClientId from nautilus_trader.model.data import DataType self.subscribe_data( data_type=DataType(DeribitVolatilityIndex, metadata={"index_name": "btc_usd"}), client_id=ClientId.from_str("DERIBIT"), ) ``` ## Rate limiting Deribit uses credit-based and endpoint-specific rate limits. The official Deribit limits are authoritative, and they can vary by endpoint, account tier, and current venue policy. The adapter adds local token buckets to reduce avoidable throttling, but it does not replace Deribit's own server-side checks. ### HTTP limits | Bucket / key | Adapter bucket | Notes | |--------------------|-------------------------|----------------------------------------------------| | `deribit:global` | 20 req/sec, 100 burst | Default bucket for non‑matching HTTP requests. | | `deribit:orders` | 5 req/sec, 20 burst | Matching‑engine HTTP bucket for low‑level clients. | | `deribit:account` | 5 req/sec, no burst | Account information endpoints. | ### WebSocket limits | Operation | Adapter bucket | Notes | |-----------------------|-----------------------|--------------------------------------------| | Subscribe/unsubscribe | 3 req/sec, 10 burst | Subscription operations. | | Order operations | 5 req/sec, 20 burst | Buy, sell, edit, and cancel via WebSocket. | :::note The Nautilus adapter uses WebSocket for order submission (not HTTP) for lower latency. Order operations are rate-limited by `DERIBIT_WS_ORDER_QUOTA` (5 req/sec, 20 burst). ::: ### Credit-based system details Deribit replenishes non-matching-engine credits continuously. Current public documentation lists the default non-matching-engine pool as follows: **Non-matching engine requests:** | Parameter | Value | Notes | |------------------|--------------------|---------------------------------| | Cost per request | 500 credits | Each API call consumes credits. | | Maximum pool | 50,000 credits | Allows 100 request burst. | | Refill rate | 10,000 credits/sec | ~20 sustained requests/second. | **Matching engine requests (default tier):** | Parameter | Value | Notes | |----------------|----------------|----------------------------------| | Sustained rate | 5 requests/sec | Continuous rate limit. | | Burst capacity | 20 requests | Maximum burst before throttling. | Higher matching-engine limits are available for market makers and high-volume traders based on 7-day trading volume tiers. Some Deribit endpoints have stricter method-specific limits. For example, current venue docs list `public/get_instruments` at 1 request per second with a 50 request burst, and subscription methods at about 3.3 requests per second with a 10 request burst. Keep `product_types` scoped to the families you need and avoid repeated full instrument reloads in live systems. The Nautilus adapter implements broad token bucket rate limiters configured as: - `DERIBIT_HTTP_REST_QUOTA`: 20 req/sec with 100 burst (non-matching HTTP) - `DERIBIT_HTTP_ORDER_QUOTA`: 5 req/sec with 20 burst (matching-engine HTTP) - `DERIBIT_HTTP_ACCOUNT_QUOTA`: 5 req/sec with no burst (account HTTP) - `DERIBIT_WS_ORDER_QUOTA`: 5 req/sec with 20 burst (matching-engine WebSocket) - `DERIBIT_WS_SUBSCRIPTION_QUOTA`: 3 req/sec with 10 burst (subscribe and unsubscribe) For more details, see the [Rate Limits article](https://support.deribit.com/hc/en-us/articles/25944617523357-Rate-Limits). :::warning Deribit returns error code `10028` (too_many_requests) when you exceed the allowed quota. Repeated violations may result in temporary throttling. ::: ## Connection management ### Platform limits | Limit | Current Deribit guidance | |-----------------------------------------|--------------------------| | Active sessions per API key or login | 16 | | Web app connections per browser session | 2 | ### Session-based authentication The adapter uses **separate WebSocket sessions** for data and execution clients, each with its own authentication scope: | Client | Session Name | Purpose | |------------------|----------------------|-----------------------------------------------------| | Data client | `nautilus-data` | Market data subscriptions (raw feeds require auth). | | Execution client | `nautilus-execution` | Order operations (buy, sell, edit, cancel). | **Authentication flow:** 1. WebSocket connects to Deribit. 2. Client authenticates using `client_signature` grant type with session scope. 3. Tokens are refreshed before expiry. 4. On reconnection, re-authentication is retried with exponential backoff (up to 3 attempts). If all attempts fail, only public channel subscriptions are restored. This session-based approach allows: - Independent token management per client type. - Isolated failure domains (data auth failure does not affect execution). - Clear audit trail in Deribit's session logs. ### Best practices The adapter follows Deribit's [recommended connection practices](https://support.deribit.com/hc/en-us/articles/25944603459613): 1. **Uses WebSocket subscriptions** for real-time data instead of REST polling, resulting in fewer requests, lower latency, and reduced rate limit consumption. 2. **Authenticates all connections** when credentials are provided. Authenticated users benefit from higher rate limits and are less likely to be IP rate-limited. 3. **Implements heartbeats** (30 second interval by default) to maintain connection health and detect disconnections early. 4. **Handles reconnection** automatically with re-authentication and subscription recovery. :::tip Always provide API credentials even for public data access. Authenticated connections have higher rate limits, and Deribit contacts authenticated clients before applying restrictions during high-load periods. ::: :::note The adapter uses a 30 second heartbeat interval by default. Deribit requires WebSocket heartbeat intervals to be at least 10 seconds. ::: ## Authentication Deribit uses API key authentication with HMAC-SHA256 signatures for private endpoints. To create API credentials: 1. Log into your Deribit account at [deribit.com](https://www.deribit.com) (or [test.deribit.com](https://test.deribit.com) for testnet). 2. Navigate to **Account** -> **API**. 3. Click **Add new key** and configure permissions: - Enable **read** for market data access - Enable **trade** for order execution - Enable **wallet** if you need account balance access 4. Note down your **Client ID** (API key) and **Client Secret** (API secret). :::warning Keep your API secret secure. Never share it or commit it to version control. ::: ### API key scopes Each API key on Deribit is assigned a default access scope, which defines the maximum permissions. Configure appropriate permissions when [creating your API key](https://support.deribit.com/hc/en-us/articles/26268257333661): | Scope | Required For | |--------------------|----------------------------------------| | `account:read` | Account information, portfolio data. | | `trade:read` | View orders and positions. | | `trade:read_write` | Place, modify, and cancel orders. | | `wallet:read` | View balances and transaction history. | **Recommended minimum for trading:** `account:read`, `trade:read_write`, `wallet:read` :::tip Follow the principle of least privilege. For data-only access (market data, no trading), create a read-only key without `trade:read_write`. ::: ## Testnet Deribit provides a testnet environment for testing strategies without real funds. To use the testnet, set `environment=DeribitEnvironment.TESTNET` in your client configuration: ```python from nautilus_trader.adapters.deribit import DeribitDataClientConfig from nautilus_trader.adapters.deribit import DeribitEnvironment from nautilus_trader.adapters.deribit import DeribitExecClientConfig from nautilus_trader.adapters.deribit import DeribitProductType from nautilus_trader.model import AccountId from nautilus_trader.model import TraderId product_types = [DeribitProductType.FUTURE] trader_id = TraderId.from_str("TRADER-001") account_id = AccountId.from_str("DERIBIT-001") data_config = DeribitDataClientConfig( product_types=product_types, environment=DeribitEnvironment.TESTNET, ) exec_config = DeribitExecClientConfig( trader_id=trader_id, account_id=account_id, product_types=product_types, environment=DeribitEnvironment.TESTNET, ) ``` When testnet mode is enabled: - HTTP requests use `https://test.deribit.com`. - WebSocket connections use `wss://test.deribit.com/ws/api/v2`. - Loads credentials from `DERIBIT_TESTNET_API_KEY` and `DERIBIT_TESTNET_API_SECRET` environment variables. :::note Testnet API keys are separate from production keys. Create API keys specifically for the testnet through the testnet interface at [test.deribit.com](https://test.deribit.com). ::: ## Configuration ### Data client configuration options | Option | Default | Description | |------------------------------------|------------|--------------------------------------------------------------------| | `api_key` | `None` | Deribit API key. Loads from environment variables when omitted. | | `api_secret` | `None` | Deribit API secret. Loads from environment variables when omitted. | | `product_types` | `[FUTURE]` | Product types to load. | | `environment` | `MAINNET` | Environment enum (`MAINNET` or `TESTNET`). | | `base_url_http` | `None` | Override for the HTTP JSON-RPC base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `http_timeout_secs` | `60` | Request timeout in seconds for HTTP calls. | | `max_retries` | `3` | Maximum retry attempts for recoverable errors. | | `retry_delay_initial_ms` | `1,000` | Initial delay in milliseconds before retrying. | | `retry_delay_max_ms` | `10,000` | Maximum delay in milliseconds between retries. | | `heartbeat_interval_secs` | `30` | WebSocket heartbeat interval. | | `update_instruments_interval_mins` | `60` | Interval in minutes between instrument refreshes. | | `auto_load_missing_instruments` | `False` | Lazy‑load uncached instruments on subscribe. | #### Lazy-load on subscribe `subscribe_*` commands look up the instrument in the local cache before sending the WebSocket subscribe so the handler can parse the inbound frames. With `auto_load_missing_instruments = False` (the default), a subscribe for an instrument that was not preloaded (because of the configured `product_types`) returns an error up front rather than silently succeeding and dropping subsequent frames at the handler. Set `auto_load_missing_instruments = True` to instead fetch the instrument over HTTP on the first subscribe, seed the WebSocket handler cache, and then forward the subscribe. HTTP failures are logged and the WebSocket subscribe is skipped. ### Execution client configuration options | Option | Default | Description | |--------------------------|------------|--------------------------------------------------------------------| | `trader_id` | Required | Nautilus trader ID for generated reports and events. | | `account_id` | Required | Nautilus account ID for generated reports and events. | | `api_key` | `None` | Deribit API key. Loads from environment variables when omitted. | | `api_secret` | `None` | Deribit API secret. Loads from environment variables when omitted. | | `product_types` | `[FUTURE]` | Product types to load. | | `environment` | `MAINNET` | Environment enum (`MAINNET` or `TESTNET`). | | `base_url_http` | `None` | Override for the HTTP JSON-RPC base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `http_timeout_secs` | `60` | Request timeout in seconds for HTTP calls. | | `max_retries` | `3` | Maximum retry attempts for recoverable errors. | | `retry_delay_initial_ms` | `1,000` | Initial delay in milliseconds before retrying. | | `retry_delay_max_ms` | `10,000` | Maximum delay in milliseconds between retries. | Rust configs also expose `transport_backend`. The default is `Sockudo` when the `transport-sockudo` Cargo feature is enabled, and `Tungstenite` otherwise. The Python bindings use the compiled default. ### Production configuration Below is an example live node using Deribit data and execution clients: ```python from nautilus_trader.adapters.deribit import DeribitDataClientConfig from nautilus_trader.adapters.deribit import DeribitDataClientFactory from nautilus_trader.adapters.deribit import DeribitEnvironment from nautilus_trader.adapters.deribit import DeribitExecClientConfig from nautilus_trader.adapters.deribit import DeribitExecutionClientFactory from nautilus_trader.adapters.deribit import DeribitProductType from nautilus_trader.common import Environment from nautilus_trader.live import LiveNode from nautilus_trader.model import AccountId from nautilus_trader.model import TraderId product_types = [DeribitProductType.FUTURE] trader_id = TraderId.from_str("TRADER-001") account_id = AccountId.from_str("DERIBIT-001") node = ( LiveNode.builder("DERIBIT-001", trader_id, Environment.LIVE) .add_data_client( None, DeribitDataClientFactory(), DeribitDataClientConfig( product_types=product_types, environment=DeribitEnvironment.MAINNET, api_key=None, api_secret=None, ), ) .add_exec_client( None, DeribitExecutionClientFactory(), DeribitExecClientConfig( trader_id=trader_id, account_id=account_id, product_types=product_types, environment=DeribitEnvironment.MAINNET, api_key=None, api_secret=None, ), ) .build() ) ``` ### API credentials There are multiple options for supplying your credentials to the Deribit clients. Either pass the corresponding values to the configuration objects, or set the following environment variables: For Deribit live (production) clients: - `DERIBIT_API_KEY` - `DERIBIT_API_SECRET` For Deribit testnet clients: - `DERIBIT_TESTNET_API_KEY` - `DERIBIT_TESTNET_API_SECRET` :::tip We recommend using environment variables to manage your credentials. ::: ### Product types The `product_types` configuration option controls which Deribit product families are loaded. Available options via the `DeribitProductType` enum: - `DeribitProductType.FUTURE` - Perpetual and dated futures. - `DeribitProductType.OPTION` - Call and put options. - `DeribitProductType.SPOT` - Spot trading pairs. - `DeribitProductType.FUTURE_COMBO` - Future spread instruments. - `DeribitProductType.OPTION_COMBO` - Option spread instruments. Example loading multiple product types: ```python from nautilus_trader.adapters.deribit import DeribitDataClientConfig from nautilus_trader.adapters.deribit import DeribitProductType config = DeribitDataClientConfig( product_types=[ DeribitProductType.FUTURE, DeribitProductType.OPTION, ], # ... other config ) ``` ### Base URL overrides It's possible to override the default base URLs for both HTTP and WebSocket APIs: | Environment | HTTP URL | WebSocket URL | |-------------|----------------------------|------------------------------------| | Production | `https://www.deribit.com` | `wss://www.deribit.com/ws/api/v2` | | Testnet | `https://test.deribit.com` | `wss://test.deribit.com/ws/api/v2` | ## Server infrastructure Deribit's matching engine is located in **Equinix LD4, Slough, UK**. For latency-sensitive strategies, consider hosting in or near London. Colocation and cross-connect options are available directly from Deribit for institutional clients. For most users connecting via internet, the adapter's built-in retry logic, heartbeat monitoring, and automatic reconnection handling provide reliable connectivity. For more details, see the [Server Infrastructure article](https://support.deribit.com/hc/en-us/articles/25944617582877). ## Contributing :::info For additional features or to contribute to the Deribit adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Derive Source: https://nautilustrader.io/docs/latest/integrations/derive/ Derive (formerly Lyra) is a decentralized derivatives venue offering European-style options and cash-settled perpetual swaps, and one of the largest on-chain options markets. Trading runs against a per-user smart-contract wallet on the Derive Chain, so collateral stays in the user's custody while orders match through the venue's orderbook. The Derive Chain is an optimistic rollup that settles to Ethereum. Orders match off chain and settle on chain, pairing orderbook execution with self-custody. Orders are authorized with EIP-712 typed-data signatures from a session key scoped to a subaccount, which keeps the signing key separate from the wallet owner and lets users rotate or revoke access without moving funds. ## Examples Rust example testers live in [`crates/adapters/derive/examples/`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/derive/examples/). ## Overview The Derive adapter is implemented in Rust under `crates/adapters/derive`. It exposes: - `DeriveHttpClient`: Low-level REST connectivity to `api.lyra.finance` (mainnet) or `api-demo.lyra.finance` (testnet). - `DeriveWebSocketClient`: JSON-RPC WebSocket transport with subscription tracking, reconnect, and signed order entry (the WebSocket Trading API). - `DeriveInstrumentProvider`: Per-currency instrument fetch and caching. - `DeriveDataClient`: Live market data client. - `DeriveDataClientFactory`: Data client factory for the live node builder. - `DeriveExecutionClient`: Live execution client for signed order, cancel, query, and report flows. - `DeriveExecutionClientFactory`: Execution client factory for the live node builder. Execution flows use EIP-712 typed-data signing against the Derive Chain per-action module contracts. ## Derive documentation Derive publishes API documentation at [docs.derive.xyz](https://docs.derive.xyz). Refer to it alongside this guide for additional details. ## Products | Product type | Supported | Notes | |------------------------|-----------|----------------------------------------------------------------------| | ERC-20 spot | ✓ | USDC‑quoted pairs such as `ETH-USDC`; parsed as `CurrencyPair`. | | Perpetual swaps | ✓ | Cash‑settled in USDC, with per‑currency listings such as `ETH-PERP`. | | Options (calls / puts) | ✓ | European‑style options using `{CURRENCY}-{EXPIRY}-{STRIKE}-{C|P}`. | ## Symbology Derive instruments use the native venue symbol with the venue suffix `.DERIVE`: - Spot: `ETH-USDC.DERIVE` (base currency, quote currency). - Perpetual: `ETH-PERP.DERIVE`, `BTC-PERP.DERIVE`. - Option: `ETH-20260626-3000-C.DERIVE` (currency, expiry, strike, kind). The first hyphen-separated segment of the symbol is the underlying currency. The provider fetches `public/get_instruments` once per currency, so subscribing to a new currency triggers a lazy REST fetch when `auto_load_missing_instruments` is enabled (the default). The adapter routes on the venue `instrument_type` (`perp`, `option`, `erc20`), not on the symbol suffix, so spot pairs need no special symbology parsing. Spot reuses the same Trade-module signing path as perps and options; the in-repo fixtures under `crates/adapters/derive/test_data/spot/` capture the spot instrument, order book, ticker, and trade field shapes the parser and execution paths are pinned to. :::warning Spot trading has had less live exercise than perpetuals and options. Testnet accepts and cancels a passive `ETH-USDC` limit order at the `0.1 ETH` minimum amount, and mainnet place/cancel has been exercised manually. Public spot trade channels (`trades.erc20.ETH`, `trades.ETH-USDC`) subscribe successfully but can be low-volume, so expect sparse trade frames. ::: ## Environments Configure the environment with the `DeriveEnvironment` enum on either client config. | Environment | Config | REST | WebSocket | |-------------|------------------------------|---------------------------------|----------------------------------| | Mainnet | `DeriveEnvironment::Mainnet` | `https://api.lyra.finance` | `wss://api.lyra.finance/ws` | | Testnet | `DeriveEnvironment::Testnet` | `https://api-demo.lyra.finance` | `wss://api-demo.lyra.finance/ws` | Testnet is a separate chain with its own session keys and balances; mainnet and testnet API keys are not interchangeable. Public market data (book, ticker, trades) does not require credentials. EIP-712 Protocol Constants (`DOMAIN_SEPARATOR`, `ACTION_TYPEHASH`, per-action module addresses) for both networks are shipped in `crates/adapters/derive/src/common/consts.rs` and tracked against Derive's [Protocol Constants reference](https://docs.derive.xyz/reference/protocol-constants). `DeriveExecClientConfig::domain_separator`, `action_typehash`, and `trade_module_address` accept per-instance overrides that take precedence over the shipped values. ## Testnet onboarding Derive labels the demo environment "testnet" in the web app and "demo" in the API hostname. This guide uses "testnet" to match the dashboard and our `DeriveEnvironment::Testnet` enum. Steps to reach a position where the execution client can submit a signed order: 1. **Sign in to the testnet dashboard.** Open [testnet.derive.xyz](https://testnet.derive.xyz) and connect an EVM wallet (MetaMask, WalletConnect, social login, etc.). This is the owner EOA that authorises the smart- contract wallet below. 2. **Register the Derive Chain smart-contract wallet.** First sign-in deploys a per-user smart-contract wallet on the Derive testnet chain. The address shown under "Developers" -> "Derive Wallet" is the `wallet_address` (and the `X-LYRAWALLET` header) the client uses. It is distinct from the EOA you just connected. 3. **Create a subaccount.** Open a subaccount under the wallet (Standard Margin is the simplest mode for test trading). The integer id is the `subaccount_id` the client signs each `private/order` request against. 4. **Generate a session key.** Under "Developers" -> "Session Keys", create a session key scoped to the subaccount and copy the raw secp256k1 private key. This is the `session_key` value; it never leaves the client and is redacted from `Debug` output. Session keys can be rotated or revoked from the same panel. 5. **Fund the subaccount via the faucet.** The testnet dashboard exposes a USDC faucet that drips test collateral. Deposit into the subaccount so the on-chain balance shows non-zero collateral; the API will reject orders until the subaccount has enough margin for the requested size. 6. **Set the environment variables.** Export the three values the client reads in testnet mode (or pass them on `DeriveExecClientConfig`, where the config field wins): ```bash export DERIVE_TESTNET_WALLET_ADDRESS="0x..." # Derive Chain smart-contract wallet export DERIVE_TESTNET_SESSION_PRIVATE_KEY="0x..." # secp256k1 session-key private key export DERIVE_TESTNET_SUBACCOUNT_ID="12345" # integer subaccount id ``` ### Minimum funding There is no fixed venue minimum. The matching engine accepts any order that satisfies the subaccount's initial-margin requirement for the resulting position. Treat these as practical floors for the smallest viable test: - **Smoke test (submit and cancel, no fills):** any positive USDC balance covers the signed-order plumbing. - **Round-trip an `ETH-PERP` fill:** budget for the worst-case slippage-adjusted notional plus the initial-margin cushion. For one contract at $3500 and the venue's ~10% IM, that is roughly $350 collateral plus $400 cushion. Around $1000 USDC is a comfortable working balance for a first-fill test. - **Options:** options carry higher IM than perps. Pull `public/get_instrument` for the option, multiply the contract size by mark price, then add the option-specific IM (visible on the instrument response) before sizing the deposit. Use the `private/get_subaccount` endpoint after funding to confirm `initial_margin`/`maintenance_margin` headroom against the order you plan to submit; the adapter's `query_account` command emits this snapshot as an `AccountState` event so the strategy layer can gate trading on it. ## Mainnet onboarding Mainnet onboarding mirrors testnet against the production dashboard. Use real funds. 1. **Sign in to the mainnet dashboard.** Open [derive.xyz](https://derive.xyz) and connect the EVM owner wallet (MetaMask, WalletConnect, social login, etc.). First sign-in deploys your Derive Chain smart-contract wallet. 2. **Copy the wallet address.** Under "Developers" -> "Derive Wallet", copy the smart-contract wallet address. This is the `wallet_address` the client signs against; it is **distinct** from the EOA you signed in with. Verify on the Derive Chain explorer that the address has contract code (EOAs do not). 3. **Create or pick a subaccount.** Open a subaccount under the wallet (Standard Margin is the simplest mode; switch to Portfolio Margin only once you understand the cross-margin semantics). The integer id is the `subaccount_id`. 4. **Generate a mainnet session key.** Under "Developers" -> "Session Keys", create a session key scoped to the subaccount and copy the raw secp256k1 private key. Session keys can be rotated or revoked from the same panel; prefer short-lived keys for exploratory tester runs. 5. **Fund the subaccount.** Deposit USDC (or supported collateral) into the subaccount via the dashboard's deposit flow. Confirm via `private/get_subaccount` (or the adapter's `query_account`) that `collaterals_value` and `initial_margin` headroom cover the intended order before submitting. 6. **Set the environment variables.** Export the three mainnet values (or pass them on `DeriveExecClientConfig`, where the config field wins): ```bash export DERIVE_WALLET_ADDRESS="0x..." # Derive Chain smart-contract wallet export DERIVE_SESSION_PRIVATE_KEY="0x..." # secp256k1 session-key private key export DERIVE_SUBACCOUNT_ID="12345" # integer subaccount id ``` All three Rust examples (`node_data_tester`, `node_exec_tester`, `node_delta_neutral`) pin the network with a `const DERIVE_ENVIRONMENT: DeriveEnvironment = DeriveEnvironment::Testnet;` literal; flip it to `DeriveEnvironment::Mainnet` for real-funds runs. Production deployments select the network via `DeriveDataClientConfig::environment` / `DeriveExecClientConfig::environment`. ## Capabilities ### Market data | Capability | Supported | Notes | |--------------------------------|-----------|-------------------------------------------------------------------------| | Request instrument (REST) | ✓ | `public/get_instrument`; loads one instrument into the local cache. | | Request all instruments (REST) | ✓ | `public/get_instruments`; fetches each currency in `currencies`. | | Instrument subscription | - | *Not supported.* Use the configured REST refresh interval. | | Order book deltas (L2_MBP) | ✓ | Channel: `orderbook.{instrument}.{group}.{depth}`. | | Order book depth10 (L2_MBP) | ✓ | Same order book channel with `depth=10`. | | Order book at interval | - | *Not supported.* Maintain interval books from deltas locally. | | Order book snapshot (REST) | - | *Not supported.* Not exposed by the adapter. | | Historical book deltas (REST) | - | *Not supported.* Not exposed by the adapter. | | Quotes (`ticker_slim`) | ✓ | Channel: `ticker_slim.{instrument}.{interval}`. | | Quote snapshot (REST) | ✓ | One‑shot `public/get_tickers`; emits a single `QuoteTick`. | | Historical quotes (REST) | - | *Not supported.* The venue exposes ticker snapshots only. | | Trades | ✓ | Channel: `trades.{instrument_type}.{currency}`. | | Historical trades (REST) | ✓ | `public/get_trade_history`; honors `start`, `end`, and `limit`. | | Bars / OHLC (REST) | ✓ | `public/get_tradingview_chart_data`; minute, hour, day, and week bars. | | Bars / OHLC (WS) | - | *Not supported.* The venue has no candle subscription channel. | | Mark price stream | ✓ | Derived from `ticker_slim`; shares the quote subscription. | | Index price stream | ✓ | Derived from `ticker_slim`; shares the quote subscription. | | Funding rate stream | ✓ | Derived from `perp_details.funding_rate` on perp tickers. | | Funding rate history (REST) | ✓ | `public/get_funding_rate_history` for perpetuals. | | Instrument status | - | *Not supported.* Ticker payloads include `is_active`. | | Instrument close | - | *Not supported.* Option settlement is REST-only. | | Option greeks | ✓ | Derived from `option_pricing` on option tickers. | | Option chain | ✓ | Aggregated from quotes and greeks; `public/get_tickers` bootstraps ATM. | `request_instrument` calls `public/get_instrument` for the requested `InstrumentId` and caches the returned definition before emitting the response. The cached instrument carries the precision and increment fields used by later quote, trade, book, and bar parsing. Derive exposes book deltas and depth10 snapshots through the same `orderbook.{instrument}.{group}.{depth}` channel family. `subscribe_book_deltas` publishes snapshot deltas as `OrderBookDeltas`, while `subscribe_book_depth10` fixes `depth=10` and publishes `OrderBookDepth10` snapshots. ### Execution Order placement, cancellation, modification, query, and report generation use Derive's EIP-712 self-custodial signing flow. Order-entry writes (`private/order`, `private/cancel`, `private/cancel_all`, `private/replace`) go over the WebSocket Trading API on the same authenticated session that streams account, order, trade, and balance state through the private channels (`{subaccount_id}.orders`, `{subaccount_id}.trades`, `{subaccount_id}.balances`). The signed EIP-712 body is identical regardless of transport. :::note The HTTP order-entry endpoints remain available on `DeriveHttpClient` for tooling and tests, but the live execution client routes all writes over the WebSocket Trading API. Report generation, account refresh, and instrument lookups still use REST. ::: Perpetuals, options, and ERC-20 spot pairs all use the Derive Trade module. Spot has no separate signing path, and reconciliation treats spot instruments like other instrument classes except for the reduce-only guard described below. The adapter supports ordinary `private/order` requests: `LIMIT` and `MARKET` orders with `GTC`, `IOC`, or `FOK` time-in-force values. It also supports Derive trigger orders for the Nautilus-native stop and if-touched order types listed below. Unsupported Nautilus order types are rejected before signing, so they cannot fill at the venue. Market orders require a cached quote before submission. After the async submit task resolves the instrument, it refreshes the current ticker snapshot and derives the signed slippage-bound `limit_price` from that refreshed quote. #### Conditional orders Derive trigger orders use the WebSocket-only `private/trigger_order` endpoint, not the normal `private/order` endpoint. The venue stores them with `order_status=untriggered` until its trigger worker submits the signed child order. Reconciliation therefore reads both `private/get_open_orders` and `private/get_trigger_orders`. Derive mainnet requires trigger-order signatures to expire 30 to 90 days from venue time. The adapter signs trigger orders with a fixed 31-day expiry; `signature_expiry_secs` still controls ordinary `private/order` and `private/replace` writes, and must be greater than the 300s venue minimum. | Nautilus order type | Supported | Derive `order_type` | Derive `trigger_type` | Notes | |---------------------|-----------|---------------------|-----------------------|-------------------------------| | `StopMarket` | ✓ | `market` | `stoploss` | Uses trigger price as bound. | | `StopLimit` | ✓ | `limit` | `stoploss` | Sends limit and trigger price. | | `MarketIfTouched` | ✓ | `market` | `takeprofit` | Uses trigger price as bound. | | `LimitIfTouched` | ✓ | `limit` | `takeprofit` | Sends limit and trigger price. | | `MarketToLimit` | - | - | - | *Not supported by Derive*. | | Trailing stops | - | - | - | *Not supported by Derive*. | | TWAP / algo / RFQ | - | - | - | *Not exposed by this adapter*. | The adapter maps Nautilus `TriggerType::Default` and `TriggerType::MarkPrice` to Derive `trigger_price_type=mark`. Derive's current error-code reference states that index and last-trade trigger price types are not supported yet, so `IndexPrice`, `LastPrice`, `BidAsk`, and other trigger price types are rejected locally before signing. Derive error `11054` states that trigger orders cannot replace or be replaced. The adapter therefore rejects Nautilus modify requests for trigger orders with an `OrderModifyRejected` event; cancel and resubmit for trigger updates. Derive validates the trigger price side and rejects a trigger that does not sit beyond the current price in the expected direction with error `11051`. The trigger price is fixed when the order is signed, so a tight offset on a fast-moving or high-priced instrument can drift onto the wrong side before the venue receives the order. Size the trigger offset to comfortably exceed expected price movement during submission (for `ETH-PERP`, tens of dollars rather than a few cents); a too-tight offset produces spurious `11051` rejections. #### Execution instructions | Instruction | Supported | Derive value | Notes | |---------------|-----------|---------------|---------------------------------------------------------------| | `post_only` | ✓ | `post_only` | Requires `GTC`; rejects if the order would take liquidity. | | `reduce_only` | ✓ | `reduce_only` | Perps and options, market or `IOC`/`FOK` only; spot denied. | #### Time in force Derive documents `gtc`, `post_only`, `fok`, and `ioc` as its `time_in_force` values. The adapter rejects Nautilus values with no Derive equivalent before signing. Derive exposes post-only as a `time_in_force` value, so `post_only` cannot combine with `IOC` or `FOK`. | Time in force | Supported | Derive value | Notes | |----------------|-----------|--------------|----------------------------| | `GTC` | ✓ | `gtc` | Good Till Canceled. | | `IOC` | ✓ | `ioc` | Immediate or Cancel. | | `FOK` | ✓ | `fok` | Fill or Kill. | | `GTD` | - | - | *Not supported by Derive*. | | `DAY` | - | - | *Not supported by Derive*. | | `AT_THE_OPEN` | - | - | *Not supported by Derive*. | | `AT_THE_CLOSE` | - | - | *Not supported by Derive*. | #### Spot reduce-only orders Derive spot has no position concept, so a reduce-only spot order can never reduce anything. The venue always rejects it with error `11025`; the adapter avoids that round-trip when it knows the instrument is spot. Cached spot instruments are denied with `OrderDenied`; lazily resolved spot instruments are rejected with `OrderRejected` during submit. Reduce-only orders for perpetuals and options still reach the venue, where the outcome depends on the subaccount's position state. The `derive-flatten` bin closes derivative positions only and never spot, since flattening a spot balance would dump the base asset into a different quote. Derive only honors `reduce_only` on market orders or non-resting limits (`IOC`/`FOK`). A resting `GTC` or post-only limit with `reduce_only` is rejected by the venue with error `11024 Reduce only not supported with this time in force`. As a result a Nautilus bracket whose take-profit leg is a reduce-only `GTC` limit cannot rest on Derive: the entry and stop-loss legs submit, but the take-profit is rejected. Use a reduce-only `IOC`/`FOK` close or a non-reduce-only take-profit when targeting Derive. #### Order rejection semantics State-changing writes (`submit_order`, `modify_order`, `cancel_order`) are sent once over the WebSocket Trading API and are not replayed. The adapter keys terminal vs ambiguous handling off the WebSocket request outcome. It emits a terminal rejection event (`OrderRejected`, `OrderModifyRejected`, `OrderCancelRejected`) for definitive venue failures: - Signed-action rejections such as invalid params, insufficient margin, or unknown orders. - Venue business codes such as `11009 Zero liquidity`. - Post-only crossing rejections (`11008 Post only order cannot cross the market`), reported as `OrderRejected` with `due_post_only=true`. - Rate-limit responses (`-32000 Rate limit exceeded`), where the gateway rejects the request before the matching engine sees it. For post-only orders that reach the venue, Derive rejects a crossing order with JSON-RPC `11008` and message `Post only order cannot cross the market`. The adapter marks that terminal rejection with `due_post_only=true`; if a WebSocket/order-report rejection carries the same reason, the tracked order path applies the same classification. Local rejections for unsupported post-only IOC/FOK combinations are not marked `due_post_only` because they do not represent a venue crossing rejection. For ambiguous write outcomes, the adapter emits no terminal event and lets WebSocket reconciliation or later status reports settle the state. The ambiguous set is deliberately narrow: - `-32603`, a generic JSON-RPC internal error. - A response that cannot be decoded (the action may have been processed). - Request timeouts, dropped responses on reconnect, and transport errors. This distinction protects both sides of the order lifecycle. A false terminal rejection can make the engine treat a live order as rejected; a false ambiguous outcome can leave an unplaced order hanging in `Submitted` forever because no WebSocket frame will arrive. ## Subscription parameters `subscribe_book_deltas` and `subscribe_book_depth10` accept these `subscribe_params` keys: | Key | Type | Default | Allowed | |----------|--------|---------|----------------------| | `group` | string | `"1"` | `"1"`, `"10"`, `"100"` | | `depth` | string | `"10"` | `"1"`, `"10"`, `"20"`, `"100"` | `subscribe_quotes` accepts: | Key | Type | Default | Allowed | |------------|--------|----------|-------------------| | `interval` | string | `"1000"` | `"100"`, `"1000"` | Unknown values are rejected at subscribe time. ### Shared ticker subscription Quotes, mark prices, index prices, funding rates, and option greeks are all derived from the same `ticker_slim.{instrument}.{interval}` WebSocket subscription. The adapter reference-counts the underlying WS subscribe call: the first feed subscribed for an instrument opens the channel and the last unsubscribe closes it. As a consequence, the `interval` from the first subscribe wins; subsequent feeds subscribing with a different interval share the existing channel. Mark prices, index prices, funding rates, and option greeks read fields from the ticker payload. Both the full ticker shape and the compact `SlimEnvelope` shape carry these fields, so the derived feeds work on either: `mark_price` and `index_price` are required (a frame missing them fails to deserialize and is logged, rather than silently dropped), while `funding_rate` and `option_pricing` are optional and present only for the relevant instrument class. The quote feed always works because bid/ask are present in both shapes. Funding rates are only meaningful for perpetuals, and option greeks only for options. Subscribing the wrong feed for an instrument's class (e.g. funding rates for an option) is accepted and the WebSocket subscription opens, but the parser returns no events for that feed because the venue payload lacks the relevant fields (`perp_details` for non-perps, `option_pricing` for non-options). Verify the instrument class before subscribing to derivative- specific feeds. ## Configuration ### Data client configuration options Class/struct: `DeriveDataClientConfig`. | Option | Default | Description | |------------------------------------|-----------|-------------| | `base_url_rest` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `environment` | `Mainnet` | Network selector (`MAINNET` or `TESTNET` in Python). | | `http_timeout_secs` | `10` | REST request timeout in seconds. | | `ws_timeout_secs` | `30` | WebSocket connect and idle timeout in seconds. | | `update_instruments_interval_mins` | `60` | Interval in minutes between instrument refreshes. | | `currencies` | `[]` | Currencies to bulk‑load on connect. Empty means lazy‑load on demand. | | `include_expired` | `false` | Include expired option rows from `public/get_instruments`. | | `auto_load_missing_instruments` | `true` | Lazy‑load unknown instruments before subscribe or request commands. | | `transport_backend` | `Sockudo` | WebSocket transport when `transport-sockudo` is enabled. | ### Execution client configuration options Class/struct: `DeriveExecClientConfig`. | Option | Default | Description | |------------------------------------|-----------|-------------| | `wallet_address` | `None` | Derive Chain smart‑contract wallet address. Falls back to env vars below. | | `session_key` | `None` | secp256k1 session‑key private key. Falls back to env vars below. | | `subaccount_id` | `None` | Derive subaccount id. Falls back to env vars below. | | `base_url_rest` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `environment` | `Mainnet` | Network selector (`MAINNET` or `TESTNET` in Python). | | `http_timeout_secs` | `10` | REST request timeout in seconds. | | `max_retries` | `3` | Retry attempts for recoverable reads and definitive non‑write paths. | | `retry_delay_initial_ms` | `100` | Initial retry delay in milliseconds. | | `retry_delay_max_ms` | `5000` | Maximum retry delay in milliseconds. | | `max_fee_per_contract` | `None` | Per‑contract USDC fee cap signed into each order. | | `domain_separator` | `None` | Optional EIP-712 domain separator override. | | `action_typehash` | `None` | Optional EIP-712 action typehash override. | | `trade_module_address` | `None` | Optional Trade module contract address override. | | `signature_expiry_secs` | `600` | Order/replace TTL; must be >300s. Trigger orders use fixed 31-day TTL. | | `market_order_slippage_bps` | `50` | Slippage bound for market‑order limit prices. | | `max_matching_requests_per_second` | `None` | Max matching‑engine write requests/sec (create/cancel/replace). Defaults to the Trader‑tier limit of 1 when unset; raise for Market Maker accounts. | | `transport_backend` | `Sockudo` | WebSocket transport when `transport-sockudo` is enabled. | The default transport falls back to `Tungstenite` when the build disables the `transport-sockudo` feature. The `wallet_address`, `session_key`, and `subaccount_id` fall back to environment variables when unset: | Field | Mainnet variable | Testnet variable | |------------------|------------------------------|--------------------------------------| | `wallet_address` | `DERIVE_WALLET_ADDRESS` | `DERIVE_TESTNET_WALLET_ADDRESS` | | `session_key` | `DERIVE_SESSION_PRIVATE_KEY` | `DERIVE_TESTNET_SESSION_PRIVATE_KEY` | | `subaccount_id` | `DERIVE_SUBACCOUNT_ID` | `DERIVE_TESTNET_SUBACCOUNT_ID` | The session key is the secp256k1 private key registered on the wallet for API signing. The `session_key` field is redacted in `Debug` output and Python `repr`. ### Python v2 live node Rust-backed Python v2 nodes use `LiveNode.builder(...)` and pass concrete factory instances. The execution factory needs `DeriveExecFactoryConfig`, which wraps the trader and account identifiers with the underlying `DeriveExecClientConfig`. ```python from decimal import Decimal from nautilus_trader.adapters.derive import DeriveDataClientConfig from nautilus_trader.adapters.derive import DeriveDataClientFactory from nautilus_trader.adapters.derive import DeriveEnvironment from nautilus_trader.adapters.derive import DeriveExecClientConfig from nautilus_trader.adapters.derive import DeriveExecFactoryConfig from nautilus_trader.adapters.derive import DeriveExecutionClientFactory from nautilus_trader.common import Environment from nautilus_trader.live import LiveNode from nautilus_trader.model import AccountId from nautilus_trader.model import TraderId trader_id = TraderId("TESTER-001") data_config = DeriveDataClientConfig( environment=DeriveEnvironment.TESTNET, currencies=["ETH", "BTC"], ) exec_config = DeriveExecClientConfig( environment=DeriveEnvironment.TESTNET, max_fee_per_contract=Decimal("1000"), ) exec_factory_config = DeriveExecFactoryConfig( trader_id, AccountId("DERIVE-001"), exec_config, ) node = ( LiveNode.builder("DERIVE-001", trader_id, Environment.LIVE) .add_data_client(None, DeriveDataClientFactory(), data_config) .add_exec_client(None, DeriveExecutionClientFactory(), exec_factory_config) .build() ) ``` Do not pass `DeriveExecClientConfig` directly to `add_exec_client`; the Derive execution factory requires the wrapped `DeriveExecFactoryConfig` so it can create the `ExecutionClientCore` with the correct trader and account identifiers. ### Rust data client ```rust use nautilus_derive::{ common::enums::DeriveEnvironment, config::DeriveDataClientConfig, }; let config = DeriveDataClientConfig { environment: DeriveEnvironment::Testnet, currencies: vec!["ETH".to_string(), "BTC".to_string()], ..Default::default() }; ``` Notable fields: - `currencies`: which currencies to bulk-load on connect. Empty means lazy-load per subscribe. - `include_expired`: include expired option rows from `public/get_instruments`. - `auto_load_missing_instruments`: lazy-load on subscribe when an instrument is unknown. - `update_instruments_interval_mins`: REST refresh interval (default 60 minutes). - `http_timeout_secs`, `ws_timeout_secs`: transport timeouts. ### Rust execution client ```rust use nautilus_derive::{ common::enums::DeriveEnvironment, config::DeriveExecClientConfig, }; let config = DeriveExecClientConfig { wallet_address: Some("0x...".to_string()), session_key: Some("0x...".to_string()), subaccount_id: Some(1), environment: DeriveEnvironment::Testnet, ..Default::default() }; ``` ## Known limitations - `request_instruments` requires at least one configured currency in `DeriveDataClientConfig::currencies`; the venue's `public/get_instruments` endpoint is scoped per-currency and the adapter does not enumerate the currency universe. - The `data_client.rs` integration test asserts the set, not the order, of recorded REST calls because `fetch_instrument_definitions` issues the `perp` and `option` requests in parallel via `tokio::try_join!`. - Venue does not push instrument status, instrument close, or candle subscriptions; the ticker payload carries margin parameters and `is_active`, and bars are REST-only. - The book snapshot REST endpoint and historical book deltas / historical quote endpoints are not exposed by the venue. See the capabilities table above. - Derive's official REST docs mark `public/get_ticker` as deprecated in favor of `public/get_tickers` as of December 1, 2025. The adapter uses `public/get_tickers` for quote snapshots and option-chain forward-price bootstrap. # dYdX Source: https://nautilustrader.io/docs/latest/integrations/dydx/ dYdX is one of the largest decentralized cryptocurrency exchanges for crypto derivative products. This integration supports live market data ingestion and order execution with dYdX v4, running on its own Cosmos SDK application-specific blockchain (dYdX Chain) with CometBFT consensus. The order book and matching engine run on-chain as part of the validator process. Orders are submitted as Cosmos transactions via gRPC and settled each block. An Indexer service exposes REST and WebSocket APIs for market data and account state. This is the Rust-backed adapter with Python bindings. ## Installation :::note No additional installation extras are required. The adapter is implemented in Rust and compiled into the core `nautilus_trader` package automatically during the build. ::: ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/dydx/). ## Overview This adapter is implemented in Rust with Python bindings via PyO3. It provides direct integration with dYdX's Indexer API (REST/WebSocket) for market data and gRPC for Cosmos SDK transaction submission, without requiring external client libraries. ### Product support | Product Type | Data Feed | Trading | Notes | |-------------------|-----------|---------|----------------------------------------| | Perpetual Futures | ✓ | ✓ | All perpetuals are USDC‑settled. | | Spot | - | - | dYdX offers spot on Solana; not supported by this adapter. | | Options | - | - | *Not available on dYdX*. | :::note This adapter supports perpetual futures only. All markets are quoted in USD and settled in USDC. ::: ## Chain architecture Unlike centralized exchanges (CEXs) that expose a single REST/WebSocket API, dYdX v4 runs on its own **Cosmos SDK application-specific blockchain**. This means every trade is a Cosmos transaction that goes through consensus, and the adapter must manage sequences, gas, and block-height-based expiration. ### Transport layers The adapter communicates through three independent transport layers: ``` ┌─────────────────────────────────────────────┐ │ dYdX v4 Chain │ │ │ ┌──────────┐ HTTP │ ┌──────────────────────┐ │ │ │───────────►│ │ Indexer (read-only) │ │ │ │ WebSocket │ │ - REST API │ │ │ Nautilus │───────────►│ │ - Streaming API │ │ │ Adapter │ │ └──────────────────────┘ │ │ │ gRPC │ ┌──────────────────────┐ │ │ │───────────►│ │ Validator (write) │ │ └──────────┘ │ │ - Cosmos Tx submit │ │ │ │ - Sequence mgmt │ │ │ └──────────────────────┘ │ └─────────────────────────────────────────────┘ ``` | Layer | Target | Direction | Purpose | |-----------|-----------|------------|------------------------------------------------------| | HTTP | Indexer | Read‑only | Instrument metadata, historical data, account state. | | WebSocket | Indexer | Read‑only | Real‑time market data, order/fill/position updates. | | gRPC | Validator | Write | Order placement, cancellation, and batch operations. | ### Block-based settlement dYdX blocks are produced approximately every **~0.5 seconds** (actual times vary). The adapter includes a `BlockTimeMonitor` that tracks observed block times from the WebSocket feed to dynamically estimate `seconds_per_block`. This estimate is used to convert time-based order expiry into block-height offsets for short-term orders. ## Architecture The dYdX v4 adapter includes multiple components which can be used together or separately: - `DydxHttpClient`: Rust-backed HTTP client for Indexer REST API queries. - `DydxWebSocketClient`: Rust-backed WebSocket client for real-time market data and account updates. - `DydxGrpcClient`: Rust-backed gRPC client for Cosmos SDK transaction submission. - `DydxInstrumentProvider`: Instrument parsing and loading functionality. - `DydxDataClient`: Market data feed manager. - `DydxExecutionClient`: Account management and trade execution gateway. - `DydxDataClientFactory`: Factory for dYdX v4 data clients (used by the trading node builder). - `DydxExecutionClientFactory`: Factory for dYdX v4 execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to work with these lower level components directly. ::: :::warning[First-time account activation] A dYdX v4 trading account (sub-account 0) is created only after the wallet's first deposit or trade. Until then, every gRPC/Indexer query returns `NOT_FOUND`, so `DydxExecutionClient.connect()` fails. Before starting a live `TradingNode`, send any positive amount of USDC or other supported collateral from the same wallet on the same network (mainnet/testnet). Once the transaction has finalised (a few blocks), restart the node and the client will connect cleanly. ::: ## Troubleshooting ### `StatusCode.NOT_FOUND` account not found **Cause:** The wallet/sub-account has never been funded and therefore does not yet exist on-chain. **Fix:** 1. Deposit any positive amount of USDC to sub-account 0 on the correct network. 2. Wait for finality (roughly 30 seconds on mainnet, longer on testnet). 3. Restart the `TradingNode`; the connection should now succeed. :::tip In unattended deployments, wrap the `connect()` call in an exponential-backoff loop so the client retries until the deposit appears. ::: ## Symbology dYdX uses specific symbol conventions for perpetual futures contracts. ### Symbol format Format: `{Base}-USD-PERP` All perpetuals on dYdX are: - Quoted in USD - Settled in USDC - Use the `.DYDX` venue suffix in Nautilus Examples: - `BTC-USD-PERP.DYDX` - Bitcoin perpetual futures - `ETH-USD-PERP.DYDX` - Ethereum perpetual futures - `SOL-USD-PERP.DYDX` - Solana perpetual futures To subscribe in your strategy: ```python InstrumentId.from_str("BTC-USD-PERP.DYDX") InstrumentId.from_str("ETH-USD-PERP.DYDX") ``` :::info The `-PERP` suffix is appended for consistency with other adapters and future-proofing. While dYdX currently only supports perpetuals, this naming convention allows for potential expansion to other product types. ::: ## Orders capability dYdX supports perpetual futures trading with a full set of order types and execution features. The Rust adapter automatically classifies orders as short-term or long-term based on time-in-force and expiry, so no manual tagging is needed. ### Order types | Order Type | Perpetuals | Notes | |------------------------|------------|----------------------------------------------------| | `MARKET` | ✓ | Immediate execution at best available price. | | `LIMIT` | ✓ | | | `STOP_MARKET` | ✓ | Stop‑loss conditional order, always long‑term. | | `STOP_LIMIT` | ✓ | Conditional order, always long‑term. | | `MARKET_IF_TOUCHED` | ✓ | Take‑profit market order, triggers on price touch. | | `LIMIT_IF_TOUCHED` | ✓ | Take‑profit limit order, triggers on price touch. | | `TRAILING_STOP_MARKET` | - | *Not supported*. | ### Execution instructions | Instruction | Perpetuals | Notes | |---------------|------------|--------------------------------------------------------------------------------------| | `post_only` | ✓ | Supported on LIMIT, STOP_LIMIT, and LIMIT_IF_TOUCHED orders. A post‑only order priced to cross the spread is **accepted then immediately canceled** by the venue (not rejected with a reason). | | `reduce_only` | ✓ | Passed for all order types. dYdX enforces this as a **fill‑time clamp**, not a placement‑time precondition: a reduce‑only order placed against no position will still fill normally. | ### Time in force options | Time in force | Perpetuals | Notes | |---------------|------------|----------------------------------------------------------------------------| | `GTC` | ✓ | Good Till Canceled. | | `GTD` | ✓ | Good Till Date. The venue reports expiry as a cancel event; the adapter maps this to `OrderExpired` (not `OrderCanceled`) when the order's `expire_time` has passed. | | `IOC` | ✓ | Immediate or Cancel. | | `FOK` | - | *Deprecated by dYdX v4*. The chain rejects FOK orders with `code=48`; the adapter generates `OrderDenied` locally and does not broadcast. | | `DAY` | - | *Not supported*. The adapter generates `OrderDenied` locally and does not broadcast. | ### Advanced order features | Feature | Perpetuals | Notes | |--------------------|------------|------------------| | Order modification | - | Not supported. dYdX supports short‑term order [replacement](https://docs.dydx.xyz/concepts/trading/limit-orderbook#replacements) (same ID, higher GTB); not yet exposed as `ModifyOrder`. | | Bracket/OCO orders | - | *Not supported*. | | Iceberg orders | - | *Not supported*. | ### Batch operations | Operation | Perpetuals | Notes | |--------------|------------|------------------------------------------------------------------------------------------------------------------------| | Batch submit | ✓ | Supported for long‑term `LIMIT` orders. Short‑term orders are submitted individually. | | Batch modify | - | *Not supported*. | | Batch cancel | ✓ | Partitioned: short‑term orders use `MsgBatchCancel` (single gRPC call), long‑term orders use batched `MsgCancelOrder`. | ### Position management | Feature | Perpetuals | Notes | |------------------|------------|-------------------------------| | Query positions | ✓ | Real‑time position updates. | | Position mode | - | Netting only (see below). | | Leverage control | ✓ | Per‑market leverage settings. | | Margin mode | - | Cross margin only. | :::note dYdX supports netting (one position per instrument) at the venue level. The adapter currently operates in `NETTING` mode only. Hedging support is planned for a future version. ::: ### Order querying | Feature | Perpetuals | Notes | |----------------------|------------|--------------------------------| | Query open orders | ✓ | List all active orders. | | Query order history | ✓ | Historical order data. | | Order status updates | ✓ | Real‑time order state changes. | | Trade history | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Perpetuals | Notes | |--------------------|------------|--------------------------------------------------| | Order lists | - | *Not supported*. | | OCO orders | - | *Not supported*. | | Bracket orders | - | *Not supported*. | | Conditional orders | ✓ | Stop, take‑profit market, and take‑profit limit. | ### Equity tier limit dYdX caps each subaccount at a fixed number of **simultaneous open conditional orders** based on the account's equity tier (e.g., 10 conditional orders for the standard tier). Submitting an additional conditional order beyond the cap is rejected on‑chain with `code=10001` and a log message of the form `Opening order would exceed equity tier limit of N`. Cancel existing conditional orders before placing more, or split strategies across subaccounts. ### MIT and LIT round‑tripping dYdX's protocol uses a single `TAKE_PROFIT` order type with a price (`subticks`) and trigger price; whether it behaves as market‑on‑trigger or limit‑on‑trigger is implicit in the price. The adapter submits Nautilus `MARKET_IF_TOUCHED` as a take‑profit with the price set to the 5% pay‑through worst‑case, and `LIMIT_IF_TOUCHED` as a take‑profit at the user's limit price. Both forms are returned by the Indexer as `"type":"TAKE_PROFIT"`. On reconciliation, the adapter compares the parsed limit price against the configured pay‑through tolerance to recover the original Nautilus order type. If the price is within the pay‑through band of the oracle, the order is reconciled as `MARKET_IF_TOUCHED`; otherwise it is reconciled as `LIMIT_IF_TOUCHED`. ### Liquidation and ADL (deleveraging) handling dYdX v4 applies two sequential risk mechanisms: 1. **Liquidation** runs when an account drops below its maintenance margin. Positions close against the insurance fund within a bounded spread from the oracle price. 2. **Deleveraging (ADL)** activates when either liquidation cannot fully restore collateralisation, or when a large oracle jump drives an account negative in a single step. Deleveraging closes the undercollateralised position against randomly selected offsetting accounts. The indexer exposes the classification via the `type` field on each `Fill` record (`DydxFillType`): | `type` | Meaning | |----------------|-------------------------------------------------------| | `LIMIT` | Normal fill. | | `LIQUIDATED` | Taker side of a liquidation (undercollateralised). | | `LIQUIDATION` | Maker side of a liquidation (insurance fund). | | `DELEVERAGED` | Taker side of a deleveraging (ADL closure). | | `OFFSETTING` | Maker side of a deleveraging (offsetting account). | The adapter logs a warning with instrument, side, size, and price for each liquidation / deleveraging fill, then emits the `FillReport` through the normal path. `DydxPerpetualPositionStatus::Liquidated` closes out the corresponding position report. Upstream references: - [Liquidations](https://docs.dydx.xyz/concepts/trading/liquidations) - [Contract loss mechanisms (deleveraging)](https://help.dydx.trade/en/articles/166973-contract-loss-mechanisms-on-dydx-chain) ### Order classification dYdX classifies every order into one of three on-chain categories. The Rust adapter automatically determines the category based on time-in-force and expiry, so no manual configuration is required. | Category | Placement | Expiry | Typical use | |-----------------|-------------|-------------------|-----------------------------------------------| | Short‑term | In‑memory | Block height | IOC/FOK, or orders expiring within 40 blocks. | | Long‑term | On‑chain | Timestamp (UTC) | GTC/GTD with expiry beyond the short‑term window (~20s at ~0.5s/block). | | Conditional | On‑chain | Timestamp (UTC) | Stop‑loss and take‑profit triggers. | At the protocol level, **all dYdX orders are limit orders**. The `MARKET` order type is a Nautilus convenience that the adapter implements as an aggressive IOC limit order priced well through the book. This means market orders follow the same `Submitted > Accepted > Filled` lifecycle as limit orders (an `OrderAccepted` event is expected before the fill). See the [dYdX order documentation](https://docs.dydx.xyz/concepts/trading/orders) for full protocol-level details on short-term vs stateful order mechanics. #### Short-term orders Short-term orders live **in validator memory only** and expire by block height (max 40 blocks, roughly ~20 seconds at ~0.5s/block). They are the fastest order type on dYdX because they skip on-chain storage. **Properties**: - **IOC and FOK are always short-term**, regardless of other parameters - **GTD orders** are automatically classified as short-term when the expiry falls within the dynamic short-term window (`40 blocks × seconds_per_block`) - Use Good-Til-Block (GTB) for replay protection instead of Cosmos SDK sequences - Can be broadcast **concurrently** (no semaphore, cached sequence) - Expire silently without generating cancel events - Cannot be batched in a single transaction (one `MsgPlaceOrder` per tx) #### Long-term orders Long-term (stateful) orders are **stored on-chain** and expire by UTC timestamp. They generate explicit cancel events when they expire or are cancelled. **Properties**: - **GTC** orders default to 90-day expiration (protocol limit is 95 days) - **GTD** orders use the user-provided expiry timestamp - Require proper Cosmos SDK sequence management (serialized via semaphore) - Must be broadcast **serially** with incrementing sequence numbers - Can be batched in a single transaction #### Conditional orders Conditional orders (stop-loss, take-profit) are **always stored on-chain** and triggered by price conditions on the validator. **Properties**: - Always use timestamp-based expiry (default 90 days for GTC, protocol limit 95 days) - Always use the long-term broadcast path (serialized with semaphore) - Include `StopMarket`, `StopLimit`, `TakeProfitMarket`, and `TakeProfitLimit` #### Automatic routing The adapter determines order lifetime automatically using the `BlockTimeMonitor`: ``` max_short_term_secs = SHORT_TERM_ORDER_MAXIMUM_LIFETIME (40) × seconds_per_block ``` If the order's time until expiry is within `max_short_term_secs`, it is routed as short-term. Otherwise, it is routed as long-term. No manual configuration is needed. #### MARKET order implementation dYdX has no native market order type. The adapter implements `MARKET` orders as aggressive **IOC limit orders** priced at: - **Buy**: `oracle_price × (1 + 0.05)` (5% above oracle) - **Sell**: `oracle_price × (1 - 0.05)` (5% below oracle) This 5% slippage buffer (`DEFAULT_MARKET_ORDER_SLIPPAGE = 0.05`) sets the worst-case price (the "pay-through price"). Because the order is IOC, unfilled slippage is not consumed. The buffer is intentionally wide to maximize fill probability across volatile conditions. ### Client order ID encoding dYdX requires `u32` client IDs on-chain, but Nautilus uses string-based `ClientOrderId` values (e.g., `O-20260220-031943-001-000-51`). The adapter encodes these bidirectionally so that orders can be reconciled across restarts without persisted state. For the standard O-format (`O-YYYYMMDD-HHMMSS-TTT-SSS-CCC`), the encoding is deterministic: | dYdX field | Bits | Contents | |-------------------|------|----------------------------------------------------| | `client_id` | 32 | `[trader:10][strategy:10][count:12]` (unique key). | | `client_metadata` | 32 | Seconds since 2020-01-01 UTC (timestamp). | Because the encoding is deterministic, the adapter can decode any reconciled order back to its original `ClientOrderId` string without needing a database or mapping file. Non-standard `ClientOrderId` formats (custom strings, plain numbers) fall back to sequential allocation with an in-memory reverse map. These IDs can only be decoded within the same session. #### Restart collision prevention On restart, Nautilus resets the internal order counter based on the number of reconciled orders, which may be lower than the highest counter value used in the previous session (e.g., if some orders have expired from the API response). This can cause a new order to produce the same `client_id` as a previous session's order, resulting in a duplicate venue order UUID. The adapter prevents this by registering every `client_id` seen during reconciliation. If a new O-format encoding produces a `client_id` that was already used, the encoder logs a warning and falls back to sequential allocation. Sequential allocation also skips any registered values. :::note This protection is automatic and requires no user configuration. The warning log `[ENCODER] client_id ... collides with reconciled order` is informational. The order will still be submitted successfully with an alternative ID. ::: ## Broadcasting and retry strategy ### Short-term broadcast Short-term orders use Good-Til-Block (GTB) for replay protection. The chain's `ClobDecorator` ante handler skips Cosmos SDK sequence checking for short-term messages, so: - **No semaphore**: broadcasts are fully concurrent - **Cached sequence**: no increment or allocation needed - **No retry**: if the broadcast fails, it fails immediately - Benign cancel errors are treated as success (see below) ### Long-term broadcast Long-term and conditional orders require proper Cosmos SDK sequence management: - **Semaphore** with 1 permit serializes all long-term broadcasts - **Exponential backoff**: 500ms -> 1s -> 2s -> 4s (max 5 retries) - **10-second total budget** prevents indefinite retry loops - On sequence mismatch, the sequence is **resynced from chain** before retry ### Sequence mismatch detection | Error code | Source | Meaning | |------------|----------------------|--------------------------------------------------| | `code=32` | Cosmos SDK | Account sequence mismatch | | `code=104` | dYdX authenticator | Signature verification failed (sequence‑related) | Both trigger automatic resync + retry via the `RetryManager`. ### Benign cancel errors These errors during short-term cancel operations are treated as **success**: | Error code | Meaning | |-------------|----------------------------------------------------------------| | `code=19` | Transaction already in mempool cache (duplicate tx) | | `code=9` | Cancel already exists in memclob with >= GoodTilBlock | | `code=3006` | Order to cancel does not exist (already filled/expired/cancelled) | ### Batch cancel partitioning When cancelling multiple orders, the adapter partitions them by lifetime: 1. **Short-term orders**: single `MsgBatchCancel` via `broadcast_short_term()` 2. **Long-term orders**: batched `MsgCancelOrder` messages via `broadcast_with_retry()` This ensures each group uses the appropriate broadcast strategy. ## Funding rates dYdX perpetual futures use a fixed 1-hour funding interval. The adapter sets `interval` to `60` (minutes) on all `FundingRateUpdate` objects for both WebSocket and historical funding data. ## Rate limiting ### gRPC rate limiting The adapter rate-limits gRPC `broadcast_tx` calls to prevent `ResourceExhausted` (429) errors from validator nodes. | Setting | Default | Description | |-------------------------------|---------|-------------------------------------------| | `grpc_rate_limit_per_second` | `4` | Maximum gRPC broadcast requests per second. Set to `None` to disable. | ### Provider limits Known rate limits for public gRPC providers: | Provider | Limit | Notes | |------------|--------------------|-----------------| | Polkachu | 300 req/min (~5/s) | | | KingNodes | 250 req/min (~4.2/s) | | | AutoStake | 4 req/s | | The default of 4 req/s is conservative and works across all public providers. ### Multiple gRPC URL fallback The execution config's `grpc_endpoint` field overrides the primary gRPC endpoint. It is a config-struct field and is not a parameter of the Python `DydxExecClientConfig` constructor. When `grpc_endpoint` is unset, the adapter uses the default public nodes for the selected network with built-in fallback across the public validator list. Explicit multi-URL fallback via user config is not currently exposed on the Python config. ## Price and size quantization dYdX uses integer-based quantization for prices and sizes. The adapter handles all conversions automatically via `OrderMessageBuilder`, but understanding the parameters helps with debugging. ### Market parameters | Parameter | Description | |--------------------------------|----------------------------------------------------------| | `atomic_resolution` | Exponent for converting human‑readable size to quantums | | `quantum_conversion_exponent` | Exponent for converting quantums to tokens | | `step_base_quantums` | Minimum order size step in quantums | | `subticks_per_tick` | Price granularity within each tick | ### Market order pricing Market orders use the oracle price with a 5% slippage buffer (the "pay-through price"): - **Buy**: `oracle_price × 1.05` - **Sell**: `oracle_price × 0.95` The oracle price is cached from the Indexer and refreshed periodically. ### Automatic handling All price and size quantization is handled automatically by `OrderMessageBuilder`. No manual conversion is needed when submitting orders through Nautilus. ## Data subscriptions The v4 adapter supports the following data subscriptions: | Data type | Subscription | Historical request | Notes | |----------------------|--------------|--------------------|-------------------------------------------------| | Trade ticks | ✓ | ✓ | | | Quote ticks | ✓ | - | Synthesized from order book top‑of‑book. | | Order book deltas | ✓ | - | L2 depth only. | | Order book snapshots | - | ✓ | One‑time snapshot via HTTP request. | | Bars | ✓ | ✓ | See supported resolutions below. | | Mark prices | ✓ | - | Via markets channel. | | Index prices | ✓ | - | Via markets channel. | | Funding rates | ✓ | ✓ | Real‑time via markets channel, history via HTTP. | | Instrument status | ✓ | - | Via markets channel. | ### Supported bar resolutions | Resolution | dYdX candle | |------------|-------------| | 1-MINUTE | `1MIN` | | 5-MINUTE | `5MINS` | | 15-MINUTE | `15MINS` | | 30-MINUTE | `30MINS` | | 1-HOUR | `1HOUR` | | 4-HOUR | `4HOURS` | | 1-DAY | `1DAY` | ## Subaccounts dYdX supports multiple subaccounts per wallet address, allowing segregation of trading strategies and risk management within a single wallet. ### Key concepts - Each wallet address can have multiple numbered subaccounts (0, 1, 2, ..., 127). - Subaccount 0 is the **default** and is automatically created on first deposit. - Each subaccount maintains its own: - Positions - Open orders - Collateral balance - Margin requirements ### Configuration Specify the subaccount number in the execution client config: ```python config = TradingNodeConfig( exec_clients={ "DYDX": DydxExecClientConfig( subaccount_number=0, # Default subaccount ), }, ) ``` :::note Most users will use subaccount `0` (the default). Advanced users can configure multiple execution clients for different subaccounts to implement strategy segregation or risk isolation. ::: ## Testnet setup The dYdX testnet (`dydx-testnet-4`) is a full replica of mainnet for testing strategies without risking real funds. All default testnet endpoints are resolved automatically when `network=DydxNetwork.TESTNET`. ### 1. Create a testnet wallet **Option A: Via the dYdX testnet web app (easiest)** 1. Go to [v4.testnet.dydx.exchange](https://v4.testnet.dydx.exchange) 2. Connect with MetaMask, Keplr, Phantom, or WalletConnect 3. A dYdX account is generated automatically 4. Export your secret phrase: click your address (top-right) and select "Export secret phrase" **Option B: Use an existing secp256k1 private key** Any 32-byte hex-encoded secp256k1 private key will work. The adapter derives the `dydx1...` address from the key automatically using Cosmos bech32 encoding. ### 2. Fund the testnet account A subaccount must be funded before the adapter can connect (see [First-time account activation](#architecture)). **Via the testnet web app:** Click the deposit/recharge button on [v4.testnet.dydx.exchange](https://v4.testnet.dydx.exchange) to receive testnet USDC automatically. **Via the faucet API directly:** ```bash # Fund subaccount 0 with 2000 USDC curl -X POST https://faucet.v4testnet.dydx.exchange/faucet/tokens \ -H "Content-Type: application/json" \ -d '{"address": "dydx1...", "subaccountNumber": 0, "amount": 2000}' # Fund native tokens (for gas fees) curl -X POST https://faucet.v4testnet.dydx.exchange/faucet/native-token \ -H "Content-Type: application/json" \ -d '{"address": "dydx1..."}' ``` ### 3. Set environment variables ```bash export DYDX_TESTNET_WALLET_ADDRESS="dydx1..." export DYDX_TESTNET_PRIVATE_KEY="0x..." # hex-encoded, 0x prefix optional ``` ### 4. Configure the trading node Set `network=DydxNetwork.TESTNET` on both data and execution clients: ```python from nautilus_trader.adapters.dydx import DydxNetwork config = TradingNodeConfig( ..., # Omitted data_clients={ DYDX: DydxDataClientConfig( wallet_address=None, # Falls back to DYDX_TESTNET_WALLET_ADDRESS env var instrument_provider=InstrumentProviderConfig(load_all=True), network=DydxNetwork.TESTNET, ), }, exec_clients={ DYDX: DydxExecClientConfig( wallet_address=None, # Falls back to DYDX_TESTNET_WALLET_ADDRESS env var private_key=None, # Falls back to DYDX_TESTNET_PRIVATE_KEY env var subaccount_number=0, instrument_provider=InstrumentProviderConfig(load_all=True), network=DydxNetwork.TESTNET, ), }, ) ``` ### Testnet endpoints Default testnet endpoints are used automatically. Override via the `http_endpoint`, `ws_endpoint`, or `grpc_endpoint` config-struct fields on the execution config if needed (these are not Python constructor parameters). | Service | Default URL | |-----------|------------------------------------------------------| | HTTP | `https://indexer.v4testnet.dydx.exchange` | | WebSocket | `wss://indexer.v4testnet.dydx.exchange/v4/ws` | | gRPC | `https://test-dydx-grpc.kingnodes.com:443` (primary) | | Faucet | `https://faucet.v4testnet.dydx.exchange` | | Web app | `https://v4.testnet.dydx.exchange` | ### Mainnet endpoints Default mainnet endpoints are used automatically. Override via the `http_endpoint`, `ws_endpoint`, or `grpc_endpoint` config-struct fields on the execution config if needed (these are not Python constructor parameters). | Service | Default URL | |-----------|-----------------------------------------------------| | HTTP | `https://indexer.dydx.trade` | | WebSocket | `wss://indexer.dydx.trade/v4/ws` | | gRPC | `https://dydx-ops-grpc.kingnodes.com:443` (primary) | ## Configuration Configure the dYdX adapter through the trading node configuration. Execution clients support environment variable fallbacks for credentials. Data clients use public endpoints and do not require wallet credentials. ### Data client configuration options | Option | Default | Description | |---------------------------|-----------|---------------------------------------------------------------------------------------------| | `wallet_address` | `None` | Legacy Python config field. The public data client does not use wallet credentials. | | `network` | `None` | `DydxNetwork.MAINNET` or `DydxNetwork.TESTNET`. | | `bars_timestamp_on_close` | `True` | If bar `ts_event` should be the bar close time. Set `False` to use venue‑native open time. | | `base_url_http` | `None` | HTTP API endpoint override. `None` selects the default for the selected network. | | `base_url_ws` | `None` | WebSocket endpoint override. `None` selects the default for the selected network. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `max_retries` | `3` | Maximum retry attempts for REST / WebSocket recovery. | | `retry_delay_initial_ms` | `100` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `5,000` | Maximum delay (milliseconds) between retries. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | `base_url_http` and `base_url_ws` are config-struct fields and are not parameters of the Python `DydxDataClientConfig` constructor. ### Execution client configuration options | Option | Default | Description | |--------------------------------|-----------|----------------------------------------------------------------------------------------------------| | `wallet_address` | `None` | dYdX wallet address. Falls back to `DYDX_WALLET_ADDRESS` / `DYDX_TESTNET_WALLET_ADDRESS` env var. | | `subaccount_number` | `0` | Subaccount number (0-127). Subaccount 0 is the default. | | `private_key` | `None` | Hex‑encoded private key for signing. Falls back to `DYDX_PRIVATE_KEY` / `DYDX_TESTNET_PRIVATE_KEY`.| | `authenticator_ids` | `None` | List of authenticator IDs for permissioned key trading (institutional setups). | | `network` | `None` | `DydxNetwork.MAINNET` or `DydxNetwork.TESTNET`. | | `http_endpoint` | `None` | HTTP client custom endpoint override. `None` selects the default for the selected network. | | `ws_endpoint` | `None` | WebSocket client custom endpoint override. `None` selects the default for the selected network. | | `grpc_endpoint` | `None` | gRPC client custom endpoint override. `None` selects the default for the selected network. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `max_retries` | `3` | Maximum retry attempts for submit/cancel/modify order operations. | | `retry_delay_initial_ms` | `1,000` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10,000` | Maximum delay (milliseconds) between retries. | | `grpc_rate_limit_per_second` | `4` | Maximum gRPC requests per second. Set to `None` to disable. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | `http_endpoint`, `ws_endpoint`, and `grpc_endpoint` are config-struct fields and are not parameters of the Python `DydxExecClientConfig` constructor. ### Basic setup Configure a live `TradingNode` to include dYdX data and execution clients: ```python from nautilus_trader.adapters.dydx import DydxDataClientConfig from nautilus_trader.adapters.dydx import DydxExecClientConfig from nautilus_trader.adapters.dydx import DydxNetwork from nautilus_trader.adapters.dydx.constants import DYDX from nautilus_trader.config import InstrumentProviderConfig from nautilus_trader.config import TradingNodeConfig config = TradingNodeConfig( ..., # Omitted data_clients={ DYDX: DydxDataClientConfig( wallet_address=None, # Falls back to env var instrument_provider=InstrumentProviderConfig(load_all=True), network=DydxNetwork.MAINNET, ), }, exec_clients={ DYDX: DydxExecClientConfig( wallet_address=None, # Falls back to env var private_key=None, # Falls back to env var subaccount_number=0, instrument_provider=InstrumentProviderConfig(load_all=True), network=DydxNetwork.MAINNET, ), }, ) ``` Then, create a `TradingNode` and register the client factories: ```python from nautilus_trader.adapters.dydx import DydxDataClientFactory from nautilus_trader.adapters.dydx import DydxExecutionClientFactory from nautilus_trader.adapters.dydx.constants import DYDX from nautilus_trader.live.node import TradingNode node = TradingNode(config=config) node.add_data_client_factory(DYDX, DydxDataClientFactory) node.add_exec_client_factory(DYDX, DydxExecutionClientFactory) node.build() ``` ### API credentials Credentials can be passed directly via the Python config (`wallet_address`, `private_key`) or resolved automatically from environment variables based on the configured `network`. #### Environment variables | Variable | Network | Description | |---------------------------------|----------|------------------------------------------------| | `DYDX_WALLET_ADDRESS` | Mainnet | Bech32-encoded wallet address (`dydx1...`). | | `DYDX_PRIVATE_KEY` | Mainnet | Hex‑encoded secp256k1 private key for signing. | | `DYDX_TESTNET_WALLET_ADDRESS` | Testnet | Testnet wallet address (`dydx1...`). | | `DYDX_TESTNET_PRIVATE_KEY` | Testnet | Testnet private key. | #### Resolution priority 1. Value passed in the Python config (if non-empty) 2. Environment variable selected by `network` ### Permissioned key trading #### What are API Trading Keys API Trading Keys let you delegate trading to a separate signing key without sharing your main wallet's seed phrase. The API key can place trades using all available margin in the owner's cross-margin account, but cannot withdraw funds or transfer assets. #### Creating an API key 1. In the dYdX web app, navigate to **More > API Trading Keys** 2. Click **Generate New API Key** 3. Save the **API Wallet Address** and **Private Key** (shown once, not stored by dYdX) 4. Click **Authorize API Key** (this registers the key on-chain as an authenticator) 5. The key is now active and can be used for trading See the [dYdX API Trading Keys guide](https://docs.dydx.xyz/concepts/trading/api-trading-keys) for full details on creating and managing API keys. #### Adapter configuration There are two ways to configure the adapter for API Trading Key usage: **Auto-resolution (recommended):** Set the API key's private key as `DYDX_PRIVATE_KEY` and the owner's wallet address as `DYDX_WALLET_ADDRESS`. The adapter detects the mismatch during connect and automatically queries the chain for matching authenticator IDs. No manual ID configuration needed. ```python config = DydxExecClientConfig( wallet_address="dydx1owner...", # Owner account (holds margin) private_key="0xapikey...", # API Trading Key private key # authenticator_ids resolved automatically ) ``` **Manual override:** If you know the authenticator IDs (e.g., from the dYdX TypeScript client), pass them directly to skip auto-resolution: ```python config = DydxExecClientConfig( wallet_address="dydx1owner...", private_key="0xapikey...", authenticator_ids=[1, 2], # Skip auto-resolution ) ``` :::note API Trading Keys only work with **cross-margin** accounts and cross markets. Isolated margin is not supported. ::: ## Order books Order books can be maintained at full depth or top-of-book quotes depending on the subscription. The venue does not provide quotes directly. Instead, the adapter subscribes to order book deltas and synthesizes quotes for the `DataEngine` when there is a top-of-book price or size change. Only L2 (MBP) book type is supported. ## Contributing :::info For additional features or to contribute to the dYdX adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Hyperliquid Source: https://nautilustrader.io/docs/latest/integrations/hyperliquid/ [Hyperliquid](https://hyperliquid.gitbook.io/hyperliquid-docs) is a decentralized perpetual futures and spot exchange built on the Hyperliquid L1, a purpose-built blockchain optimized for trading. HyperCore provides a fully on-chain order book and matching engine. This integration supports live market data ingest and order execution on Hyperliquid. ## Overview This adapter is implemented in Rust with Python bindings. It provides direct integration with Hyperliquid's REST and WebSocket APIs without requiring external client libraries. The Hyperliquid adapter includes multiple components: - `HyperliquidHttpClient`: Low-level HTTP API connectivity. - `HyperliquidWebSocketClient`: Low-level WebSocket API connectivity. - `HyperliquidInstrumentProvider`: Instrument parsing and loading functionality. - `HyperliquidDataClient`: Market data feed manager. - `HyperliquidExecutionClient`: Account management and trade execution gateway. - `HyperliquidDataClientFactory`: Factory for Hyperliquid data clients (used by the trading node builder). - `HyperliquidExecutionClientFactory`: Factory for Hyperliquid execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as shown below) and won't need to work directly with these lower-level components. ::: ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/hyperliquid/). ## Builder code attribution Submitted mainnet orders carry the NautilusTrader builder code at a **zero fee rate**, so attribution adds no trading cost. This helps us gauge real usage of the integration and prioritize ongoing maintenance. Users who attribute order flow may also qualify for direct support through the [Institutional](https://nautilustrader.io/institutional/) tier when trading at scale. You may opt out of attribution with `include_builder_attribution: false` in serialized config, or `include_builder_attribution=False` in Python. The builder address is omitted from orders in three cases: - **Testnet**: Hyperliquid testnet rejects orders that include a builder address the wallet has not explicitly approved (faucet-funded testnet wallets typically have no approval), so testnet orders never include the builder. - **Vault trading** (`vault_address` configured): Hyperliquid does not allow vaults to approve builder fees, so including the builder address would cause the exchange to reject the order. - **Attribution disabled** (`include_builder_attribution=False`): Users who choose not to attribute their order flow can disable builder attribution explicitly. ```python from nautilus_trader.adapters.hyperliquid import HyperliquidExecClientConfig config = HyperliquidExecClientConfig( include_builder_attribution=False, ) ``` ### Builder fee approval Hyperliquid requires a one-time `ApproveBuilderFee` approval before orders can carry the builder address: orders from a wallet that has never approved a builder fee are rejected with the reason `Builder fee has not been approved` (any prior approval, including at a 0% rate, satisfies the check). The approval must be signed by the master wallet's private key, which the adapter does not hold in agent (API) wallet setups, so it runs as a one-time script rather than at execution client startup. The 0% max fee rate permits attribution only: no builder fee is ever charged, and raising the rate would require a new approval signed by you. Run the approval script once per wallet (reads `HYPERLIQUID_PK`, or `HYPERLIQUID_TESTNET_PK` with `HYPERLIQUID_TESTNET=true`): ```bash cargo run -p nautilus-hyperliquid --bin hyperliquid-builder-fee-approve ``` Or from Python: ```python from nautilus_trader.adapters.hyperliquid import builder_fee_approve builder_fee_approve() ``` ### Revoking the approval Use revocation to cap a previously approved builder fee at 0% (for example, an approval from a version that charged builder fees). Revocation caps the fee; it does not remove the approval record, so attribution continues unless `include_builder_attribution` is disabled. ```bash cargo run -p nautilus-hyperliquid --bin hyperliquid-builder-fee-revoke ``` Or from Python: ```python from nautilus_trader.adapters.hyperliquid import builder_fee_revoke builder_fee_revoke() ``` The Rust scripts print a summary of the action and pause for an Enter keypress before signing; abort with `Ctrl+C` if anything in the summary looks wrong, or pass `--yes` to skip the prompt. The Python bindings do not prompt: review the active environment variables yourself before calling. ## Testnet setup Hyperliquid provides a testnet environment for testing strategies with mock funds. :::info **Mainnet account required.** Hyperliquid's testnet faucet only works for wallets that have previously deposited on mainnet. You must fund a mainnet account first before you can obtain testnet USDC. ::: ### Getting testnet funds To receive testnet USDC, you must first have deposited on **mainnet** using the same wallet address: 1. Visit the [Hyperliquid mainnet portal](https://app.hyperliquid.xyz/) and make a deposit with your wallet. 2. Visit the [testnet faucet](https://app.hyperliquid-testnet.xyz/drip) using the same wallet. 3. Claim 1,000 mock USDC from the faucet. :::note **Email wallet users**: Email login generates different addresses for mainnet vs testnet. To use the faucet, export your email wallet from mainnet, import it into MetaMask or Rabby, then connect the extension to testnet. ::: ### Creating a testnet account 1. Visit the [Hyperliquid testnet portal](https://app.hyperliquid-testnet.xyz/). 2. Connect your wallet (MetaMask, WalletConnect, or email). 3. The testnet automatically creates an account for your wallet address. ### Exporting your private key To use your testnet account with NautilusTrader, you need to export your wallet's private key: **MetaMask:** 1. Click the three dots menu next to your account. 2. Select "Account details". 3. Click "Show private key". 4. Enter your password and copy the private key. :::warning **Never share your private keys.** Store private keys securely using environment variables; never commit them to version control. ::: ### Setting environment variables Set your testnet credentials as environment variables: ```bash export HYPERLIQUID_TESTNET_PK="your_private_key_here" # Optional: for vault trading export HYPERLIQUID_TESTNET_VAULT="vault_address_here" ``` The adapter automatically loads these when `environment=HyperliquidEnvironment.TESTNET` in the configuration. :::warning **Agent / API wallets**: if `HYPERLIQUID_TESTNET_PK` is an [agent wallet](#agent-wallets) approved under a master account (the typical setup when you create an API wallet on the Hyperliquid UI), you must also set `HYPERLIQUID_ACCOUNT_ADDRESS` to the master account address. Without it, `OrderStatusReport` requests and WebSocket user feeds come back empty even though orders are live on the venue. See [GH-4010](https://github.com/nautechsystems/nautilus_trader/issues/4010). ::: ## Product support Hyperliquid offers linear perpetual futures, HIP-3 builder-deployed perpetuals, native spot markets, and HIP-4 binary outcome markets. | Product Type | Data Feed | Trading | Notes | |-------------------|-----------|---------|---------------------------------------------------------| | Spot | ✓ | ✓ | Native spot markets. | | Perpetual Futures | ✓ | ✓ | USDC‑settled linear perps (validator‑operated). | | HIP‑3 Perpetuals | ✓ | ✓ | Builder‑deployed perps with per‑dex collateral. Opt‑in. | | HIP‑4 Outcomes | ✓ | ✓ | USDH‑settled binary outcomes. Opt‑in. | :::note Standard Hyperliquid perpetuals are settled in USDC. HIP-3 dexes may settle in their own collateral token, such as USDH, USDE, or USDT0, while keeping Nautilus symbols quoted as `USD`. Spot markets are standard currency pairs. See [HIP-3 builder-deployed perpetuals](#hip-3-builder-deployed-perpetuals) and [HIP-4 outcome markets](#hip-4-outcome-markets) for configuration and opt-in details. Hyperliquid's current API docs mark `outcomeMeta` as testnet-only, so HIP-4 discovery depends on that payload being available from the selected environment. ::: ## Symbology Hyperliquid uses a specific symbol format for instruments: ### Spot markets Format: `{Base}-{Quote}-SPOT` Examples: - `PURR-USDC-SPOT` - PURR/USDC spot pair - `HYPE-USDC-SPOT` - HYPE/USDC spot pair To subscribe in your strategy: ```python InstrumentId.from_str("PURR-USDC-SPOT.HYPERLIQUID") ``` :::note Spot instruments may include vault tokens (prefixed with `vntls:`). These are automatically handled by the instrument provider. ::: ### Perpetual futures Format: `{Base}-USD-PERP` Examples: - `BTC-USD-PERP` - Bitcoin perpetual futures - `ETH-USD-PERP` - Ethereum perpetual futures - `SOL-USD-PERP` - Solana perpetual futures To subscribe in your strategy: ```python InstrumentId.from_str("BTC-USD-PERP.HYPERLIQUID") InstrumentId.from_str("ETH-USD-PERP.HYPERLIQUID") ``` ### HIP-3 perpetuals Format: `{dex}:{Asset}-USD-PERP` [HIP-3](https://hyperliquid.gitbook.io/hyperliquid-docs/hyperliquid-improvement-proposals-hips/hip-3-builder-deployed-perpetuals) markets use a dex prefix separated by a colon. The dex name identifies which builder-deployed perp dex the market belongs to. Examples: - `xyz:TSLA-USD-PERP` - Tesla perp on trade.xyz - `xyz:GOLD-USD-PERP` - Gold perp on trade.xyz - `flx:NVDA-USD-PERP` - Nvidia perp on Felix - `vntl:SPACEX-USD-PERP` - SpaceX perp on Ventuals To subscribe in your strategy: ```python InstrumentId.from_str("xyz:TSLA-USD-PERP.HYPERLIQUID") ``` ### HIP-4 outcome side tokens Format: `{outcome_index}-{YES|NO}-OUTCOME.HYPERLIQUID`, where `outcome_index` is the `outcome` field from `outcomeMeta` and the middle segment names the binary side. The `-OUTCOME` suffix is symmetric with `-PERP` / `-SPOT`. [HIP-4](https://hyperliquid.gitbook.io/hyperliquid-docs/hyperliquid-improvement-proposals-hips/hip-4-outcome-markets) side tokens are binary contracts that settle in USDH at `0` (loser) or `1` (winner). The Nautilus symbol uses the human-readable form above; the wire `raw_symbol` uses the venue coin form `#{encoding}` (where `encoding = 10 * outcome_index + side`, `side` is `0` for Yes / `1` for No), which is what `l2Book` and `allMids` accept. Examples (outcome 25): - `25-YES-OUTCOME.HYPERLIQUID`: Yes side. Encoding `250`, wire coin `#250`, token name `+250`, action asset id `100_000_250`. - `25-NO-OUTCOME.HYPERLIQUID`: No side. Encoding `251`, wire coin `#251`, token name `+251`, action asset id `100_000_251`. To subscribe in your strategy: ```python InstrumentId.from_str("25-YES-OUTCOME.HYPERLIQUID") ``` :::note The outcome universe cycles. Each settlement removes the resolved outcome from `outcomeMeta`, and the venue's next listing advances the index. Inspect the live universe with `curl -s -X POST https://api.hyperliquid.xyz/info -d '{"type":"outcomeMeta"}'`. ::: See [HIP-4 outcome markets](#hip-4-outcome-markets) for the trading flow, settlement, and current limitations. ## HIP-3 builder-deployed perpetuals [HIP-3](https://hyperliquid.gitbook.io/hyperliquid-docs/hyperliquid-improvement-proposals-hips/hip-3-builder-deployed-perpetuals) allows qualified deployers to launch permissionless perp dexes on Hyperliquid. These markets include equities (TSLA, NVDA, AAPL), commodities (gold, crude oil), indices (S&P 500), and pre-IPO tokens (SpaceX, OpenAI). In a live `TradingNode`, HIP-3 perpetuals load automatically alongside standard perpetuals at connect: the adapter fetches every perp dex (standard and builder-deployed) from `allPerpMetas`, so no additional client configuration is required. To narrow the loaded set, filter with `InstrumentProviderConfig`: ```python instrument_provider=InstrumentProviderConfig( load_all=True, filters={"market_types": ["perp_hip3"]}, ) ``` For direct `HyperliquidHttpClient` usage, the HIP-3 perp dexes are excluded unless you opt in through `load_instrument_definitions`: ```python from nautilus_trader.adapters.hyperliquid import HyperliquidEnvironment from nautilus_trader.adapters.hyperliquid import HyperliquidHttpClient client = HyperliquidHttpClient.from_env(HyperliquidEnvironment.MAINNET) instruments = await client.load_instrument_definitions( include_spot=True, include_perps=True, include_perps_hip3=True, include_outcomes=False, ) ``` ### Differences from standard perpetuals HIP-3 markets trade on the same HyperCore matching engine and use the same order API. The key differences are: - **Higher fees**: 2x standard perp fees by default. The deployer receives half. - **Isolated margin**: HIP-3 markets default to isolated-only margin. - **Per-dex collateral**: Each HIP-3 dex declares its settlement token through its `collateralToken` entry in `allPerpMetas`. Nautilus resolves that token through `spotMeta` and keeps the symbol's quote leg as `USD`. If a non-USDC collateral token cannot resolve from `spotMeta`, instrument loading returns an error rather than falling back to USDC. - **Deployer-managed oracles**: The deployer operates the oracle feed, not validators. - **Growth mode**: Some dexes enable growth mode, which reduces protocol fees by 90%. For full protocol details, see the Hyperliquid docs: - [HIP-3 proposal](https://hyperliquid.gitbook.io/hyperliquid-docs/hyperliquid-improvement-proposals-hips/hip-3-builder-deployed-perpetuals) - [HIP-3 deployer actions](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/hip-3-deployer-actions) - [Asset IDs](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/asset-ids) - [Fees](https://hyperliquid.gitbook.io/hyperliquid-docs/trading/fees) ### Wildcard character sanitization Some HIP-3 dexes deploy assets whose venue names contain `*` or `?` bytes (for example `dex:STREAMABCD****-USD-PERP`). Those bytes collide with the Nautilus message bus pattern syntax (`*` = zero-or-more, `?` = one-char) and would corrupt subscription routing if embedded in topic strings unchanged. The Hyperliquid adapter substitutes both bytes with `x` when constructing the `InstrumentId.symbol`, so a HIP-3 asset named `dex:STREAMABCD****` is exposed to strategies as: ```python InstrumentId.from_str("dex:STREAMABCDxxxx-USD-PERP.HYPERLIQUID") ``` The substitution applies only to the Nautilus-internal symbol used in topics, caches, logs, and config. The venue-official name is preserved on the instrument's `raw_symbol` field for HTTP and WebSocket wire calls, and order submissions reference the numeric asset index, so the round-trip with Hyperliquid is unaffected. When subscribing to a HIP-3 instrument with wildcard bytes in its venue name, use the sanitized form. Symbols without `*` or `?` are passed through unchanged. The substitution is lossy: two distinct venue names such as `dex:FOO*` and `dex:FOO?` would normalize onto the same Nautilus symbol. The instrument loader detects collisions, keeps the first definition, and logs a warning with the dropped venue name; the dropped instrument will not be tradeable through Nautilus until the venue rename resolves the collision. ## HIP-4 outcome markets [HIP-4](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/asset-ids#outcomes) markets are fully-collateralized binary contracts. Each market has two side tokens (Yes / No) that settle to `1 USDH` (winner) or `0 USDH` (loser) on the resolution date. Hyperliquid's current API docs expose outcome metadata through `outcomeMeta` and mark that endpoint as testnet-only. The adapter treats outcome metadata as best-effort and skips HIP-4 instruments when the venue does not return that payload. ### Loading outcome instruments In a live `TradingNode`, outcome instruments load automatically (best-effort) when the venue exposes `outcomeMeta`; current Hyperliquid docs mark that metadata endpoint as testnet-only, and the adapter skips HIP-4 instruments when the payload is unavailable. No client configuration is required. For direct `HyperliquidHttpClient` usage, opt in through `load_instrument_definitions`: ```python from nautilus_trader.adapters.hyperliquid import HyperliquidEnvironment from nautilus_trader.adapters.hyperliquid import HyperliquidHttpClient client = HyperliquidHttpClient.from_env(HyperliquidEnvironment.TESTNET) instruments = await client.load_instrument_definitions( include_spot=True, include_perps=True, include_perps_hip3=False, include_outcomes=True, ) ``` The provider emits two `BinaryOption` instruments per outcome (one per side), denominated in USDH. Symbols use the form `{outcome_index}-{YES|NO}-OUTCOME.HYPERLIQUID`. `expiration_ns` is parsed from the venue description (`expiry:YYYYMMDD-HHMM`, UTC). Standalone binaries carry their own expiry; named and fallback outcomes inherit from their parent question. Defaults: `0.0001` per tick, `0.01` per lot. Each instrument's `BinaryOption.info` carries the parsed venue metadata as a key/value map (consumed via `info["key"]` in Python or `Params.get_str(...)` in Rust). Derived identifiers are always populated; description-derived fields appear when the venue includes them. | Field | Source | Notes | |--------------------|--------------------------------|---------------------------------------------------| | `outcome_index` | derived | `outcome` from `outcomeMeta` | | `outcome_side` | derived | `0` = Yes, `1` = No | | `side_name` | derived | `"Yes"` or `"No"` | | `encoding` | derived | `10 * outcome_index + side` | | `asset_id` | derived | `100_000_000 + encoding` | | `market_name` | `outcomeMeta.outcomes[*].name` | venue market label | | `class` | description | `priceBinary` or `priceBucket` | | `underlying` | description | underlying asset code | | `expiry` | description | `YYYYMMDD-HHMM` UTC | | `target_price` | description | binary settlement threshold | | `period` | description | recurrence period (e.g. `1d`, `3m`) | | `price_thresholds` | description | comma‑separated thresholds (bucket markets) | | `named_index` | named‑outcome description | position in parent `named_outcomes` array | | `is_fallback` | fallback‑outcome description | `true` for the `other` outcome of a question | | `question` | parent question | question id | | `question_name` | parent question | question label | | `question_*` | parent question description | every parsed question field, `question_` prefixed | Description keys are lowered from venue camelCase to snake_case (`targetPrice` -> `target_price`, `priceThresholds` -> `price_thresholds`). Values are kept as strings to preserve wire fidelity; numeric identifiers (`outcome_index`, `outcome_side`, `encoding`, `asset_id`, `question`, `named_index`) are stored as JSON numbers. ### Settlement currency Outcomes settle in USDH (token index 360, traded on the `USDH/USDC` spot pair `@230`). The adapter registers USDH at 8-decimal precision on first outcome instrument creation, so `BinaryOption.currency`, `quote_currency`, and the commission currency on zero-fee outcome fills all resolve to USDH. USDH spot balances merge with the perp clearinghouse view, so `AccountState` carries USDH alongside USDC and any other non-zero spot holdings. ### Trading flow Outcome side tokens (`{outcome_index}-{YES|NO}-OUTCOME.HYPERLIQUID`) trade through the standard order path. Submit `SubmitOrder` as you would for any perp or spot instrument; the execution client routes it through the same `Order` action against the venue's `#{encoding}` orderbook (where `encoding = 10 * outcome_index + outcome_side`). No HIP-4-specific call is needed. Settlement is venue-driven; see [Settlement dispatch](#settlement-dispatch). #### Advanced workflows The full `userOutcome` action set is reachable directly on `HyperliquidHttpClient` (Rust and PyO3) for strategies that need to manage side-token inventory off-book: ```python from decimal import Decimal from nautilus_trader.core.nautilus_pyo3 import HyperliquidEnvironment from nautilus_trader.core.nautilus_pyo3 import HyperliquidHttpClient client = HyperliquidHttpClient.from_env(HyperliquidEnvironment.MAINNET) # Mint matched Yes + No side tokens from USDH (e.g. dual-side market making) await client.submit_split_outcome(50, Decimal("1.0")) # Burn a matched Yes + No pair back to USDH (amount=None merges the max) await client.submit_merge_outcome(50, None) # Multi-outcome priceBucket helpers await client.submit_merge_question(9, None) await client.submit_negate_outcome(9, 52, Decimal("1.0")) ``` | Action | Use case | |-------------------------|----------| | `submit_split_outcome` | Mint paired Yes + No tokens from quote (initial market making, dual‑side hedges) | | `submit_merge_outcome` | Burn a matched Yes + No pair back to quote without crossing the spread | | `submit_merge_question` | Close a full multi‑outcome basket back to quote atomically | | `submit_negate_outcome` | Convert No shares of one outcome into Yes shares of every other in the same question | For directional bets the ordinary `SubmitOrder` path is sufficient; the methods above are only needed when you want to create or destroy side-token inventory off-book. ### Order constraints Outcome side tokens behave like spot tokens (no margin, no funding, no liquidation). The execution client rejects features that don't apply: - `reduce_only` orders. - Trigger order types (`StopMarket`, `StopLimit`, `MarketIfTouched`, `LimitIfTouched`, trailing stops). `Limit` and `Market` orders with `GTC`, `IOC`, or `ALO` time-in-force are supported. The venue minimum is 10 USDH notional; size `order_qty` so that `order_qty * limit_price >= 10`. ### Settlement dispatch At expiry the venue closes held side-token balances and emits a `Settlement` fill per side. The adapter consumes these through the standard user-fills stream (HTTP poll and WebSocket); no synthetic dispatch runs. Each settlement fill: - `order_side = SELL`, zero commission. - Price `1` USDH for the winning side, `0` for the loser. - Surfaces as a `FillReport`. - Also emits `OrderFilled` when WebSocket dispatch links the position to a tracked order. Covers standalone `priceBinary` outcomes and multi-outcome `priceBucket` questions uniformly. ### Position reconciliation HIP-4 side tokens arrive on `spotClearinghouseState` with `coin` set to the `+E` token form and no `token` field. The adapter: - Treats `SpotBalance.token` as optional during deserialization. - Resolves `+E` / `#E` coins to their `BinaryOption` instrument when generating `PositionStatusReport`s. - Skips the perp clearinghouse fetch when the position-status filter is an outcome instrument (outcomes never appear in `assetPositions`). ### Multi-outcome (priceBucket) markets The venue exposes multi-outcome markets via the top-level `questions` array in `outcomeMeta`. Each question references a fallback outcome plus a sequence of named outcomes whose individual descriptions point back at the question via `index:N`. Each side token is modeled as an independent `BinaryOption` instrument; the `submit_merge_question` and `submit_negate_outcome` actions on `HyperliquidHttpClient` operate at the question level for basket close and cross-outcome rotation. ## Instrument provider The instrument provider supports filtering when loading instruments via `InstrumentProviderConfig(filters=...)`: | Filter key | Type | Description | |-----------------------------|-------------|---------------------------------------------| | `market_types` (or `kinds`) | `list[str]` | `"perp"`, `"perp_hip3"`, or `"spot"`. | | `bases` | `list[str]` | Base currency codes, e.g. `["BTC", "ETH"]`. | | `quotes` | `list[str]` | Quote currency codes, e.g. `["USDC"]`. | | `symbols` | `list[str]` | Full symbols, e.g. `["BTC-USD-PERP"]`. | Example loading only perpetual instruments: ```python instrument_provider=InstrumentProviderConfig( load_all=True, filters={"market_types": ["perp"]}, ) ``` ## Data subscriptions The adapter supports the following data subscriptions. All perpetual data types (mark prices, index prices, funding rates) apply to both standard and HIP-3 perps. | Data type | Sub. | Snapshot | Hist. | Nautilus type | Notes | |-------------------|------|----------|-------|-------------------------------|---------------------------------------| | Trade ticks | ✓ | - | ✓ | `TradeTick` | WebSocket trades; `recentTrades`. | | Quote ticks | ✓ | - | - | `QuoteTick` | Best bid/offer. | | Order book deltas | ✓ | ✓ | - | `OrderBookDelta` | L2 snapshots. | | Order book depth | ✓ | - | - | `OrderBookDepth10` | Top-10 L2 snapshots. | | Bars | ✓ | - | ✓ | `Bar` | Supported intervals below. | | Mark prices | ✓ | - | - | `MarkPriceUpdate` | Perpetual mark price ticks. | | Index prices | ✓ | - | - | `IndexPriceUpdate` | Underlying reference prices. | | Funding rates | ✓ | - | ✓ | `FundingRateUpdate` | `fundingHistory` endpoint. | | Open interest | ✓ | - | - | `HyperliquidOpenInterest` | Custom data from `activeAssetCtx`. | | All mids | ✓ | - | - | `HyperliquidAllMids` | Custom data from `allMids`. | | All dex contexts | ✓ | - | - | `HyperliquidAllDexsAssetCtxs` | Custom data from `allDexsAssetCtxs`. | :::note Historical quote requests are not supported. Historical trade requests use the `recentTrades` info endpoint, which returns a recent snapshot of public trades (newest first) with no time range. `request_trades` filters that snapshot to the requested `[start, end]` window and applies `limit` by keeping the most recent trades. When the request reaches below the snapshot's oldest trade, the adapter logs a warning and serves the available subset (or an empty response). The endpoint depends on the Hyperliquid indexer: self-hosted `/info` nodes return HTTP 422, which the adapter treats as no coverage and answers with an empty response. Real-time trades remain available via the WebSocket `trades` channel. ::: ### Order book precision controls The `l2Book` subscription accepts optional `nSigFigs` and `mantissa` parameters that thin the venue-side book aggregation. The adapter forwards them when passed through `subscribe_params` on book deltas and depth subscriptions. Hyperliquid accepts `nSigFigs` values `2`, `3`, `4`, `5`, or omitted for full precision. `mantissa` is only valid when `nSigFigs=5` and accepts `1`, `2`, or `5`. ```python from nautilus_trader.model.data import BookType self.subscribe_order_book_deltas( instrument_id=instrument_id, book_type=BookType.L2_MBP, params={"n_sig_figs": 5, "mantissa": 2}, ) ``` Omitting both params subscribes to the full-depth book. ### Hyperliquid specific data The adapter emits two Hyperliquid-specific custom data types: - `HyperliquidAllMids` from the WebSocket `allMids` feed. Each update carries all currently reported mid prices in one payload. - `HyperliquidAllDexsAssetCtxs` from the WebSocket `allDexsAssetCtxs` feed. Each update carries normalized per-instrument asset-context entries across the default perp dex and HIP-3 builder dexes. - `HyperliquidOpenInterest` from the shared `activeAssetCtx` feed used by mark prices, index prices, and funding rates. | Field | Type | Description | |------------|------------------|----------------------------------------------------------| | `mids` | `dict[str, str]` | Instrument ID to mid price mapping. | | `ts_event` | `int` | UNIX timestamp in nanoseconds when the update occurred. | | `ts_init` | `int` | UNIX timestamp in nanoseconds when the object was built. | Subscribe from an actor or strategy with `DataType(HyperliquidAllMids)`. For HIP-3 dex-specific streams, pass the venue dex in `metadata["dex"]`: ```python from nautilus_trader.adapters.hyperliquid.constants import HYPERLIQUID_CLIENT_ID from nautilus_trader.adapters.hyperliquid.data import HyperliquidAllMids from nautilus_trader.model.data import DataType self.subscribe_data( data_type=DataType(HyperliquidAllMids, metadata={"dex": "hyperliquid"}), client_id=HYPERLIQUID_CLIENT_ID, ) ``` `HyperliquidOpenInterest` carries the latest open interest for one perpetual instrument. Subscribe with the canonical Nautilus `instrument_id` in `metadata["instrument_id"]`: | Field | Type | Description | |-----------------|----------------|-----------------------------------------------------------------------------| | `instrument_id` | `InstrumentId` | Canonical Nautilus instrument ID. | | `open_interest` | `Decimal` | Open interest parsed for direct arithmetic use. | | `ts_event` | `int` | UNIX timestamp in nanoseconds when the update occurred. Mirrors `ts_init`. | | `ts_init` | `int` | UNIX timestamp in nanoseconds when the object was built. | ```python from nautilus_trader.adapters.hyperliquid import HYPERLIQUID_CLIENT_ID from nautilus_trader.adapters.hyperliquid import HyperliquidOpenInterest from nautilus_trader.model.data import DataType self.subscribe_data( data_type=DataType( HyperliquidOpenInterest, metadata={"instrument_id": str(self.instrument_id)}, ), client_id=HYPERLIQUID_CLIENT_ID, ) ``` `HyperliquidOpenInterest` reuses the same single underlying `activeAssetCtx` venue subscription that already backs mark prices, index prices, and funding rates for the same coin. Adding OI does not open a second parallel `activeAssetCtx` subscription. In a Python strategy running inside a `TradingNode`, the payload is delivered to `on_data` as the concrete custom data type itself: ```python from decimal import Decimal from nautilus_trader.adapters.hyperliquid import HyperliquidOpenInterest def on_data(self, data) -> None: if isinstance(data, HyperliquidOpenInterest): if data.open_interest > Decimal("1000"): self.log.info(f"OI {data.instrument_id} -> {data.open_interest}") ``` `HyperliquidAllDexsAssetCtxs` exposes a whole-feed aggregate rather than one topic per instrument, so strategies subscribe once and filter the normalized entries they need: | Field | Type | Description | |-------------------|-----------------------------------|----------------------------------------------------------------------------| | `dex` | `str` | Perp dex identifier from Hyperliquid `perpDexs`. `""` is the default dex. | | `instrument_id` | `InstrumentId` | Canonical Nautilus instrument ID for the entry. | | `mark_price` | `Price` | Current mark price. | | `oracle_price` | `Price` | Current oracle / index reference price. | | `prev_day_price` | `Price` | Previous day reference price from the venue payload. | | `mid_price` | `Price \| None` | Mid price when present in the venue payload. | | `impact_prices` | `HyperliquidImpactPrices \| None` | Best bid / ask impact prices when present. | | `funding_rate` | `Decimal` | Funding rate parsed for direct arithmetic use. | | `open_interest` | `Decimal` | Open interest parsed for direct arithmetic use. | | `premium` | `Decimal \| None` | Premium when present in the venue payload. | | `day_ntl_volume` | `Decimal` | 24h notional volume. | | `day_base_volume` | `Decimal` | 24h base volume. | | `ts_event` | `int` | UNIX timestamp in nanoseconds when the update occurred. Mirrors `ts_init`. | | `ts_init` | `int` | UNIX timestamp in nanoseconds when the object was built. | The underlying Hyperliquid wire payload arrives as `ctxs: [[dex, ctxs[]], ...]`. The adapter decodes that live venue format and normalizes it into the per-entry output shown below before the strategy sees the data. The adapter does not invent `dex` values. It bootstraps the ordered dex universe from Hyperliquid `meta` / `allPerpMetas` and resolves builder dex identifiers from the live `perpDexs` info endpoint. The empty string `""` represents Hyperliquid's default perp dex; non-empty values such as `xyz`, `flx`, or `vntl` are venue-defined builder dex identifiers. The mapping is resolved from the instruments loaded at connect, and the feed is positional (no per-entry coin name), so perps listed later only appear after a reconnect. A context-count mismatch for a dex logs a warning to reconnect; entries stay aligned positionally, which is correct for appended listings. ```python from nautilus_trader.adapters.hyperliquid import HYPERLIQUID_CLIENT_ID from nautilus_trader.adapters.hyperliquid import HyperliquidAllDexsAssetCtxs from nautilus_trader.model.data import DataType self.subscribe_data( data_type=DataType(HyperliquidAllDexsAssetCtxs), client_id=HYPERLIQUID_CLIENT_ID, ) def on_data(self, data) -> None: if isinstance(data, HyperliquidAllDexsAssetCtxs): for entry in data.entries: if entry.dex == "xyz": self.log.info(f"{entry.instrument_id} OI={entry.open_interest}") ``` ### Supported bar intervals | Resolution | Hyperliquid candle | |------------|--------------------| | 1-MINUTE | `1m` | | 3-MINUTE | `3m` | | 5-MINUTE | `5m` | | 15-MINUTE | `15m` | | 30-MINUTE | `30m` | | 1-HOUR | `1h` | | 2-HOUR | `2h` | | 4-HOUR | `4h` | | 8-HOUR | `8h` | | 12-HOUR | `12h` | | 1-DAY | `1d` | | 3-DAY | `3d` | | 1-WEEK | `1w` | | 1-MONTH | `1M` | ## Orders capability Hyperliquid supports a full set of order types and execution options. :::note In the tables below, "Perpetuals" covers both standard validator-operated perps and HIP-3 builder-deployed perps. The same order types, time-in-force options, and execution instructions apply to both. ::: ### Order types | Order Type | Perpetuals | Spot | Notes | |---------------------|------------|------|-----------------------------------------------------| | `MARKET` | ✓ | ✓ | IOC limit with configurable slippage from best BBO. | | `LIMIT` | ✓ | ✓ | | | `STOP_MARKET` | ✓ | ✓ | Stop loss orders. | | `STOP_LIMIT` | ✓ | ✓ | Stop loss with limit execution. | | `MARKET_IF_TOUCHED` | ✓ | ✓ | Take profit at market. | | `LIMIT_IF_TOUCHED` | ✓ | ✓ | Take profit with limit execution. | :::info Conditional orders (stop and if-touched) are implemented using Hyperliquid's native trigger order functionality with automatic TP/SL mode detection. All trigger orders are evaluated against the [mark price](https://hyperliquid.gitbook.io/hyperliquid-docs/trading/robust-price-indices). ::: :::note Market orders require cached quote data. The adapter uses the best ask (for buys) or best bid (for sells) with a configurable slippage buffer (default 50 bps). Prices are rounded to Hyperliquid's price constraints before submission. Ensure you subscribe to quotes for any instrument you intend to trade with market orders. The slippage buffer is controlled by `market_order_slippage_bps` on `HyperliquidExecClientConfig` (default 50 bps) and can be overridden per-order via the `market_order_slippage_bps` key in `SubmitOrder.params`. ::: :::note `STOP_MARKET` and `MARKET_IF_TOUCHED` orders do not carry a limit price. The adapter derives one from the trigger price with the same configurable slippage buffer (default 50 bps), rounds to 5 significant figures, and clamps to the venue decimal limit (ceiling for buys, floor for sells). This guarantees Hyperliquid's `limit_px >= trigger_px` (buys) / `limit_px <= trigger_px` (sells) constraint. ::: :::warning **Price normalization is enabled by default.** Hyperliquid enforces a maximum of 5 significant figures on order prices, plus a per-asset decimal limit based on `szDecimals` (`6 - szDecimals` for perps, `8 - szDecimals` for spot). For example, if ETH is trading at $2,600 (4 integer digits), only 1 decimal place is allowed despite the instrument having `price_precision=2`. By default, the adapter normalizes all outgoing limit and trigger prices to 5 significant figures and clamps them to the instrument price precision to prevent order rejections. This means your submitted prices may shift slightly. To disable this and take full control of price formatting, set `normalize_prices=False` in your `HyperliquidExecClientConfig`. If you disable normalization, you can apply the same rounding in your strategy: ```python from decimal import Decimal, ROUND_DOWN def round_to_sig_figs(price: Decimal, sig_figs: int = 5) -> Decimal: if price == 0: return Decimal(0) shift = sig_figs - int(price.adjusted()) - 1 if shift <= 0: factor = Decimal(10) ** (-shift) return (price / factor).to_integral_value() * factor return round(price, shift) ``` ::: ### Time in force | Time in force | Perpetuals | Spot | Notes | |---------------|------------|------|----------------------| | `GTC` | ✓ | ✓ | Good Till Canceled. | | `IOC` | ✓ | ✓ | Immediate or Cancel. | | `FOK` | - | - | *Not supported*. | | `GTD` | - | - | *Not supported*. | ### Execution instructions | Instruction | Perpetuals | Spot | Notes | |---------------|------------|------|----------------------------------| | `post_only` | ✓ | ✓ | Equivalent to ALO time in force. | | `reduce_only` | ✓ | ✓ | Close‑only orders. | :::info Post-only orders that would immediately match are rejected by Hyperliquid. The adapter detects this and generates an `OrderRejected` event. Post-only orders are routed through Hyperliquid's ALO (Add-Liquidity-Only) lane. ::: ### Order operations | Operation | Perpetuals | Spot | Notes | |-------------------|------------|------|-------------------------------------------------------| | Submit order | ✓ | ✓ | Single order submission. | | Submit order list | ✓ | ✓ | Batch order submission (single API call). | | Modify order | ✓ | ✓ | Requires venue order ID. | | Cancel order | ✓ | ✓ | Cancel by client order ID. | | Cancel all orders | ✓ | ✓ | Single batched `cancelByCloid` for open orders. | | Batch cancel | ✓ | ✓ | Single batched `cancelByCloid` for the provided list. | :::info When the venue returns an authoritative per-order rejection inside a batch-cancel response (for example `MissingOrder` for an already-terminal order), the adapter emits a per-order `OrderCancelRejected` event and leaves the other cancels intact. Whole-request failures with unknown venue outcome do not carry this per-order evidence. ::: :::info Orders placed outside NautilusTrader (e.g. via the Hyperliquid web UI or another client) are detected and tracked as external orders. They appear in order status reports and position reconciliation. ::: ### Modify as cancel-replace Hyperliquid implements order modification as a **cancel-replace**. The `modify` action on the [exchange endpoint](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/exchange-endpoint#modify-an-order) cancels the original order (old `oid`) and opens a replacement with a new `oid`. Both legs share the same client order ID (`cloid`). The modify HTTP response only confirms success. The [`orderUpdates` WebSocket subscription](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/websocket/subscriptions) then delivers an `ACCEPTED(new_oid)` status report, followed by a `CANCELED(old_oid)` for the original leg. The Rust-native `HyperliquidExecutionClient` (used through `HyperliquidExecutionClientFactory`) runs detection, deduplication, and event promotion on the Rust side via the [`WsDispatchState`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/hyperliquid/src/websocket/dispatch.rs) owned by the execution client. On submission the client registers an `OrderIdentity` (strategy, instrument, side, type, quantity, last-known price) keyed by `client_order_id`. Each inbound status report or fill is routed through the dispatch: tracked orders emit typed `OrderEventAny::*` events via `ExecutionEventEmitter::send_order_event`; external orders fall back to the raw `OrderStatusReport` / `FillReport` so the engine can reconcile. The dispatch compares the report's `venue_order_id` against the last cached value for the `cloid`; when they differ it promotes the `ACCEPTED` to `OrderUpdated` and suppresses the paired stale cancel: ```mermaid sequenceDiagram participant Strategy participant ExecClient as HyperliquidExecutionClient (Rust) participant Dispatch as WsDispatchState (Rust) participant HTTP as Hyperliquid HTTP participant WS as Hyperliquid WS Strategy->>ExecClient: ModifyOrder(cloid, old_oid) ExecClient->>HTTP: POST /exchange { action: "modify", oid: old_oid } HTTP-->>ExecClient: { status: "ok" } ExecClient->>Dispatch: mark_pending_modify(cloid, old_oid) WS-->>ExecClient: ACCEPTED(new_oid, cloid) ExecClient->>Dispatch: dispatch_order_event() Dispatch->>Dispatch: cached_voi != new_oid -> promote to OrderUpdated,
clear_pending_modify, record_venue_order_id(new_oid) Dispatch-->>Strategy: OrderUpdated(venue_order_id=new_oid) WS-->>ExecClient: CANCELED(old_oid, cloid) ExecClient->>Dispatch: dispatch_order_event() Dispatch->>Dispatch: cached_voi != old_oid -> Skip (stale cancel) ``` If Hyperliquid delivers `CANCELED(old_oid)` before `ACCEPTED(new_oid)` for an in-flight modify, the pending-modify marker lets the dispatch drop the old leg's cancel and still route the subsequent `ACCEPTED` through the `OrderUpdated` path. The marker is only set after a confirmed HTTP success, so a failed modify never leaves stale race state. Because detection otherwise relies on the cached `venue_order_id`, the adapter also recovers a modify that times out on the HTTP call but still reaches the venue: the eventual WS `ACCEPTED(new_oid)` sees the old cached `oid` and translates to `OrderUpdated`. See [GH-3827](https://github.com/nautechsystems/nautilus_trader/issues/3827). The same marker guards the inflight query and single-order reconcile paths. While a modify is in flight, `query_order` and `generate_order_status_report` drop a `Canceled` for the superseded leg, so an out-of-band status probe that resolves the old `oid` before the replacement appears cannot terminate the live order. A non-cancel status for the old leg (such as a late `Filled`) is still forwarded so reconciliation can recover it. These paths also promote the replacement. Hyperliquid lists the replacement under the same `cloid` with a new `oid` in `frontendOpenOrders`, so when the replacement `ACCEPTED(new_oid)` was dropped on the WebSocket and no fill has arrived, the query resolves it by `cloid` and promotes it to `OrderUpdated` directly (rebinding the `cloid` to `new_oid` and clearing the pending-modify marker). The order is therefore not left bound to the canceled leg, and subsequent modifies and cancels target the live replacement. See [GH-4270](https://github.com/nautechsystems/nautilus_trader/issues/4270). :::note One narrow edge case remains when all three conditions occur together: 1. The modify HTTP call raises (transport timeout or connection error). 2. Hyperliquid still processes the modify on the exchange side. 3. Hyperliquid delivers `CANCELED(old_oid)` before `ACCEPTED(new_oid)` on the WebSocket. Under (1) the pending-modify marker is not installed, so the early `CANCELED(old_oid)` emits as `OrderCanceled` before the replacement `ACCEPTED(new_oid)` arrives. The periodic reconciliation cycle restores the correct order state against the exchange. ::: A `FillReport` for the replacement leg can also race ahead of `ACCEPTED(new_oid)`. The dispatch buffers such fills (when the pending-modify marker is set and the report's `oid` does not match the cached value) and drains them on the matching `ACCEPTED`, so `OrderFilled` always follows the promoting `OrderUpdated` against up-to-date state. See [GH-3972](https://github.com/nautechsystems/nautilus_trader/issues/3972). :::note A chained-modify edge case is deferred: if a delayed fill from a *prior* leg arrives during a *new* in-flight modify and that new modify then fails, the buffered fill is stranded until terminal cleanup. Reconciliation (`request_fill_reports`) recovers it. Fully closing this requires additional design work (retired-VOI tracking or drain on modify-failure paths). ::: ## Order books Order books are maintained via L2 WebSocket subscription. Each message delivers a full-depth snapshot (clear + rebuild), not incremental deltas. :::note There is a limitation of one order book per instrument per trader instance. ::: ## Account and position management `AccountState` merges perp margin and spot balances. Perp margin and cross-margin usage come from `clearinghouseState`; non-zero spot tokens (USDC, USDH, HYPE, vault tokens, HIP-4 outcome side tokens, etc.) come from `spotClearinghouseState`. USDC is deduplicated when the perp summary is present. Standard perps default to cross margin; HIP-3 perps default to isolated. On connect, the execution client reconciles orders, fills, and positions against Hyperliquid's clearinghouse state. Spot positions are reconstructed from held balances (long-only); HIP-4 side tokens reconcile against their matching `BinaryOption` instruments. :::note Leverage is managed directly through the Hyperliquid web UI or API, not through the adapter. Set your desired leverage per instrument on Hyperliquid before trading. ::: ## Liquidation and ADL handling Hyperliquid signals venue-initiated closures through two WebSocket surfaces on the `userEvents` subscription: - **`liquidation` event**: emitted when an account is liquidated. Carries a `liquidation ID`, liquidator address, liquidated user, liquidated notional position, and liquidated account value. The adapter logs these at warning level for operator visibility. - **Fill-level `liquidation` metadata**: each entry in the `fills` array can carry an optional `liquidation` object with `method`, `markPx`, and `liquidatedUser`. The `method` value is either `market` (liquidated into the book) or `backstop` (closed against the backstop vault, the equivalent of an ADL close when the insurance mechanism steps in). The adapter emits the standard `FillReport` for each liquidation fill. The liquidation metadata is logged alongside the fill so you can correlate closures to venue-side events. No strategy-side changes are required; existing risk and reconciliation logic runs over these fills as for any other TAKER fill. Upstream references: - [WebSocket `userEvents` (`liquidation` and `FillLiquidation`)](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/websocket/subscriptions) - [Liquidation mechanics](https://hyperliquid.gitbook.io/hyperliquid-docs/trading/liquidations) ## Connection management The adapter automatically reconnects on WebSocket disconnection using exponential backoff (starting at 250ms, up to 5s). On reconnect, all active subscriptions are resubscribed automatically, and order book snapshots are rebuilt. No manual intervention is required. A heartbeat ping is sent every 30 seconds to keep the connection alive (Hyperliquid closes idle connections after 60 seconds). ## API credentials There are two options for supplying your credentials to the Hyperliquid clients. Either pass the corresponding values to the configuration objects, or set the following environment variables: For Hyperliquid mainnet clients, you can set: - `HYPERLIQUID_PK` - `HYPERLIQUID_VAULT` (optional, for vault trading) For Hyperliquid testnet clients, you can set: - `HYPERLIQUID_TESTNET_PK` - `HYPERLIQUID_TESTNET_VAULT` (optional, for vault trading) For agent (API) wallet trading on either environment, you can also set: - `HYPERLIQUID_ACCOUNT_ADDRESS` (master account address; shared between mainnet and testnet) :::tip We recommend using environment variables to manage your credentials. ::: ## Agent wallets Hyperliquid lets a master account approve an [agent wallet](https://hyperliquid.gitbook.io/hyperliquid-docs/for-developers/api/nonces-and-api-wallets) (also called an API wallet or sub-key) that signs orders on the master's behalf. Orders signed by the agent belong to the master account, not to the agent's address. If your `HYPERLIQUID_PK` (or `HYPERLIQUID_TESTNET_PK`) is an agent wallet, you must also set `account_address` (or the `HYPERLIQUID_ACCOUNT_ADDRESS` environment variable) to the master account address. Otherwise the adapter queries the agent's address for balances, orders, and WebSocket events, which owns nothing, and submitted orders will never reconcile (no `OrderStatusReport`, no fills surfaced). The execution factory resolves one account address and passes that same value to REST account queries and WebSocket user subscriptions. Signing still uses the configured private key, and vault trading still sends `vaultAddress` in the signed exchange payload when `vault_address` is set. Explicit config values take precedence over environment variables. Environment variables fill only omitted config values. Resolution order for the execution account address used by info queries and WebSocket subscriptions: 1. `account_address` (master account when using an agent wallet). 2. `vault_address` (vault sub-account). 3. `HYPERLIQUID_ACCOUNT_ADDRESS`. 4. `HYPERLIQUID_VAULT` or `HYPERLIQUID_TESTNET_VAULT`. 5. The address derived from the private key (the wallet itself). :::note `HYPERLIQUID_ACCOUNT_ADDRESS` is a single env var shared by both mainnet and testnet (unlike `HYPERLIQUID_PK` / `HYPERLIQUID_TESTNET_PK`). If your agent wallet is approved under the same master address on both environments, one value covers both. ::: :::tip Email-login wallets generate different addresses for mainnet and testnet, so the master address may differ. In that case, prefer setting `account_address` explicitly in `HyperliquidExecClientConfig` per environment rather than relying on the shared environment variable. ::: ## Vault trading Hyperliquid supports [vault trading](https://hyperliquid.gitbook.io/hyperliquid-docs/trading/vaults), where a wallet operates on behalf of a vault (sub-account). Orders are signed with the wallet's private key but include the vault address in the signature payload. To trade via a vault, set the `vault_address` in your execution client config (or set the `HYPERLIQUID_VAULT` / `HYPERLIQUID_TESTNET_VAULT` environment variable). :::warning For normal vault trading, leave `account_address` unset so `vault_address` becomes the account address used for REST queries and WebSocket user subscriptions. If both `account_address` and `vault_address` are set, `account_address` wins for queries and subscriptions, while `vault_address` still goes into the signed exchange payload. ::: ## Funding rates Hyperliquid perpetual futures use a fixed 1-hour funding interval. The adapter sets `interval` to `60` (minutes) on all `FundingRateUpdate` objects. ## Rate limiting The adapter implements a token bucket rate limiter for Hyperliquid's REST API with a capacity of 1200 weight per minute. HTTP info requests are automatically retried with exponential backoff (full jitter) on rate limit (429) and server error (5xx) responses. For WebSocket post trading requests, the adapter caps simultaneous inflight messages at 100 to match the venue limit. ## Configuration ### Data client configuration options | Option | Default | Description | |---------------------|-----------|-------------------------------------------------| | `environment` | `None` | Environment enum (`MAINNET` or `TESTNET`). | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `http_timeout_secs` | `60` | Timeout (seconds) applied to REST calls. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |--------------------------------|-----------|-------------------------------------------------------------------------------------------| | `private_key` | `None` | EVM private key; loaded from `HYPERLIQUID_PK` or `HYPERLIQUID_TESTNET_PK` when omitted. | | `vault_address` | `None` | Vault address; loaded from `HYPERLIQUID_VAULT` or `HYPERLIQUID_TESTNET_VAULT` if omitted. | | `account_address` | `None` | Main account address for agent wallet trading; loaded from `HYPERLIQUID_ACCOUNT_ADDRESS`. | | `environment` | `None` | Environment enum (`MAINNET` or `TESTNET`); resolves to `MAINNET` when unset. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `max_retries` | `3` | Maximum retry attempts for submit, cancel, or modify order requests. | | `retry_delay_initial_ms` | `100` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `5000` | Maximum delay (milliseconds) between retries. | | `http_timeout_secs` | `60` | Timeout (seconds) applied to REST calls. | | `normalize_prices` | `True` | Normalize order prices to 5 significant figures before submission. | | `include_builder_attribution` | `True` | Include zero‑fee Nautilus builder attribution on eligible mainnet orders. | | `market_order_slippage_bps` | `50` | Slippage buffer (bps) applied to MARKET and stop trigger derivations. Overridable per‑order via `SubmitOrder.params`. | | `outcome_settlement_poll_secs` | `0` | HIP‑4 `outcomeMeta` settlement poll interval (seconds). Rust‑only; venue `Settlement` fills cover settlement, so polling is disabled by default. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | :::note `outcome_settlement_poll_secs` is the only Rust-only option: it is not exposed on the `HyperliquidExecClientConfig` Python constructor and always uses its default. The `max_retries`, `retry_delay_initial_ms`, and `retry_delay_max_ms` fields are accepted on both the Rust and Python config but are not yet consumed by the execution client (its HTTP client is constructed with only the request timeout and proxy). ::: ### Configuration example ```python from nautilus_trader.adapters.hyperliquid import HYPERLIQUID from nautilus_trader.adapters.hyperliquid import HyperliquidDataClientConfig from nautilus_trader.adapters.hyperliquid import HyperliquidEnvironment from nautilus_trader.adapters.hyperliquid import HyperliquidExecClientConfig from nautilus_trader.config import InstrumentProviderConfig from nautilus_trader.config import TradingNodeConfig config = TradingNodeConfig( data_clients={ HYPERLIQUID: HyperliquidDataClientConfig( instrument_provider=InstrumentProviderConfig(load_all=True), environment=HyperliquidEnvironment.TESTNET, ), }, exec_clients={ HYPERLIQUID: HyperliquidExecClientConfig( private_key=None, # Loads from HYPERLIQUID_TESTNET_PK env var vault_address=None, # Optional: loads from HYPERLIQUID_TESTNET_VAULT instrument_provider=InstrumentProviderConfig(load_all=True), environment=HyperliquidEnvironment.TESTNET, normalize_prices=True, # Rounds prices to 5 significant figures ), }, ) ``` :::note When `environment=HyperliquidEnvironment.TESTNET`, the adapter automatically uses testnet environment variables (`HYPERLIQUID_TESTNET_PK` and `HYPERLIQUID_TESTNET_VAULT`) instead of mainnet variables. ::: Then, create a `TradingNode` and add the client factories: ```python from nautilus_trader.adapters.hyperliquid import HYPERLIQUID from nautilus_trader.adapters.hyperliquid import HyperliquidDataClientFactory from nautilus_trader.adapters.hyperliquid import HyperliquidExecutionClientFactory from nautilus_trader.live.node import TradingNode # Instantiate the live trading node with a configuration node = TradingNode(config=config) # Register the client factories with the node node.add_data_client_factory(HYPERLIQUID, HyperliquidDataClientFactory) node.add_exec_client_factory(HYPERLIQUID, HyperliquidExecutionClientFactory) # Finally build the node node.build() ``` ## Contributing :::info For additional features or to contribute to the Hyperliquid adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Interactive Brokers Source: https://nautilustrader.io/docs/latest/integrations/ib/ Interactive Brokers (IB) is a trading platform providing market access across a wide range of financial instruments, including stocks, options, futures, currencies, bonds, funds, and cryptocurrencies. NautilusTrader offers an adapter to integrate with IB using their [Trader Workstation (TWS) API](https://ibkrcampus.com/ibkr-api-page/trader-workstation-api/) through their Python library, [ibapi](https://github.com/nautechsystems/ibapi). The TWS API is an interface to IB's standalone trading applications: TWS and IB Gateway. Both can be downloaded from the IB website. If you haven't installed TWS or IB Gateway yet, refer to the [Initial Setup](https://ibkrcampus.com/ibkr-api-page/trader-workstation-api/#tws-download) guide. In NautilusTrader, you'll establish a connection to one of these applications via the `InteractiveBrokersClient`. Alternatively, you can start with a [dockerized version](https://github.com/gnzsnz/ib-gateway-docker) of the IB Gateway, which is particularly useful when deploying trading strategies on a hosted cloud platform. This requires having [Docker](https://www.docker.com/) installed on your machine, along with the [docker](https://pypi.org/project/docker/) Python package, which NautilusTrader conveniently includes as an extra package. :::note The standalone TWS and IB Gateway applications require manually inputting username, password, and trading mode (live or paper) at startup. The dockerized version of the IB Gateway handles these steps programmatically. ::: ## Installation To install NautilusTrader with Interactive Brokers (and Docker) support: ```bash uv pip install "nautilus_trader[ib,docker]" ``` To build from source with all extras (including IB and Docker): ```bash uv sync --all-extras ``` :::note Because IB does not provide wheels for `ibapi`, NautilusTrader [repackages](https://pypi.org/project/nautilus-ibapi/) it for release on PyPI. ::: ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/interactive_brokers/). ## Getting started Before implementing your trading strategies, make sure that either TWS (Trader Workstation) or IB Gateway is running. You can log in to one of these standalone applications with your credentials, or connect programmatically via `DockerizedIBGateway`. :::warning Configure TWS or IB Gateway to return market data timestamps in UTC before connecting NautilusTrader. This setting must be enabled by the user in TWS/IB Gateway, as NautilusTrader is designed to work with UTC timestamps. ::: ### Connection methods There are two primary ways to connect to Interactive Brokers: 1. **Connect to an existing TWS or IB Gateway instance** 2. **Use the dockerized IB Gateway (recommended for automated deployments)** ### Default ports Interactive Brokers uses different default ports depending on the application and trading mode: | Application | Paper Trading | Live Trading | |-------------|---------------|--------------| | TWS | 7497 | 7496 | | IB Gateway | 4002 | 4001 | ### Establish connection to an existing gateway or TWS When connecting to a pre-existing gateway or TWS, specify the `ibg_host` and `ibg_port` parameters in both the `InteractiveBrokersDataClientConfig` and `InteractiveBrokersExecClientConfig`: ```python from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersDataClientConfig from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersExecClientConfig # Example for TWS paper trading (default port 7497) data_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=1, ) exec_config = InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=1, account_id="DU123456", # Your paper trading account ID ) ``` ### Establish connection to Dockerized IB Gateway For automated deployments, the dockerized gateway is recommended. Supply `dockerized_gateway` with an instance of `DockerizedIBGatewayConfig` in both client configurations. The `ibg_host` and `ibg_port` parameters are not needed as they're managed automatically. ```python from nautilus_trader.adapters.interactive_brokers.config import DockerizedIBGatewayConfig from nautilus_trader.adapters.interactive_brokers.gateway import DockerizedIBGateway gateway_config = DockerizedIBGatewayConfig( username="your_username", # Or set TWS_USERNAME env var password="your_password", # Or set TWS_PASSWORD env var trading_mode="paper", # "paper" or "live" read_only_api=True, # Set to False to allow order execution timeout=300, # Startup timeout in seconds ) # This may take a short while to start up, especially the first time gateway = DockerizedIBGateway(config=gateway_config) gateway.start() # Confirm you are logged in print(gateway.is_logged_in(gateway.container)) # Inspect the logs print(gateway.container.logs()) ``` ### Environment variables To supply credentials to the Interactive Brokers Gateway, either pass the `username` and `password` to the `DockerizedIBGatewayConfig`, or set the following environment variables: - `TWS_USERNAME`: Your IB account username. - `TWS_PASSWORD`: Your IB account password. - `TWS_ACCOUNT`: Your IB account ID (used as the fallback for `account_id`). ### Connection management The adapter includes connection management features: - **Automatic reconnection**: Configure retries with the `IB_MAX_CONNECTION_ATTEMPTS` environment variable. - **Connection timeout**: Adjust the timeout with the `connection_timeout` parameter (default: 300 seconds). - **Connection watchdog**: Monitor connection health and trigger reconnection automatically when required. - **Graceful error handling**: Handle diverse connection scenarios with error classification. ## Overview The Interactive Brokers adapter provides an integration with IB's TWS API. The adapter includes several major components: ### Core components - **`InteractiveBrokersClient`**: The central client that executes TWS API requests using `ibapi`. Manages connections, handles errors, and coordinates all API interactions. - **`InteractiveBrokersDataClient`**: Connects to the Gateway for streaming market data including quotes, trades, and bars. - **`InteractiveBrokersExecutionClient`**: Handles account information, order management, and trade execution. - **`InteractiveBrokersInstrumentProvider`**: Retrieves and manages instrument definitions, including support for options and futures chains. - **`HistoricInteractiveBrokersClient`**: Provides methods for retrieving instruments and historical data, useful for backtesting and research. ### Supporting components - **`DockerizedIBGateway`**: Manages dockerized IB Gateway instances for automated deployments. - **Configuration classes**: Provide configuration options for all components. - **Factory classes**: Create and configure client instances with the necessary dependencies. ### Supported asset classes The adapter supports trading across all major asset classes available through Interactive Brokers: - **Equities**: Stocks, ETFs, and equity options. - **Fixed income**: Bonds and bond funds. - **Derivatives**: Futures, options, and warrants. - **Foreign exchange**: Spot FX and FX forwards. - **Cryptocurrencies**: Bitcoin, Ethereum, and other digital assets. - **Commodities**: Physical commodities and commodity futures. - **Indices**: Index products and index options. ## The Interactive Brokers client The `InteractiveBrokersClient` is the central component of the IB adapter, overseeing a range of functions. These include establishing and maintaining connections, handling API errors, executing trades, and gathering various types of data such as market data, contract/instrument data, and account details. The `InteractiveBrokersClient` is divided into specialized mixin classes, each handling a specific responsibility. ### Client architecture The client uses a mixin-based architecture where each mixin handles a specific aspect of the IB API: #### Connection management (`InteractiveBrokersClientConnectionMixin`) - Establishes and maintains socket connections to TWS/Gateway. - Handles connection timeouts and reconnection logic. - Manages connection state and health monitoring. - Supports configurable reconnection attempts via `IB_MAX_CONNECTION_ATTEMPTS` environment variable. #### Error handling (`InteractiveBrokersClientErrorMixin`) - Processes all API errors and warnings. - Categorizes errors by type (client errors, connectivity issues, request errors). - Handles subscription and request-specific error scenarios. - Provides error logging and debugging information. #### Account management (`InteractiveBrokersClientAccountMixin`) - Retrieves account information and balances. - Manages position data and portfolio updates. - Handles multi-account scenarios. - Processes account-related notifications. #### Contract/instrument management (`InteractiveBrokersClientContractMixin`) - Retrieves contract details and specifications. - Handles instrument searches and lookups. - Manages contract validation and verification. - Supports complex instrument types (options chains, futures chains). #### Market data management (`InteractiveBrokersClientMarketDataMixin`) - Handles real-time and historical market data subscriptions. - Processes quotes, trades, and bar data. - Manages market data type settings (real-time, delayed, frozen). - Handles tick-by-tick data and market depth. #### Order management (`InteractiveBrokersClientOrderMixin`) - Processes order placement, modification, and cancellation. - Handles order status updates and execution reports. - Manages order validation and error handling. - Supports complex order types and conditions. ### Key features - **Asynchronous operation**: All operations are fully asynchronous using Python's asyncio. - **Error handling**: Error categorization and handling. - **Connection resilience**: Automatic reconnection with configurable retry logic. - **Message processing**: Efficient message queue processing for high-throughput scenarios. - **State management**: Proper state tracking for connections, subscriptions, and requests. :::tip To troubleshoot TWS API incoming message issues, consider starting at the `InteractiveBrokersClient._process_message` method, which acts as the primary gateway for processing all messages received from the API. ::: ## Symbology The `InteractiveBrokersInstrumentProvider` supports three methods for constructing `InstrumentId` instances, which can be configured via the `symbology_method` enum in `InteractiveBrokersInstrumentProviderConfig`. ### Symbology methods #### 1. Simplified symbology (`IB_SIMPLIFIED`) - default When `symbology_method` is set to `IB_SIMPLIFIED` (the default setting), the system uses intuitive, human-readable symbology rules: **Format Rules by Asset Class:** - **Forex**: `{symbol}/{currency}.{exchange}` - Example: `EUR/USD.IDEALPRO` - **Stocks**: `{localSymbol}.{primaryExchange}` - Spaces in localSymbol are replaced with hyphens - Example: `BF-B.NYSE`, `SPY.ARCA` - **Futures**: `{localSymbol}.{exchange}` - Individual contracts use single digit years - Example: `ESM4.CME`, `CLZ7.NYMEX` - **Continuous Futures**: `{symbol}.{exchange}` - Represents front month, automatically rolling - Example: `ES.CME`, `CL.NYMEX` - **Options on Futures (FOP)**: `{localSymbol}.{exchange}` - Format: `{symbol}{month}{year} {right}{strike}` - Example: `ESM4 C4200.CME` - **Options**: `{localSymbol}.{exchange}` - All spaces removed from localSymbol - Example: `AAPL230217P00155000.SMART` - **Indices**: `^{localSymbol}.{exchange}` - Example: `^SPX.CBOE`, `^NDX.NASDAQ` - **Bonds**: `{localSymbol}.{exchange}` - Example: `912828XE8.SMART` - **Cryptocurrencies**: `{symbol}/{currency}.{exchange}` - Example: `BTC/USD.PAXOS`, `ETH/USD.PAXOS` #### 2. Raw symbology (`IB_RAW`) Setting `symbology_method` to `IB_RAW` enforces stricter parsing rules that align directly with the fields defined in the IB API. This method provides maximum compatibility across all regions and instrument types: **Format Rules:** - **CFDs**: `{localSymbol}={secType}.IBCFD` - **Commodities**: `{localSymbol}={secType}.IBCMDTY` - **Default for Other Types**: `{localSymbol}={secType}.{exchange}` **Examples:** - `IBUS30=CFD.IBCFD` - `XAUUSD=CMDTY.IBCMDTY` - `EUR.USD=CASH.IDEALPRO` - `AAPL=STK.SMART` This configuration ensures explicit instrument identification and supports instruments from any region, especially those with non-standard symbology where simplified parsing may fail. ### MIC venue conversion The adapter supports converting Interactive Brokers exchange codes to Market Identifier Codes (MIC) for standardized venue identification: #### `convert_exchange_to_mic_venue` When set to `True`, the adapter automatically converts IB exchange codes to their corresponding MIC codes: ```python instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( convert_exchange_to_mic_venue=True, # Enable MIC conversion symbology_method=SymbologyMethod.IB_SIMPLIFIED, ) ``` **Examples of MIC Conversion:** - `CME` -> `XCME` (Chicago Mercantile Exchange) - `NASDAQ` -> `XNAS` (Nasdaq Stock Market) - `NYSE` -> `XNYS` (New York Stock Exchange) - `LSE` -> `XLON` (London Stock Exchange) #### `symbol_to_mic_venue` Symbol-prefix to MIC venue overrides. Applied **first** in venue resolution, independent of `convert_exchange_to_mic_venue`. When a contract's symbol matches a configured prefix, that MIC venue is used; otherwise resolution uses exchange (and optionally MIC conversion if `convert_exchange_to_mic_venue` is True). Useful for OPT contracts with exchange SMART (e.g. SPX -> XCBO) and for aligning with databento-style instrument IDs. ```python instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( symbol_to_mic_venue={ "SPX": "XCBO", # OPT with exchange SMART -> XCBO "ES": "XCME", # All ES futures/options use CME MIC "SPY": "ARCX", # SPY specifically uses ARCA }, ) # convert_exchange_to_mic_venue can be True or False; symbol_to_mic_venue is applied first ``` #### Venue resolution and `_process_contract_details` When loading instruments via `IBContract`, the provider passes `venue=None` into `_process_contract_details`, so each contract detail gets its own venue (via `symbol_to_mic_venue`, validExchanges, and MIC conversion). Callers that pass a single venue string still get one venue for all details. To get per-detail resolution when you have mixed or SMART-routed results, pass `venue=None`. ### Supported instrument formats The adapter supports various instrument formats based on Interactive Brokers' contract specifications: #### Futures month codes - **F** = January, **G** = February, **H** = March, **J** = April - **K** = May, **M** = June, **N** = July, **Q** = August - **U** = September, **V** = October, **X** = November, **Z** = December #### Supported exchanges by asset class **Futures Exchanges:** - `CME`, `CBOT`, `NYMEX`, `COMEX`, `KCBT`, `MGE`, `NYBOT`, `SNFE` **Options Exchanges:** - `SMART` (IB's smart routing) **Forex Exchanges:** - `IDEALPRO` (IB's forex platform) **Cryptocurrency Exchanges:** - `PAXOS` (IB's crypto platform) **CFD/Commodity Exchanges:** - `IBCFD`, `IBCMDTY` (IB's internal routing) ### Choosing the right symbology method - **Use `IB_SIMPLIFIED`** (default) for most use cases - provides clean, readable instrument IDs - **Use `IB_RAW`** when dealing with complex international instruments or when simplified parsing fails - **Enable `convert_exchange_to_mic_venue`** when you need standardized MIC venue codes for compliance or data consistency ## Instruments and contracts In Interactive Brokers, a NautilusTrader `Instrument` corresponds to an IB [Contract](https://ibkrcampus.com/ibkr-api-page/trader-workstation-api/#contracts). The adapter handles two types of contract representations: ### Contract types #### Basic contract (`IBContract`) - Contains essential contract identification fields - Used for contract searches and basic operations - Cannot be directly converted to a NautilusTrader `Instrument` #### Contract details (`IBContractDetails`) - Contains contract information including: - Order types supported - Trading hours and calendar - Margin requirements - Price increments and multipliers - Market data permissions - Can be converted to a NautilusTrader `Instrument` - Required for trading operations ### Contract discovery To search for contract information, use the [IB Contract Information Center](https://pennies.interactivebrokers.com/cstools/contract_info/). ### Loading instruments There are two primary methods for loading instruments: Interactive Brokers does not support loading the full IB instrument universe with `load_all=True`. Configure `load_ids` or `load_contracts` for the instruments a node needs at startup, or request an instrument explicitly before subscribing to its market data. #### 1. Using `load_ids` (recommended) Use `symbology_method=SymbologyMethod.IB_SIMPLIFIED` (default) with `load_ids` for clean, intuitive instrument identification: For FX instruments, use slash-separated symbols such as `EUR/USD.IDEALPRO`. The dotted local symbol form belongs to raw symbology, for example `EUR.USD=CASH.IDEALPRO`. ```python from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersInstrumentProviderConfig from nautilus_trader.adapters.interactive_brokers.config import SymbologyMethod instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( symbology_method=SymbologyMethod.IB_SIMPLIFIED, load_ids=frozenset([ "EUR/USD.IDEALPRO", # Forex "SPY.ARCA", # Stock "ESM24.CME", # Future "BTC/USD.PAXOS", # Crypto "^SPX.CBOE", # Index ]), ) ``` #### 2. Using `load_contracts` (for complex instruments) Use `load_contracts` with `IBContract` instances for complex scenarios like options/futures chains: ```python from nautilus_trader.adapters.interactive_brokers.common import IBContract # Load options chain for specific expiry options_chain_expiry = IBContract( secType="IND", symbol="SPX", exchange="CBOE", build_options_chain=True, lastTradeDateOrContractMonth='20240718', ) # Load options chain for date range options_chain_range = IBContract( secType="IND", symbol="SPX", exchange="CBOE", build_options_chain=True, min_expiry_days=0, max_expiry_days=30, ) # Load futures chain futures_chain = IBContract( secType="CONTFUT", exchange="CME", symbol="ES", build_futures_chain=True, ) instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( load_contracts=frozenset([ options_chain_expiry, options_chain_range, futures_chain, ]), ) ``` ### IBContract examples by asset class ```python from nautilus_trader.adapters.interactive_brokers.common import IBContract # Stocks IBContract(secType='STK', exchange='SMART', primaryExchange='ARCA', symbol='SPY') IBContract(secType='STK', exchange='SMART', primaryExchange='NASDAQ', symbol='AAPL') # Bonds IBContract(secType='BOND', secIdType='ISIN', secId='US03076KAA60') IBContract(secType='BOND', secIdType='CUSIP', secId='912828XE8') # Individual Options IBContract(secType='OPT', exchange='SMART', symbol='SPY', lastTradeDateOrContractMonth='20251219', strike=500, right='C') # Options Chain (loads all strikes/expirations) IBContract(secType='STK', exchange='SMART', primaryExchange='ARCA', symbol='SPY', build_options_chain=True, min_expiry_days=10, max_expiry_days=60) # CFDs IBContract(secType='CFD', symbol='IBUS30') IBContract(secType='CFD', symbol='DE40EUR', exchange='SMART') # Individual Futures IBContract(secType='FUT', exchange='CME', symbol='ES', lastTradeDateOrContractMonth='20240315') # Futures Chain (loads all expirations) IBContract(secType='CONTFUT', exchange='CME', symbol='ES', build_futures_chain=True) # Options on Futures (FOP) - Individual IBContract(secType='FOP', exchange='CME', symbol='ES', lastTradeDateOrContractMonth='20240315', strike=4200, right='C') # Options on Futures Chain (loads all strikes/expirations) IBContract(secType='CONTFUT', exchange='CME', symbol='ES', build_options_chain=True, min_expiry_days=7, max_expiry_days=60) # Forex IBContract(secType='CASH', exchange='IDEALPRO', symbol='EUR', currency='USD') IBContract(secType='CASH', exchange='IDEALPRO', symbol='GBP', currency='JPY') # Cryptocurrencies IBContract(secType='CRYPTO', symbol='BTC', exchange='PAXOS', currency='USD') IBContract(secType='CRYPTO', symbol='ETH', exchange='PAXOS', currency='USD') # Indices IBContract(secType='IND', symbol='SPX', exchange='CBOE') IBContract(secType='IND', symbol='NDX', exchange='NASDAQ') # Commodities IBContract(secType='CMDTY', symbol='XAUUSD', exchange='SMART') ``` ### Advanced configuration options ```python # Options chain with custom exchange IBContract( secType="STK", symbol="AAPL", exchange="SMART", primaryExchange="NASDAQ", build_options_chain=True, options_chain_exchange="CBOE", # Use CBOE for options instead of SMART min_expiry_days=7, max_expiry_days=45, ) # Futures chain with specific months IBContract( secType="CONTFUT", exchange="NYMEX", symbol="CL", # Crude Oil build_futures_chain=True, min_expiry_days=30, max_expiry_days=180, ) ``` ### Continuous futures For continuous futures contracts (using `secType='CONTFUT'`), the adapter creates instrument IDs using just the symbol and venue: ```python # Continuous futures examples IBContract(secType='CONTFUT', exchange='CME', symbol='ES') # -> ES.CME IBContract(secType='CONTFUT', exchange='NYMEX', symbol='CL') # -> CL.NYMEX # With MIC venue conversion enabled instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( convert_exchange_to_mic_venue=True, ) # Results in: # ES.XCME (instead of ES.CME) # CL.XNYM (instead of CL.NYMEX) ``` **Continuous Futures vs Individual Futures:** - **Continuous**: `ES.CME` - Represents the front month contract, automatically rolls - **Individual**: `ESM4.CME` - Specific March 2024 contract :::note When using `build_options_chain=True` or `build_futures_chain=True`, the `secType` and `symbol` should be specified for the underlying contract. The adapter will automatically discover and load all related derivative contracts within the specified expiry range. ::: ## Option spreads Interactive Brokers supports option spreads through BAG contracts, which combine multiple option legs into a single tradeable instrument. NautilusTrader provides support for creating, loading, and trading option spreads. ### Creating option spread instrument IDs Option spreads are created using the `new_generic_spread_id()` function, which combines individual option legs with their respective ratios: ```python from nautilus_trader.model.identifiers import InstrumentId, new_generic_spread_id # Create individual option instrument IDs call_leg = InstrumentId.from_str("SPY C400.SMART") put_leg = InstrumentId.from_str("SPY P390.SMART") # Create a 1:1 call spread (long call, short call) call_spread_id = new_generic_spread_id([ (call_leg, 1), # Long 1 contract (put_leg, -1), # Short 1 contract ]) # Create a 1:2 ratio spread ratio_spread_id = new_generic_spread_id([ (call_leg, 1), # Long 1 contract (put_leg, 2), # Long 2 contracts ]) ``` ### Dynamic spread loading Option spreads must be requested before they can be traded or subscribed to for market data. Use the `request_instrument()` method to dynamically load spread instruments: ```python # In your strategy's on_start method def on_start(self): # Request the spread instrument self.request_instrument(spread_id) def on_instrument(self, instrument): # Handle the loaded spread instrument self.log.info(f"Loaded spread: {instrument.id}") # Now you can subscribe to market data self.subscribe_quote_ticks(instrument.id) # And place orders order = self.order_factory.market( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(1), time_in_force=TimeInForce.DAY, ) self.submit_order(order) ``` ### Spread trading requirements 1. **Load individual legs first**: Ensure the individual option legs are available before creating spreads. 2. **Request the spread instrument**: Use `request_instrument()` to load the spread before trading. 3. **Subscribe to market data**: Request quote ticks after the spread is loaded. 4. **Place orders**: Any order type can be used once the spread is available. ## Historical data and backtesting The `HistoricInteractiveBrokersClient` provides methods for retrieving historical data from Interactive Brokers for backtesting and research purposes. ### Supported data types - **Bar data**: OHLCV bars with time, tick, and volume aggregations. - **Tick data**: Trade ticks and quote ticks with microsecond precision. - **Instrument data**: Complete contract specifications and trading rules. ### Historical data client ```python from nautilus_trader.adapters.interactive_brokers.historical.client import HistoricInteractiveBrokersClient from ibapi.common import MarketDataTypeEnum # Initialize the client client = HistoricInteractiveBrokersClient( host="127.0.0.1", port=7497, client_id=1, market_data_type=MarketDataTypeEnum.DELAYED_FROZEN, # Use delayed data if no subscription log_level="INFO" ) # Connect to TWS/Gateway await client.connect() ``` ### Retrieving instruments #### Basic instrument retrieval ```python from nautilus_trader.adapters.interactive_brokers.common import IBContract # Define contracts contracts = [ IBContract(secType="STK", symbol="AAPL", exchange="SMART", primaryExchange="NASDAQ"), IBContract(secType="STK", symbol="MSFT", exchange="SMART", primaryExchange="NASDAQ"), IBContract(secType="CASH", symbol="EUR", currency="USD", exchange="IDEALPRO"), ] # Request instrument definitions instruments = await client.request_instruments(contracts=contracts) ``` #### Option chain retrieval with catalog storage You can download entire option chains using `request_instruments` in your strategy, with the added benefit of saving the data to the catalog using `update_catalog=True`: ```python # In your strategy's on_start method def on_start(self): self.request_instruments( venue=IB_VENUE, update_catalog=True, params={ "ib_contracts": ( # SPY options { "secType": "STK", "symbol": "SPY", "exchange": "SMART", "primaryExchange": "ARCA", "build_options_chain": True, "min_expiry_days": 7, "max_expiry_days": 30, }, # QQQ options { "secType": "STK", "symbol": "QQQ", "exchange": "SMART", "primaryExchange": "NASDAQ", "build_options_chain": True, "min_expiry_days": 7, "max_expiry_days": 30, }, # ES futures options { "secType": "CONTFUT", "exchange": "CME", "symbol": "ES", "build_options_chain": True, "min_expiry_days": 0, "max_expiry_days": 60, }, # SPX index options { "secType": "IND", "symbol": "SPX", "exchange": "CBOE", "build_options_chain": True, "min_expiry_days": 0, "max_expiry_days": 5, }, # ES futures chain and futures options { "secType": "CONTFUT", "exchange": "CME", "symbol": "ES", "build_futures_chain": True, "build_options_chain": True, "min_expiry_days": 0, "max_expiry_days": 2, }, # ESTX50 index options (Eurex) { "secType": "IND", "exchange": "EUREX", "symbol": "ESTX50", "build_options_chain": True, "min_expiry_days": 0, "max_expiry_days": 2, }, ), }, ) ``` ### Retrieving historical bars ```python import datetime # Request historical bars bars = await client.request_bars( bar_specifications=[ "1-MINUTE-LAST", # 1-minute bars using last price "5-MINUTE-MID", # 5-minute bars using midpoint "1-HOUR-LAST", # 1-hour bars using last price "1-DAY-LAST", # Daily bars using last price ], start_date_time=datetime.datetime(2023, 11, 1, 9, 30), end_date_time=datetime.datetime(2023, 11, 6, 16, 30), tz_name="America/New_York", contracts=contracts, use_rth=True, # Regular Trading Hours only timeout=120, # Request timeout in seconds ) ``` ### Retrieving historical ticks ```python # Request historical tick data (use tick_type="TRADES" or "BID_ASK" for quote ticks) ticks = await client.request_ticks( tick_type="TRADES", start_date_time=datetime.datetime(2023, 11, 6, 9, 30), end_date_time=datetime.datetime(2023, 11, 6, 16, 30), tz_name="America/New_York", contracts=contracts, use_rth=True, timeout=120, ) ``` ### Bar specifications The adapter supports various bar specifications: #### Time-based bars - `"1-SECOND-LAST"`, `"5-SECOND-LAST"`, `"10-SECOND-LAST"`, `"15-SECOND-LAST"`, `"30-SECOND-LAST"` - `"1-MINUTE-LAST"`, `"2-MINUTE-LAST"`, `"3-MINUTE-LAST"`, `"5-MINUTE-LAST"`, `"10-MINUTE-LAST"`, `"15-MINUTE-LAST"`, `"20-MINUTE-LAST"`, `"30-MINUTE-LAST"` - `"1-HOUR-LAST"`, `"2-HOUR-LAST"`, `"3-HOUR-LAST"`, `"4-HOUR-LAST"`, `"8-HOUR-LAST"` - `"1-DAY-LAST"`, `"1-WEEK-LAST"`, `"1-MONTH-LAST"` #### Price types - `LAST` - Last traded price - `MID` - Midpoint of bid/ask - `BID` - Bid price - `ASK` - Ask price ### Complete example ```python import asyncio import datetime from nautilus_trader.adapters.interactive_brokers.common import IBContract from nautilus_trader.adapters.interactive_brokers.historical.client import HistoricInteractiveBrokersClient from nautilus_trader.persistence.catalog import ParquetDataCatalog async def download_historical_data(): # Initialize client client = HistoricInteractiveBrokersClient( host="127.0.0.1", port=7497, client_id=5, ) # Connect await client.connect() await asyncio.sleep(2) # Allow connection to stabilize # Define contracts contracts = [ IBContract(secType="STK", symbol="AAPL", exchange="SMART", primaryExchange="NASDAQ"), IBContract(secType="CASH", symbol="EUR", currency="USD", exchange="IDEALPRO"), ] # Request instruments instruments = await client.request_instruments(contracts=contracts) # Request historical bars bars = await client.request_bars( bar_specifications=["1-HOUR-LAST", "1-DAY-LAST"], start_date_time=datetime.datetime(2023, 11, 1, 9, 30), end_date_time=datetime.datetime(2023, 11, 6, 16, 30), tz_name="America/New_York", contracts=contracts, use_rth=True, ) # Request tick data ticks = await client.request_ticks( tick_type="TRADES", start_date_time=datetime.datetime(2023, 11, 6, 14, 0), end_date_time=datetime.datetime(2023, 11, 6, 15, 0), tz_name="America/New_York", contracts=contracts, ) # Save to catalog catalog = ParquetDataCatalog("./catalog") catalog.write_data(instruments) catalog.write_data(bars) catalog.write_data(ticks) print(f"Downloaded {len(instruments)} instruments") print(f"Downloaded {len(bars)} bars") print(f"Downloaded {len(ticks)} ticks") # Disconnect await client.disconnect() # Run the example if __name__ == "__main__": asyncio.run(download_historical_data()) ``` ### Data limitations Be aware of Interactive Brokers' historical data limitations: - **Rate Limits**: IB enforces rate limits on historical data requests - **Data Availability**: Historical data availability varies by instrument and subscription level - **Market Data Permissions**: Some data requires specific market data subscriptions - **Time Ranges**: Maximum lookback periods vary by bar size and instrument type ### Best practices 1. **Use Delayed Data**: For backtesting, `MarketDataTypeEnum.DELAYED_FROZEN` is often sufficient 2. **Batch Requests**: Group multiple instruments in single requests when possible 3. **Handle Timeouts**: Set appropriate timeout values for large data requests 4. **Respect Rate Limits**: Add delays between requests to avoid hitting rate limits 5. **Validate Data**: Always check data quality and completeness before backtesting :::warning Interactive Brokers enforces pacing limits; excessive historical-data or order requests trigger pacing violations and IB can disable the API session for several minutes. ::: ## Live trading Live trading with Interactive Brokers requires setting up a `TradingNode` that incorporates both `InteractiveBrokersDataClient` and `InteractiveBrokersExecutionClient`. These clients depend on the `InteractiveBrokersInstrumentProvider` for instrument management. ### Architecture overview The live trading setup consists of three main components: 1. **InstrumentProvider**: Manages instrument definitions and contract details 2. **DataClient**: Handles real-time market data subscriptions 3. **ExecutionClient**: Manages orders, positions, and account information ### InstrumentProvider configuration The `InteractiveBrokersInstrumentProvider` provides access to financial instrument data from IB. It supports loading individual instruments, options chains, and futures chains. #### Basic configuration ```python from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersInstrumentProviderConfig from nautilus_trader.adapters.interactive_brokers.config import SymbologyMethod from nautilus_trader.adapters.interactive_brokers.common import IBContract instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( symbology_method=SymbologyMethod.IB_SIMPLIFIED, build_futures_chain=False, # Set to True if fetching futures chains build_options_chain=False, # Set to True if fetching options chains min_expiry_days=10, # Minimum days to expiry for derivatives max_expiry_days=60, # Maximum days to expiry for derivatives convert_exchange_to_mic_venue=False, # Use MIC codes for venue mapping cache_validity_days=1, # Cache instrument data for 1 day load_ids=frozenset([ # Individual instruments using simplified symbology "EUR/USD.IDEALPRO", # Forex "BTC/USD.PAXOS", # Cryptocurrency "SPY.ARCA", # Stock ETF "V.NYSE", # Individual stock "ESM4.CME", # Future contract (single digit year) "^SPX.CBOE", # Index ]), load_contracts=frozenset([ # Complex instruments using IBContract IBContract(secType='STK', symbol='AAPL', exchange='SMART', primaryExchange='NASDAQ'), IBContract(secType='CASH', symbol='GBP', currency='USD', exchange='IDEALPRO'), ]), ) ``` #### Advanced configuration for derivatives ```python # Configuration for options and futures chains advanced_config = InteractiveBrokersInstrumentProviderConfig( symbology_method=SymbologyMethod.IB_SIMPLIFIED, build_futures_chain=True, # Enable futures chain loading build_options_chain=True, # Enable options chain loading min_expiry_days=7, # Load contracts expiring in 7+ days max_expiry_days=90, # Load contracts expiring within 90 days load_contracts=frozenset([ # Load SPY options chain IBContract( secType='STK', symbol='SPY', exchange='SMART', primaryExchange='ARCA', build_options_chain=True, ), # Load ES futures chain IBContract( secType='CONTFUT', exchange='CME', symbol='ES', build_futures_chain=True, ), ]), ) ``` #### Filtering security types Use `filter_sec_types` to ignore specific IB `secType` values. Any contract whose `secType` matches an entry in this frozenset is skipped with a warning (for example unsupported types such as `WAR` or `IOPT`): ```python instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( load_ids=frozenset(["SPY.ARCA"]), filter_sec_types=frozenset({"WAR", "IOPT"}), # Opt out from unsupported asset types ) ``` ### Integration with external data providers The Interactive Brokers adapter can be used alongside other data providers for enhanced market data coverage. When using multiple data sources: - Use consistent symbology methods across providers - Consider using `convert_exchange_to_mic_venue=True` for standardized venue identification - Ensure instrument cache management is handled properly to avoid conflicts ### Data client configuration The `InteractiveBrokersDataClient` interfaces with IB for streaming and retrieving real-time market data. Upon connection, it configures the [market data type](https://ibkrcampus.com/ibkr-api-page/trader-workstation-api/#delayed-market-data) and loads instruments based on the `InteractiveBrokersInstrumentProviderConfig` settings. #### Supported data types - **Quote Ticks**: Real-time bid/ask prices and sizes - **Trade Ticks**: Real-time trade prices and volumes - **Bar Data**: Real-time OHLCV bars (1-second to 1-day intervals) - **Market Depth**: Level 2 order book data (where available) #### Market data types Interactive Brokers supports several market data types: - `REALTIME`: Live market data (requires market data subscriptions) - `DELAYED`: 15-20 minute delayed data (free for most markets) - `DELAYED_FROZEN`: Delayed data that doesn't update (useful for testing) - `FROZEN`: Last known real-time data (when market is closed) #### Basic data client configuration ```python from nautilus_trader.adapters.interactive_brokers.config import IBMarketDataTypeEnum from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersDataClientConfig data_client_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=7497, # TWS paper trading port ibg_client_id=1, use_regular_trading_hours=True, # RTH only for stocks market_data_type=IBMarketDataTypeEnum.DELAYED_FROZEN, # Use delayed data ignore_quote_tick_size_updates=False, # Include size-only updates instrument_provider=instrument_provider_config, connection_timeout=300, # 5 minutes request_timeout_secs=60, # 1 minute ) ``` #### Advanced data client configuration ```python # Configuration for production with real-time data production_data_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=4001, # IB Gateway live trading port ibg_client_id=1, use_regular_trading_hours=False, # Include extended hours market_data_type=IBMarketDataTypeEnum.REALTIME, # Real-time data ignore_quote_tick_size_updates=True, # Reduce tick volume handle_revised_bars=True, # Handle bar revisions instrument_provider=instrument_provider_config, dockerized_gateway=dockerized_gateway_config, # If using Docker connection_timeout=300, request_timeout_secs=60, ) ``` ### Data client configuration options | Option | Default | Description | |---------------------------------|-------------------------------------------------|-------------| | `instrument_provider` | `InteractiveBrokersInstrumentProviderConfig()` | Instrument provider settings controlling which contracts load at startup. | | `ibg_host` | `127.0.0.1` | Hostname or IP for TWS/IB Gateway. | | `ibg_port` | `None` | Port for TWS/IB Gateway (`7497`/`7496` for TWS, `4002`/`4001` for IBG). | | `ibg_client_id` | `1` | Unique client identifier used when connecting to TWS/IB Gateway. | | `use_regular_trading_hours` | `True` | Request bars limited to regular trading hours when `True`. | | `market_data_type` | `REALTIME` | Market data feed type (`REALTIME`, `DELAYED`, `DELAYED_FROZEN`, etc.). | | `ignore_quote_tick_size_updates`| `False` | Suppress quote ticks where only size changes when `True`. | | `handle_revised_bars` | `False` | When `True`, processes bar revisions from IB (bars can be updated after initial publication). | | `dockerized_gateway` | `None` | Optional `DockerizedIBGatewayConfig` for containerized setups. | | `connection_timeout` | `300` | Seconds to wait for the initial API connection. | | `request_timeout_secs` | `60` | Seconds to wait for historical data requests before timing out. | #### Notes - **`use_regular_trading_hours`**: When `True`, only requests data during regular trading hours. Primarily affects bar data for stocks. - **`ignore_quote_tick_size_updates`**: When `True`, filters out quote ticks where only the size changed (not price), reducing data volume. - **`handle_revised_bars`**: When `True`, processes bar revisions from IB (bars can be updated after initial publication). - **`connection_timeout`**: Maximum time to wait for initial connection establishment. - **`request_timeout_secs`**: Maximum time to wait for historical data requests. ### Execution client configuration options | Option | Default | Description | |-----------------------------------------|-------------------------------------------------|-------------| | `instrument_provider` | `InteractiveBrokersInstrumentProviderConfig()` | Instrument provider settings controlling which contracts load at startup. | | `ibg_host` | `127.0.0.1` | Hostname or IP for TWS/IB Gateway. | | `ibg_port` | `None` | Port for TWS/IB Gateway (`7497`/`7496` for TWS, `4002`/`4001` for IBG). | | `ibg_client_id` | `1` | Unique client identifier used when connecting to TWS/IB Gateway. | | `account_id` | `None` | Interactive Brokers account identifier (falls back to `TWS_ACCOUNT` env var). | | `dockerized_gateway` | `None` | Optional `DockerizedIBGatewayConfig` for containerized setups. | | `connection_timeout` | `300` | Seconds to wait for the initial API connection. | | `request_timeout_secs` | `60` | Seconds to wait for request responses (contract details, etc.). | | `fetch_all_open_orders` | `False` | When `True`, pulls open orders for every API client ID (not just this session). | | `track_option_exercise_from_position_update` | `False` | Subscribe to real‑time position updates to detect option exercises when `True`. | ### Execution client configuration The `InteractiveBrokersExecutionClient` handles trade execution, order management, account information, and position tracking. It provides order lifecycle management and real-time account updates. #### Supported functionality - **Order Management**: Place, modify, and cancel orders - **Order Types**: Market, limit, stop, stop-limit, trailing stop, and more - **Account Information**: Real-time balance and margin updates - **Position Tracking**: Real-time position updates and P&L - **Trade Reporting**: Execution reports and fill notifications - **Risk Management**: Pre-trade risk checks and position limits #### Supported order types The adapter supports most Interactive Brokers order types: - **Market Orders**: `OrderType.MARKET` - **Limit Orders**: `OrderType.LIMIT` - **Stop Orders**: `OrderType.STOP_MARKET` - **Stop-Limit Orders**: `OrderType.STOP_LIMIT` - **Market-If-Touched**: `OrderType.MARKET_IF_TOUCHED` - **Limit-If-Touched**: `OrderType.LIMIT_IF_TOUCHED` - **Trailing Stop Market**: `OrderType.TRAILING_STOP_MARKET` - **Trailing Stop Limit**: `OrderType.TRAILING_STOP_LIMIT` - **Market-on-Close**: `OrderType.MARKET` with `TimeInForce.AT_THE_CLOSE` - **Limit-on-Close**: `OrderType.LIMIT` with `TimeInForce.AT_THE_CLOSE` #### Time in force options - **Day Orders**: `TimeInForce.DAY` - **Good-Till-Canceled**: `TimeInForce.GTC` - **Immediate-or-Cancel**: `TimeInForce.IOC` - **Fill-or-Kill**: `TimeInForce.FOK` - **Good-Till-Date**: `TimeInForce.GTD` - **At-the-Open**: `TimeInForce.AT_THE_OPEN` - **At-the-Close**: `TimeInForce.AT_THE_CLOSE` #### Batch operations | Operation | Supported | Notes | |--------------------|-----------|----------------------------------------------| | Batch Submit | ✓ | Submit multiple orders in single request. | | Batch Modify | ✓ | Modify multiple orders in single request. | | Batch Cancel | ✓ | Cancel multiple orders in single request. | #### Position management | Feature | Supported | Notes | |--------------------|-----------|----------------------------------------------| | Query positions | ✓ | Real‑time position updates. | | Position mode | ✓ | Net vs separate long/short positions. | | Leverage control | ✓ | Account‑level margin requirements. | | Margin mode | ✓ | Portfolio vs individual margin. | #### Order querying | Feature | Supported | Notes | |--------------------|-----------|----------------------------------------------| | Query open orders | ✓ | List all active orders. | | Query order history | ✓ | Historical order data. | | Order status updates| ✓ | Real‑time order state changes. | | Trade history | ✓ | Execution and fill reports. | #### Contingent orders | Feature | Supported | Notes | |--------------------|-----------|----------------------------------------------| | Order lists | ✓ | Atomic multi‑order submission. | | OCO orders | ✓ | One‑Cancels‑Other with customizable OCA types (1, 2, 3). | | Bracket orders | ✓ | Parent‑child order relationships. | | Conditional orders | ✓ | Advanced order conditions and triggers. | #### Basic execution client configuration ```python from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersExecClientConfig from nautilus_trader.config import RoutingConfig exec_client_config = InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, # TWS paper trading port ibg_client_id=1, account_id="DU123456", # Your IB account ID (paper or live) instrument_provider=instrument_provider_config, connection_timeout=300, routing=RoutingConfig(default=True), # Route all orders through this client ) ``` #### Advanced execution client configuration ```python # Production configuration with dockerized gateway production_exec_config = InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=4001, # IB Gateway live trading port ibg_client_id=1, account_id=None, # Will use TWS_ACCOUNT environment variable instrument_provider=instrument_provider_config, dockerized_gateway=dockerized_gateway_config, connection_timeout=300, routing=RoutingConfig(default=True), ) ``` #### Account ID configuration The `account_id` parameter is crucial and must match the account logged into TWS/Gateway: ```python # Option 1: Specify directly in config exec_config = InteractiveBrokersExecClientConfig( account_id="DU123456", # Paper trading account # ... other parameters ) # Option 2: Use environment variable import os os.environ["TWS_ACCOUNT"] = "DU123456" exec_config = InteractiveBrokersExecClientConfig( account_id=None, # Will use TWS_ACCOUNT env var # ... other parameters ) ``` #### Order params The execution adapter supports `params["exchange"]` on order submit, order list submit, and order modification commands. Use it to override the IB contract exchange for routing the current order while preserving the cached instrument contract: ```python self.submit_order(order, params={"exchange": "IEX"}) ``` Leave `exchange` unset, or set it to an empty string, to use the cached contract exchange. #### Order tags and advanced features The adapter supports IB-specific order parameters through order tags: ```python from nautilus_trader.adapters.interactive_brokers.common import IBOrderTags # Create order with IB-specific parameters order_tags = IBOrderTags( allOrNone=True, # All-or-none order ocaGroup="MyGroup1", # One-cancels-all group ocaType=1, # Cancel with block activeStartTime="20240315 09:30:00 EST", # GTC activation time activeStopTime="20240315 16:00:00 EST", # GTC deactivation time goodAfterTime="20240315 09:35:00 EST", # Good after time ) # Apply tags to an order order = order_factory.limit( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(100), price=instrument.make_price(100.0), tags=[order_tags.value], ) ``` #### OCA (one-cancels-all) orders The adapter provides support for OCA orders through explicit configuration using `IBOrderTags`: ### Basic OCA configuration All OCA functionality must be explicitly configured using `IBOrderTags`: ```python from nautilus_trader.adapters.interactive_brokers.common import IBOrderTags # Create OCA configuration oca_tags = IBOrderTags( ocaGroup="MY_OCA_GROUP", ocaType=1, # Type 1: Cancel All with Block (recommended) ) # Apply to bracket orders bracket_order = order_factory.bracket( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(100), tp_price=instrument.make_price(110.0), sl_trigger_price=instrument.make_price(90.0), tp_tags=[oca_tags.value], # Must explicitly add OCA tags sl_tags=[oca_tags.value], # Must explicitly add OCA tags ) ``` ### Advanced OCA configuration You can specify different OCA types and behaviors using `IBOrderTags`: ```python from nautilus_trader.adapters.interactive_brokers.common import IBOrderTags # Create custom OCA configuration custom_oca_tags = IBOrderTags( ocaGroup="MY_CUSTOM_GROUP", ocaType=2, # Use Type 2: Reduce with Block ) # Apply to individual orders order = order_factory.limit( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(100), price=instrument.make_price(100.0), tags=[custom_oca_tags.value], ) ``` ### OCA types Interactive Brokers supports three OCA types: | Type | Name | Behavior | Use Case | |------|------|----------|----------| | **1** | Cancel All with Block | Cancel all remaining orders with block protection | **Default** - Safest option, prevents overfills | | **2** | Reduce with Block | Proportionally reduce remaining orders with block protection | Partial fills with overfill protection | | **3** | Reduce without Block | Proportionally reduce remaining orders without block protection | Fastest execution, higher overfill risk | #### Multiple orders in same OCA group ```python # Create multiple orders with the same OCA group oca_tags = IBOrderTags( ocaGroup="MULTI_ORDER_GROUP", ocaType=3, # Use Type 3: Reduce without Block ) order1 = order_factory.limit( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(50), price=instrument.make_price(99.0), tags=[oca_tags.value], ) order2 = order_factory.limit( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(50), price=instrument.make_price(101.0), tags=[oca_tags.value], ) ``` ### OCA configuration requirements OCA functionality is **only** available through explicit configuration: 1. **IBOrderTags Required** - OCA settings must be explicitly specified in order tags 2. **No Automatic Detection** - `ContingencyType.OCO` and `ContingencyType.OUO` do not automatically create OCA groups 3. **Manual Configuration** - All OCA groups and types must be manually specified ### Conditional orders The adapter supports Interactive Brokers conditional orders through the `conditions` parameter in `IBOrderTags`. Conditional orders allow you to specify criteria that must be met before an order is transmitted or cancelled. #### Supported condition types - **Price Conditions**: Trigger based on price movements of a specific instrument - **Time Conditions**: Trigger at a specific date and time - **Volume Conditions**: Trigger based on trading volume thresholds - **Execution Conditions**: Trigger when trades occur for a specific instrument - **Margin Conditions**: Trigger based on account margin levels - **Percent Change Conditions**: Trigger based on percentage price changes #### Basic conditional order example ```python from nautilus_trader.adapters.interactive_brokers.common import IBOrderTags # Create a price condition: trigger when SPY goes above $250 price_condition = { "type": "price", "conId": 265598, # SPY contract ID "exchange": "SMART", "isMore": True, # Trigger when price is greater than threshold "price": 250.00, "triggerMethod": 0, # Default trigger method "conjunction": "and", } # Create order tags with condition order_tags = IBOrderTags( conditions=[price_condition], conditionsCancelOrder=False, # Transmit order when condition is met ) # Apply to order order = order_factory.limit( instrument_id=instrument.id, order_side=OrderSide.BUY, quantity=instrument.make_qty(100), price=instrument.make_price(251.00), tags=[order_tags.value], ) ``` #### Multiple conditions with logic ```python # Create multiple conditions with AND/OR logic conditions = [ { "type": "price", "conId": 265598, "exchange": "SMART", "isMore": True, "price": 250.00, "triggerMethod": 0, "conjunction": "and", # AND with next condition }, { "type": "time", "time": "20250315-09:30:00", "isMore": True, "conjunction": "or", # OR with next condition }, { "type": "volume", "conId": 265598, "exchange": "SMART", "isMore": True, "volume": 10000000, "conjunction": "and", }, ] order_tags = IBOrderTags( conditions=conditions, conditionsCancelOrder=False, ) ``` #### Condition parameters **Price Condition:** - `conId`: Contract ID of the instrument to monitor - `exchange`: Exchange to monitor (e.g., "SMART", "NASDAQ") - `isMore`: True for >=, False for <= - `price`: Price threshold - `triggerMethod`: 0=Default, 1=DoubleBidAsk, 2=Last, 3=DoubleLast, 4=BidAsk, 7=LastBidAsk, 8=MidPoint **Time Condition:** - `time`: Time string in UTC format "YYYYMMDD-HH:MM:SS" (e.g., "20250315-09:30:00") - `isMore`: True for after time, False for before time **Volume Condition:** - `conId`: Contract ID of the instrument to monitor - `exchange`: Exchange to monitor - `isMore`: True for >=, False for <= - `volume`: Volume threshold **Execution Condition:** - `symbol`: Symbol to monitor for trades - `secType`: Security type (e.g., "STK", "OPT", "FUT") - `exchange`: Exchange to monitor **Margin Condition:** - `percent`: Margin cushion percentage threshold - `isMore`: True for >=, False for <= **Percent Change Condition:** - `conId`: Contract ID of the instrument to monitor - `exchange`: Exchange to monitor - `isMore`: True for >=, False for <= - `changePercent`: Percentage change threshold #### Complete example: all condition types ```python # Example showing all 6 supported condition types from nautilus_trader.adapters.interactive_brokers.common import IBOrderTags # 1. Price Condition - trigger when ES futures > 6000 price_condition = { "type": "price", "conId": 495512563, # ES futures contract ID "exchange": "CME", "isMore": True, "price": 6000.0, "triggerMethod": 0, "conjunction": "and", } # 2. Time Condition - trigger at specific time time_condition = { "type": "time", "time": "20250315-09:30:00", # UTC format "isMore": True, "conjunction": "and", } # 3. Volume Condition - trigger when volume > 100,000 volume_condition = { "type": "volume", "conId": 495512563, "exchange": "CME", "isMore": True, "volume": 100000, "conjunction": "and", } # 4. Execution Condition - trigger when SPY trades execution_condition = { "type": "execution", "symbol": "SPY", "secType": "STK", "exchange": "SMART", "conjunction": "and", } # 5. Margin Condition - trigger when margin cushion > 75% margin_condition = { "type": "margin", "percent": 75, "isMore": True, "conjunction": "and", } # 6. Percent Change Condition - trigger when price changes > 5% percent_change_condition = { "type": "percent_change", "conId": 495512563, "exchange": "CME", "changePercent": 5.0, "isMore": True, "conjunction": "and", } # Use any combination of conditions order_tags = IBOrderTags( conditions=[price_condition, time_condition], # Multiple conditions conditionsCancelOrder=False, # Transmit when conditions met ) ``` #### Order behavior Set `conditionsCancelOrder` to control what happens when conditions are met: - `False`: Transmit the order when conditions are satisfied - `True`: Cancel the order when conditions are satisfied #### Implementation notes - **All 6 condition types are fully supported** and tested with live Interactive Brokers orders - **Price conditions** work correctly despite a known bug in the ibapi library where `PriceCondition.__str__` is incorrectly decorated as a property - **Time conditions** use UTC format with dash separator (`YYYYMMDD-HH:MM:SS`) for reliable parsing - **Conjunction logic** allows complex condition combinations using "and"/"or" operators ### Complete trading node configuration Setting up a complete trading environment involves configuring a `TradingNodeConfig` with all necessary components. Here are examples for different scenarios. #### Paper trading configuration ```python import os from nautilus_trader.adapters.interactive_brokers.common import IB from nautilus_trader.adapters.interactive_brokers.common import IB_VENUE from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersDataClientConfig from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersExecClientConfig from nautilus_trader.adapters.interactive_brokers.config import InteractiveBrokersInstrumentProviderConfig from nautilus_trader.adapters.interactive_brokers.config import IBMarketDataTypeEnum from nautilus_trader.adapters.interactive_brokers.config import SymbologyMethod from nautilus_trader.adapters.interactive_brokers.factories import InteractiveBrokersLiveDataClientFactory from nautilus_trader.adapters.interactive_brokers.factories import InteractiveBrokersLiveExecClientFactory from nautilus_trader.config import LiveDataEngineConfig from nautilus_trader.config import LoggingConfig from nautilus_trader.config import RoutingConfig from nautilus_trader.config import TradingNodeConfig from nautilus_trader.live.node import TradingNode # Instrument provider configuration instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( symbology_method=SymbologyMethod.IB_SIMPLIFIED, load_ids=frozenset([ "EUR/USD.IDEALPRO", "GBP/USD.IDEALPRO", "SPY.ARCA", "QQQ.NASDAQ", "AAPL.NASDAQ", "MSFT.NASDAQ", ]), ) # Data client configuration data_client_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=7497, # TWS paper trading ibg_client_id=1, use_regular_trading_hours=True, market_data_type=IBMarketDataTypeEnum.DELAYED_FROZEN, instrument_provider=instrument_provider_config, ) # Execution client configuration exec_client_config = InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, # TWS paper trading ibg_client_id=1, account_id="DU123456", # Your paper trading account instrument_provider=instrument_provider_config, routing=RoutingConfig(default=True), ) # Trading node configuration config_node = TradingNodeConfig( trader_id="PAPER-TRADER-001", logging=LoggingConfig(log_level="INFO"), data_clients={IB: data_client_config}, exec_clients={IB: exec_client_config}, data_engine=LiveDataEngineConfig( time_bars_timestamp_on_close=False, # IB standard: use bar open time validate_data_sequence=True, # Discard out-of-sequence bars ), timeout_connection=90.0, timeout_reconciliation=5.0, timeout_portfolio=5.0, timeout_disconnection=5.0, timeout_post_stop=2.0, ) # Create and configure the trading node node = TradingNode(config=config_node) node.add_data_client_factory(IB, InteractiveBrokersLiveDataClientFactory) node.add_exec_client_factory(IB, InteractiveBrokersLiveExecClientFactory) node.build() if __name__ == "__main__": try: node.run() finally: node.dispose() ``` When `validate_data_sequence=True`, subscribe to live bars via the `request_bars()` `callback` so the stream starts only after history has loaded; see [Working with bars: request vs. subscribe](../concepts/data.md#working-with-bars-request-vs-subscribe). ## Live trading with Dockerized gateway ```python from nautilus_trader.adapters.interactive_brokers.config import DockerizedIBGatewayConfig # Dockerized gateway configuration dockerized_gateway_config = DockerizedIBGatewayConfig( username=os.environ.get("TWS_USERNAME"), password=os.environ.get("TWS_PASSWORD"), trading_mode="live", # "paper" or "live" read_only_api=False, # Allow order execution timeout=300, ) # Data client with dockerized gateway data_client_config = InteractiveBrokersDataClientConfig( ibg_client_id=1, use_regular_trading_hours=False, # Include extended hours market_data_type=IBMarketDataTypeEnum.REALTIME, instrument_provider=instrument_provider_config, dockerized_gateway=dockerized_gateway_config, ) # Execution client with dockerized gateway exec_client_config = InteractiveBrokersExecClientConfig( ibg_client_id=1, account_id=os.environ.get("TWS_ACCOUNT"), # Live account ID instrument_provider=instrument_provider_config, dockerized_gateway=dockerized_gateway_config, routing=RoutingConfig(default=True), ) # Live trading node configuration config_node = TradingNodeConfig( trader_id="LIVE-TRADER-001", logging=LoggingConfig(log_level="INFO"), data_clients={IB: data_client_config}, exec_clients={IB: exec_client_config}, data_engine=LiveDataEngineConfig( time_bars_timestamp_on_close=False, validate_data_sequence=True, ), ) ``` ### Multi-client configuration For advanced setups, you can configure multiple clients with different purposes: ```python # Separate data and execution clients with different client IDs data_client_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=1, # Data client uses ID 1 market_data_type=IBMarketDataTypeEnum.REALTIME, instrument_provider=instrument_provider_config, ) exec_client_config = InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=2, # Execution client uses ID 2 account_id="DU123456", instrument_provider=instrument_provider_config, routing=RoutingConfig(default=True), ) ``` ### Multiple IB execution clients for different accounts NautilusTrader supports using multiple Interactive Brokers execution clients simultaneously, each connected to a different IB account. This is useful when you need to trade with multiple accounts, such as: - Separate accounts for different strategies - Paper trading and live trading accounts running simultaneously - Multiple managed accounts under the same IB login To configure multiple IB execution clients, provide multiple entries in the `exec_clients` dictionary with unique keys. Each entry specifies a different `account_id`: ```python from nautilus_trader.adapters.interactive_brokers.config import ( InteractiveBrokersDataClientConfig, InteractiveBrokersExecClientConfig, InteractiveBrokersInstrumentProviderConfig, SymbologyMethod, IBMarketDataTypeEnum, ) from nautilus_trader.live.config import TradingNodeConfig, RoutingConfig, LoggingConfig from nautilus_trader.model.identifiers import AccountId, Venue, ClientId # Shared instrument provider configuration instrument_provider_config = InteractiveBrokersInstrumentProviderConfig( symbology_method=SymbologyMethod.IB_SIMPLIFIED, ) # Data client (shared across all accounts) data_client_config = InteractiveBrokersDataClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=1, market_data_type=IBMarketDataTypeEnum.REALTIME, instrument_provider=instrument_provider_config, ) # Configuration for multiple IB execution clients config_node = TradingNodeConfig( trader_id="MULTI-ACCOUNT-001", logging=LoggingConfig(log_level="INFO"), # Single data client shared across accounts data_clients={ "IB": data_client_config, }, # Multiple execution clients, one per account exec_clients={ # First account: Paper trading account "IB-PAPER": InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=2, # Unique IB API client ID account_id="DU123456", # Paper trading account ID instrument_provider=instrument_provider_config, routing=RoutingConfig(default=False), # Not default ), # Second account: Live trading account "IB-LIVE": InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=3, # Unique IB API client ID account_id="U987654", # Live account ID instrument_provider=instrument_provider_config, routing=RoutingConfig(default=True), # Set as default ), # Third account: Another managed account "IB-ACCOUNT3": InteractiveBrokersExecClientConfig( ibg_host="127.0.0.1", ibg_port=7497, ibg_client_id=4, # Unique IB API client ID account_id="U456789", # Another account ID instrument_provider=instrument_provider_config, routing=RoutingConfig(default=False), ), }, ) ``` **Key points for multiple IB execution clients:** 1. **Unique keys**: Each entry in `exec_clients` must have a unique key (e.g., `"IB-PAPER"`, `"IB-LIVE"`). This key becomes the `account_issuer` for that client. 2. **Unique client IDs**: Each execution client must use a different `ibg_client_id` (2, 3, 4, etc.). IB Gateway/TWS requires each API connection to use a unique client ID. 3. **Account ID**: Each execution client must specify a different `account_id` matching the account logged into IB Gateway/TWS. 4. **Account identifiers**: The system creates `AccountId` instances like: - `AccountId("IB-PAPER-DU123456")` - `AccountId("IB-LIVE-U987654")` - `AccountId("IB-ACCOUNT3-U456789")` 5. **Routing**: Orders and queries are automatically routed to the correct execution client based on: - Explicit `client_id` in the command - `account_id` issuer (for `QueryAccount` commands or orders with account_id set) - Default client (if one is marked with `routing=RoutingConfig(default=True)`) 6. **Portfolio queries**: When querying portfolio properties, you can specify either: - `account_id` for account-specific queries: `portfolio.realized_pnls(account_id=AccountId("IB-PAPER-DU123456"))` - `venue` for aggregated queries across all accounts with that venue: `portfolio.realized_pnls(venue=Venue("IB-PAPER"))` **Example: Using multiple IB execution clients in a strategy:** ```python from nautilus_trader.model.identifiers import AccountId, ClientId from nautilus_trader.trading.strategy import Strategy class MultiAccountStrategy(Strategy): """Example strategy using multiple IB accounts.""" def on_start(self): # Define account IDs for easy reference self.paper_account = AccountId("IB-PAPER-DU123456") self.live_account = AccountId("IB-LIVE-U987654") # Query paper account balance paper_account_state = self.cache.account(self.paper_account) if paper_account_state: self.log.info(f"Paper account balance: {paper_account_state.balance_total()}") # Query live account balance live_account_state = self.cache.account(self.live_account) if live_account_state: self.log.info(f"Live account balance: {live_account_state.balance_total()}") def submit_order_to_paper(self, order): """Submit order to paper trading account.""" self.submit_order(order, client_id=ClientId("IB-PAPER")) def submit_order_to_live(self, order): """Submit order to live trading account.""" self.submit_order(order, client_id=ClientId("IB-LIVE")) def check_paper_pnl(self, instrument_id): """Check realized PnL for paper account.""" pnl = self.portfolio.realized_pnl( instrument_id=instrument_id, account_id=self.paper_account ) return pnl def check_live_pnl(self, instrument_id): """Check realized PnL for live account.""" pnl = self.portfolio.realized_pnl( instrument_id=instrument_id, account_id=self.live_account ) return pnl ``` **Example: Querying account information with multiple IB clients:** ```python from nautilus_trader.model.identifiers import AccountId # Query specific account paper_account = cache.account(AccountId("IB-PAPER-DU123456")) live_account = cache.account(AccountId("IB-LIVE-U987654")) # Query account using account_id (preferred method) paper_account_by_id = cache.account(AccountId("IB-PAPER-DU123456")) # Alternative: Query account using account_id parameter (also works) paper_account_via_account_id = cache.account_for_venue( account_id=AccountId("IB-PAPER-DU123456") ) # Query portfolio properties by account paper_realized_pnl = portfolio.realized_pnl( instrument_id=instrument_id, account_id=AccountId("IB-PAPER-DU123456") ) # Query portfolio properties aggregated across all IB accounts # Note: This aggregates across all accounts with the same venue all_ib_realized_pnl = portfolio.realized_pnls(venue=Venue("IB")) ``` ### Running the trading node ```python def run_trading_node(): """Run the trading node with proper error handling.""" node = None try: # Create and build node node = TradingNode(config=config_node) node.add_data_client_factory(IB, InteractiveBrokersLiveDataClientFactory) node.add_exec_client_factory(IB, InteractiveBrokersLiveExecClientFactory) node.build() # Add your strategies here # node.trader.add_strategy(YourStrategy()) # Run the node node.run() except KeyboardInterrupt: print("Shutting down...") except Exception as e: print(f"Error: {e}") finally: if node: node.dispose() if __name__ == "__main__": run_trading_node() ``` ### Additional configuration options #### Environment variables Set these environment variables for easier configuration: ```bash export TWS_USERNAME="your_ib_username" export TWS_PASSWORD="your_ib_password" export TWS_ACCOUNT="your_account_id" export IB_MAX_CONNECTION_ATTEMPTS="5" # Optional: limit reconnection attempts ``` #### Logging configuration ```python # Enhanced logging configuration logging_config = LoggingConfig( log_level="INFO", log_level_file="DEBUG", log_file_format="json", # JSON format for structured logging log_component_levels={ "InteractiveBrokersClient": "DEBUG", "InteractiveBrokersDataClient": "INFO", "InteractiveBrokersExecutionClient": "INFO", }, ) ``` You can find additional examples here: ## Troubleshooting ### Common connection issues #### Connection refused - **Cause**: TWS/Gateway not running or wrong port - **Solution**: Verify TWS/Gateway is running and check port configuration - **Default Ports**: TWS (7497/7496), IB Gateway (4002/4001) #### Authentication errors - **Cause**: Incorrect credentials or account not logged in - **Solution**: Verify username/password and ensure account is logged into TWS/Gateway #### Client ID conflicts - **Cause**: Multiple clients using the same client ID - **Solution**: Use unique client IDs for each connection #### Market data permissions - **Cause**: Insufficient market data subscriptions - **Solution**: Use `IBMarketDataTypeEnum.DELAYED_FROZEN` for testing or subscribe to required data feeds ### Error codes Interactive Brokers uses specific error codes. Common ones include: - **200**: No security definition found - **201**: Order rejected - reason follows - **202**: Order cancelled - **300**: Can't find EId with ticker ID - **354**: Requested market data is not subscribed - **2104**: Market data farm connection is OK - **2106**: HMDS data farm connection is OK ### Performance optimization #### Reduce data volume ```python # Reduce quote tick volume by ignoring size-only updates data_config = InteractiveBrokersDataClientConfig( ignore_quote_tick_size_updates=True, # ... other config ) ``` #### Connection management ```python # Set reasonable timeouts config = InteractiveBrokersDataClientConfig( connection_timeout=300, # 5 minutes request_timeout_secs=60, # 1 minute # ... other config ) ``` #### Memory management - Use appropriate bar sizes for your strategy - Limit the number of simultaneous subscriptions - Consider using historical data for backtesting instead of live data ### Best practices #### Security - Never hardcode credentials in source code - Use environment variables for sensitive information - Use paper trading for development and testing - Set `read_only_api=True` for data-only applications #### Development workflow 1. **Start with Paper Trading**: Always test with paper trading first 2. **Use Delayed Data**: Use `DELAYED_FROZEN` market data for development 3. **Implement Proper Error Handling**: Handle connection losses and API errors gracefully 4. **Monitor Logs**: Enable appropriate logging levels for debugging 5. **Test Reconnection**: Test your strategy's behavior during connection interruptions #### Production deployment - Use dockerized gateway for automated deployments - Implement proper monitoring and alerting - Set up log aggregation and analysis - Use real-time data subscriptions only when necessary - Implement circuit breakers and position limits #### Order management - Always validate orders before submission - Implement proper position sizing - Use appropriate order types for your strategy - Monitor order status and handle rejections - Implement timeout handling for order operations ### Debugging tips #### Enable debug logging ```python logging_config = LoggingConfig( log_level="DEBUG", log_component_levels={ "InteractiveBrokersClient": "DEBUG", }, ) ``` #### Monitor connection status ```python # Check connection status in your strategy if not self.data_client.is_connected: self.log.warning("Data client disconnected") ``` #### Validate instruments ```python # Ensure instruments are loaded before trading instruments = self.cache.instruments() if not instruments: self.log.error("No instruments loaded") ``` ### Support and resources - **IB API Documentation**: [TWS API Guide](https://ibkrcampus.com/ibkr-api-page/trader-workstation-api/) - **NautilusTrader Examples**: [GitHub Examples](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/interactive_brokers) - **IB Contract Search**: [Contract Information Center](https://pennies.interactivebrokers.com/cstools/contract_info/) - **Market Data Subscriptions**: [IB Market Data](https://www.interactivebrokers.com/en/pricing/market-data-pricing.php) ## Contributing :::info For additional features or to contribute to the Interactive Brokers adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Integrations Source: https://nautilustrader.io/docs/latest/integrations/ NautilusTrader uses modular *adapters* to connect to trading venues and data providers, translating raw APIs into a unified interface and normalized domain model. The following integrations are currently supported: | Name | ID | Type | Status | Docs | | :--------------------------------------------------------------------------- | :-------------------- | :---------------------- | :------------------------------------------------------ | :----------------------- | | [AX Exchange](https://architect.exchange) | `AX` | Perpetuals Exchange | ![status](https://img.shields.io/badge/stable-green) | [Guide](architect_ax.md) | | [Betfair](https://betfair.com) | `BETFAIR` | Sports Betting Exchange | ![status](https://img.shields.io/badge/stable-green) | [Guide](betfair.md) | | [Binance](https://binance.com) | `BINANCE` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](binance.md) | | [Coinbase](https://coinbase.com) | `COINBASE` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](coinbase.md) | | [BitMEX](https://www.bitmex.com) | `BITMEX` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](bitmex.md) | | [Bybit](https://www.bybit.com) | `BYBIT` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](bybit.md) | | [Databento](https://databento.com) | `DATABENTO` | Data Provider | ![status](https://img.shields.io/badge/stable-green) | [Guide](databento.md) | | [Deribit](https://www.deribit.com) | `DERIBIT` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](deribit.md) | | [Derive](https://www.derive.xyz) | `DERIVE` | Crypto Exchange (DEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](derive.md) | | [dYdX](https://dydx.exchange/) | `DYDX` | Crypto Exchange (DEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](dydx.md) | | [Hyperliquid](https://hyperliquid.xyz) | `HYPERLIQUID` | Crypto Exchange (DEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](hyperliquid.md) | | [Lighter](https://lighter.xyz) | `LIGHTER` | Crypto Exchange (DEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](lighter.md) | | [Interactive Brokers](https://www.interactivebrokers.com) | `INTERACTIVE_BROKERS` | Brokerage (multi‑venue) | ![status](https://img.shields.io/badge/stable-green) | [Guide](ib.md) | | [Kraken](https://kraken.com) | `KRAKEN` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](kraken.md) | | [OKX](https://okx.com) | `OKX` | Crypto Exchange (CEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](okx.md) | | [Polymarket](https://polymarket.com) | `POLYMARKET` | Prediction Market (DEX) | ![status](https://img.shields.io/badge/stable-green) | [Guide](polymarket.md) | | [Tardis](https://tardis.dev) | `TARDIS` | Crypto Data Provider | ![status](https://img.shields.io/badge/stable-green) | [Guide](tardis.md) | - **ID**: The default client ID for the integrations adapter clients. - **Type**: The type of integration (often the venue type). ## Status - `planned`: Planned for future development. - `building`: Under construction and likely not in a usable state. - `beta`: Completed to a minimally working state and in a 'beta' testing phase. - `stable`: Stabilized feature set and API, the integration has been tested by both developers and users to a reasonable level (some bugs may still remain). ## Implementation goals The primary goal of NautilusTrader is to provide a unified trading system for use with a variety of integrations. To support the widest range of trading strategies, priority will be given to *standard* functionality: - Requesting historical market data. - Streaming live market data. - Reconciling execution state. - Submitting standard order types with standard execution instructions. - Modifying existing orders (if possible on an exchange). - Canceling orders. The implementation of each integration aims to meet the following criteria: - Low-level client components should match the exchange API as closely as possible. - The full range of an exchange's functionality (where applicable to NautilusTrader) should *eventually* be supported. - Exchange specific data types will be added to support the functionality and return types which are reasonably expected by a user. - Actions unsupported by an exchange or NautilusTrader will be logged as a warning or error when invoked. ## API unification All integrations must conform to NautilusTrader’s system API, requiring normalization and standardization: - Symbols should use the venue’s native symbol format unless disambiguation is required (e.g., Binance Spot vs. Binance Futures). - Timestamps must use UNIX epoch nanoseconds. If milliseconds are used, field/property names should explicitly end with `_ms`. # Kraken Source: https://nautilustrader.io/docs/latest/integrations/kraken/ Kraken offers spot and derivatives trading across a wide range of digital assets. This integration connects to Kraken Pro and supports live market data ingest and order execution for Kraken Spot and Kraken Derivatives (Futures). ## Overview This adapter is implemented in Rust with Python bindings for ease of use in Python-based workflows. It does not require external Kraken client libraries; the core components are compiled as a static library and linked automatically during the build. This guide assumes a trader is setting up for both live market data feeds and trade execution. The Kraken adapter includes multiple components, which can be used together or separately depending on the use case. - `KrakenSpotRawHttpClient` and `KrakenFuturesRawHttpClient`: Low-level HTTP API connectivity. - `KrakenSpotHttpClient` and `KrakenFuturesHttpClient`: Higher-level HTTP clients with instrument caching and reconciliation support. - `KrakenInstrumentProvider`: Instrument parsing and loading functionality. - `KrakenDataClient`: Market data feed manager. - `KrakenExecutionClient`: Account management and trade execution gateway. - `KrakenDataClientFactory`: Factory for Kraken data clients (used by the trading node builder). - `KrakenExecutionClientFactory`: Factory for Kraken execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to work directly with these lower-level components. ::: ## Examples You can find live example scripts in the [examples/live/kraken] directory. [examples/live/kraken]: https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/kraken/ ## Kraken documentation Kraken provides detailed documentation for users: - [Kraken API Documentation](https://docs.kraken.com/api/) - [Kraken Spot REST API](https://docs.kraken.com/api/docs/guides/spot-rest-intro) - [Kraken Futures REST API](https://docs.kraken.com/api/docs/futures-api) Refer to the Kraken documentation in conjunction with this NautilusTrader integration guide. ## Products Kraken supports two primary product categories: | Product Type | Supported | Notes | |--------------------------|-----------|-----------------------------------------------------------| | Spot | ✓ | Standard cryptocurrency pairs with margin support. | | Futures (Perpetual) | ✓ | Inverse (`PI_`) and USD-margined (`PF_`) perpetual swaps. | | Futures (Dated/Flex) | ✓ | Fixed maturity (`FI_`) and flex (`FF_`) contracts. | :::note **Single product type per client**: Each Kraken data or execution client is configured for a single `product_type` (`SPOT` or `FUTURES`); a single client does not span both markets. ::: ## Bar streaming ### Supported intervals The Kraken adapter supports real-time bar (OHLC) streaming for Spot markets via WebSocket. The following intervals are available: | Interval | BarType specification | |------------|-----------------------| | 1 minute | `1-MINUTE-LAST` | | 5 minutes | `5-MINUTE-LAST` | | 15 minutes | `15-MINUTE-LAST` | | 30 minutes | `30-MINUTE-LAST` | | 1 hour | `1-HOUR-LAST` | | 4 hours | `4-HOUR-LAST` | | 1 day | `1-DAY-LAST` | | 1 week | `1-WEEK-LAST` | | 15 days | `15-DAY-LAST` | :::note **Futures limitation**: Kraken Futures does not support bar streaming via WebSocket. Use `request_bars()` for historical bar data instead. ::: ### Bar emission latency Kraken's WebSocket OHLC channel pushes updates for the *current* (incomplete) bar on every trade. Unlike some exchanges (e.g., Binance), Kraken does not provide an "is_closed" indicator to signal when a bar is complete. To avoid emitting partial/incomplete bars, the adapter buffers the current bar and only emits it when the next bar period begins (i.e., when a message with a new `interval_begin` timestamp arrives). This means: - Bars are emitted with a delay of up to one bar period. - For 1-minute bars, the maximum delay is ~1 minute. - The emitted bar data is complete and final. We chose this approach over timer-based emission because: - Timer-based emission could miss the final update before the bar closes. - Kraken's updates are not guaranteed to arrive at exact interval boundaries. - Buffering preserves data integrity at the cost of latency. :::warning If bar latency matters for your strategy, consider using trade tick data and aggregating bars locally with `BarAggregator`. ::: :::tip For most use cases, we recommend using `INTERNAL` bar aggregation (subscribing to trades and aggregating bars locally) rather than `EXTERNAL` exchange-provided bars: - Bars are emitted immediately when complete, with no buffering delay. - Consistent behavior across all exchanges, simplifying multi-venue strategies. ::: ## Symbology ### Bitcoin symbol format (BTC vs XBT) Kraken uses different Bitcoin symbol conventions across their APIs: | Market | Symbol Format | Example | Notes | |---------|---------------|--------------------|---------------------------------------------| | Spot | `BTC` | `BTC/USD.KRAKEN` | Adapter normalizes XBT to BTC at load time. | | Futures | `XBT` | `PI_XBTUSD.KRAKEN` | Uses Kraken's native XBT format. | :::note Kraken's REST API returns `XBT` for Bitcoin (following ISO 4217 conventions for supranational currencies), but their WebSocket v2 API requires the `BTC` format. The adapter automatically normalizes spot symbols to `BTC` when loading instruments, whether XBT appears as the base currency (e.g., `XBT/USD` to `BTC/USD`) or quote currency (e.g., `ETH/XBT` to `ETH/BTC`). Futures retain Kraken's native `XBT` format. ::: ### Spot markets NautilusTrader uses ISO 4217-A3 format for Kraken Spot instrument symbols, which provides a standardized representation across exchanges. The adapter handles translation to Kraken's native format internally. **Instrument ID format:** ```python InstrumentId.from_str("BTC/USD.KRAKEN") # Spot BTC/USD InstrumentId.from_str("ETH/USD.KRAKEN") # Spot ETH/USD InstrumentId.from_str("SOL/USD.KRAKEN") # Spot SOL/USD InstrumentId.from_str("BTC/USDT.KRAKEN") # Spot BTC/USDT InstrumentId.from_str("ETH/BTC.KRAKEN") # Spot ETH/BTC (normalized from ETH/XBT) ``` ### Futures markets Kraken Futures instruments use a specific naming convention with prefixes: - `PI_` - Perpetual Inverse contracts (e.g., `PI_XBTUSD`) - `PF_` - Perpetual Fixed-margin contracts (e.g., `PF_XBTUSD`) - `FI_` - Fixed maturity Inverse contracts (e.g., `FI_XBTUSD_230929`) - `FF_` - Flex futures contracts **Instrument ID format:** ```python InstrumentId.from_str("PI_XBTUSD.KRAKEN") # Perpetual inverse BTC InstrumentId.from_str("PI_ETHUSD.KRAKEN") # Perpetual inverse ETH InstrumentId.from_str("PF_XBTUSD.KRAKEN") # Perpetual fixed-margin BTC ``` ## Data capability ### Subscriptions (real-time) | Data type | Spot | Futures | Notes | |------------------------|------|---------|----------------------------------------| | `QuoteTick` | ✓ | ✓ | Derived from ticker channel. | | `TradeTick` | ✓ | ✓ | | | `OrderBookDeltas` | ✓ | ✓ | Spot L2/L3 and Futures L2 updates. | | `OrderBookDepth10` | - | - | Use `OrderBookDeltas` with depth `10`. | | `Bar` | ✓ | - | Spot WS OHLC channel. See bar section. | | `MarkPriceUpdate` | - | ✓ | From futures ticker feed. | | `IndexPriceUpdate` | - | ✓ | From futures ticker feed. | | `FundingRateUpdate` | - | ✓ | Perpetuals only. | | `InstrumentStatus` | ✓ | ✓ | Python adapter polls instrument refreshes. | ### Requests (historical) | Data type | Spot | Futures | Notes | |------------------------|------|---------|----------------------------------------| | `TradeTick` | ✓ | ✓ | | | `Bar` | ✓ | ✓ | | | `OrderBook` (snapshot) | ✓ | ✓ | Via HTTP depth endpoint. | | `FundingRateUpdate` | - | ✓ | Client‑side start/end/limit filtering. | ## L3 order book (market-by-order) Kraken exposes Spot per-order book data via the WebSocket v2 `level3` channel at `wss://ws-l3.kraken.com/v2`. This gives venue order IDs, per-order quantities, and true incremental events (`add`, `modify`, `delete`). The adapter hashes each venue order ID into the `u64` `BookOrder.order_id` field used by NautilusTrader. ### Prerequisites L3 subscriptions require Spot API credentials because Kraken's `level3` channel is authenticated. Set them in `KrakenDataClientConfig` or via `KRAKEN_SPOT_API_KEY` and `KRAKEN_SPOT_API_SECRET`: ```python from nautilus_trader.adapters.kraken.config import KrakenDataClientConfig config = KrakenDataClientConfig( api_key="YOUR_KEY", api_secret="YOUR_SECRET", ) ``` Then subscribe with `book_type=BookType.L3_MBO`: ```python from nautilus_trader.model.enums import BookType await client.subscribe_book_deltas( instrument_id=instrument_id, book_type=BookType.L3_MBO, depth=1000, # valid: 10, 100, 1000 ) ``` Valid depths are `10`, `100`, and `1000`. A `depth` of `0` uses `1000`. ### CRC32 checksum validation By default, the adapter validates the CRC32 checksum on each L3 snapshot and update when Kraken provides one. On mismatch, it emits a `Clear` delta, clears local L3 state, refreshes the auth token, and resubscribes so Kraken sends a fresh snapshot. To disable validation for benchmarking: ```python config = KrakenDataClientConfig( api_key="...", api_secret="...", validate_l3_checksum=False, ) ``` ### Storage recommendations `OrderBookDelta` already carries `order_id: u64` in its Arrow schema, so L3 data is stored identically to L2 in the `ParquetDataCatalog`. L3 generates significantly more events per instrument than L2. Recommended settings: - Lower chunk size (e.g. `chunk_size=50_000`) for faster parallel reads. - Enable `zstd` compression in catalog config. - Use per-instrument path partitioning (enabled by default). ## Orders capability ### Order types | Order type | Spot | Futures | Notes | |------------------------|------|---------|-----------------------------------------------| | `MARKET` | ✓ | ✓ | Immediate execution at market price. | | `LIMIT` | ✓ | ✓ | Execution at specified price or better. | | `STOP_MARKET` | ✓ | ✓ | Conditional market order (stop‑loss). | | `MARKET_IF_TOUCHED` | ✓ | ✓ | Conditional market order (take‑profit). | | `STOP_LIMIT` | ✓ | ✓ | Conditional limit order (stop‑loss‑limit). | | `LIMIT_IF_TOUCHED` | ✓ | ✓ | Maps to `take_profit` with `limit_price`. | | `TRAILING_STOP_MARKET` | ✓ | - | Trailing stop with `trailing_offset`. | | `TRAILING_STOP_LIMIT` | ✓ | - | Trailing stop‑limit with `limit_offset`. | ### Time in force | Time in Force | Spot | Futures | Notes | |---------------|------|---------|-----------------------------------------------------| | `GTC` | ✓ | ✓ | Good Till Canceled. | | `GTD` | ✓ | - | Good Till Date (Spot only, requires `expire_time`). | | `IOC` | ✓ | ✓ | Immediate or Cancel. | | `FOK` | ✓ | - | Spot limit orders only. | :::note **Market orders** are inherently immediate and do not support time-in-force. `IOC` only applies to limit-type orders. ::: ### Execution instructions | Instruction | Spot | Futures | Notes | |------------------|------|---------|----------------------------------------------------------------------| | `post_only` | ✓ | ✓ | Available for limit orders. | | `reduce_only` | ✓ | ✓ | Spot requires `spot_account_type=Margin` (margin orders only). | | `quote_quantity` | ✓ | - | Spot only. Volume in quote currency (`viqc`). | | `display_qty` | ✓ | - | Spot only. Iceberg orders (`displayvol`). | ### Trigger types Conditional orders (stop, take-profit, trailing stop) support a trigger price reference on Spot: | Trigger Type | Spot | Futures | Notes | |---------------|------|---------|--------------------------------------------| | `LAST_PRICE` | ✓ | ✓ | Default. Last traded price. | | `INDEX_PRICE` | ✓ | ✓ | Broader market index price. | | `MARK_PRICE` | - | ✓ | Futures only. | :::note The adapter rejects unsupported trigger types (e.g., `BID_ASK`) at submission time rather than silently coercing them. ::: ### Batch operations | Operation | Spot | Futures | Notes | |--------------|------|---------|---------------------------------------------------------| | Batch Submit | ✓ | ✓ | Spot chunks at 15 orders. Futures chunks at 10. | | Batch Modify | - | ✓ | Futures HTTP helper only. Execution sends one command. | | Batch Cancel | ✓ | ✓ | Auto‑chunks into batches of 50. | :::note **Cancel all orders**: - Order side filtering is not supported; all orders are canceled regardless of side. - Spot: Cancels all open orders across all symbols. - Futures: Requires an `instrument_id`; cancels orders for that symbol only. ::: ### Position management | Feature | Spot | Futures | Notes | |------------------|------|---------|---------------------------------------------------------| | Query positions | ✓ | ✓ | Spot margin via `OpenPositions`; spot cash opt‑in. | | Position mode | - | - | Single position per instrument. | | Leverage control | ✓ | ✓ | Spot tiers; per‑order `params={"leverage": N}`. | | Margin mode | ✓ | ✓ | Spot/Futures cross margin; no isolated spot margin. | ### Order querying | Feature | Spot | Futures | Notes | |----------------------|------|---------|----------------------------------------------| | Query open orders | ✓ | ✓ | List all active orders. | | Query order history | ✓ | ✓ | Historical order data with pagination. | | Order status updates | ✓ | ✓ | Real‑time order state changes via WebSocket. | | Trade history | ✓ | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Spot | Futures | Notes | |---------------------|------|---------|------------------------------------------| | Order lists | - | - | *Not supported*. | | OCO orders | - | - | *Not supported*. | | Bracket orders | - | - | *Not supported*. | | Conditional orders | ✓ | ✓ | Stop and take‑profit orders. | ## Order routing (Spot) The Spot execution client routes `submit_order`, `modify_order`, `cancel_order`, and `submit_order_list` through Kraken's authenticated WebSocket v2 trade channel by default, falling back to REST when the WebSocket is inactive. Set `use_ws_trade=False` on `KrakenExecClientConfig` to route all order operations through REST. ### Order shapes routed via REST Some Spot order shapes always route via REST. They split into two categories: shapes Kraken's WS v2 API does not support at all, and shapes the WS API supports but this adapter does not yet encode. **Kraken WS v2 limitation:** | Shape | Reason | |---------------------------|--------------------------------------------------------------| | Unsupported trigger types | `triggers.reference` accepts only `last` and `index`. | | Mixed‑symbol order lists | `batch_add` requires a single shared symbol. | **Not yet encoded by this adapter (follow-up work, currently REST):** | Shape | Notes | |-----------------------------|--------------------------------------------------------------------------------------| | `FOK` time in force | Encodable as the `FOK` time in force, but the builder routes REST. | | Trailing stop / stop‑limit | Encodable via `triggers.price` + `triggers.price_type`, but the builder routes REST. | | Iceberg (`display_qty`) | Encodable as `order_type: "iceberg"` + `display_qty`, but the builder routes REST. | | Quote‑quantity orders | Buy market quote‑qty maps to `cash_order_qty`; routed REST today. | The per-call `params={"use_ws_trade": False}` override forces a single command through REST regardless of the configured default. Set it on `SubmitOrder`, `ModifyOrder`, `CancelOrder`, or `SubmitOrderList`. ### WebSocket request timeout When a WebSocket round-trip exceeds `ws_request_timeout_secs` (default `5`) the dispatcher treats the command outcome as unknown and leaves the order in its current in-flight state: - Submit / batch_add: the dispatcher may send a best-effort compensating `cancel_order` over the same WebSocket so a delayed venue acceptance is not left as an orphan order. - Modify: the order remains in `PENDING_UPDATE`. - Cancel: the order remains in `PENDING_CANCEL`. The timeout does not emit `OrderRejected`, `OrderModifyRejected`, or `OrderCancelRejected` by itself. If the venue actually accepted the command, WebSocket order updates or the live execution reconciliation engine (`open_check_interval_secs`) are the recovery path. :::tip Set `ws_request_timeout_secs` comfortably above your observed round-trip latency (the default `5` is roughly 25x typical) so the timeout only fires under genuine network failure. ::: ### WebSocket order-routing options `KrakenExecClientConfig` exposes: | Option | Default | Description | |---------------------------|---------|---------------------------------------------------------------| | `use_ws_trade` | `True` | Route orders via WS when the trade channel is active. | | `ws_request_timeout_secs` | `5` | WS round‑trip timeout before marking command outcome unknown. | ## Reconciliation The Kraken adapter provides reconciliation capabilities for both Spot and Futures markets, allowing traders to synchronize their local state with the exchange state at startup or during operation. ### Spot reconciliation **Order status reports:** - Open orders: Fetches all currently active orders. - Closed orders: Fetches historical orders with pagination support. - Time-bounded queries: Supports filtering by start/end timestamps. **Fill reports:** - Trade history: Fetches execution history with pagination. - Time-bounded queries: Supports filtering by start/end timestamps. - All fill types: Market, limit, and conditional order fills. **Margin position reports** (when `spot_account_type=Margin`): - Open positions: Fetched from `POST /0/private/OpenPositions` and aggregated by (pair, side) into `PositionStatusReport` entries. - Synthetic FLAT cleanup: If the local cache has an open spot margin position that no longer appears on the venue (Kraken omits closed positions from `OpenPositions`), the adapter emits a synthetic FLAT report on the next position-check tick so the engine reconciles to closed. - Margin balances: `POST /0/private/TradeBalance` is called alongside the account-state refresh; used margin populates `MarginBalance.initial`, remaining metrics flow into `AccountState.info` (see Spot margin trading). ### Futures reconciliation **Order status reports:** - Open orders: Fetches all currently active futures orders. - Historical orders: Fetches closed and filled orders when `open_only=False`. - Order events: Full order lifecycle history via `/api/history/v2/orders` endpoint. **Fill reports:** - Fill history: Fetches all execution reports. - Time filtering: Client-side filtering by start/end timestamps (parses RFC3339 timestamps). - All fill types: Maker and taker fills with fee information. **Position status reports:** - Open positions: Fetches all active futures positions. - Real-time data: Includes unrealized funding, average price, and position size. :::note **Futures time filtering**: The Kraken Futures fills endpoint does not support server-side time range filtering. The adapter implements client-side filtering by parsing `fillTime` fields and comparing against requested start/end timestamps. ::: ### Spot position reports (cash mode) In cash mode, the Kraken adapter can optionally report wallet balances as position status reports for spot instruments. This feature is disabled by default and must be explicitly enabled via configuration. Margin-mode accounts should leave it disabled and rely on `OpenPositions` instead (see Spot margin trading). **How it works:** - When enabled, wallet balances are converted to `PositionStatusReport` objects. - Positive balances are reported as `LONG` positions. - Only instruments matching the configured quote currency are reported (default: `USDT`). - This prevents duplicate reports when the same asset is available with multiple quote currencies (e.g., BTC/USD, BTC/USDT, BTC/EUR). **Configuration:** ```python exec_clients={ KRAKEN: { "use_spot_position_reports": True, "spot_positions_quote_currency": "USDT", # Default }, } ``` :::warning **Use with caution**: Enabling spot position reports may lead to unintended behavior if your strategy is not designed to handle spot positions. For example, a strategy that expects to close positions may attempt to sell your wallet holdings. ::: ## Spot margin trading Kraken Spot supports leveraged trading on selected pairs. Per-pair availability and the valid leverage tiers are advertised by Kraken on the instruments endpoint as `AssetPairInfo.leverage_buy` and `leverage_sell`; the adapter caches these at instrument-load time and validates the requested tier before order submission. Margin trading is enabled per-execution-client via `spot_account_type`, with per-order `leverage` params. ### Configuration ```python from nautilus_trader.adapters.kraken import KrakenExecClientConfig from nautilus_trader.model.enums import AccountType exec_clients = { KRAKEN: KrakenExecClientConfig( spot_account_type=AccountType.MARGIN, default_leverage=3, # optional config-level default margin_balance_asset="ZGBP", # optional summary-display asset ), } ``` `margin_balance_asset` controls only the denomination of the account-summary metrics returned by Kraken's `TradeBalance` endpoint (equity, free margin, used margin, etc.). Per-position figures from `OpenPositions` are always in the traded pair's quote currency. ### Per-order leverage Override the configured default on a single order via `params`: ```python order = strategy.order_factory.limit( instrument_id=BTC_USD, order_side=OrderSide.BUY, quantity=Quantity.from_str("0.01"), price=Price.from_str("50000.00"), params={"leverage": 5}, ) ``` The adapter validates the requested tier against `AssetPairInfo.leverage_buy` / `leverage_sell` for the pair before submitting; an invalid tier produces an `OrderDenied` event and never hits the venue. ### Reduce-only Margin orders can carry `reduce_only=True`; Kraken rejects the order if no matching position exists. Cash orders ignore the flag. ### Account state When `spot_account_type=Margin`, the adapter calls Kraken's `TradeBalance` endpoint and surfaces the result in two places: - `MarginBalance.initial`: used margin (`m`). - `AccountState.info` dict: full `TradeBalance` snapshot: - `equity`: net equity - `free_margin`: equity minus used margin - `unrealized_pnl`: open-position P&L - `margin_level`: equity / used margin (%) when positions are open - `trade_balance`: collateral on deposit - `equivalent_balance`: combined-currency wallet equivalent - `cost_basis`, `valuation`, `unexecuted_value`, `used_margin`: raw `TradeBalance` fields - `asset`: resolved denominating asset (e.g. `USD`, `GBP`) A single INFO log line is emitted on every account state refresh: ```text Margin metrics: equity=1234.56 GBP, free_margin=1100.00, unrealized_pnl=12.34 ``` Strategies read the values via `account_state.info["equity"]`, etc. ### Position reconciliation Open spot margin positions are surfaced via `POST /0/private/OpenPositions` on each `position_check_interval_secs` tick. Closed positions on the venue that still appear open in the local cache are reconciled to FLAT on the next sweep. This path is independent of `use_spot_position_reports` (which is wallet-derived, cash-mode-only). ## Funding rates The adapter receives funding rate data from the [Ticker](https://docs.kraken.com/api/docs/futures-api/websocket/ticker) WebSocket feed, which provides `relative_funding_rate` and `next_funding_rate_time` for perpetual futures. The `interval` field on `FundingRateUpdate` is `None` for Kraken because the ticker feed does not include a funding interval field and the Kraken API documentation does not specify a fixed funding period. ## Rate limiting The adapter implements automatic rate limiting to comply with Kraken's API requirements. | Endpoint Type | Limit (requests/sec) | Notes | |-----------------------|----------------------|--------------------------------------| | Spot REST (global) | 5 | Global rate limit for Spot API. | | Futures REST (global) | 5 | Global rate limit for Futures API. | :::info Kraken uses a counter-based rate limiting system with tier-dependent limits: - **Starter tier**: 15 max counter, -0.33/sec decay - **Intermediate tier**: 20 max counter, -0.5/sec decay - **Pro tier**: 20 max counter, -1/sec decay Ledger/trade history calls add +2 to the counter; other calls add +1. ::: :::warning Kraken may temporarily block IP addresses that exceed rate limits. The adapter automatically queues requests when limits are approached. ::: ### Reconciliation interval guidance The execution engine's `open_check_interval_secs` and `position_check_interval_secs` settings create sustained REST API load that can exhaust Kraken's counter-based rate limit, especially on the Starter tier where the counter decays at only 0.33/sec. Each open-order check generates 1-3 REST calls (+1 or +2 counter each), and at short intervals the counter overflows before it can decay, causing `EAPI:Rate limit exceeded` errors. Recommended settings for Kraken: ```python exec_engine=LiveExecEngineConfig( reconciliation=True, open_check_interval_secs=30.0, # 30s minimum for Starter tier position_check_interval_secs=120.0, # 2 minutes ) ``` Higher-tier accounts with faster counter decay can use shorter intervals. If you see `EAPI:Rate limit exceeded` errors in the logs, increase these intervals or reduce `max_requests_per_second` in the adapter config. ## Configuration The product type for each client is specified via the `product_type` option. ### Data client configuration options | Option | Default | Description | |---------------------------|-----------|----------------------------------------------------------------| | `product_type` | `SPOT` | Product type for this client (`SPOT` or `FUTURES`). | | `environment` | `LIVE` | Trading environment (`LIVE` or `DEMO`); demo only for Futures. | | `api_key` | `None` | API key; loaded from environment variables when omitted. | | `api_secret` | `None` | API secret; loaded from environment variables when omitted. | | `base_url` | `None` | Override for the Kraken REST base URL. | | `ws_public_url` | `None` | Override for the public WebSocket URL. | | `ws_private_url` | `None` | Override for the private WebSocket URL. | | `ws_l3_url` | `None` | Override for the Spot L3 WebSocket URL. | | `validate_l3_checksum` | `True` | Validate Kraken Spot L3 checksums and resync on mismatch. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `timeout_secs` | `30` | HTTP request timeout in seconds. | | `heartbeat_interval_secs` | `30` | WebSocket heartbeat interval in seconds. | | `ws_idle_timeout_ms` | `10000` | Idle timeout for the Spot v2 WebSocket; `0` disables. | | `max_requests_per_second` | `None` | Override rate limit; default is 5 req/s. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |---------------------------------|-----------|-----------------------------------------------------------------------| | `api_key` | required | Kraken API key. | | `api_secret` | required | Kraken API secret. | | `product_type` | `SPOT` | Product type for this client (`SPOT` or `FUTURES`). | | `environment` | `LIVE` | Trading environment (`LIVE` or `DEMO`); demo only for Futures. | | `base_url` | `None` | Override for the Kraken REST base URL. | | `ws_url` | `None` | Override for the Kraken WebSocket URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `timeout_secs` | `30` | HTTP request timeout in seconds. | | `heartbeat_interval_secs` | `30` | WebSocket heartbeat interval in seconds. | | `max_requests_per_second` | `None` | Override rate limit; default is 5 req/s. | | `spot_account_type` | `CASH` | Account type for spot trading; `MARGIN` enables leverage and reports. | | `default_leverage` | `None` | Default spot margin leverage sent as `"N:1"` when set. | | `use_spot_position_reports` | `False` | Report wallet balances as positions; cash mode only. | | `spot_positions_quote_currency` | `"USDT"` | Quote currency filter for spot wallet position reports. | | `margin_balance_asset` | `None` | Summary asset for `TradeBalance`; `None` defaults to `ZUSD`. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | For spot margin, `default_leverage` applies when an order has no per-order leverage param. `margin_balance_asset` only changes the `TradeBalance` summary denomination; per-position figures remain in the pair's quote currency. ### Demo environment setup To test with Kraken Futures demo (paper trading): 1. Sign up at [https://demo-futures.kraken.com](https://demo-futures.kraken.com) and generate API credentials. 2. Set environment variables with your demo credentials: - `KRAKEN_FUTURES_DEMO_API_KEY` - `KRAKEN_FUTURES_DEMO_API_SECRET` 3. Configure the adapter with `environment=KrakenEnvironment.DEMO` and `product_type=KrakenProductType.FUTURES`. ```python from nautilus_trader.adapters.kraken import KRAKEN from nautilus_trader.adapters.kraken import KrakenEnvironment from nautilus_trader.adapters.kraken import KrakenProductType config = TradingNodeConfig( ..., # Omitted data_clients={ KRAKEN: { "environment": KrakenEnvironment.DEMO, "product_type": KrakenProductType.FUTURES, }, }, exec_clients={ KRAKEN: { "environment": KrakenEnvironment.DEMO, "product_type": KrakenProductType.FUTURES, }, }, ) ``` ### Production configuration The most common use case is to configure a live `TradingNode` to include Kraken data and execution clients. Add a `KRAKEN` section to your client configuration(s): ```python from nautilus_trader.adapters.kraken import KRAKEN from nautilus_trader.adapters.kraken import KrakenEnvironment from nautilus_trader.adapters.kraken import KrakenProductType from nautilus_trader.live.node import TradingNode config = TradingNodeConfig( ..., # Omitted data_clients={ KRAKEN: { "environment": KrakenEnvironment.LIVE, "product_type": KrakenProductType.SPOT, }, }, exec_clients={ KRAKEN: { "environment": KrakenEnvironment.LIVE, "product_type": KrakenProductType.SPOT, }, }, ) ``` Then, create a `TradingNode` and add the client factories: ```python from nautilus_trader.adapters.kraken import KRAKEN from nautilus_trader.adapters.kraken import KrakenDataClientFactory from nautilus_trader.adapters.kraken import KrakenExecutionClientFactory from nautilus_trader.live.node import TradingNode # Instantiate the live trading node with a configuration node = TradingNode(config=config) # Register the client factories with the node node.add_data_client_factory(KRAKEN, KrakenDataClientFactory) node.add_exec_client_factory(KRAKEN, KrakenExecutionClientFactory) # Finally build the node node.build() ``` ### API credentials There are two options for supplying your credentials to the Kraken clients. Either pass the corresponding `api_key` and `api_secret` values to the configuration objects, or set the following environment variables: | Environment Variable | Description | |----------------------------------|------------------------------------------| | `KRAKEN_SPOT_API_KEY` | API key for Kraken Spot live trading. | | `KRAKEN_SPOT_API_SECRET` | API secret for Kraken Spot live trading. | | `KRAKEN_FUTURES_API_KEY` | Kraken Futures live API key. | | `KRAKEN_FUTURES_API_SECRET` | Kraken Futures live API secret. | | `KRAKEN_FUTURES_DEMO_API_KEY` | API key for Kraken Futures (demo). | | `KRAKEN_FUTURES_DEMO_API_SECRET` | API secret for Kraken Futures (demo). | :::note **Demo environment**: Only Kraken Futures offers a demo environment (`https://demo-futures.kraken.com`) for testing without real funds. Kraken Spot does not have a demo or testnet environment. ::: :::tip We recommend using environment variables to manage your credentials. ::: When starting the trading node, you'll receive immediate confirmation of whether your credentials are valid and have trading permissions. ## Contributing :::info For additional features or to contribute to the Kraken adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Lighter Source: https://nautilustrader.io/docs/latest/integrations/lighter/ [Lighter](https://lighter.xyz) is a decentralized central-limit-order-book exchange for spot and perpetual futures. The venue settles through an Ethereum zero-knowledge rollup, while matching and sequencing run off-chain. The NautilusTrader Lighter adapter is implemented by the `nautilus-lighter` crate. It provides Rust data and execution clients, typed REST and WebSocket models, and an in-tree L2 transaction signer for the venue's Schnorr / ECgFp5 signing flow. ## Overview The adapter consists of the following main components: - `LighterRawHttpClient`: low-level REST client for the public and account endpoints. - `LighterHttpClient`: domain client which parses instruments, trades, books, orders, and account state into Nautilus model types. - `LighterWebSocketClient`: reconnecting WebSocket client for public market and private account streams. - `LighterDataClient`: Nautilus data client for instruments, trades, quotes, and L2 MBP books. - `LighterExecutionClient`: Nautilus execution client for account streams, order submission, modification, cancellation, and reconciliation reports. - `LighterDataClientFactory` and `LighterExecutionClientFactory`: live-node factory wiring. The Python surface is intentionally narrow. The Python extension exposes configuration, environment selection, factory classes, and integrator revocation; data and execution clients are consumed through the Rust trait surface. ## Examples The adapter includes Python v2 and Rust live-node examples. The Python examples live in [`python/examples/lighter/`](https://github.com/nautechsystems/nautilus_trader/tree/develop/python/examples/lighter/) and default to a dry build: they build the node, register the tester, and exit unless `--run` is passed. ```bash cd python .venv/bin/python examples/lighter/data_tester.py --lighter-environment testnet .venv/bin/python examples/lighter/exec_tester.py --lighter-environment testnet ``` Pass `--run` to connect to Lighter. The execution tester remains in `dry_run` mode unless `--live-orders` is also passed. ```bash cd python .venv/bin/python examples/lighter/data_tester.py \ --lighter-environment mainnet \ --instrument BTC-PERP.LIGHTER \ --run .venv/bin/python examples/lighter/exec_tester.py \ --lighter-environment mainnet \ --instrument DOGE-PERP.LIGHTER \ --run ``` Rust examples live under `crates/adapters/lighter/examples/`. The data tester connects when run. The execution tester also connects when run, but defaults to `DRY_RUN = true`; set it to `false` in the source only when you intend to submit real orders: ```bash cargo run --example lighter-data-tester --package nautilus-lighter --features examples cargo run --example lighter-exec-tester --package nautilus-lighter --features examples ``` :::warning Examples can connect to live venues. Execution examples with live order flow enabled can submit orders when pointed at a funded mainnet account. Review the selected instrument, quantity, and environment before running them. ::: For emergency account cleanup, `cargo run --bin lighter-flatten -p nautilus-lighter` cancels open orders and closes positions for the configured Lighter account. It scans all registered markets, so the standard 60 req/min REST quota can make the run take several minutes. Review the active account and positions first because it is account-wide, not strategy-scoped. ## Product support | Product type | Data feed | Trading | Notes | |-------------------|-----------|---------|--------------------------------------------------------------| | Spot | ✓ | ✓ | Spot markets using Lighter market indexes 2048-4094. | | Perpetual futures | ✓ | ✓ | Linear perpetual markets using Lighter market indexes 0-254. | | Dated futures | - | - | *Not supported*. | | Options | - | - | *Not supported*. | ## Limitations The current adapter scope is deliberately narrower than the venue's full transaction surface: - Grouped order lists, OCO/OTO groups, bracket orders, TWAP, trailing stops, and iceberg display size are not implemented. - Native batch submit and native batch cancel are wired for independent order operations only. Batch submit sends independent `L2CreateOrder` txs in one `sendTxBatch`, and batch cancel signs independent `L2CancelOrder` txs in one `sendTxBatch`. Both are capped at 15 txs per batch. - Grouped venue orders remain out of scope: batch submit does not use `CreateGroupedOrders` and does not provide atomic OCO/OTO or bracket grouping. - `CancelAllOrders` uses cached open orders for the requested instrument. The adapter does not use Lighter's native account-wide cancel-all transaction because it can affect unrelated markets. - Spot trading supports market and limit orders. Conditional stop-loss and take-profit orders are limited to perpetual markets. - Account state and position reports come from private WebSocket streams. `query_account` and position status generation replay the latest cached stream state. - Unscoped order reconciliation is bounded to configured or observed active markets to avoid a full venue-wide fan-out under the standard REST quota. - Historical trade requests use the public `recentTrades` endpoint and need no credentials. ## Symbology Lighter identifies markets by numeric `market_index` values. The adapter bootstraps the mapping from `GET /api/v1/orderBookDetails`, then converts the raw venue symbol into a Nautilus `InstrumentId`. | Venue product | Nautilus symbol format | Example | Notes | |--------------------|-------------------------------|-------------------------|------------------------------| | Perpetual futures | `{BASE}-PERP.LIGHTER` | `BTC-PERP.LIGHTER` | Raw venue symbol `BTC`. | | Spot | `{BASE}/{QUOTE}-SPOT.LIGHTER` | `ETH/USDC-SPOT.LIGHTER` | Raw venue symbol `ETH/USDC`. | The suffix disambiguates spot and perpetual listings. Spot symbols keep the quoted venue pair, while outbound requests strip the suffix and use the cached `market_index`. ## Environments | Environment | REST URL | WebSocket URL | Chain ID | |-------------|---------------------------------------|--------------------------------------------|----------| | Mainnet | `https://mainnet.zklighter.elliot.ai` | `wss://mainnet.zklighter.elliot.ai/stream` | 304 | | Testnet | `https://testnet.zklighter.elliot.ai` | `wss://testnet.zklighter.elliot.ai/stream` | 300 | Use `LighterEnvironment::Mainnet` or `LighterEnvironment::Testnet` in data and execution configuration. URL overrides are available for private gateways or local test fixtures. ## Integrator attribution Submitted create and modify order transactions carry the NautilusTrader integrator account index in Lighter's `L2TxAttributes`. This helps us gauge real usage of the integration and prioritize ongoing maintenance. Maker and taker integrator fees are set to zero, so attribution adds no trading cost. Lighter requires an `ApproveIntegrator` approval before these attributes can be attached to orders. During startup, the execution client submits the required **zero-fee** approval for the configured L2 account. ### Revoking the approval Use revocation as cleanup when leaving the adapter. It sends the same `ApproveIntegrator` tx with `approval_expiry = 0` and every max fee set to zero; the next execution-client startup records a fresh zero-fee approval. ```bash export LIGHTER_API_KEY_INDEX=5 export LIGHTER_API_SECRET=REPLACE_ME export LIGHTER_ACCOUNT_INDEX=123456 cargo run -p nautilus-lighter --bin lighter-integrator-revoke # mainnet cargo run -p nautilus-lighter --bin lighter-integrator-revoke testnet # testnet ``` Script source: [`crates/adapters/lighter/bin/integrator_revoke.rs`](https://github.com/nautechsystems/nautilus_trader/blob/develop/crates/adapters/lighter/bin/integrator_revoke.rs). ```python # Python (PyO3 binding) - reads the same env vars as the Rust bin from nautilus_trader.adapters.lighter import revoke_lighter_integrator from nautilus_trader.adapters.lighter import LighterEnvironment await revoke_lighter_integrator() # mainnet (default) await revoke_lighter_integrator(LighterEnvironment.TESTNET) # testnet ``` The Rust script prints a summary of the action and pauses for an Enter keypress before signing or sending; abort with `Ctrl+C` before that point if anything in the summary looks wrong. The Python binding does not prompt: review the active env vars yourself before calling. ## Data subscriptions | Data type | Sub. | Snapshot | Hist. | Nautilus type | Notes | |----------------------|--------------|----------|-------|---------------------|----------------------------------------------------------| | Instrument metadata | Cache replay | ✓ | - | `InstrumentAny` | Loaded from `orderBookDetails`. | | Trade ticks | ✓ | - | ✓ | `TradeTick` | WebSocket trades; public `recentTrades` REST history. | | Quote ticks | ✓ | - | - | `QuoteTick` | Best bid and ask ticker stream. | | Order book deltas | ✓ | ✓ | - | `OrderBookDeltas` | `L2_MBP` only. | | Order book depth10 | ✓ | - | - | `OrderBookDepth10` | Live top-10 view from maintained book; no REST snapshot. | | Order book snapshots | - | ✓ | - | `OrderBook` | REST snapshot, max depth 250. | | Mark prices | ✓ | - | - | `MarkPriceUpdate` | Perp market stats stream. | | Index prices | ✓ | - | - | `IndexPriceUpdate` | Market and spot stats streams. | | Funding rates | ✓ | - | ✓ | `FundingRateUpdate` | Current estimates and REST hourly history. | | Bars | ✓ | - | ✓ | `Bar` | WebSocket candle stream; REST history for backfill. | | Instrument status | REST | ✓ | - | `InstrumentStatus` | `active` / `inactive` snapshots. | Only `BookType::L2_MBP` is accepted for book-delta and depth10 subscriptions. Other book types return an error before subscribing. The WebSocket order book initializes only from `subscribed/order_book`. If an `update/order_book` arrives before that snapshot, the adapter drops it and waits for the real snapshot because incremental updates do not contain the full visible book. Depth10 subscriptions use the same WebSocket `order_book` stream as deltas. The adapter emits a refreshed top-10 view after each accepted snapshot or incremental update. Bar subscriptions use the venue's `candle/{market_id}/{resolution}` WebSocket channel. Lighter batches in-progress updates for the open bar every ~500 ms; the adapter emits a Nautilus `Bar` only when the candle start timestamp advances, so consumers see one event per closed period. The in-progress cache is cleared on reconnect and on unsubscribe. The stream supports `1m`, `5m`, `15m`, `30m`, `1h`, `4h`, `12h`, and `1d`. `1w` is REST-only via `request_bars`; subscribing to a `1-WEEK` bar type returns an error. Instrument status subscriptions replay the latest cached `orderBookDetails` status when available and otherwise fetch a REST snapshot. Lighter does not expose a WebSocket status-change stream. Funding-rate subscriptions use `market_stats.current_funding_rate`, which is Lighter's estimate for the upcoming payment. Historical funding-rate requests use `/api/v1/fundings` at `1h` resolution and map settled rows to `FundingRateUpdate` with `interval=60`. The REST `direction` field controls the sign: `long` stays positive because longs pay shorts, while `short` is mapped negative because shorts pay longs. The adapter does not use account-specific `positionFunding` payloads for public funding history. Trade subscriptions use the public WebSocket trade stream. Historical trade requests use the public `/api/v1/recentTrades` endpoint, which needs no credentials; the adapter clamps the request to the venue per-call cap and filters the returned ticks to the requested time range. ### Unsupported data requests `request_quotes` is not implemented. Lighter exposes best bid and offer data through the WebSocket `ticker` stream, but the REST endpoints available to the adapter do not provide a timestamped quote snapshot or quote history that can map safely to `QuoteTick`. `request_book_depth` is not implemented. The documented REST book endpoints do not provide a venue event timestamp for `OrderBookDepth10.ts_event`; use `subscribe_book_depth10` for a live depth10 stream or `request_book_snapshot` for a REST `OrderBook` snapshot. ## Orders capability ### Order identification Lighter uses a numeric venue order index and a caller-supplied `client_order_index`. The adapter derives the Lighter `client_order_index` from the Nautilus `ClientOrderId` and keeps a local map so private WebSocket reports can recover the original client order ID. Query paths can use either the Nautilus client order ID or the numeric venue order ID when the required mapping is available. ### Order types | Order type | Perpetuals | Spot | Notes | |------------------------|------------|------|---------------------------------------------------------| | `MARKET` | ✓ | ✓ | Cap derived from cached far‑side quote + slippage. | | `LIMIT` | ✓ | ✓ | Requires a limit price. | | `STOP_MARKET` | ✓ | - | Perp only; cap derived from `trigger_price` + slippage. | | `STOP_LIMIT` | ✓ | - | Perp only; maps to Lighter stop‑loss limit orders. | | `MARKET_IF_TOUCHED` | ✓ | - | Perp only; cap derived from `trigger_price` + slippage. | | `LIMIT_IF_TOUCHED` | ✓ | - | Perp only; maps to Lighter take‑profit limit orders. | | `MARKET_TO_LIMIT` | - | - | *Not supported*. | | `TRAILING_STOP_MARKET` | - | - | *Not supported*. | | `TRAILING_STOP_LIMIT` | - | - | *Not supported*. | | `TWAP` | - | - | *Not supported*; no Nautilus mapping. | Conditional order types are available for perpetual markets only. Spot conditional orders are denied locally because Lighter rejects them at the venue. Conditional order types must include a `trigger_price`. `STOP_MARKET` and `MARKET_IF_TOUCHED` are denied upfront if the trigger is missing, and all conditional types are denied if the trigger truncates to `0` ticks at the instrument's price precision. Lighter's market-style orders require a worst-acceptable `price` field on the wire. The adapter derives it automatically: `MARKET` orders read the cached far-side `QuoteTick` (ask for buys, bid for sells); `STOP_MARKET` and `MARKET_IF_TOUCHED` use the order's `trigger_price`. The base is widened by `market_order_slippage_bps` (default 50 bps = 0.5%), rounded conservatively at the instrument's price precision (ceil for buys, floor for sells). A `MARKET` order submitted before the strategy has subscribed to quotes is denied with a clear error. Override per order via `SubmitOrder.params["market_order_slippage_bps"]`. ### Contingent orders | Feature | Perpetuals | Spot | Notes | |---------------------------------|------------|------|--------------------------------------------------------| | Stop‑loss market | ✓ | - | `STOP_MARKET` maps to Lighter `STOP_LOSS`. | | Stop‑loss limit | ✓ | - | `STOP_LIMIT` maps to Lighter `STOP_LOSS_LIMIT`. | | Take‑profit market | ✓ | - | `MARKET_IF_TOUCHED` maps to Lighter `TAKE_PROFIT`. | | Take‑profit limit | ✓ | - | `LIMIT_IF_TOUCHED` maps to `TAKE_PROFIT_LIMIT`. | | Trigger price | ✓ | - | Required for every supported conditional order. | | Trigger price type | - | - | *Not supported*; no trigger source selector. | | Grouped order lists | - | - | *Not supported*. | | OCO / OTO orders | - | - | *Not supported*. | | Bracket orders | - | - | *Not supported*. | | `CreateGroupedOrders` | - | - | *Not supported*; native batches use independent txs. | ### Order options | Option | Perpetuals | Spot | Notes | |------------------|------------|------|----------------------------------------------------------------------------| | `post_only` | ✓ | ✓ | Maps to Lighter's post‑only time‑in‑force. | | `reduce_only` | ✓ | - | Passed through to `CreateOrder`; use only to reduce an existing position. | | `quote_quantity` | - | - | *Not supported*; submit base quantity instead. | | `display_qty` | - | - | *Not supported*; Lighter exposes no iceberg display quantity field. | ### Adapter order params | Param | Perpetuals | Spot | Notes | |--------------------------------------------|------------|------|-----------------------------------------------------| | `market_order_slippage_bps` | ✓ | ✓ | Overrides the config default for market‑style caps. | | `post_only` through `SubmitOrder.params` | - | - | *Not supported*; use the Nautilus order flag. | | `reduce_only` through `SubmitOrder.params` | - | - | *Not supported*; use the Nautilus order flag. | ### Time in force | Time in force | Perpetuals | Spot | Notes | |----------------|------------|------|------------------------------------------------------------------------------| | `GTC` | ✓ | ✓ | Limit‑style uses `GoodTillTime`; market‑style uses `IOC`. | | `DAY` | ✓ | ✓ | Limit‑style and conditional orders use a positive order expiry. | | `GTD` | ✓ | ✓ | Limit‑style and conditional orders use the supplied Nautilus expiry. | | `IOC` | ✓ | ✓ | Plain `MARKET`/`LIMIT` use expiry `0`; conditional limit uses trigger expiry. | | `FOK` | - | - | *Not supported*. | | `AT_THE_OPEN` | - | - | *Not supported*. | | `AT_THE_CLOSE` | - | - | *Not supported*. | For `MARKET`, `STOP_MARKET`, and `MARKET_IF_TOUCHED`, the adapter maps the wire time-in-force to Lighter `ImmediateOrCancel` because the venue rejects market-style orders sent as `GoodTillTime`. Plain `MARKET` orders set `OrderExpiry = 0`. Conditional market orders (`STOP_MARKET` and `MARKET_IF_TOUCHED`) keep a positive `OrderExpiry` so the trigger can rest, and the wire `ImmediateOrCancel` applies only after the trigger fires. Nautilus `IOC` cannot be represented for conditional market orders, so the adapter denies it locally with a clear error. Conditional limit orders (`STOP_LIMIT` and `LIMIT_IF_TOUCHED`) can use Nautilus `IOC`: the trigger rests with a positive `OrderExpiry`, and the child limit order uses Lighter `ImmediateOrCancel` after the trigger fires. When no explicit GTD expiry is supplied, limit-style `GTC`, `DAY`, and `GTD` orders default to the current time plus 28 days. Conditional `GTC`, `DAY`, and limit-style `IOC` orders use the same default expiry. The venue rejects `-1` as an invalid expiry for these TIFs. Lighter requires order expiry timestamps to be at least 5 minutes and at most 30 days from submission, so live GTD tests must stay inside that venue window. ### Execution instructions | Instruction | Perpetuals | Spot | Notes | |---------------|------------|------|--------------------------------------------------------------| | `post_only` | ✓ | ✓ | Overrides the TIF and sends Lighter `PostOnly`. | | `reduce_only` | ✓ | - | Position‑reducing flag for existing derivative positions. | Use `post_only` on limit-style orders. The adapter does not synthesize maker-only market orders. Live mainnet testing confirms `reduce_only=true` for closing perpetual positions. Invalid reduce-only opens can be dropped by Lighter without a venue order report; the adapter reconciles them as `INFLIGHT_TIMEOUT` rather than a venue-supplied rejection reason. ### Advanced order features | Feature | Perpetuals | Spot | Notes | |----------------------|------------|------|-------------------------------------------------------------| | Order modification | ✓ | ✓ | Modify quantity, price, and trigger price on a live order. | | Bracket orders | - | - | *Not supported*. | | Iceberg orders | - | - | *Not supported*. | | Trailing stops | - | - | *Not supported*. | | Pegged orders | - | - | *Not supported*. | | TWAP orders | - | - | *Not supported*; no Nautilus mapping. | | Leverage update | ✓ | - | Perp only; submits a signed `UpdateLeverage` tx. | | Native cancel‑all | - | - | *Not supported*; adapter scopes cancel‑all per instrument. | | Dead man's switch | - | - | *Not supported*. | ### Order operations | Operation | Perpetuals | Spot | Notes | |---------------------|------------|------|-------------------------------------------------------------| | Submit order | ✓ | ✓ | Sends a signed `L2CreateOrder` transaction over WebSocket. | | Submit order list | ✓ | ✓ | Batches independent `L2CreateOrder` txs only. | | Modify order | ✓ | ✓ | Sends a signed `ModifyOrder`; reports may restate accepts. | | Cancel order | ✓ | ✓ | Sends a signed `L2CancelOrder` transaction. | | Cancel all orders | ✓ | ✓ | Iterates cached open orders for the requested instrument. | | Set leverage | ✓ | - | Perp only; submits a signed `UpdateLeverage` tx. | | Batch cancel orders | ✓ | ✓ | Batches independent `L2CancelOrder` txs only. | | Native batch submit | ✓ | ✓ | Uses one `sendTxBatch`, capped at 15 create txs. | | Native batch cancel | ✓ | ✓ | Uses one `sendTxBatch`, capped at 15 cancel txs. | | Query order | ✓ | ✓ | Requires credentials and REST lookup. | | Query account | ✓ | ✓ | Replays the latest private WebSocket account state. | | Mass status | ✓ | ✓ | Bounded to account‑active markets from WS and REST reports. | The native venue `CancelAllOrders` transaction is account-wide. The adapter deliberately cancels cached open orders per instrument to avoid touching unrelated markets. `SubmitOrderList` and `BatchCancelOrders` use `sendTxBatch` for independent operations. They do not create grouped venue orders, do not provide atomic OCO/OTO or bracket semantics, and do not use account-wide `CancelAllOrders` for scoped cancels. The `sendTxBatch` response exposes one top-level API code and a `tx_hash` list; it does not expose per-order API rejection fields. A successful batch response queues the signed transactions, then private account streams report the final per-order submit, cancel, fill, and reject outcomes. `UpdateLeverage` is exposed as `LighterExecutionClient::update_leverage(instrument_id, initial_margin_fraction, margin_mode)`. The `initial_margin_fraction` is in venue ticks (1e-4 fraction): `500` is 5% initial margin (20x leverage), `1000` is 10% (10x), and so on. `UpdateLeverage` has no oracle test vectors in this repo; the body field order is pinned against the cgo header from the upstream signer, and the wire format was verified by submitting a signed tx to Lighter mainnet that the sequencer accepted. ### Order querying and reconciliation | Feature | Perpetuals | Spot | Notes | |----------------------|------------|------|--------------------------------------------------------------| | Query open orders | ✓ | ✓ | REST `accountActiveOrders` scoped by market. | | Query order history | ✓ | ✓ | REST `accountInactiveOrders` with cursor pagination. | | Order status updates | ✓ | ✓ | Private WebSocket order streams plus status reports. | | Trade history | ✓ | ✓ | REST `trades`; credentials are required for account history. | | Fill reports | ✓ | ✓ | REST and private WebSocket trade payloads. | | Position reports | ✓ | - | Perp only; replays cached position stream. | | Account state | ✓ | ✓ | Replays the cached merged account state snapshot. | | Mass status | ✓ | ✓ | Combines orders, fills, and cached positions. | ## Account and position management Authenticated execution clients subscribe to these private streams: - `account_all_orders`: order status reports. - `account_all_trades`: fill reports. - `account_all_positions`: position snapshots. - `account_all_assets`: per-asset balance snapshots (spot balance plus perp collateral). - `user_stats`: perp-account margin rollup (collateral and available balance). The adapter merges `account_all_assets` and `user_stats` into a single account state and emits it only after both streams have delivered their first frame. The execution client requires credentials before connecting because private account streams and nonce refresh are mandatory. A client can be constructed without credentials, but live execution will not connect until `private_key`, `account_index`, and `api_key_index` resolve. Perpetual positions are reported in netting mode: one position per market. Spot balances arrive through account asset state rather than position reports. Each `account_all_positions` frame is treated as a venue snapshot. If a new frame omits a previously cached market, the adapter emits a flat position report for that instrument; an empty `positions` map clears all cached perpetual positions and emits a flat report for each. Rows present in the frame whose market cannot be mapped or whose position cannot be parsed are retained by market ID instead: parsed rows still update the cache, and cached markets absent from both the parsed rows and skipped market IDs still flatten. | Feature | Perpetuals | Spot | Notes | |-------------------------|------------|------|--------------------------------------------------------------| | Account balances | ✓ | ✓ | Merged assets + `user_stats`, replayed from cache on query. | | Position snapshots | ✓ | - | Perp only; `account_all_positions` stream. | | Netting positions | ✓ | - | One Nautilus position per perpetual market. | | Cross margin | ✓ | - | Passed through `LighterPositionMarginMode::Cross`. | | Isolated margin | ✓ | - | Passed through `LighterPositionMarginMode::Isolated`. | | Leverage updates | ✓ | - | Signed `UpdateLeverage` transaction. | | Spot margin / borrowing | - | - | *Not supported*. | | Deposits / withdrawals | - | - | Use venue tools or Lighter APIs outside the trading adapter. | ## Liquidation and ADL handling | Event or field | Support | Notes | |-----------------------------|---------|-------------------------------------------------------------------| | Liquidation trades | ✓ | Account trade rows can parse as fills, with no special event. | | Deleverage trades | ✓ | Account trade rows can parse as fills, with no special event. | | Liquidation price reporting | - | *Not supported*; reports omit this field. | | ADL event stream | - | *Not supported*. | ## Funding rates Perpetual `market_stats` frames emit `MarkPriceUpdate`, `IndexPriceUpdate`, and `FundingRateUpdate` events. Spot `spot_market_stats` frames emit `IndexPriceUpdate` events. Historical funding-rate requests use the public `/api/v1/fundings` endpoint and emit `FundingRateUpdate` responses for settled hourly rows. The adapter paginates the requested range across pages, so a wide window is not truncated to a single page; an explicit `limit` still caps the number of rows returned. ## Account tiers Lighter assigns each account a tier that governs latency, rate limits, and fees. Standard is the zero-fee default; the higher tiers are opt-in on the venue and trade fees for lower latency and higher throughput. The execution client detects the tier on connect (via `GET /api/v1/account`) and logs it in blue, including the raw `account_type` code for any tier this adapter does not yet recognize. Detection is informational only: the adapter never raises rate limits on its own, because the higher venue limits require registering the caller IP with Lighter, so a higher tier does not by itself guarantee the higher limit is active for your connection. | Tier | Latency (maker / taker) | REST weighted limit | `sendTx` limit | Fees (maker / taker) | Notes | |----------|-------------------------|---------------------|----------------------|---------------------------|-----------------------------------------| | Standard | 200 ms / 300 ms | 60 req/min | 60 req/min | 0 / 0 | Zero‑fee default tier. | | Premium | 0 ms / 140-200 ms | 24,000 req/min | 4,000-40,000 req/min | 0.28-0.40 / 1.96-2.80 bps | Lowest latency; scales with staked LIT. | | Plus | 200 ms / 300 ms | 120,000 req/min | 8,000 req/min | 0.5 / 0.5 bps | Raised limits, standard latency. | | Builder | - | 240,000 req/min | - | - | Highest REST throughput. | Premium latency, fees, and `sendTx` throughput scale with staked LIT, and the schedule can change; see the Lighter docs for the current figures. To actually use a higher tier's limits, register the caller IP with Lighter and set the quota explicitly (see [Rate limiting](#rate-limiting)). ## Rate limiting Lighter applies rate limits to both IP address and L1 address. The execution client detects the account tier on connect and logs it (see [Account tiers](#account-tiers)), but it does not raise limits automatically. By default both clients use the conservative standard-account quotas. To use a higher tier's throughput, register the caller IP with Lighter and set the quota explicitly in the client config: - `rest_quota_per_min`: REST read-bucket quota in requests per minute. Unset keeps 60 req/min. Available on both the data and execution clients. - `sendtx_quota_per_min`: transaction quota in requests per minute, metered in a bucket separate from reads. Unset keeps it at the standard 60 req/min, independent of `rest_quota_per_min`. Execution client only. Higher-tier REST quotas are weighted in the venue docs. The adapter REST limiter does not model per-endpoint weights; it paces one token per HTTP call through a shared REST bucket and a route bucket. When setting `rest_quota_per_min` above the standard default, use an effective request rate for the endpoint mix you plan to call, not the raw weighted quota. For example, Lighter documents `/api/v1/trades` and `/api/v1/recentTrades` with weight 600, so a 24,000 weighted req/min premium limit equates to 40 of those calls per minute. The venue meters transactions per account across both transports in one bucket. The execution client enforces `sendtx_quota_per_min` with a single shared limiter across both paths it submits on: the WebSocket `sendTx` path (single order submit, cancel, modify, leverage) and the HTTP `sendTx` / `sendTxBatch` endpoints (native batch submit/cancel and the startup integrator approval). Their combined rate therefore stays under the one venue limit. The data and execution clients share one WebSocket message limiter per venue URL, so their combined send rate honors the venue's per-IP cap. It paces non-transaction control frames such as subscribe, unsubscribe, and resubscribe requests at the venue's documented 200 messages/minute. Subscribe dispatch is additionally bounded by a closed-loop inflight gate: the feed handler holds back queued subscribes once the unacknowledged count reaches its cap (35), keeping it below Lighter's 50-message per-IP inflight ceiling while subscriptions fan out at startup and reconnect. A rate cap alone cannot bound this, because the unacknowledged count tracks venue acknowledgement latency rather than emission rate. `sendTx` is not counted against the WebSocket client-message bucket. | Scope | Venue limit | Adapter behavior | |----------------------------------------|-----------------------------|------------------------------------------------------| | REST, standard account | 60 req/min | Default; set `rest_quota_per_min` to override. | | REST, premium account | 24,000 weighted req/min | Logged; set `rest_quota_per_min` to use it. | | REST, plus account | 120,000 weighted req/min | Logged; set `rest_quota_per_min` to use it. | | REST, builder account | 240,000 weighted req/min | Logged; set `rest_quota_per_min` to use it. | | `sendTx` / `sendTxBatch`, standard | 60 req/min | Singles use `sendTx`; batches use `sendTxBatch`. | | `sendTx` / `sendTxBatch`, plus | 8,000 req/min | Set `sendtx_quota_per_min` to use it. | | `sendTx` / `sendTxBatch`, premium | 4,000-40,000 req/min | Set `sendtx_quota_per_min` (scales with staked LIT). | | Default transaction type limit | 40 req/min | Applies to tx types not covered by volume quota. | | `L2UpdateLeverage` transaction limit | 40 req/min | Relevant to `update_leverage`. | | Pending orders | 500/account, 16/market | Venue limit; adapter does not pre‑count it. | | Active orders | 1,500/account, 1,000/market | Venue limit; adapter does not pre‑count it. | Common REST endpoint weights from the official docs: | Endpoint group | Weight | Adapter behavior | |----------------------------------------|--------|-------------------------------------------------| | `sendTx`, `sendTxBatch`, `nextNonce` | 6 | Tx calls use tx limiter; `nextNonce` uses REST. | | `accountInactiveOrders` | 100 | Adapter counts one REST token per HTTP call. | | `trades`, `recentTrades` | 600 | Adapter counts one REST token per HTTP call. | | Other endpoints | 300 | Adapter counts one REST token per HTTP call. | | Endpoint or transport | Limit | Notes | |----------------------------------------|------------|----------------------------------------------------| | `/api/v1/trades` | 100 rows | Adapter paginates reconciliation at this cap. | | `/api/v1/accountInactiveOrders` | 100 rows | Adapter follows `next_cursor` at this cap. | | `/api/v1/orderBookOrders` | 250 levels | Snapshot depth is clamped to the venue cap. | | `/api/v1/candles` | 500 rows | Adapter caps REST bar pages at this venue maximum. | | `/api/v1/fundings` | 100 rows | Adapter paginates funding pages at this venue cap. | | WebSocket connections | 200 / IP | Venue limit. | | WebSocket subscriptions / connection | 500 | Venue limit. | | WebSocket unique accounts / connection | 500 | Venue limit. | | WebSocket connections / minute | 80 | Venue limit. | | WebSocket client messages / minute | 200 | Adapter paces non‑tx control frames at this cap. | | WebSocket inflight messages | 50 | Adapter caps non‑tx control‑frame burst at 50. | | `sendTxBatch` batch size | 15 txs | Applies to native HTTP submit and cancel batches. | | WebSocket keepalive | 2 minutes | Adapter sends heartbeats every 30 seconds. | | WebSocket outbound command queue | Not capped | Paced before writes; no queue‑depth cap. | Premium volume quota is a separate venue constraint for `L2CreateOrder`, `L2CancelAllOrders`, `L2ModifyOrder`, and `L2CreateGroupedOrders`. The adapter does not inspect remaining quota; use venue account tools if a strategy depends on premium or plus limits. See Lighter's [Volume Quota](https://apidocs.lighter.xyz/docs/volume-quota-program) documentation for the current quota rules and replenishment figures. ## Volume quota and no-fill quoting Lighter's volume quota affects quote-refresh strategies differently from transport rate limits. Create, modify, native cancel-all, and grouped-order transactions spend volume quota, while the venue replenishes quota from completed trading volume and any free allowance the program provides. A market-making smoke test that repeatedly quotes around mid without fills can therefore exhaust quota even when the WebSocket and `sendTx` rate limiters are working as intended. For live tests on Lighter, avoid treating indefinite no-fill quoting as a neutral connectivity check. Prefer slower one-sided quoting, wider refresh thresholds, testnet coverage, or a bounded strategy that deliberately earns enough small fills to replenish the quota it spends. ## Connection management The WebSocket client sends heartbeats every 30 seconds and reconnects with exponential backoff from 250 milliseconds up to 30 seconds. Private account subscriptions use Lighter auth tokens with an 8-hour maximum TTL; the adapter refreshes tokens 15 minutes before expiry and resubscribes account channels. On execution reconnect, the adapter refreshes the nonce baseline through `GET /api/v1/nextNonce` before it resumes signed transaction dispatch. Within a session, the adapter manages transaction nonces locally: venue confirmations advance the allocation window, and rejected or failed transactions roll back or trigger a resync from `GET /api/v1/nextNonce`, so order flow recovers from nonce desyncs without a reconnect. `LighterExecutionClient::connect()` waits up to 30 seconds for every account stream (`account_all_orders`, `account_all_trades`, `account_all_positions`, `account_all_assets`, `user_stats`) to deliver its first frame before returning. Lighter has no REST endpoint for account or position state, so the WebSocket frames are the only ground truth: returning earlier would let strategies race the venue's initial state and find the venue order id lookup table or position cache empty. The gate clears any prior-session position and account caches at the start of each connect attempt so a reconnect cycle observes the new session's frames, not stale data. By contrast, transparent WebSocket reconnects do not re-enter `connect()`: they keep cached positions until the next `account_all_positions` frame, then apply the same position snapshot replacement rules. ## API credentials Lighter signing requires all three credential values: - Account index: numeric Lighter account identifier. - API key index: numeric API key slot. Use the user-created key index assigned by Lighter, avoid reserved low indexes, and do not use `255`; it is an `apikeys` query sentinel, not a signing key. - API private key: 40-byte hex private key, with or without a `0x` prefix. Config values take precedence. A missing config field, or a blank API private key (empty or whitespace only), falls back to the corresponding environment variable for the selected environment. | Environment | API key index | API private key | Account index | |-------------|---------------------------------|------------------------------|---------------------------------| | Mainnet | `LIGHTER_API_KEY_INDEX` | `LIGHTER_API_SECRET` | `LIGHTER_ACCOUNT_INDEX` | | Testnet | `LIGHTER_TESTNET_API_KEY_INDEX` | `LIGHTER_TESTNET_API_SECRET` | `LIGHTER_TESTNET_ACCOUNT_INDEX` | Execution rejects incomplete credentials. The data client runs without credentials: its subscriptions and REST requests (instruments, book, trades, bars, funding) all use public endpoints. ## Configuration ### Data client configuration options | Option | Default | Description | |------------------------------------|-----------|-----------------------------------------------------| | `base_url_http` | `None` | Optional REST URL override. | | `base_url_ws` | `None` | Optional WebSocket URL override. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket. | | `environment` | `Mainnet` | `LighterEnvironment::Mainnet` or `Testnet`. | | `account_index` | `None` | Lighter account index for authenticated REST data. | | `api_key_index` | `None` | Lighter API key slot for authenticated REST data. | | `private_key` | `None` | Hex private key for REST auth tokens. | | `http_timeout_secs` | `60` | HTTP request timeout in seconds. | | `ws_timeout_secs` | `30` | WebSocket connect timeout in seconds. | | `update_instruments_interval_mins` | `60` | Instrument metadata refresh interval in minutes. | | `rest_quota_per_min` | `None` | REST quota override; unset keeps 60 req/min. | | `transport_backend` | Default | WebSocket transport backend. | ### Execution client configuration options | Option | Default | Description | |-----------------------------|---------------|------------------------------------------------------------| | `trader_id` | `TRADER-001` | Nautilus trader identifier. | | `account_id` | `LIGHTER-001` | Nautilus account identifier for the venue. | | `account_index` | `None` | Lighter account index. | | `api_key_index` | `None` | Lighter API key slot. | | `private_key` | `None` | Hex private key for auth and L2 transaction signing. | | `base_url_http` | `None` | Optional REST URL override. | | `base_url_ws` | `None` | Optional WebSocket URL override. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket. | | `environment` | `Mainnet` | `LighterEnvironment::Mainnet` or `Testnet`. | | `http_timeout_secs` | `60` | HTTP request timeout in seconds. | | `ws_timeout_secs` | `30` | WebSocket connect timeout in seconds. | | `market_order_slippage_bps` | `50` | Slippage cap (bps) for `MARKET` / `STOP_MARKET` / `MIT`. | | `rest_quota_per_min` | `None` | REST quota override; unset keeps 60 req/min. | | `sendtx_quota_per_min` | `None` | Transaction quota override; unset keeps 60 req/min. | | `transport_backend` | Default | WebSocket transport backend. | ### Configuration example ```rust use nautilus_lighter::{ common::enums::LighterEnvironment, config::{LighterDataClientConfig, LighterExecClientConfig}, }; let data_config = LighterDataClientConfig::builder() .environment(LighterEnvironment::Testnet) .build(); let exec_config = LighterExecClientConfig::builder() .trader_id(trader_id) .account_id(account_id) .environment(LighterEnvironment::Testnet) .build(); ``` The execution config above resolves credentials from the matching testnet environment variables. Set `account_index`, `api_key_index`, and `private_key` directly to override environment lookup. Use `LiveExecEngineConfig.reconciliation_instrument_ids` to scope reconciliation to specific Nautilus instruments. On accounts with long history, also set `reconciliation_lookback_mins` on `LiveExecEngineConfig` to bound inactive order and fill replay to the window your strategy needs. ## Official documentation - Get started: - Trading and signing: - API keys: - Rate limits: - Volume quota: - Data structures, constants, and errors: - REST OpenAPI: - WebSocket reference: ## Contributing :::info For additional features or to contribute to the Lighter adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # OKX Source: https://nautilustrader.io/docs/latest/integrations/okx/ Founded in 2017, OKX is a cryptocurrency exchange that offers spot, margin, perpetual swap, futures, options, spread, and event contract trading. This integration supports live market data ingest and order execution on OKX. ## Overview This adapter is written in Rust, with optional Python bindings for Python workflows. It does not require external OKX client libraries. The core components are compiled as a static library and linked automatically during the build. ## Examples Live example scripts are available in [examples/live/okx](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/okx/). ### Product support | Product | Instrument source | Data | Exec | Notes | |-----------------|------------------------------|------|------|-------------------------------------------| | Spot | `public/instruments` | Yes | Yes | Spot trading pairs. | | Margin | `public/instruments` | Yes | Yes | Spot instruments with margin or leverage. | | Perpetual swaps | `public/instruments` | Yes | Yes | Linear and inverse contracts. | | Futures | `public/instruments` | Yes | Yes | Dated futures contracts. | | Options | `public/instruments` | Yes | Yes | Limit‑style order execution. | | Spreads | `sprd/spreads` | Yes | Yes | Snapshots, quotes, trades on business WS. | | Event contracts | `event-contract/*` endpoints | Yes | Yes | Parsed as Nautilus `BinaryOption`. | Relevant OKX docs: - [Get instruments](https://www.okx.com/docs-v5/en/#public-data-rest-api-get-instruments). - [Get Spreads (Public)](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-get-spreads-public). - [Spread trading place order](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-place-order). - [Event contract series](https://www.okx.com/docs-v5/en/#public-data-rest-api-get-series). :::note **Options support**: The adapter supports options market data, venue-provided Greeks (`subscribe_option_greeks`), and order execution for options instruments. See the [Options trading](#options-trading) section below for details and the [Options](../concepts/options.md) guide for subscription patterns. ::: :::info **Instrument multipliers**: For derivatives (`SWAP`, `FUTURES`, `OPTION`), instrument multipliers are calculated as the product of OKX's `ctMult` and `ctVal` fields. This keeps position sizing aligned with OKX contract size and value. ::: The OKX adapter includes multiple components, which can be used separately or together: - `OKXHttpClient`: Low-level HTTP API connectivity. - `OKXWebSocketClient`: Low-level WebSocket API connectivity. - `OKXInstrumentProvider`: Instrument parsing and loading functionality. - `OKXDataClient`: Market data feed manager. - `OKXExecutionClient`: Account management and trade execution gateway. - `OKXLiveDataClientFactory`: Factory for OKX data clients (used by the trading node builder). - `OKXLiveExecClientFactory`: Factory for OKX execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as shown below), and won't need to work directly with these lower-level components. ::: ## Symbology OKX uses specific symbol conventions for different instrument types. Add the `.OKX` suffix when referencing instruments in Nautilus, for example `BTC-USDT.OKX`. ### Symbol format by instrument type #### SPOT Format: `{BaseCurrency}-{QuoteCurrency}` Examples: - `BTC-USDT` - Bitcoin against USDT (Tether) - `BTC-USDC` - Bitcoin against USDC - `ETH-USDT` - Ethereum against USDT - `SOL-USDT` - Solana against USDT To subscribe to spot Bitcoin USD in your strategy: ```python InstrumentId.from_str("BTC-USDT.OKX") # For USDT-quoted spot InstrumentId.from_str("BTC-USDC.OKX") # For USDC-quoted spot ``` #### SWAP (perpetual swaps) Format: `{BaseCurrency}-{QuoteCurrency}-SWAP` Examples: - `BTC-USDT-SWAP` - Bitcoin perpetual swap (linear, USDT-margined) - `BTC-USD-SWAP` - Bitcoin perpetual swap (inverse, coin-margined) - `ETH-USDT-SWAP` - Ethereum perpetual swap (linear) - `ETH-USD-SWAP` - Ethereum perpetual swap (inverse) Linear vs inverse contracts: - **Linear** (USDT-margined): Uses stablecoins like USDT as margin. - **Inverse** (coin-margined): Uses the base cryptocurrency as margin. #### FUTURES (dated futures) Format: `{BaseCurrency}-{QuoteCurrency}-{YYMMDD}` Examples: - `BTC-USD-251226` - Bitcoin futures expiring December 26, 2025 - `ETH-USD-251226` - Ethereum futures expiring December 26, 2025 - `BTC-USD-250328` - Bitcoin futures expiring March 28, 2025 Note: Futures are typically inverse contracts (coin-margined). #### SPREADS Format: `{Leg1InstrumentId}_{Leg2InstrumentId}` Examples: - `BTC-USDT_BTC-USDT-SWAP` - Spread between BTC-USDT spot and BTC-USDT perpetual swap - `ETH-USD-SWAP_ETH-USD-231229` - Spread between ETH-USD perpetual swap and dated future Set `load_spreads=True` on the data client to load live OKX spread instruments from the OKX [Get Spreads (Public)](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-get-spreads-public) endpoint. The adapter maps each OKX `sprdId` to a Nautilus spread instrument ID with the `.OKX` venue suffix. Concise spread instrument notes: - Spread market data streams on the OKX business WebSocket: quotes (`sprd-bbo-tbt`), trades (`sprd-public-trades`), and 5-level book snapshots (`sprd-books5`). Spreads have no incremental book channel, so each `sprd-books5` update is a full snapshot delivered through the order book subscription (flagged as a snapshot, not incremental L2 deltas). - Current OKX live spread discovery returns spot, swap, and futures leg combinations. - The parser can represent option-leg spread definitions if OKX exposes them through the same spread endpoint. - OKX option RFQ and block trading workflows are separate from the Nitro spread order book API and are not routed by this spread path. #### OPTIONS Format: `{BaseCurrency}-{QuoteCurrency}-{YYMMDD}-{Strike}-{Type}` Examples: - `BTC-USD-251226-100000-C` - Bitcoin call option, $100,000 strike, expiring December 26, 2025 - `BTC-USD-251226-100000-P` - Bitcoin put option, $100,000 strike, expiring December 26, 2025 - `ETH-USD-251226-4000-C` - Ethereum call option, $4,000 strike, expiring December 26, 2025 Where: - `C` = Call option - `P` = Put option #### EVENTS OKX event contract instrument IDs use the market ID returned by the OKX instruments API. The adapter represents these markets as Nautilus `BinaryOption` instruments. Example: - `BTC-ABOVE-DAILY-260224-1600-65000` - Event contract market in the `BTC-ABOVE-DAILY` series. ### Common questions **Q: How do I subscribe to spot Bitcoin USD?** A: Use `BTC-USDT.OKX` for USDT-margined spot or `BTC-USDC.OKX` for USDC-margined spot. **Q: What's the difference between BTC-USDT-SWAP and BTC-USD-SWAP?** A: `BTC-USDT-SWAP` is a linear perpetual (USDT-margined), while `BTC-USD-SWAP` is an inverse perpetual (BTC-margined). **Q: How do I know which contract type to use?** A: Check the `contract_types` parameter in the configuration: - For linear contracts: `OKXContractType.LINEAR`. - For inverse contracts: `OKXContractType.INVERSE`. **Q: How do I load event contracts?** A: Use `OKXInstrumentType.EVENTS`. To scope loading, pass OKX `seriesId` values such as `BTC-ABOVE-DAILY` through `instrument_families`. ## Orders capability Below are the order types, execution instructions, and time-in-force options supported for linear perpetual swap products on OKX. ### WebSocket order identification OKX WebSocket order operations use `instIdCode` (a numeric instrument identifier) instead of the string `instId` parameter. The adapter resolves `instIdCode` values from the instrument definitions fetched during startup and caches them for the session lifetime. If the instrument cache is empty (e.g. because of a failed bootstrap), order submissions fail with a clear error. ### Client order ID requirements :::note OKX has specific requirements for client order IDs: - **No hyphens allowed**: OKX does not accept hyphens (`-`) in client order IDs. - Maximum length: 32 characters. - Allowed characters: letters and numbers only. When configuring your strategy, ensure you set: ```python use_hyphens_in_client_order_ids=False ``` ::: ### Order types | Order type | Linear perpetual swap | Notes | |------------------------|-----------------------|---------------------------------------------------------------| | `MARKET` | ✓ | Immediate execution at market price. Supports quote quantity. | | `MARKET_TO_LIMIT` | ✓ | Market order converted to IOC limit. | | `LIMIT` | ✓ | Execution at specified price or better. | | `STOP_MARKET` | ✓ | Conditional market order through OKX algo orders. | | `STOP_LIMIT` | ✓ | Conditional limit order through OKX algo orders. | | `MARKET_IF_TOUCHED` | ✓ | Conditional market order through OKX algo orders. | | `LIMIT_IF_TOUCHED` | ✓ | Conditional limit order through OKX algo orders. | | `TRAILING_STOP_MARKET` | ✓ | Trailing stop market order through OKX advance algo orders. | :::info **Conditional orders**: `STOP_MARKET`, `STOP_LIMIT`, `MARKET_IF_TOUCHED`, `LIMIT_IF_TOUCHED`, and `TRAILING_STOP_MARKET` use OKX algo orders. The `TRAILING_STOP_MARKET` path uses OKX's advance algo order API (`move_order_stop`) and requires the `cancel-advance-algos` endpoint for cancellation. ::: ### Spread orders OKX spread instruments use a separate spread trading order book and API family. The execution client currently routes spread orders by spread instrument ID, for example `ETH-USD-SWAP_ETH-USD-231229.OKX`, through the HTTP `/api/v5/sprd/*` endpoints. The adapter uses OKX's spread REST endpoints for submit, cancel, mass cancel, order status, and trade reports. It subscribes to the OKX business WebSocket [`sprd-orders` channel](https://www.okx.com/docs-v5/en/#spread-trading-websocket-private-channel-order-channel) for live spread order updates. OKX `sprd-orders` WebSocket updates do not include fee fields. Live spread fill reports emitted from that channel use zero commission; historical and reconciliation fill reports from the REST [`sprd/trades` endpoint](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-get-trades) include OKX fee data. Supported spread order instructions: - `LIMIT` with GTC time-in-force. - `LIMIT` with IOC time-in-force. - `LIMIT` with post-only execution. Spread order lists, conditional orders, FOK time-in-force, and modify requests are not supported by the OKX spread trading API path. Relevant OKX docs: - [Spread order placement](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-place-order). - [Spread order details](https://www.okx.com/docs-v5/en/#spread-trading-rest-api-get-order-details). - [Spread order channel](https://www.okx.com/docs-v5/en/#spread-trading-websocket-private-channel-order-channel). ### Quantity semantics for spot margin trading When using spot margin trading (`use_spot_margin=True`), OKX interprets order quantities differently depending on the order side: - **Limit** orders interpret `quantity` as the number of base currency units. - **Market SELL** orders also use base-unit quantities. - **Market BUY** orders interpret `quantity` as quote notional (e.g., USDT). :::warning **When submitting spot margin market BUY orders**, set `quote_quantity=True` on the order (or pre-compute the quote-denominated amount). The OKX execution client denies base-denominated market buy orders for spot margin to prevent unintended fills. **On the first fill**, the order quantity is automatically updated from the quote quantity to the actual base quantity received, reflecting the executed trade. ::: ```python # Spot margin market BUY with quote quantity (spend $100 USDT) order = strategy.order_factory.market( instrument_id=instrument_id, order_side=OrderSide.BUY, quantity=instrument.make_qty(100.0), quote_quantity=True, # Interpret as USDT notional ) strategy.submit_order(order) ``` ### Execution instructions | Instruction | Linear perpetual swap | Notes | |---------------|-----------------------|------------------------| | `post_only` | ✓ | Only for limit orders. | | `reduce_only` | ✓ | Only for derivatives. | ### Time in force | Time in force | Linear perpetual swap | Notes | |---------------|-----------------------|---------------------------------------------------| | `GTC` | ✓ | Good Till Canceled. | | `FOK` | ✓ | Fill or Kill. | | `IOC` | ✓ | Immediate or Cancel. | | `GTD` | - | *No native OKX order time‑in‑force.* | :::note **GTD (Good Till Date) time in force**: OKX supports request expiry through `expTime`, but that is a request timeout rather than a native order expiry instruction. If you need GTD functionality, use Nautilus's strategy-managed GTD feature. It handles order expiration by canceling the order at the specified expiry time. ::: ### Batch operations | Operation | Linear perpetual swap | Notes | |--------------------|-----------------------|-------------------------------------------| | Batch Submit | ✓ | Submit multiple orders in single request. | | Batch Modify | ✓ | Modify multiple orders in single request. | | Batch Cancel | ✓ | Cancel multiple orders in single request. | ### Position management | Feature | Linear perpetual swap | Notes | |-------------------|-----------------------|------------------------------------------------------| | Query positions | ✓ | Real‑time position updates. | | Position mode | ✓ | Net vs Long/Short mode (see below). | | Leverage control | ✓ | Dynamic leverage adjustment per instrument. | | Margin mode | ✓ | Supports cash, isolated, and cross modes. | #### Position modes OKX supports two position modes for derivatives trading: - **Net mode** (netting): One position per instrument. Buy and sell orders net against each other. This is the default and recommended mode for most traders. - **Long/Short mode** (hedging): Separate long and short positions for the same instrument. This mode supports simultaneous long and short exposure. :::note Position mode must be configured through the OKX web or app interface and applies account-wide. The adapter detects the current position mode and handles position reporting accordingly. ::: ### Trade modes and margin configuration OKX's unified account system supports different trade modes for spot and derivatives trading. The adapter determines the correct trade mode from your configuration and instrument type. :::note **Important**: Configure the account mode first through the OKX Web or app interface. The API cannot set the account mode for the first time. ::: For more details on OKX account modes and margin, see the [OKX Account Mode documentation](https://www.okx.com/docs-v5/en/#overview-account-mode). #### Trade modes overview OKX supports multiple account modes. For orders, the adapter selects one of the `cash`, `isolated`, or `cross` trade modes from your configuration: | Mode | Used for | Leverage | Borrowing | Configuration | |----------------|--------------------------------|----------|-----------|---------------------------------------| | **`cash`** | Spot trading without leverage. | - | - | Default when `use_spot_margin=False`. | | **`isolated`** | Spot margin or derivatives. | ✓ | ✓ | `margin_mode=ISOLATED`. | | **`cross`** | Spot margin or derivatives. | ✓ | ✓ | `margin_mode=CROSS`. | #### Configuration-based trade mode selection The adapter selects the trade mode from: 1. **Instrument type** (`SPOT` vs other OKX instrument types). 2. **Configuration settings** (`use_spot_margin` for `SPOT`, `margin_mode` otherwise). ##### For SPOT trading ```python # Simple SPOT trading without leverage (uses 'cash' mode) exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.SPOT,), use_spot_margin=False, # Default - simple SPOT # ... other config ), } # SPOT trading WITH margin/leverage (uses 'isolated' or 'cross' mode) exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.SPOT,), use_spot_margin=True, # Enable margin trading for SPOT margin_mode=OKXMarginMode.ISOLATED, # Or CROSS for shared margin # ... other config ), } ``` ##### For non-spot trading ```python # Derivatives with isolated margin (default - uses 'isolated' mode) exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.SWAP,), margin_mode=OKXMarginMode.ISOLATED, # Or omit - ISOLATED is default # ... other config ), } # Derivatives with cross margin (uses 'cross' mode) exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.SWAP,), margin_mode=OKXMarginMode.CROSS, # Share margin across all positions # ... other config ), } ``` ##### For mixed SPOT and derivatives trading When trading both SPOT and derivatives instruments, the adapter determines the trade mode per order based on the instrument being traded: ```python # Mixed SPOT + SWAP configuration exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.SPOT, OKXInstrumentType.SWAP), use_spot_margin=True, # Applies to SPOT orders only margin_mode=OKXMarginMode.CROSS, # Applies to SWAP orders only # ... other config ), } ``` **How it works:** - **SPOT orders** use `cross` mode because `use_spot_margin=True` and `margin_mode=CROSS`. - **SWAP orders** use `cross` mode because `margin_mode=CROSS`. - Each order gets the correct `tdMode` based on its instrument type. - No manual intervention is required. This supports strategies that trade across instrument types with different margin configuration, such as: - Spot-futures arbitrage strategies. - Delta-neutral strategies combining spot and perpetual swaps. - Market making across spot and derivatives markets. :::warning **Manual trade mode override**: You can override the trade mode per order with `params={"td_mode": "..."}`. This bypasses adapter selection and can lead to order rejection when the value does not match the instrument type, such as `isolated` for spot instruments. Only use manual override for requirements that cannot be met through configuration. ::: #### Benefits of configuration-based approach - **Type-safe**: Configuration is validated at startup before placing any orders. - **Automatic**: The adapter chooses the mode based on instrument type and intent. - **Clear**: Field names explain intent, such as `use_spot_margin` vs `td_mode`. - **Safe**: Incompatible combinations are rejected before they reach OKX. - **Backwards compatible**: Default values preserve existing behavior. ### Order querying | Feature | Linear perpetual swap | Notes | |----------------------|-----------------------|-------------------------------------------| | Query open orders | ✓ | List all active orders. | | Query order history | ✓ | Historical order data. | | Order status updates | ✓ | Real‑time order state changes. | | Trade history | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Linear perpetual swap | Notes | |--------------------|-----------------------|---------------------------------------| | Order lists | ✓ | Batch via WS; regular orders only. | | OCO orders | ✓ | One‑Cancels‑Other orders. | | Bracket orders | ✓ | Stop loss + take profit combinations. | | Conditional orders | ✓ | Stop and limit‑if‑touched orders. | #### Conditional order architecture Conditional orders (OKX algo orders) use a hybrid architecture: - **Submission**: HTTP REST API (`/api/v5/trade/order-algo`). - **Status updates**: WebSocket business endpoint (`/ws/v5/business`) on the `orders-algo` channel. - **Cancellation**: HTTP REST API with algo order ID tracking. This design ensures: - Immediate submission acknowledgment through HTTP. - Real-time status updates through WebSocket. - Proper order lifecycle management with algo order ID mapping. #### Supported conditional order types | Order type | Trigger types | Notes | |------------------------|-------------------|-------------------------------------------| | `STOP_MARKET` | Last, Mark, Index | Market execution when triggered. | | `STOP_LIMIT` | Last, Mark, Index | Limit order placement when triggered. | | `MARKET_IF_TOUCHED` | Last, Mark, Index | Market execution when price touched. | | `LIMIT_IF_TOUCHED` | Last, Mark, Index | Limit order placement when price touched. | | `TRAILING_STOP_MARKET` | Last, Mark, Index | Trailing stop with callback ratio. | #### Trigger price types Conditional orders support different trigger price sources: - **Last price** (`TriggerType.LAST_PRICE`): Uses the last traded price (default). - **Mark price** (`TriggerType.MARK_PRICE`): Uses the mark price. - **Index price** (`TriggerType.INDEX_PRICE`): Uses the underlying index price. ```python # Example: Stop loss using mark price trigger stop_order = order_factory.stop_market( instrument_id=instrument_id, order_side=OrderSide.SELL, quantity=Quantity.from_str("0.1"), trigger_price=Price.from_str("45000.0"), trigger_type=TriggerType.MARK_PRICE, # Use mark price for trigger ) strategy.submit_order(stop_order) ``` ## Risk management ### Liquidation and ADL event handling The OKX adapter detects exchange-initiated risk management events: - **Liquidation orders**: When the exchange liquidates a position, the adapter detects the liquidation category and logs warnings with order details. These orders continue through the normal order and fill pipeline. - **Auto-deleveraging (ADL)**: When OKX closes your position to offset a counterparty's liquidation, the adapter detects and logs the ADL event with position details. Detection is driven by the `category` field on the order record. The recognized values are: | `category` | Meaning | |-------------------------|------------------------------------------------------| | `full_liquidation` | Full position liquidation. | | `partial_liquidation` | Partial position liquidation. | | `adl` | Auto‑deleveraging close. | | `delivery` | Contract delivery at expiry. | | `normal` / other values | Regular order flow. | Detection runs on both paths: - WebSocket `orders` channel (live order/fill updates). - HTTP `GET /api/v5/trade/orders-history` and `orders-history-archive` (used during reconciliation and cold-start mass status). :::info **Liquidation and ADL events are logged at WARNING level** with details including order ID, instrument, and state. Monitor these logs as part of your risk management process. The adapter handles these exchange-generated orders, emits the relevant `OrderFilled` events, and updates positions. Your strategy code does not need a separate path. ::: Upstream references: - [Order channel and `category` field](https://www.okx.com/docs-v5/en/#order-book-trading-trade-ws-order-channel) - [Auto-Deleveraging mechanism](https://www.okx.com/help/okx-contract-auto-deleveraging-adl) - [Liquidation mechanism](https://www.okx.com/help/introduction-to-liquidation) ## Options trading The OKX adapter supports trading options (`OPTION` instrument type) with some differences from other derivatives. OKX options are inverse contracts settled in the underlying cryptocurrency. For full API details see the [OKX Options Trading documentation](https://www.okx.com/docs-v5/en/#order-book-trading-trade-post-place-order). ### Supported order types Only limit-style orders are supported. OKX does not allow market orders for options. | Order type | Supported | Notes | |------------|-----------|---------------------------------------------------| | `LIMIT` | ✓ | Standard limit order. | | `MARKET` | - | Rejected by the adapter before reaching the API. | Options support FOK and IOC time-in-force. OKX uses a dedicated `op_fok` order type for options FOK orders; the adapter handles this mapping automatically. Conditional/algo orders (`STOP_MARKET`, `STOP_LIMIT`, `MARKET_IF_TOUCHED`, `LIMIT_IF_TOUCHED`, `TRAILING_STOP_MARKET`) are not supported for options and are denied. ### Pricing modes Options orders can be priced in three mutually exclusive ways. Pass the pricing mode via order `params`: | Mode | Parameter | Description | |-------|-----------|---------------------------------------------------------| | Price | (default) | Standard limit price in the contract's currency. | | USD | `px_usd` | Price in USD terms. | | IV | `px_vol` | Price in implied volatility (1.0 = 100%). | ```python # Price in USD order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTC-USD-250328-50000-C.OKX"), order_side=OrderSide.BUY, quantity=Quantity.from_int(1), price=Price.from_str("0"), # Placeholder; px_usd takes precedence params={"px_usd": "100.5"}, ) # Price in implied volatility order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTC-USD-250328-50000-C.OKX"), order_side=OrderSide.BUY, quantity=Quantity.from_int(1), price=Price.from_str("0"), # Placeholder; px_vol takes precedence params={"px_vol": "0.55"}, ) ``` When modifying an order, the same `px_usd` or `px_vol` params can be passed to the modify command to amend the price in the original pricing mode. ### Option Greeks OKX publishes two parallel greek sets on the `opt-summary` channel: - **Black-Scholes (`BLACK_SCHOLES`)**: Greeks denominated in USD. Matches the convention used by the Deribit and Bybit adapters. - **Price-adjusted (`PRICE_ADJUSTED`)**: Greeks denominated in the underlying coin units. Matches OKX's native contract convention. By default the adapter emits both on every `opt-summary` tick. Each emitted `OptionGreeks` carries a `convention` field set to `GreeksConvention.BLACK_SCHOLES` or `GreeksConvention.PRICE_ADJUSTED`, so receivers can branch per message. To narrow the stream, pass `params["greeks_convention"]` on subscribe: - Single string: `"BLACK_SCHOLES"` or `"PRICE_ADJUSTED"` (case-insensitive). - List of strings: `["BLACK_SCHOLES", "PRICE_ADJUSTED"]`. - Omitted: adapter emits both. Unknown entries log a warning and are skipped. If every requested entry is unknown, the adapter falls back to emitting both. ```python # Default (both conventions, receiver branches) self.subscribe_option_greeks(instrument_id) def on_option_greeks(self, greeks: OptionGreeks) -> None: if greeks.convention == GreeksConvention.BLACK_SCHOLES: self._handle_bs(greeks) else: self._handle_pa(greeks) ``` ```python # Single-convention narrowing self.subscribe_option_greeks( instrument_id, params={"greeks_convention": "PRICE_ADJUSTED"}, ) ``` ```python # Explicit list (equivalent to the default when both are listed) self.subscribe_option_greeks( instrument_id, params={"greeks_convention": ["BLACK_SCHOLES", "PRICE_ADJUSTED"]}, ) ``` :::note The data engine deduplicates option-greeks subscriptions by `instrument_id`, so if two actors on one node subscribe to the same instrument with different single conventions only the first one reaches the adapter. The second actor gets the first actor's convention set. Workaround: either actor can subscribe without `params` (or with the full list) to receive both streams and filter locally on `greeks.convention`. ::: ### Position Greeks The adapter exposes position-level Black-Scholes Greeks (`delta_bs`, `gamma_bs`, `theta_bs`, `vega_bs`) from OKX position data. These are available through the standard position reporting pipeline. ### Restrictions - `reduce_only` is not applicable to options and is automatically stripped. - Position side defaults to `Net`. ### Configuration Options require the `instrument_families` config parameter to scope which underlyings to load: ```python config = TradingNodeConfig( data_clients={ OKX: OKXDataClientConfig( instrument_types=(OKXInstrumentType.OPTION,), instrument_families=("BTC-USD", "ETH-USD"), instrument_provider=InstrumentProviderConfig(load_all=True), ), }, exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.OPTION,), instrument_families=("BTC-USD", "ETH-USD"), margin_mode=OKXMarginMode.CROSS, ), }, ) ``` ## Event contracts OKX exposes prediction market contracts through `instType=EVENTS`. The adapter loads these instruments as Nautilus `BinaryOption` instruments and preserves OKX metadata such as `seriesId`, `instCategory`, `instIdCode`, `state`, and `ruleType` in the instrument `info` field. ### Loading event contract instruments Use `OKXInstrumentType.EVENTS` in the data or execution client config. The `instrument_families` setting maps to OKX `seriesId` values for event contracts. When `instrument_families` is omitted, the adapter requests the event contract series list first, then requests instruments for each series. ```python config = TradingNodeConfig( data_clients={ OKX: OKXDataClientConfig( instrument_types=(OKXInstrumentType.EVENTS,), instrument_families=("BTC-ABOVE-DAILY",), instrument_provider=InstrumentProviderConfig(load_all=True), ), }, exec_clients={ OKX: OKXExecClientConfig( instrument_types=(OKXInstrumentType.EVENTS,), instrument_families=("BTC-ABOVE-DAILY",), margin_mode=OKXMarginMode.CROSS, ), }, ) ``` ### Event contract market data The low-level HTTP client exposes OKX's public event contract discovery endpoints: - `request_event_contract_series`. - `request_event_contract_events`. - `request_event_contract_markets`. The low-level WebSocket client supports the `event-contract-markets` channel through `subscribe_event_contract_markets` and `unsubscribe_event_contract_markets`. This channel publishes market status and floor-strike generation updates, has no initial snapshot, and does not include `instId`, so the adapter forwards it as raw venue JSON. :::note OKX's standard market data endpoints return YES-side data for `EVENTS`. Derive NO-side prices from YES-side prices when a strategy needs both outcomes. ::: ### Event contract trading Pass the OKX event outcome through order `params` when submitting event contract orders: ```python order = strategy.order_factory.limit( instrument_id=InstrumentId.from_str("BTC-ABOVE-DAILY-260224-1600-65000.OKX"), order_side=OrderSide.BUY, quantity=Quantity.from_int(1), price=Price.from_str("0.42"), params={"outcome": "yes"}, ) strategy.submit_order(order) ``` OKX requires `outcome` for `EVENTS` orders. It also requires `speedBump=1` for non-post-only event contract orders and amendments. The adapter validates `outcome` before sending the order and defaults `speedBump` to `1` for non-post-only event orders when it is not supplied. Settlement fills arrive with OKX order category `delivery`. The adapter recognizes this category during live order updates and reconciliation. Upstream references: - [Event contract REST endpoints](https://www.okx.com/docs-v5/en/#public-data-rest-api-get-series). - [WS channel](https://www.okx.com/docs-v5/en/#public-data-websocket-event-contract-markets-channel). - [Place order request fields](https://www.okx.com/docs-v5/en/#order-book-trading-trade-post-place-order). ## Authentication To use the OKX adapter, create API credentials in your OKX account: 1. Log into your OKX account and navigate to the API management page. 2. Create a new API key with the required permissions for trading and data access. 3. Record your API key, secret key, and passphrase. You can provide these credentials through environment variables: ```bash export OKX_API_KEY="your_api_key" export OKX_API_SECRET="your_api_secret" export OKX_API_PASSPHRASE="your_passphrase" ``` Or pass them directly in the configuration (not recommended for production). ## Demo trading OKX provides a demo trading environment for testing strategies without real funds. ### Setting up a demo account 1. Log into your OKX account at [okx.com](https://www.okx.com). 2. Navigate to **Trade** > **Demo Trading**. 3. Go to **Personal Center** within Demo Trading. 4. Select **Demo Trading API** and create a new API key. 5. Record your demo API key, secret key, and passphrase. You can provide demo credentials through environment variables: ```bash export OKX_API_KEY="your_demo_api_key" export OKX_API_SECRET="your_demo_api_secret" export OKX_API_PASSPHRASE="your_demo_passphrase" ``` ### Configuration Set `environment=OKXEnvironment.DEMO` in your client configuration: ```python from nautilus_trader.core.nautilus_pyo3 import OKXEnvironment config = TradingNodeConfig( data_clients={ OKX: OKXDataClientConfig( environment=OKXEnvironment.DEMO, # ... other config ), }, exec_clients={ OKX: OKXExecClientConfig( environment=OKXEnvironment.DEMO, # ... other config ), }, ) ``` When demo mode is enabled: - REST API requests include the `x-simulated-trading: 1` header. - WebSocket connections use demo endpoints (`wspap.okx.com`). :::note Demo API keys are separate from production keys. Create API keys for demo trading through the Demo Trading interface. Production API keys do not work in demo mode. ::: ## Regional endpoints OKX serves distinct endpoints per region, and an API key is only valid against the region where it was registered (using a key against another region's endpoints returns `API key doesn't exist`). Set `region` to select the correct endpoint set: | Region | Registered on | REST | WebSocket host | |----------|---------------|-----------------|----------------------| | `GLOBAL` | `www.okx.com` | `www.okx.com` | `ws.okx.com` | | `EEA` | `my.okx.com` | `eea.okx.com` | `wseea.okx.com` | | `US` | `app.okx.com` | `us.okx.com` | `wsus.okx.com` | `region` defaults to `GLOBAL`. For example, an EEA account: ```python from nautilus_trader.core.nautilus_pyo3 import OKXRegion config = TradingNodeConfig( data_clients={ OKX: OKXDataClientConfig( region=OKXRegion.EEA, # ... other config ), }, exec_clients={ OKX: OKXExecClientConfig( region=OKXRegion.EEA, # ... other config ), }, ) ``` `region` selects the regional defaults, and combines with `environment` to pick the demo hosts (for example `wseeapap.okx.com` for EEA demo). Explicit `base_url_http` and `base_url_ws` overrides always take precedence over the region defaults. ## Funding rates The adapter receives funding rate data from the [Funding Rate Channel](https://www.okx.com/docs-v5/en/#public-data-websocket-funding-rate-channel) WebSocket stream. OKX provides both `fundingTime` and `nextFundingTime` in each message, and the adapter computes `interval` as the difference between these two values. For historical funding rate requests, the adapter computes the interval from consecutive funding timestamps returned by the [Get Funding Rate History](https://www.okx.com/docs-v5/en/#public-data-rest-api-get-funding-rate-history) endpoint. ## Rate limiting The adapter enforces OKX's per-endpoint quotas while keeping sensible defaults for REST and WebSocket calls. ### REST limits - Internal global bucket: 250 requests per second. - Endpoint-specific quotas appear in the table below and mirror OKX's published limits where available. ### WebSocket limits - Connection establishment: 3 requests per second (per IP). - Subscription operations (subscribe/unsubscribe/login): 480 requests per hour per connection. - Order operation buckets appear in the table below and mirror OKX's published limits where available. | Operation key | Limit (req/sec) | Notes | |----------------|-----------------|-----------------------------------------------------------| | `order` | 30 | OKX 60 requests / 2 seconds. | | `cancel` | 30 | OKX 60 requests / 2 seconds. | | `amend` | 30 | OKX 60 requests / 2 seconds. | | `batch-order` | 7 | OKX 300 orders / 2 seconds, rounded down for full batches. | | `batch-cancel` | 7 | OKX 300 orders / 2 seconds, rounded down for full batches. | | `batch-amend` | 7 | OKX 300 orders / 2 seconds, rounded down for full batches. | | `mass-cancel` | 2 | OKX 5 requests / 2 seconds, rounded down. | | `algo-order` | 10 | OKX 20 requests / 2 seconds. | | `algo-cancel` | 1 | OKX 20 orders / 2 seconds, rounded down for full batches. | :::warning OKX enforces per-endpoint and per-account quotas. Exceeding them leads to HTTP 429 responses and temporary throttling on that key. ::: | Key / endpoint | Limit (req/sec) | Notes | |-----------------------------------------|-----------------|---------------------------------------------------| | `okx:global` | 250 | Adapter‑level shared bucket. | | `/api/v5/account/set-position-mode` | 2 | OKX 5 requests / 2 seconds, rounded down. | | `/api/v5/account/balance` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/account/trade-fee` | 2 | OKX 5 requests / 2 seconds, rounded down. | | `/api/v5/account/positions` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/account/positions-history` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/instruments` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/public/position-tiers` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/event-contract/series` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/event-contract/events` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/event-contract/markets` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/opt-summary` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/public/time` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/mark-price` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/public/funding-rate-history` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/market/index-tickers` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/market/books` | 20 | OKX 40 requests / 2 seconds. | | `/api/v5/market/candles` | 20 | OKX 40 requests / 2 seconds. | | `/api/v5/market/history-candles` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/market/history-trades` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/sprd/spreads` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/sprd/order` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/sprd/cancel-order` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/sprd/mass-cancel` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/sprd/orders-pending` | 5 | OKX 10 requests / 2 seconds. | | `/api/v5/sprd/orders-history` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/sprd/trades` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/trade/order` | 30 | OKX 60 requests / 2 seconds. | | `/api/v5/trade/cancel-batch-orders` | 7 | OKX 300 orders / 2 seconds, rounded down. | | `/api/v5/trade/orders-pending` | 30 | OKX 60 requests / 2 seconds. | | `/api/v5/trade/orders-history` | 20 | OKX 40 requests / 2 seconds. | | `/api/v5/trade/fills` | 30 | OKX 60 requests / 2 seconds. | | `/api/v5/trade/order-algo` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/trade/cancel-algos` | 1 | OKX 20 orders / 2 seconds. | | `/api/v5/trade/cancel-advance-algos` | 1 | Conservative bucket for advance algo cancels. | | `/api/v5/trade/amend-algos` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/trade/orders-algo-pending` | 10 | OKX 20 requests / 2 seconds. | | `/api/v5/trade/orders-algo-history` | 10 | OKX 20 requests / 2 seconds. | All keys include the `okx:global` bucket. URLs are normalized with query strings removed before rate limiting, so requests with different filters share the same quota. For order-based cancel quotas, the adapter uses request-level buckets that assume full batch sizes: 20 orders per request for regular batch cancels and 10 orders per request for algo cancels. OKX's current public docs no longer list a rate limit for `/api/v5/trade/cancel-advance-algos`, but the adapter still has an endpoint-specific bucket because the HTTP client can call that legacy path. :::info See the [OKX rate limit documentation](https://www.okx.com/docs-v5/en/#rest-api-rate-limit). ::: ## Configuration ### Configuration options The OKX data client provides the following configuration options: #### Data client | Option | Default | Description | |------------------------------------|-----------------------------|----------------------------------------------| | `instrument_types` | `(OKXInstrumentType.SPOT,)` | OKX instrument types to load. | | `contract_types` | `None` | Contract styles to load. | | `load_spreads` | `False` | Loads live spread instruments. | | `instrument_families` | `None` | Families or event `seriesId` values. | | `base_url_http` | `None` | Override for the OKX REST endpoint. | | `base_url_ws_public` | `None` | Override for the public WebSocket URL. | | `base_url_ws_business` | `None` | Override for the business WebSocket URL. | | `api_key` | `None` | Falls back to `OKX_API_KEY` when unset. | | `api_secret` | `None` | Falls back to `OKX_API_SECRET` when unset. | | `api_passphrase` | `None` | Falls back to `OKX_API_PASSPHRASE`. | | `environment` | `None` | Environment enum (`LIVE` or `DEMO`). | | `region` | `None` | Region enum (`GLOBAL`, `EEA`, or `US`). | | `http_timeout_secs` | `60` | REST market data request timeout. | | `max_retries` | `3` | Retry attempts for recoverable REST errors. | | `retry_delay_initial_ms` | `1,000` | Initial delay before retrying. | | `retry_delay_max_ms` | `10,000` | Maximum exponential backoff delay. | | `update_instruments_interval_mins` | `60` | Background instrument refresh interval. | | `vip_level` | `None` | Enables higher‑depth books by VIP tier. | | `proxy_url` | `None` | Optional HTTP and WebSocket proxy URL. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | Supported data client `instrument_types` values are `SPOT`, `MARGIN`, `SWAP`, `FUTURES`, `OPTION`, and `EVENTS`. `instrument_families` is required for `OPTION`, optional for `FUTURES`, `SWAP`, and `EVENTS`, and ignored for `SPOT` and `MARGIN`. For `EVENTS`, pass OKX `seriesId` values such as `BTC-ABOVE-DAILY`. Spread instruments use `load_spreads` instead of `instrument_types` because OKX serves them from `/api/v5/sprd/spreads`. The OKX execution client provides the following configuration options: #### Execution client | Option | Default | Description | |-----------------------------------|-----------------------------|---------------------------------------------| | `instrument_types` | `(OKXInstrumentType.SPOT,)` | Tradable OKX instrument types. | | `contract_types` | `None` | Tradable contract styles to load. | | `load_spreads` | `False` | Loads live spread instruments. | | `instrument_families` | `None` | Families or event `seriesId` values. | | `base_url_http` | `None` | Override for the OKX trading REST endpoint. | | `base_url_ws_private` | `None` | Override for the private WebSocket URL. | | `base_url_ws_business` | `None` | Override for the business WebSocket URL. | | `api_key` | `None` | Falls back to `OKX_API_KEY` when unset. | | `api_secret` | `None` | Falls back to `OKX_API_SECRET` when unset. | | `api_passphrase` | `None` | Falls back to `OKX_API_PASSPHRASE`. | | `environment` | `None` | Environment enum (`LIVE` or `DEMO`). | | `region` | `None` | Region enum (`GLOBAL`, `EEA`, or `US`). | | `margin_mode` | `None` | Margin mode (`ISOLATED` or `CROSS`). | | `use_spot_margin` | `False` | Enables spot‑style margin or leverage. | | `http_timeout_secs` | `60` | REST trading request timeout. | | `use_fills_channel` | `False` | Subscribes to fills channel (VIP5+). | | `use_mm_mass_cancel` | `False` | Uses the market‑maker bulk cancel endpoint. | | `max_retries` | `3` | Retry attempts for recoverable REST errors. | | `retry_delay_initial_ms` | `1,000` | Initial delay before retrying. | | `retry_delay_max_ms` | `10,000` | Maximum exponential backoff delay. | | `use_spot_cash_position_reports` | `False` | Generates SPOT cash positions from wallet. | | `proxy_url` | `None` | Optional HTTP and WebSocket proxy URL. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | Supported execution client `instrument_types` values are `SPOT`, `MARGIN`, `SWAP`, `FUTURES`, `OPTION`, and `EVENTS`. `instrument_families` has the same meaning for execution clients as it does for data clients. Spread instruments use OKX spread IDs instead of `instrument_types`; load them with `load_spreads=True` on the data client and, for Python v1 execution-only nodes, on the execution client before trading them. ### Manual endpoint overrides Setting `region` (see [Regional endpoints](#regional-endpoints)) selects the correct EEA or US endpoints automatically, which is the recommended approach. The explicit `base_url_*` overrides below remain available for proxies, custom routing, or endpoints not covered by a region; they take precedence over the `region` default. The EEA bases are shown as an example. | Config field | Live base | Demo base | WebSocket path | |------------------------|----------------------------|-------------------------------|-------------------| | `base_url_http` | `https://eea.okx.com` | `https://eea.okx.com` | | | `base_url_ws_public` | `wss://wseea.okx.com:8443` | `wss://wseeapap.okx.com:8443` | `/ws/v5/public` | | `base_url_ws_private` | `wss://wseea.okx.com:8443` | `wss://wseeapap.okx.com:8443` | `/ws/v5/private` | | `base_url_ws_business` | `wss://wseea.okx.com:8443` | `wss://wseeapap.okx.com:8443` | `/ws/v5/business` | For WebSocket fields, join the base and path in the same row. Use `base_url_ws_public` with data client configs and `base_url_ws_private` with execution client configs. EEA accounts must also set `base_url_ws_business` on both v2 configs because v2 does not derive the business WebSocket URL from the public or private override. Python v1 live configs expose `base_url_ws` instead of the split WebSocket fields. For those configs, set `base_url_ws` to the public EEA WebSocket URL on `OKXDataClientConfig` and to the private EEA WebSocket URL on `OKXExecClientConfig`; each client derives its business WebSocket URL from that value. See the [OKX EEA API documentation](https://my.okx.com/docs-v5/en/) for the current official endpoint list. Below is an example configuration for a live trading node using OKX data and execution clients: ```python from nautilus_trader.adapters.okx import OKX from nautilus_trader.adapters.okx import OKXDataClientConfig, OKXExecClientConfig from nautilus_trader.adapters.okx.factories import OKXLiveDataClientFactory, OKXLiveExecClientFactory from nautilus_trader.config import InstrumentProviderConfig, TradingNodeConfig from nautilus_trader.core.nautilus_pyo3 import OKXContractType from nautilus_trader.core.nautilus_pyo3 import OKXEnvironment from nautilus_trader.core.nautilus_pyo3 import OKXInstrumentType from nautilus_trader.core.nautilus_pyo3 import OKXMarginMode from nautilus_trader.live.node import TradingNode config = TradingNodeConfig( ..., data_clients={ OKX: OKXDataClientConfig( api_key=None, # Will use OKX_API_KEY env var api_secret=None, # Will use OKX_API_SECRET env var api_passphrase=None, # Will use OKX_API_PASSPHRASE env var base_url_http=None, base_url_ws=None, environment=OKXEnvironment.LIVE, instrument_provider=InstrumentProviderConfig(load_all=True), instrument_types=(OKXInstrumentType.SWAP,), contract_types=(OKXContractType.LINEAR,), ), }, exec_clients={ OKX: OKXExecClientConfig( api_key=None, api_secret=None, api_passphrase=None, base_url_http=None, base_url_ws=None, environment=OKXEnvironment.LIVE, instrument_provider=InstrumentProviderConfig(load_all=True), instrument_types=(OKXInstrumentType.SWAP,), contract_types=(OKXContractType.LINEAR,), ), }, ) node = TradingNode(config=config) node.add_data_client_factory(OKX, OKXLiveDataClientFactory) node.add_exec_client_factory(OKX, OKXLiveExecClientFactory) node.build() ``` ## Contributing :::info For additional features or to contribute to the OKX adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Polymarket Source: https://nautilustrader.io/docs/latest/integrations/polymarket/ Founded in 2020, Polymarket is a decentralized prediction market platform that enables traders to speculate on event outcomes by buying and selling outcome tokens. NautilusTrader provides a venue integration for data and execution via Polymarket's Central Limit Order Book (CLOB) API. Today the repository exposes two Polymarket implementations through the public package path `nautilus_trader.adapters.polymarket`: - The Python adapter, which uses the [official Python CLOB V2 client library](https://github.com/Polymarket/py-clob-client-v2). - The Rust-native adapter surface, which NautilusTrader is consolidating toward. :::warning The two implementations overlap heavily, but they do not behave identically in every area. This guide calls out the current differences where they matter. ::: NautilusTrader supports multiple Polymarket signature types for order signing, which gives flexibility for different wallet configurations while NautilusTrader handles signing and order preparation. ## Installation To install NautilusTrader with Polymarket support: ```bash uv pip install "nautilus_trader[polymarket]" ``` To build from source with all extras (including Polymarket): ```bash uv sync --all-extras ``` ## Examples You can find live example scripts [here](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/polymarket/). ## Binary options A [binary option](https://en.wikipedia.org/wiki/Binary_option) is a type of financial exotic option contract in which traders bet on the outcome of a yes-or-no proposition. If the prediction is correct, the trader receives a fixed payout; otherwise, they receive nothing. NautilusTrader represents Polymarket outcome tokens as `BinaryOption` instruments. Polymarket uses **pUSD** as the collateral token for trading, [see below](#pusd) for more information. ## Polymarket documentation Polymarket offers resources for different audiences: - [Polymarket Learn](https://learn.polymarket.com/): Educational content and guides for users to understand the platform and how to engage with it. - [Polymarket CLOB API](https://docs.polymarket.com/trading/orders/overview): Technical documentation for developers interacting with the Polymarket CLOB API. ## Overview This guide assumes a trader is setting up for both live market data feeds and trade execution. The Polymarket integration adapter includes multiple components, which can be used together or separately depending on the use case. - `PolymarketWebSocketClient`: Low-level WebSocket API connectivity (built on top of the Nautilus `WebSocketClient` written in Rust). - `PolymarketInstrumentProvider`: Instrument parsing and loading functionality for `BinaryOption` instruments. - `PolymarketDataClient`: A market data feed manager. - `PolymarketExecutionClient`: A trade execution gateway. - `PolymarketLiveDataClientFactory`: Factory for Polymarket data clients (used by the trading node builder). - `PolymarketLiveExecClientFactory`: Factory for Polymarket execution clients (used by the trading node builder). :::note Most users will define a configuration for a live trading node (as below), and won't need to work with these lower-level components directly. ::: ### Python and Rust implementations The current docs cover both the Python adapter and the Rust-native adapter surface. The table below shows the main differences that affect behavior today. | Area | Python adapter | Rust adapter | Notes | |---------------------|-------------------------------------------------------------------------------|---------------------------------------------------------------|-------| | Public package path | `nautilus_trader.adapters.polymarket` | `nautilus_trader.adapters.polymarket` | Rust is the consolidation target. | | Order signing | Uses `py-clob-client-v2` | Native Rust signing | Python signing is slower. | | Post‑only orders | Supported for `GTC` and `GTD` only | Supported for `GTC` and `GTD` only | Both reject post‑only with market TIF (`IOC` or `FOK`). | | Batch submit | Uses `POST /orders` for batchable `SubmitOrderList` requests | Uses `POST /orders` for batchable `SubmitOrderList` requests | Both batch only independent limit orders, capped at 15 per request. | | Batch cancel | Uses `DELETE /orders` | Uses `DELETE /orders` | Both align with official Polymarket docs. | | Market unsubscribe | Sends dynamic WebSocket `unsubscribe` messages | Sends dynamic WebSocket `unsubscribe` messages | Both support subscribe and unsubscribe. | | Auto‑load retry | `auto_load_max_retries` (12), `auto_load_retry_delay_*` (5.0/15.0 secs) | Same knobs, same defaults | Both retry CLOB‑hydration / indexing‑lag misses with bounded exponential backoff plus jitter. | | Data client config | Credentials, subscription buffering, quote handling, provider config | Base URLs, timeouts, filters, new‑market discovery | Config surfaces differ materially outside of the auto‑load family. | | Exec client config | Credentials, retries, raw WS logging, experimental trade‑based order recovery | Credentials, retries, account IDs, native timeouts | Rust does not expose every Python‑only option. | ## pUSD **pUSD** is the collateral token used for trading on Polymarket. It is a standard ERC-20 token on Polygon, backed by USDC. The proxy contract address is [0xC011a7E12a19f7B1f670d46F03B03f3342E82DFB](https://polygonscan.com/address/0xC011a7E12a19f7B1f670d46F03B03f3342E82DFB) on Polygon. Direct on-chain funding wraps Polygon USDC.e (bridged USDC) into pUSD through the [CollateralOnramp](https://docs.polymarket.com/resources/contracts). The Bridge API can also deposit supported assets from other chains and credit pUSD after conversion. ## Wallets and accounts To interact with Polymarket via NautilusTrader, you'll need a **Polygon**-compatible wallet (such as MetaMask). ### Signature types Polymarket supports multiple signature types for order signing and verification: | Signature Type | Wallet Type | Description | Use Case | |----------------|--------------------------------|--------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------| | `0` | EOA (Externally Owned Account) | Standard EIP712 signatures from wallets with direct private key control. | **Default.** Direct wallet connections (MetaMask, hardware wallets, etc.). | | `1` | Email/Magic Wallet Proxy | Smart contract wallet for email‑based accounts (Magic Link). | Polymarket Proxy associated with Email/Magic accounts. Requires `funder` address. | | `2` | Browser Wallet Proxy | Modified Gnosis Safe (1-of-1 multisig) for browser wallets. | Polymarket Proxy associated with browser wallets. Enables UI verification. Requires `funder` address. | | `3` | Deposit Wallet | ERC-1271 deposit wallet flow for new API users. | Requires deposit wallet `funder`; API credentials stay bound to the signer. | :::note See also: [Proxy wallet](https://docs.polymarket.com/developers/proxy-wallet) in the Polymarket documentation for more details about signature types and proxy wallet infrastructure. ::: NautilusTrader defaults to signature type 0 (EOA) but can be configured to use any of the supported signature types via the `signature_type` configuration parameter. A single wallet address is supported per trader instance when using environment variables, or multiple wallets could be configured with multiple `PolymarketExecutionClient` instances. :::note Ensure your wallet is funded with **pUSD**, otherwise you will encounter the "not enough balance or allowance" API error when submitting orders. ::: ### Setting allowances for Polymarket contracts Before you can start trading, you need to ensure that your wallet has allowances set for Polymarket's smart contracts. You can do this by running the provided script located at `nautilus_trader/adapters/polymarket/scripts/set_allowances.py`. This script is adapted from a [gist](https://gist.github.com/poly-rodr/44313920481de58d5a3f6d1f8226bd5e) created by @poly-rodr. :::note You only need to run this script **once** per EOA wallet that you intend to use for trading on Polymarket. ::: This script automates the process of approving the necessary allowances for the Polymarket contracts. It sets approvals for the pUSD collateral token and Conditional Token Framework (CTF) contract to allow the Polymarket CLOB Exchange to interact with your funds. Before running the script, ensure the following prerequisites are met: - Install the web3 Python package: `uv pip install "web3==7.12.1"`. - Have a **Polygon**-compatible wallet funded with some POL (used for gas fees). - Set the following environment variables in your shell: - `POLYGON_PRIVATE_KEY`: Your private key for the **Polygon**-compatible wallet. - `POLYGON_PUBLIC_KEY`: Your public key for the **Polygon**-compatible wallet. Once you have these in place, the script will: - Approve the maximum possible amount of pUSD (using the `MAX_INT` value) for the Polymarket collateral token contract. - Set the approval for the CTF contract, allowing it to interact with your account for trading purposes. :::note You can also adjust the approval amount in the script instead of using `MAX_INT`, with the amount specified in *fractional units* of **pUSD**, though this has not been tested. ::: Ensure that your private key and public key are correctly stored in the environment variables before running the script. Here's an example of how to set the variables in your terminal session: ```bash export POLYGON_PRIVATE_KEY="YOUR_PRIVATE_KEY" export POLYGON_PUBLIC_KEY="YOUR_PUBLIC_KEY" ``` Run the script using: ```bash python nautilus_trader/adapters/polymarket/scripts/set_allowances.py ``` ### Script breakdown The script performs the following actions: - Connects to the Polygon network via an RPC URL (). - Signs and sends a transaction to approve the maximum pUSD allowance for Polymarket contracts. - Sets approval for the CTF contract to manage Conditional Tokens on your behalf. - Repeats the approval process for specific addresses like the Polymarket CLOB Exchange and Neg Risk adapter. This allows Polymarket to interact with your funds when executing trades and ensures smooth integration with the CLOB Exchange. ## API keys To trade with Polymarket, you'll need to generate API credentials. Follow these steps: 1. Ensure the following environment variables are set: - `POLYMARKET_PK`: Your private key for signing transactions. - `POLYMARKET_FUNDER`: The wallet address (public key) on the **Polygon** network used for funding trades on Polymarket. 2. Run the script using: ```bash python nautilus_trader/adapters/polymarket/scripts/create_api_key.py ``` The script will generate and print API credentials, which you should save to the following environment variables: - `POLYMARKET_API_KEY` - `POLYMARKET_API_SECRET` - `POLYMARKET_PASSPHRASE` These can then be used for Polymarket client configurations: - `PolymarketDataClientConfig` - `PolymarketExecClientConfig` ## Configuration When setting up NautilusTrader to work with Polymarket, it’s crucial to properly configure the necessary parameters, particularly the private key. **Key parameters**: - `private_key`: The private key for your wallet used to sign orders. The interpretation depends on your `signature_type` configuration. If not explicitly provided in the configuration, it will automatically source the `POLYMARKET_PK` environment variable. - `funder`: The **pUSD** funding wallet address used for funding trades. If not provided, will source the `POLYMARKET_FUNDER` environment variable. - API credentials: You will need to provide the following API credentials to interact with the Polymarket CLOB: - `api_key`: If not provided, will source the `POLYMARKET_API_KEY` environment variable. - `api_secret`: If not provided, will source the `POLYMARKET_API_SECRET` environment variable. - `passphrase`: If not provided, will source the `POLYMARKET_PASSPHRASE` environment variable. API credentials are created from the private-key signer for L2 authentication. For `POLY_1271`, the deposit wallet remains the `funder`, but it is not the L2 auth address. - `auto_load_missing_instruments` (default `True`): Controls whether subscribe and request commands for an instrument that is not already in the cache trigger an ad-hoc load via the Gamma API. When disabled, subscribing to an uncached instrument returns an error. See [Runtime instrument loading](#runtime-instrument-loading). - `auto_load_debounce_ms` (default `100`): The window (milliseconds) over which concurrent auto-load requests are coalesced into a single batched Gamma call. :::tip We recommend using environment variables to manage your credentials. ::: ## Orders capability Polymarket operates as a prediction market with a more limited set of order types and instructions compared to traditional exchanges. ### Order types | Order Type | Binary Options | Notes | |------------------------|----------------|---------------------------------------------------------------------------| | `MARKET` | ✓ | **BUY orders require quote quantity**, SELL orders require base quantity. | | `LIMIT` | ✓ | | | `STOP_MARKET` | - | *Not supported by Polymarket*. | | `STOP_LIMIT` | - | *Not supported by Polymarket*. | | `MARKET_IF_TOUCHED` | - | *Not supported by Polymarket*. | | `LIMIT_IF_TOUCHED` | - | *Not supported by Polymarket*. | | `TRAILING_STOP_MARKET` | - | *Not supported by Polymarket*. | ### Quantity semantics Polymarket interprets order quantities differently depending on the order type *and* side: - **Limit** orders interpret `quantity` as the number of conditional tokens (base units). - **Market SELL** orders also use base-unit quantities. - **Market BUY** orders interpret `quantity` as quote notional in **pUSD**. As a result, a market buy order submitted with a base-denominated quantity will execute far more size than intended. When submitting market BUY orders, set `quote_quantity=True` on the order. The Python SDK or Rust adapter converts the quote amount (pUSD) to the signed base-unit share amount before posting to the CLOB. The Polymarket execution client denies base-denominated market buys to prevent unintended fills. ```python # Market BUY with quote quantity (spend $10 pUSD) order = strategy.order_factory.market( instrument_id=instrument_id, order_side=OrderSide.BUY, quantity=instrument.make_qty(10.0), time_in_force=TimeInForce.IOC, # Maps to Polymarket FAK quote_quantity=True, # Interpret as pUSD notional ) strategy.submit_order(order) ``` ### Execution instructions | Instruction | Binary Options | Notes | |---------------|----------------|------------------------------------------------------| | `post_only` | ✓ | Supported for limit orders with `GTC` or `GTD` only. | | `reduce_only` | - | *Not supported by Polymarket*. | ### Time-in-force options Polymarket calls the `POST /order` field `orderType`. In NautilusTrader, this maps to `TimeInForce`. The valid combinations depend on the Nautilus order type: | Nautilus TIF | Polymarket `orderType` | Nautilus order scope | Notes | |--------------|------------------------|----------------------|-------| | `GTC` | `GTC` | `LIMIT` only | Good‑Til‑Cancelled; rests on the book. | | `GTD` | `GTD` | `LIMIT` only | Good‑Til‑Date; rests until expiration, fill, or cancel. | | `FOK` | `FOK` | `LIMIT` or `MARKET` | Fill the full size immediately or cancel the whole order. | | `IOC` | `FAK` | `LIMIT` or `MARKET` | Fill available size immediately and cancel the remainder. | :::note Polymarket uses `FAK` (Fill-And-Kill) for the semantics NautilusTrader calls `IOC` (Immediate or Cancel). Polymarket docs classify `FOK` and `FAK` as market order types, while `GTC` and `GTD` are limit order types. For Nautilus `MARKET` orders, both adapters accept only `IOC` and `FOK`; `GTC` and `GTD` are valid for resting `LIMIT` orders only. ::: :::note A marketable order (any `FOK`/`FAK` order, or a `BUY` that crosses the book) must be worth at least **1 pUSD** in notional value, otherwise the venue rejects it with `invalid amount for a marketable BUY order … min size: $1`. Resting `GTC`/`GTD` limit orders are bounded only by the 5‑share minimum. ::: :::note The venue reports `GTD` expiry as an `OrderCanceled` event (not `OrderExpired`), and Polymarket applies an internal expiration buffer of roughly one minute, so a `GTD` order rests for about a minute less than the requested duration before the venue cancels it. ::: ### Advanced order features | Feature | Binary Options | Notes | |--------------------|----------------|------------------------------------| | Order modification | - | Cancellation functionality only. | | Bracket/OCO orders | - | *Not supported by Polymarket.* | | Iceberg orders | - | *Not supported by Polymarket.* | ### Batch operations | Operation | Binary Options | Notes | |--------------|----------------|---------------------------------------------------------------------------------------------------------------------------------| | Batch Submit | ✓ | Both adapters use `POST /orders` for independent limit‑order batches (max 15 orders per request). See [Batch submit](#batch-submit). | | Batch Modify | - | *Not supported by Polymarket*. | | Batch Cancel | ✓ | Both adapters use `DELETE /orders`. | #### Batch submit `SubmitOrderList` commands are routed to Polymarket's `POST /orders` endpoint. The endpoint accepts at most 15 orders per request (`BATCH_ORDER_LIMIT`); larger lists are split into sequential 15‑order chunks. - Only `LIMIT` orders are batched. `MARKET` orders inside the list are routed to the single-order path, which signs a marketable order and submits it with `FAK` or `FOK` based on Nautilus `time_in_force`. - `reduce_only` orders, `quote_quantity` orders, and `post_only` with market TIF (`IOC` or `FOK`) are rejected before submission. - A single eligible order falls through to `POST /order` so it keeps the single‑order retry semantics; the batch path deliberately disables retry because the venue does not expose an idempotency key. - `BatchCancelOrders` is dispatched to `DELETE /orders` in one shot. ### Submit error handling Polymarket's public documentation describes successful [`POST /order`](https://docs.polymarket.com/api-reference/trade/post-a-new-order) responses with `success`, `orderID`, `status`, and `errorMsg`, and documents [API errors](https://docs.polymarket.com/resources/error-codes) as structured error responses. It does not document statusless `py-clob-client` exceptions or transport failures as venue rejections. The adapter rejects only when the response proves the order was not accepted, such as `success=false`, a documented order processing error, or another non-retryable client/API error. Transport failures, timeouts, ambiguous retry exhaustion, statusless `PolyApiException`, malformed responses, and server-side failures keep the order submitted. The batch endpoint reports a rejected leg as `success=true` with an empty `orderID` and the reason in `errorMsg` (for example a naked sell the venue cannot accept): the adapter rejects that leg with the venue reason. A leg with no `orderID` and no reason stays submitted for reconciliation. When a rejection reason reports a post-only order crossing the book, the `OrderRejected` event sets `due_post_only=true` so strategies can distinguish it from other venue rejections. For unknown outcomes, both adapters derive the expected Polymarket order hash from the signed EIP-712 order when possible and cache it as the `VenueOrderId`. Later WebSocket order events (or reconciliation reports) then attach to the local `ClientOrderId` instead of becoming external orders. Quote-quantity market BUY orders still apply the signed quote-to-base quantity update on the unknown path. Cancels requested while submit outcome is unknown are deferred until the expected venue order ID is known, and fill tracking is registered under that ID. ### Position management | Feature | Binary Options | Notes | |------------------|----------------|-----------------------------------| | Query positions | ✓ | Current user positions from the Polymarket Data API. | | Position mode | - | Binary outcome positions only. | | Leverage control | - | No leverage available. | | Margin mode | - | No margin trading. | ### Order querying | Feature | Binary Options | Notes | |----------------------|----------------|--------------------------------| | Query open orders | ✓ | Active orders only. | | Query order history | ✓ | Limited historical data. | | Order status updates | ✓ | Real‑time order state changes. | | Trade history | ✓ | Execution and fill reports. | ### Contingent orders | Feature | Binary Options | Notes | |--------------------|----------------|-------------------------------------| | Order lists | - | Independent order batches exist, but linked contingency semantics do not. | | OCO orders | - | *Not supported by Polymarket*. | | Bracket orders | - | *Not supported by Polymarket*. | | Conditional orders | - | *Not supported by Polymarket*. | ### Precision limits Polymarket enforces different precision constraints based on tick size and `orderType`. **Binary Option instruments** typically support up to 6 decimal places for amounts (with 0.0001 tick size), but **market orders (`FAK` and `FOK`) have stricter precision requirements**: - **Market order types (`FAK` and `FOK`):** - Sell orders: maker amount limited to **2 decimal places**. - Taker amount: limited to **4 decimal places**. - The product `size × price` must not exceed **2 decimal places**. - **Resting limit order types (`GTC` and `GTD`):** More flexible precision based on market tick size. ### Tick size precision hierarchy | Tick Size | Price Decimals | Size Decimals | Amount Decimals | |-----------|----------------|---------------|-----------------| | 0.1 | 1 | 2 | 3 | | 0.01 | 2 | 2 | 4 | | 0.001 | 3 | 2 | 5 | | 0.0001 | 4 | 2 | 6 | :::note - The tick size precision hierarchy is defined in the [`py-clob-client-v2` `ROUNDING_CONFIG`](https://github.com/Polymarket/py-clob-client-v2/blob/main/py_clob_client_v2/order_builder/builder.py). - Market order precision limits (2 decimals for the size field, plus tick-derived bounds for the computed amount) come from the same `ROUNDING_CONFIG` and are enforced by `OrderBuilder.get_market_order_amounts` before signing. - Tick sizes can change dynamically during market conditions, particularly when markets become one-sided. ::: ### Tick size change handling When a market's tick size changes (`tick_size_change` WebSocket event), old book levels can be invalid on the new grid (for example `0.505` fits a `0.001` tick but not a `0.01` tick). To keep old-grid prices out of the new epoch, the adapter treats the change as a book epoch transition: 1. Publish the updated `BinaryOption` with the new `price_increment` and `price_precision`. 2. Drop the local order book for the instrument. 3. Mark the instrument as awaiting a fresh snapshot. 4. Drop incremental `price_change` book deltas until the snapshot arrives. 5. Reseed the book from the snapshot and resume normal processing. Trade ticks and the instrument update flow through unchanged. The Rust adapter keeps emitting `QuoteTick` events through the gap by reading `best_bid` and `best_ask` from each `price_change`. The Python adapter derives quotes from the local book, so quote subscribers see the same brief gap as the deltas (typically sub-second, until the venue snapshot arrives). ## Trades Trades on Polymarket can have the following statuses: - `MATCHED`: Trade has been matched and sent to the executor service by the operator. The executor service submits the trade as a transaction to the Exchange contract. - `MINED`: Trade is observed to be mined into the chain, and no finality threshold is established. - `CONFIRMED`: Trade has achieved strong probabilistic finality and was successful. - `RETRYING`: Trade transaction has failed (revert or reorg) and is being retried/resubmitted by the operator. - `FAILED`: Trade has failed and is not being retried. Once a trade is initially matched, subsequent trade status updates will be received via the WebSocket. NautilusTrader records the initial trade details in the `info` field of the `OrderFilled` event, with additional trade events stored in the cache as JSON under a custom key to retain this information. ### Trade ID derivation Polymarket does not publish a trade ID on `last_trade_price` market-data events. The adapter derives a deterministic `TradeId` from the asset ID, side, price, size, and timestamp via `determine_trade_id` (FNV-1a in Rust, blake2b in Python). For CLOB Data API trade history the adapter composes the `TradeId` from a hash suffix, an asset suffix, and a per-(transaction, asset) sequence number (format `{transactionHash[-24:]}-{asset[-4:]}-{seq:06d}`). A single Polygon transaction can settle multiple fills sharing the same `transactionHash`, so the older last-36-character form collapsed those fills to a single id and downstream catalogs silently dropped duplicates. The same venue event yields the same trade ID across replays, keeping downstream dedup intact. ## Fees Polymarket uses the formula `fee = C * feeRate * p * (1 - p)` where C is shares traded and p is the share price. Fees peak at p = 0.50 and decrease symmetrically toward the extremes. Only takers pay fees; makers pay zero. | Category | Taker `feeRate` | Maker `feeRate` | Maker rebate | |-----------------|-----------------|-----------------|--------------| | Crypto | 0.072 | 0 | 20% | | Sports | 0.03 | 0 | 25% | | Finance | 0.04 | 0 | 25% | | Politics | 0.04 | 0 | 25% | | Economics | 0.05 | 0 | 25% | | Culture | 0.05 | 0 | 25% | | Weather | 0.05 | 0 | 25% | | Other / General | 0.05 | 0 | 25% | | Mentions | 0.04 | 0 | 25% | | Tech | 0.04 | 0 | 25% | | Geopolitics | 0 | 0 | - | Fees are calculated in USDC, rounded to 5 decimal places, and applied at match time by the protocol. The smallest fee charged is 0.00001 USDC; smaller fees round to zero. :::note For the latest rates, see Polymarket's [Fees](https://docs.polymarket.com/trading/fees) documentation. ::: ### Backtest fee model For backtests, the adapter ships `PolymarketFeeModel` (a `nautilus_trader.backtest.models.FeeModel` subclass) which applies the taker fee formula above and credits passive maker fills with a rebate inferred from the market category. Polymarket pays a 20% maker rebate on Crypto markets and 25% on other fee-enabled categories (Sports, Finance, Politics, Economics, Culture, Weather, Tech, Mentions, Other), distributed daily from each market's rebate pool. Geopolitics markets are fee-free with no rebates and the model returns zero for them. ```python from nautilus_trader.adapters.polymarket.fee_model import PolymarketFeeModel # Default: maker rebates enabled fee_model = PolymarketFeeModel() # Or for taker-only strategies fee_model = PolymarketFeeModel(maker_rebates_enabled=False) ``` The model can also be configured through `BacktestVenueConfig.fee_model` via `ImportableFeeModelConfig` and `PolymarketFeeModelConfig`. Maker rebate share inference uses the instrument's category labels first, then falls back to the documented per-category fee rate when labels are absent. ## Reconciliation The Polymarket API returns either all **active** (open) orders or specific orders when queried by the Polymarket order ID (`venue_order_id`). The execution reconciliation procedure for Polymarket is as follows: - Generate order reports for all instruments with active (open) orders, as reported by Polymarket. - Generate position reports from current user positions reported by Polymarket's Data API. - Compare these reports with Nautilus execution state. - Generate missing orders to bring Nautilus execution state in line with positions reported by Polymarket. **Note**: Polymarket does not directly provide data for orders which are no longer active. The Python adapter exposes an experimental `generate_order_history_from_trades` option to fill some of this gap from trade history. The Rust adapter does not expose the same option today. :::warning An optional execution client configuration, `generate_order_history_from_trades`, is currently under development. It is not recommended for production use at this time. ::: ### Single-order recovery from trades `/data/order/{id}` only returns active orders, so a `Filled` or `Canceled` order returns an empty response. To avoid the engine resolving a local `ACCEPTED` order as `REJECTED` (which discards fills that already happened at the venue), `generate_order_status_report` falls back to `/data/trades` filtered by the venue order ID. The cached order is resolved via `client_order_id`, falling back to the cache's `venue_order_id` index when only the venue ID is known. Recovery is keyed on the cached order; without one the recovery defers to the engine rather than synthesizing an external order from trade history alone: - Cached order + recovered fills covering the cached quantity (within `DUST_SNAP_THRESHOLD` for CLOB cent-tick truncation): returns `Filled`. The engine reconciles any delta over the cached `filled_qty` via inferred fill. - Cached order + recovered fills that fall short of the cached quantity by more than dust: returns `Canceled` with the recovered `filled_qty`. The engine's CANCELED branch transitions the order at the cached `filled_qty`, so any newly recovered fills that arrived only via REST (not WS) are not applied in this rare partial-cancel case. Closing the order is preferred over leaving it stuck open; if exact fill metadata matters in this scenario the venue trade history can be reviewed manually. - Cached order, no trades: returns `Canceled` with `cancel_reason="ORDER_NOT_FOUND_AT_VENUE"`. - No cached order (regardless of trades): returns `None`; the engine's not-found-at-venue path resolves the local entry. `open_check_interval_secs` is recommended for Polymarket so the engine periodically drives this recovery path for orders whose terminal WS update was missed. ## Fill quantity normalization Polymarket reports fill quantities that drift slightly from the submitted order quantity due to protocol-level rounding: the CLOB rounds matched fills to integer cent ticks (underfill) and the V2 SDK truncates `takerAmount` to USDC scale on market-BUY quote-quantity orders (overfill, a few microshares). Both drift sources are fixed in absolute share terms, so the adapter normalizes them with a single threshold of `DUST_SNAP_THRESHOLD = 0.01` shares. Anything beyond that surfaces to the engine as a real partial fill or overfill. | Direction | Source | Adapter behaviour | |-----------|----------------------------------------|-------------------------------------------| | Overfill | V2 USDC‑scale truncation (microshares) | Snap fill DOWN to `submitted_qty` | | Underfill | CLOB cent‑tick truncation (≤ `0.01`) | Preserved; synthetic dust fill at MATCHED | `FillReport.commission` always reflects the venue-reported size, not the snapped quantity. The few-ulp difference is sub-microcent in pUSD. The fill tracker is keyed by `venue_order_id` and registered on order accept, so fill reports for orders placed in another session pass through unchanged. `DUST_SNAP_THRESHOLD` is not configurable per-strategy; it lives in `nautilus_polymarket::common::consts`. ## WebSockets The `PolymarketWebSocketClient` is built on top of the high-performance Nautilus `WebSocketClient` base class, written in Rust. ### Data The data adapter buffers the initial `market` subscriptions during the connection window and then subscribes dynamically as new instruments are requested. The client manages multiple WebSocket connections internally when the subscription count grows past the configured per-connection cap. ### Runtime instrument loading Polymarket lists thousands of active markets and new markets appear throughout the day, so preloading the full universe at startup is rarely practical. The data adapter auto-loads missing instruments on demand so that strategies can subscribe to markets that are not in the cache: - When a strategy issues `subscribe_quote_ticks`, `subscribe_trade_ticks`, `subscribe_order_book_deltas`, or `request_instrument` for an instrument that is not cached, the adapter registers the request and waits `auto_load_debounce_ms` (default 100 ms) so that concurrent requests coalesce. - It then issues a single batched Gamma API call. Batches larger than the Gamma `condition_ids` query ceiling (about 100) are split across multiple calls and merged. - Once the instruments are loaded, they are published to the data engine (populating the cache) and the deferred subscriptions open their WebSocket subscriptions atomically. A strategy that unsubscribes while the auto-load is in flight does not see a spurious subscription opened. The feature is enabled by default. Disable it by setting `auto_load_missing_instruments=False` on `PolymarketDataClientConfig`. To preload a known set of markets at startup instead, supply `load_ids`, `event_slugs`, `market_slugs`, or `event_slug_builder` on `PolymarketInstrumentProviderConfig`. Newly-minted markets pass through a CLOB hydration window of several minutes during which Gamma reports `active=true` but `GET /markets/{cid}` returns either a 404 or a 200 with empty `token_id` strings. Both adapters classify these as transient and retry the auto-load with bounded exponential backoff plus jitter. Tune the cadence with `auto_load_max_retries` (default 12), `auto_load_retry_delay_initial_secs` (default 5.0), and `auto_load_retry_delay_max_secs` (default 15.0); the defaults cap the retry window near 3 minutes. Set `auto_load_max_retries=0` to disable retry. 5-minute markets (e.g. updown crypto) can expire before the venue finishes hydrating, so budget for that or raise the cap. After the retry budget is exhausted, a condition still missing on Gamma is logged as a terminal miss. The Python adapter then picks it back up on the next `update_instruments_interval_mins` refresh; the Rust adapter leaves the subscription unresolved until the caller resubscribes. ### Market resolution events The Rust data client tracks Polymarket exposure at `condition_id` level so both YES and NO legs close together when the venue resolves the market. Position events add open Polymarket binary option instruments to an internal watchlist. Once a watched condition expires, the data client waits `resolve_poll_grace_secs`, then polls Gamma every `resolve_poll_interval_secs` until the condition resolves or `resolve_poll_max_wait_secs` elapses. Resolution uses strict winner inference: - Gamma must return a closed binary market with exactly two token IDs, two outcomes, and a binary `outcomePrices` shape. - If Gamma does not provide a strict result for the condition, the client falls back to CLOB `GET /markets/{condition_id}` and uses `tokens[].winner`. - Non-binary, ambiguous, malformed, or still-unresolved payloads are skipped. They remain on the watchlist until the poll window times out or a manual request resolves them. When the client applies a resolution, it emits one `InstrumentStatus` close and one `InstrumentClose` per tracked leg. The winner leg closes at `1`, and the losing leg closes at `0`. The close type is `InstrumentCloseType.ContractExpired`. This event closes Nautilus exposure and does not redeem tokens or claim funds on-chain. The same apply path handles WebSocket `market_resolved` events, automatic polling, and manual requests. After `resolve_poll_max_wait_secs`, automatic polling pauses the watched condition and logs it for manual recovery. Manual requests can still retry the condition later. #### Manual resolution requests Use `request_data()` with data type `PolymarketResolveRequest` to force a resolution check. The request accepts any of these params: | Param | Type | Description | |------------------|----------------------|-------------| | `condition_id` | `str` | Resolve one Polymarket condition. | | `condition_ids` | `str` or `list[str]` | Resolve one or more Polymarket conditions. | | `instrument_ids` | `str` or `list[str]` | Resolve Polymarket instrument IDs; other venues are ignored. | If a request omits all selectors, the client uses the watchlist. With automatic polling enabled, the fallback selects paused or timed-out entries. With automatic polling disabled, it selects all expired eligible entries, so operators can run the recovery flow manually. The response payload is custom data with this dictionary shape: | Key | Meaning | |------------------------------|---------| | `requested_condition_ids` | Deduplicated condition IDs checked by the request. | | `fetched_markets` | Gamma markets returned across the batched lookup. | | `resolved_markets` | Conditions with a strict Gamma result or successful CLOB fallback result. | | `skipped_non_binary_markets` | Gamma markets skipped for non‑binary or ambiguous resolution shape. | | `clob_fallback_successes` | Conditions resolved through the CLOB fallback path. | | `emitted_condition_ids` | Conditions that emitted at least one `InstrumentClose`. | | `failed_condition_ids` | Conditions where both Gamma and CLOB lookup failed. | | `used_watchlist_fallback` | Whether the request selected conditions from the watchlist. | | `timed_out_watchlist` | Timed‑out watchlist entries seen during fallback selection. | | `error` | First summary error, if one occurred. | Redemption is a separate account or execution workflow. Do not extend the data client resolution path to claim funds; it only publishes market-outcome close events into Nautilus. ### Purging instruments at runtime Polymarket auto-loads instruments on demand, so a long-running session keeps growing the cache as markets resolve, new markets appear, and strategies cycle through events. Use `cache.purge_instrument` to drop markets the strategy no longer tracks. The call removes the instrument record and every cache-owned map keyed by it (order book, quotes, trades, bars). ```python class PolymarketHousekeeping(Strategy): def on_position_closed(self, event: PositionClosed) -> None: # Drop the market once the position is closed and you have no further interest. instrument_id = event.instrument_id self.unsubscribe_quote_ticks(instrument_id) self.unsubscribe_order_book_deltas(instrument_id) self.cache.purge_instrument(instrument_id) ``` Common triggers on Polymarket: - A market resolves and produces no further trades. - An event ends and the strategy rotates off its markets. - The strategy rotates a fixed-size watchlist and drops the oldest entry. The purge skips any instrument that still has non-terminal orders (initialized, submitted, accepted, emulated, released, or inflight) or non-closed positions, so it is safe to call without coordinating with the execution client. Active WebSocket subscriptions belong to the data engine. Unsubscribe before purging if you no longer want updates. The cache also exposes `purge_order`, `purge_position`, `purge_closed_orders`, `purge_closed_positions`, and `purge_account_events` for trimming closed execution state. For long-running Polymarket nodes, schedule the bulk purges from `LiveExecEngineConfig` (15 min interval, 60 min buffer is a sensible default). See [Cache: purging cached data](../concepts/cache.md#purging-cached-data) for the full set. :::warning The caller decides when an instrument is no longer needed. Purging an instrument that another actor, strategy, or engine still relies on causes missing instrument lookups and loses market-data history. ::: ### Execution The execution adapter keeps a `user` channel connection for order and trade events and manages market subscriptions as needed for instruments seen during trading. Both the Python and Rust adapters support dynamic WebSocket subscribe and unsubscribe operations. ### Subscription limits Polymarket enforces a **maximum of 500 instruments per WebSocket connection** (undocumented limitation). When you attempt to subscribe to 501 or more instruments on a single WebSocket connection: - You will **not** receive the initial order book snapshot for each instrument. - You will only receive subsequent order book updates. NautilusTrader automatically manages WebSocket connections to handle this limitation: - The adapter defaults to **200 instrument subscriptions per connection** (configurable via `ws_max_subscriptions_per_connection` in the Python adapter; `ws_max_subscriptions` in the Rust adapter). - When the subscription count exceeds this limit, additional WebSocket connections are created automatically. - This ensures you receive complete order book data (including initial snapshots) for all subscribed instruments. :::tip If you need to subscribe to a large number of instruments (e.g., 5000+), the adapter will automatically distribute these subscriptions across multiple WebSocket connections. You can tune the per-connection limit up to 500 via `ws_max_subscriptions_per_connection` (Python) or `ws_max_subscriptions` (Rust). ::: ## Rate limiting Polymarket enforces rate limits via Cloudflare throttling. When limits are exceeded, requests are throttled on sliding windows. Sustained overshoot can still surface as HTTP 429 responses or temporary blocking. ### REST limits Polymarket changes these quotas over time. As of 2026-05-06, the official limits are: | Endpoint | Burst (10s) | Sustained (10 min) | Notes | |-------------------------------|-------------|--------------------|-------| | General rate limiting | 15,000 | - | Global documented rate limit. | | Health check (`/ok`) | 100 | - | Health endpoint. | | CLOB general | 9,000 | - | Aggregate across CLOB endpoints. | | CLOB `POST /order` | 3,500 | 36,000 | Single‑order submit. | | CLOB `POST /orders` | 1,000 | 15,000 | Batch submit (up to 15 orders per request). | | CLOB `DELETE /order` | 3,000 | 30,000 | Single‑order cancel. | | CLOB `DELETE /orders` | 1,000 | 15,000 | Batch cancel. | | CLOB `GET /balance-allowance` | 200 | - | Balance and allowance queries. | | CLOB API key endpoints | 100 | - | Key management. | | Gamma general | 4,000 | - | Aggregate across Gamma endpoints. | | Gamma `/markets` | 300 | - | Market metadata. | | Gamma `/events` | 500 | - | Event metadata. | | Data general | 1,000 | - | Aggregate across Data API endpoints. | | Data `/trades` | 200 | - | Trade history. | | Data `/positions` | 150 | - | Current positions. | ### WebSocket limits The WebSocket quotas are not part of the published REST rate-limits table. The adapter ships a configurable per-connection subscription cap (`ws_max_subscriptions_per_connection` in the Python adapter, `ws_max_subscriptions` in the Rust adapter) defaulting to 200; Polymarket previously documented an upper bound of 500 per connection. :::warning Exceeding Polymarket rate limits triggers Cloudflare throttling. Requests are queued using sliding windows rather than rejected immediately, but sustained overshoot can result in HTTP 429 responses or temporary blocking. ::: ### Data loader rate limiting The `PolymarketDataLoader` includes built-in rate limiting when using the default HTTP client. Requests are automatically throttled to 100 requests per minute by default. That is a NautilusTrader default, not Polymarket's current published limit. The current Rust HTTP clients also ship with conservative 100 requests per minute quotas. When fetching large date ranges across multiple markets: - Multiple loaders sharing the same `http_client` instance will coordinate rate limiting automatically. - For higher throughput, pass a custom `http_client` with adjusted quotas. - The loader does not implement automatic retry on 429 errors, so implement backoff if needed. :::info For the latest rate limit details, see the official Polymarket documentation: ::: ## Limitations and considerations The following limitations are currently known: - Python order signing via `py-clob-client-v2` is slow and can take around one second per order. - Reduce-only orders are not supported. - Batch submit (`POST /orders`) accepts at most 15 orders per request; the adapter splits larger `SubmitOrderList` commands into sequential 15-order chunks. ## Configuration The Python adapter and the Rust-native adapter expose different config surfaces. The tables below document both adapters in full. ### Data client options (Python v2) Class: `PolymarketDataClientConfig` in `nautilus_trader.adapters.polymarket.config`. | Option | Default | Description | |---------------------------------------|--------------|-------------| | `venue` | `POLYMARKET` | Venue identifier registered for the data client. | | `private_key` | `None` | Wallet private key; sourced from `POLYMARKET_PK` when omitted. | | `signature_type` | `0` | Signature scheme (0 = EOA, 1 = email proxy, 2 = browser wallet proxy). | | `funder` | `None` | pUSD funding wallet; sourced from `POLYMARKET_FUNDER` when omitted. | | `api_key` | `None` | API key; sourced from `POLYMARKET_API_KEY` when omitted. | | `api_secret` | `None` | API secret; sourced from `POLYMARKET_API_SECRET` when omitted. | | `passphrase` | `None` | API passphrase; sourced from `POLYMARKET_PASSPHRASE` when omitted. | | `base_url_http` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `ws_connection_initial_delay_secs` | `5` | Delay (seconds) before the first WebSocket connection to buffer subscriptions. | | `ws_connection_delay_secs` | `0.1` | Delay (seconds) between subsequent WebSocket connection attempts. | | `ws_max_subscriptions_per_connection` | `200` | Maximum instrument subscriptions per WebSocket connection (Polymarket limit is 500). | | `update_instruments_interval_mins` | `60` | Interval (minutes) between instrument catalogue refreshes. | | `compute_effective_deltas` | `False` | Compute effective order book deltas for bandwidth savings. | | `drop_quotes_missing_side` | `True` | Drop quotes with missing bid/ask prices instead of substituting boundary values. | | `auto_load_missing_instruments` | `True` | Load instruments on demand when subscribe or request commands reference uncached instruments. | | `auto_load_debounce_ms` | `100` | Debounce window (milliseconds) for coalescing concurrent runtime instrument loads. | | `auto_load_max_retries` | `12` | Maximum retry attempts on transient auto‑load failures (404 or empty `token_id` during CLOB hydration). Set to `0` to disable. | | `auto_load_retry_delay_initial_secs` | `5.0` | Initial delay (seconds) between transient auto‑load retries. | | `auto_load_retry_delay_max_secs` | `15.0` | Maximum delay (seconds) between transient auto‑load retries. | | `instrument_config` | `None` | Optional `PolymarketInstrumentProviderConfig` for instrument loading. | ### Execution client options (Python v2) Class: `PolymarketExecClientConfig` in `nautilus_trader.adapters.polymarket.config`. | Option | Default | Description | |---------------------------------------|--------------|-------------| | `venue` | `POLYMARKET` | Venue identifier registered for the execution client. | | `private_key` | `None` | Wallet private key; sourced from `POLYMARKET_PK` when omitted. | | `signature_type` | `0` | Signature scheme (0 = EOA, 1 = email proxy, 2 = browser wallet proxy). | | `funder` | `None` | pUSD funding wallet; sourced from `POLYMARKET_FUNDER` when omitted. | | `api_key` | `None` | API key; sourced from `POLYMARKET_API_KEY` when omitted. | | `api_secret` | `None` | API secret; sourced from `POLYMARKET_API_SECRET` when omitted. | | `passphrase` | `None` | API passphrase; sourced from `POLYMARKET_PASSPHRASE` when omitted. | | `base_url_http` | `None` | Override for the REST base URL. | | `base_url_ws` | `None` | Override for the WebSocket base URL. | | `base_url_data_api` | `None` | Override for the Data API base URL (default `https://data-api.polymarket.com`). | | `proxy_url` | `None` | Optional proxy URL for HTTP and WebSocket transports. | | `ws_max_subscriptions_per_connection` | `200` | Maximum instrument subscriptions per WebSocket connection (Polymarket limit is 500). | | `max_retries` | `None` | Maximum retry attempts for submit/cancel requests. | | `retry_delay_initial_ms` | `None` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `None` | Maximum delay (milliseconds) between retries. | | `ack_timeout_secs` | `5.0` | Timeout (seconds) to wait for order/trade acknowledgment from cache. | | `generate_order_history_from_trades` | `False` | Generate synthetic order history from trade reports when `True` (experimental). | | `log_raw_ws_messages` | `False` | Log raw WebSocket payloads at INFO level when `True`. | | `instrument_config` | `None` | Optional `PolymarketInstrumentProviderConfig` for instrument loading. | ### Data client options (Rust v2) Struct: `PolymarketDataClientConfig` in `crates/adapters/polymarket/src/config.rs`. | Option | Default | Description | |--------------------------------------|--------------------------------------------|-------------| | `base_url_http` | `None` (official CLOB endpoint) | Override for the CLOB REST base URL. | | `base_url_ws` | `None` (official CLOB endpoint) | Override for the CLOB WebSocket base URL. | | `base_url_rtds` | `None` (official RTDS endpoint) | Override for the real‑time data service (RTDS) base URL. | | `base_url_gamma` | `None` (official Gamma endpoint) | Override for the Gamma API base URL. | | `base_url_data_api` | `None` (`https://data-api.polymarket.com`) | Override for the Data API base URL. | | `http_timeout_secs` | `60` | HTTP request timeout (seconds). | | `ws_timeout_secs` | `30` | WebSocket connect/idle timeout (seconds). | | `ws_max_subscriptions` | `200` | Maximum instrument subscriptions per WebSocket connection. | | `update_instruments_interval_mins` | `60` | Interval (minutes) between instrument catalogue refreshes. | | `subscribe_new_markets` | `false` | Subscribe to new‑market discovery events via WebSocket when `true`. | | `new_market_fetch_max_concurrency` | `8` | Maximum concurrent instrument fetches spawned from new‑market discovery events. | | `auto_load_missing_instruments` | `true` | Load instruments on demand when subscribe or request commands reference uncached instruments. | | `auto_load_debounce_ms` | `100` | Debounce window (milliseconds) for coalescing concurrent runtime instrument loads. | | `auto_load_max_retries` | `12` | Maximum retry attempts on transient auto‑load failures (markets in the CLOB hydration window). Set to `0` to disable. | | `auto_load_retry_delay_initial_secs` | `5.0` | Initial delay (seconds) between transient auto‑load retries. | | `auto_load_retry_delay_max_secs` | `15.0` | Maximum delay (seconds) between transient auto‑load retries. | | `resolve_poll_enabled` | `true` | Automatically poll expired watched conditions for market resolution. | | `resolve_poll_interval_secs` | `30` | Interval (seconds) between automatic resolution polling attempts. | | `resolve_poll_grace_secs` | `10` | Delay (seconds) after expiry before the first automatic resolution poll. | | `resolve_poll_max_wait_secs` | `1800` | Maximum wait (seconds) after expiry before automatic polling pauses a watched condition for manual recovery. | | `instrument_config` | `None` | Optional `PolymarketInstrumentProviderConfig` controlling bootstrap loading (`load_ids`, `event_slugs`, `market_slugs`, `event_slug_builder`). | | `filters` | `[]` | Instrument filters applied during loading and discovery. | | `new_market_filter` | `None` | Optional filter applied to newly discovered markets before emission. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | The Rust data client config does not accept account credentials; authentication is handled by the execution client. Subscription buffering (`ws_connection_initial_delay_secs`) and quote handling (`compute_effective_deltas`, `drop_quotes_missing_side`) are Python-only today. ### Execution client options (Rust v2) Struct: `PolymarketExecClientConfig` in `crates/adapters/polymarket/src/config.rs`. | Option | Default | Description | |--------------------------|--------------------------------------------|-------------| | `trader_id` | default `TraderId` | Trader identifier the client registers under. | | `account_id` | `POLYMARKET-001` | Account identifier for this execution client. | | `private_key` | `None` (`POLYMARKET_PK` env) | Wallet private key for EIP-712 signing. | | `api_key` | `None` (`POLYMARKET_API_KEY` env) | CLOB API key (L2 auth). | | `api_secret` | `None` (`POLYMARKET_API_SECRET` env) | CLOB API secret (L2 auth). | | `passphrase` | `None` (`POLYMARKET_PASSPHRASE` env) | CLOB API passphrase (L2 auth). | | `funder` | `None` (`POLYMARKET_FUNDER` env) | pUSD funding wallet; for `Poly1271`, this is the deposit wallet. | | `signature_type` | `Eoa` | Signature scheme (`Eoa`, `PolyProxy`, `PolyGnosisSafe`, `Poly1271`). | | `base_url_http` | `None` (official CLOB endpoint) | Override for the CLOB REST base URL. | | `base_url_ws` | `None` (official CLOB endpoint) | Override for the CLOB WebSocket base URL. | | `base_url_data_api` | `None` (`https://data-api.polymarket.com`) | Override for the Data API base URL. | | `http_timeout_secs` | `60` | HTTP request timeout (seconds). | | `max_retries` | `3` | Maximum retry attempts for single‑order submit/cancel requests. | | `retry_delay_initial_ms` | `1000` | Initial delay (milliseconds) between retries. | | `retry_delay_max_ms` | `10000` | Maximum delay (milliseconds) between retries. | | `ack_timeout_secs` | `5` | Timeout (seconds) waiting for WebSocket order/trade acknowledgment. | | `transport_backend` | `Sockudo` | WebSocket transport backend. | The Rust execution client does not expose `generate_order_history_from_trades`, `log_raw_ws_messages`, `ws_max_subscriptions_per_connection`, or `instrument_config`. Batch submissions via `POST /orders` deliberately skip retry regardless of `max_retries`; the single-order path still retries on transient failures. ### Instrument provider configuration options The instrument provider config is passed via the `instrument_config` parameter on the data client config. | Option | Default | Description | |----------------------|---------|---------------------------------------------------------------------------------------------| | `load_all` | `False` | Load all venue instruments on start. Auto‑set to `True` when slug scopes are provided. | | `event_slugs` | `None` | Static event slugs to resolve through Gamma events. | | `market_slugs` | `None` | Static market slugs to load directly through Gamma markets. | | `event_slug_builder` | `None` | Rust‑backed `PolymarketUpDownEventSlugConfig` for predictable Up/Down event slug windows. | #### Event slug builder The Rust Python v2 adapter treats Python as a configuration, factory, and user strategy boundary. Provider, data, and execution operations run in Rust. `event_slug_builder` therefore accepts a Rust-backed `PolymarketUpDownEventSlugConfig`; it does not accept Python callable paths. Use this for predictable Polymarket Up/Down event slugs without downloading the full venue catalogue. The builder emits slugs with the pattern `{asset}-updown-{interval_mins}m-{unix_timestamp}` for the configured window of aligned periods. ```python from nautilus_trader.adapters.polymarket import PolymarketInstrumentProviderConfig from nautilus_trader.adapters.polymarket import PolymarketUpDownEventSlugConfig instrument_config = PolymarketInstrumentProviderConfig( event_slug_builder=PolymarketUpDownEventSlugConfig( assets=["btc"], interval_mins=5, periods=3, start_offset_periods=0, ), ) ``` For custom event patterns, pass explicit `event_slugs`, pass direct `market_slugs`, or add a Rust filter or builder. The Rust v2 adapter rejects Python callable `event_slug_builder` values so adapter operations do not cross into Python during live trading. ## Historical data loading The `PolymarketDataLoader` provides methods for fetching and parsing historical market data for research and backtesting purposes. The loader integrates with multiple Polymarket APIs to provide the required data. :::note All data fetching methods are **asynchronous** and must be called with `await`. The loader can optionally accept an `http_client` parameter for dependency injection (useful for testing). ::: ### Data sources The loader fetches data from three primary sources: 1. **Polymarket Gamma API** - Market metadata, instrument details, and active market listings. 2. **Polymarket CLOB API** - Market details for instrument construction. 3. **Polymarket Data API** - Historical trades and current user positions. The current loader does **not** expose helpers for CLOB price history timeseries or order book history snapshots. ### Method naming conventions The loader provides two ways to access the Polymarket APIs: | Prefix | Type | Use case | |-----------|------------------|------------------------------------------------------------------------| | `query_*` | Static methods | API exploration without an instrument. No loader instance needed. | | `fetch_*` | Instance methods | Data fetching with a configured loader. Uses the loader's HTTP client. | **Use `query_*` when** you want to explore markets, discover events, or fetch metadata before committing to a specific instrument: ```python # No loader needed: query the API directly market = await PolymarketDataLoader.query_market_by_slug("some-market") event = await PolymarketDataLoader.query_event_by_slug("some-event") ``` **Use `fetch_*` when** you have a loader instance and want to fetch data using its configured HTTP client (for coordinated rate limiting across multiple calls): ```python loader = await PolymarketDataLoader.from_market_slug("some-market") # All fetch calls share the loader's HTTP client markets = await loader.fetch_markets(active=True, limit=100) events = await loader.fetch_events(active=True) details = await loader.fetch_market_details(condition_id) ``` ### Finding markets Use the provided utility scripts to discover active markets: ```bash # List all active markets python nautilus_trader/adapters/polymarket/scripts/active_markets.py # List BTC and ETH UpDown markets specifically python nautilus_trader/adapters/polymarket/scripts/list_updown_markets.py ``` ### Basic usage The recommended way to create a loader is using the factory classmethods, which handle all the API calls and instrument creation automatically: ```python import asyncio from nautilus_trader.adapters.polymarket import PolymarketDataLoader async def main(): # Create loader from market slug (recommended) loader = await PolymarketDataLoader.from_market_slug("gta-vi-released-before-june-2026") # Loader is ready to use with instrument and token_id set print(loader.instrument) print(loader.token_id) asyncio.run(main()) ``` For events with multiple markets (e.g., temperature buckets), use `from_event_slug`: ```python # Returns a list of loaders, one per market in the event loaders = await PolymarketDataLoader.from_event_slug("highest-temperature-in-nyc-on-january-26") ``` #### Look-ahead protection for resolved markets When constructing a loader for a market that has already resolved at backtest build time, the venue payload includes the answer (`closed`, `closedTime`, `umaResolutionStatus`, per-token `winner`). A strategy that reads `cache.instrument(...).info` from `on_start` can therefore see the outcome before the simulation runs. Pass `sanitize_info=True` to either factory to redact those fields from `instrument.info` before the instrument is constructed. The redacted slice is stashed on the loader as `resolution_metadata` for post-hoc analytics (settlement PnL, Brier scoring) without leaking it into the simulation: ```python loader = await PolymarketDataLoader.from_market_slug( "some-resolved-market", sanitize_info=True, ) assert "closed" not in loader.instrument.info assert loader.resolution_metadata["closed"] is True ``` ### Discovering markets and events Use `fetch_markets()` and `fetch_events()` to discover available markets programmatically: ```python loader = await PolymarketDataLoader.from_market_slug("any-market") # List active markets markets = await loader.fetch_markets(active=True, closed=False, limit=100) for market in markets: print(f"{market['slug']}: {market['question']}") # List active events events = await loader.fetch_events(active=True, limit=50) for event in events: print(f"{event['slug']}: {event['title']}") # Get all markets within a specific event event_markets = await loader.get_event_markets("highest-temperature-in-nyc-on-january-26") ``` For quick exploration without creating a loader, use the static `query_*` methods (see [Method naming conventions](#method-naming-conventions) above). ### Fetching trade history The `load_trades()` convenience method fetches and parses historical trades in one step: ```python import pandas as pd # Load all available trades trades = await loader.load_trades() # Or filter by time range (client-side filtering) end = pd.Timestamp.now(tz="UTC") start = end - pd.Timedelta(hours=24) trades = await loader.load_trades( start=start, end=end, ) ``` Alternatively, you can fetch and parse separately using the lower-level methods: ```python condition_id = loader.condition_id # Fetch raw trades from the Polymarket Data API raw_trades = await loader.fetch_trades(condition_id=condition_id) # Parse to NautilusTrader TradeTicks trades = loader.parse_trades(raw_trades) ``` Trade data is sourced from the [Polymarket Data API](https://data-api.polymarket.com/trades), which provides real execution data including price, size, side, and on-chain transaction hash. :::note The public Data API caps offset-based pagination on high-activity markets. When this ceiling is hit the loader emits a `RuntimeWarning` and returns the trades fetched up to the cap rather than aborting the load. Use another historical data source if you need full coverage of a heavily traded market. ::: ### Complete backtest example See `examples/backtest/polymarket_simple_quoter.py` for a full example: ```python import asyncio from decimal import Decimal from nautilus_trader.adapters.polymarket import POLYMARKET_VENUE from nautilus_trader.adapters.polymarket import PolymarketDataLoader from nautilus_trader.backtest.config import BacktestEngineConfig from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.examples.strategies.ema_cross_long_only import EMACrossLongOnly from nautilus_trader.examples.strategies.ema_cross_long_only import EMACrossLongOnlyConfig from nautilus_trader.model.currencies import pUSD from nautilus_trader.model.data import BarType from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.model.identifiers import TraderId from nautilus_trader.model.objects import Money async def run_backtest(): # Initialize loader and fetch market data loader = await PolymarketDataLoader.from_market_slug("gta-vi-released-before-june-2026") instrument = loader.instrument # Load historical trades from the Polymarket Data API trades = await loader.load_trades() # Configure and run backtest config = BacktestEngineConfig(trader_id=TraderId("BACKTESTER-001")) engine = BacktestEngine(config=config) engine.add_venue( venue=POLYMARKET_VENUE, oms_type=OmsType.NETTING, account_type=AccountType.CASH, base_currency=pUSD, starting_balances=[Money(10_000, pUSD)], ) engine.add_instrument(instrument) engine.add_data(trades) bar_type = BarType.from_str(f"{instrument.id}-100-TICK-LAST-INTERNAL") strategy_config = EMACrossLongOnlyConfig( instrument_id=instrument.id, bar_type=bar_type, trade_size=Decimal("20"), ) strategy = EMACrossLongOnly(config=strategy_config) engine.add_strategy(strategy=strategy) engine.run() # Display results print(engine.trader.generate_account_report(POLYMARKET_VENUE)) # Run the backtest asyncio.run(run_backtest()) ``` **Run the complete example**: ```bash python examples/backtest/polymarket_simple_quoter.py ``` ### Helper functions The adapter provides utility functions for working with Polymarket identifiers: ```python from nautilus_trader.adapters.polymarket import get_polymarket_instrument_id # Create NautilusTrader InstrumentId from Polymarket identifiers instrument_id = get_polymarket_instrument_id( condition_id="0xcccb7e7613a087c132b69cbf3a02bece3fdcb824c1da54ae79acc8d4a562d902", token_id="8441400852834915183759801017793514978104486628517653995211751018945988243154" ) ``` ## Contributing :::info For additional features or to contribute to the Polymarket adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Tardis Source: https://nautilustrader.io/docs/latest/integrations/tardis/ Tardis provides granular data for cryptocurrency markets including tick-by-tick order book snapshots and updates, trades, open interest, funding rates, option summaries, and liquidations data for leading crypto exchanges. NautilusTrader integrates with the Tardis API, Tardis Machine WebSocket server, and Tardis CSV formats. The capabilities of this adapter include: - `TardisCSVDataLoader`: reads Tardis-format CSV files into Nautilus data, with bulk and memory-efficient streaming paths. - `TardisMachineClient`: streams live or historical replay data from Tardis Machine and converts messages into Nautilus data. - `TardisHttpClient`: requests instrument metadata from the Tardis HTTP API and parses it into Nautilus instrument definitions. - `TardisDataClient`: provides a live data client for Tardis Machine streams. - `TardisInstrumentProvider`: loads instrument definitions from the Tardis metadata API. - **Data pipeline functions**: replay historical data from Tardis Machine and write Nautilus Parquet catalog files. :::info A `TARDIS_API_KEY` is required for Nautilus instrument metadata calls. Tardis Machine uses `TM_API_KEY` for historical dates outside the free first day of each month. See also [environment variables](#environment-variables). ::: ## Overview This adapter is implemented in Rust, with optional Python bindings. It does not require any external Tardis client library dependencies. :::info There is **no** need for additional installation steps for `tardis`. The core components of the adapter are compiled as static libraries and linked during the build. ::: ## Tardis documentation Tardis provides extensive user [documentation](https://docs.tardis.dev/). We recommend also referring to the Tardis documentation in conjunction with this NautilusTrader integration guide. ## Supported formats Tardis provides *normalized* market data, a unified format consistent across supported exchanges. This normalization lets one parser handle data from any [Tardis-supported exchange](#venues). NautilusTrader does not support exchange-native Tardis market data formats in this adapter. The following normalized Tardis Machine formats are supported by NautilusTrader. See the official [Tardis data type reference](https://docs.tardis.dev/tardis-machine/data-types) for field schemas. | Tardis format | Nautilus data type | |:--------------------|:------------------------------------------------------------------| | `book_change` | `OrderBookDelta` | | `book_snapshot_*` | `OrderBookDepth10` or `OrderBookDeltas` | | `quote` | `QuoteTick` | | `quote_10s` | `QuoteTick` | | `trade` | `Trade` | | `trade_bar_*` | `Bar` | | `instrument` | `CurrencyPair`, `CryptoFuture`, `CryptoPerpetual`, `CryptoOption` | | `derivative_ticker` | `FundingRateUpdate` | | `option_summary` | `OptionGreeks`; optional `QuoteTick` from BBO fields | | `disconnect` | *Not applicable* | **Notes:** - Tardis documents `quote` as an alias for `book_snapshot_1_0ms`. - Tardis documents `quote_10s` as an alias for `book_snapshot_1_10s`. - `quote`, `quote_10s`, and one-level snapshots are parsed as `QuoteTick`. - The Rust data client also emits mark and index price updates from `derivative_ticker` messages when those values change. - Tardis `option_summary` messages include best bid/offer fields. Nautilus always maps this feed to `OptionGreeks`; set `extract_bbo_as_quotes` to `true` to also emit `QuoteTick` from those BBO fields. :::info See also the Tardis [Tardis Machine quickstart](https://docs.tardis.dev/tardis-machine/quickstart). ::: ## Bars The adapter converts Tardis trade bar intervals and suffixes to Nautilus `BarType`s. This includes the following: | Tardis suffix | Meaning | Nautilus bar aggregation | |:--------------|:----------------|:-------------------------| | `ms` | Milliseconds | `MILLISECOND` | | `s` | Seconds | `SECOND` | | `m` | Minutes | `MINUTE` | | `ticks` | Number of ticks | `TICK` | | `vol` | Volume size | `VOLUME` | ## Symbology and normalization The Tardis integration ensures compatibility with NautilusTrader's crypto exchange adapters by consistently normalizing symbols. Typically, NautilusTrader uses the native exchange naming conventions provided by Tardis. For certain exchanges, raw symbols are adjusted to adhere to Nautilus symbology normalization, as outlined below: ### Common rules - All symbols are converted to uppercase. - Market type suffixes are appended with a hyphen for some exchanges. - Original exchange symbols are preserved in the Nautilus instrument definitions `raw_symbol` field. ### Exchange-specific normalizations - **Binance**: Nautilus appends the suffix `-PERP` to all perpetual symbols. - **Bybit**: Nautilus uses product category suffixes, including `-SPOT`, `-LINEAR`, `-INVERSE`, and `-OPTION`. - **dYdX**: Nautilus appends the suffix `-PERP` to all perpetual symbols. - **Gate.io**: Nautilus appends the suffix `-PERP` to all perpetual symbols. For detailed symbology documentation per exchange: - [Binance symbology](./binance.md#symbology) - [Bybit symbology](./bybit.md#symbology) - [dYdX symbology](./dydx.md#symbology) ## Venues Some exchanges on Tardis are partitioned into multiple venues. The table below outlines the mappings between Nautilus venues and corresponding Tardis exchanges: | Nautilus venue | Tardis exchange(s) | |:------------------------|:------------------------------------------------------| | `ASCENDEX` | `ascendex` | | `BINANCE` | `binance`, `binance-dex`, `binance-futures`, `binance-options` | | `BINANCE_DELIVERY` | `binance-delivery` (*COIN‑margined contracts*) | | `BINANCE_US` | `binance-us` | | `BITFINEX` | `bitfinex`, `bitfinex-derivatives` | | `BITFLYER` | `bitflyer` | | `BITGET` | `bitget`, `bitget-futures` | | `BITMEX` | `bitmex` | | `BITNOMIAL` | `bitnomial` | | `BITSTAMP` | `bitstamp` | | `BLOCKCHAIN_COM` | `blockchain-com` | | `BYBIT` | `bybit`, `bybit-options`, `bybit-spot` | | `COINBASE` | `coinbase` | | `COINBASE_INTX` | `coinbase-international` | | `COINFLEX` | `coinflex` (*for historical research*) | | `CRYPTO_COM` | `crypto-com` | | `CRYPTOFACILITIES` | `cryptofacilities` | | `DELTA` | `delta` | | `DERIBIT` | `deribit` | | `DYDX` | `dydx` | | `DYDX_V4` | `dydx-v4` | | `FTX` | `ftx`, `ftx-us` (*historical research*) | | `GATE_IO` | `gate-io`, `gate-io-futures` | | `GEMINI` | `gemini` | | `HITBTC` | `hitbtc` | | `HUOBI` | `huobi`, `huobi-dm`, `huobi-dm-linear-swap`, `huobi-dm-options` | | `HUOBI_DELIVERY` | `huobi-dm-swap` | | `HYPERLIQUID` | `hyperliquid` | | `KRAKEN` | `kraken` | | `KUCOIN` | `kucoin`, `kucoin-futures` | | `MANGO` | `mango` | | `OKCOIN` | `okcoin` | | `OKEX` | `okex`, `okex-futures`, `okex-options`, `okex-spreads`, `okex-swap` | | `PHEMEX` | `phemex` | | `POLONIEX` | `poloniex` | | `SERUM` | `serum` (*historical research*) | | `STAR_ATLAS` | `star-atlas` | | `UPBIT` | `upbit` | | `WOO_X` | `woo-x` | Tardis also exposes legacy Binance exchanges such as `binance-european-options` and `binance-jersey`. ## Environment variables The following environment variables are used by Tardis and NautilusTrader. - `TM_API_KEY`: API key for the Tardis Machine. - `TARDIS_API_KEY`: API key for NautilusTrader Tardis clients. - `TARDIS_MACHINE_WS_URL` (optional): WebSocket URL for the `TardisMachineClient`. - `TARDIS_BASE_URL` (optional): Base URL for the `TardisHttpClient` in NautilusTrader. - `NAUTILUS_PATH` (optional): Parent directory containing the `catalog/` subdirectory for replay output. The Tardis instruments metadata API requires bearer-token authorization and is available to active pro and business Tardis subscriptions. ## Running Tardis Machine historical replays The [Tardis Machine Server](https://docs.tardis.dev/tardis-machine/quickstart) is a locally runnable server with built-in data caching. It provides tick-level historical and consolidated real-time cryptocurrency market data through HTTP and WebSocket APIs. You can perform complete Tardis Machine WebSocket replays of historical data and output the results in Nautilus Parquet format, using either Python or Rust. Since the function is implemented in Rust, performance is consistent whether run from Python or Rust. The end-to-end `run_tardis_machine_replay` data pipeline function uses a specified [configuration](#configuration) to execute the following steps: - Connect to the Tardis Machine server. - Request and parse all necessary instrument definitions from the Tardis instruments metadata API. - Stream all requested instruments and data types for the specified time ranges from Tardis Machine. - For each instrument, data type and date (UTC), generate a catalog-compatible `.parquet` file. - Disconnect from the Tardis Machine server, and terminate the program. **File naming convention** Files are written one per day, per instrument, using ISO 8601 timestamp ranges: - **Format**: `{start_timestamp}_{end_timestamp}.parquet` - **Example**: `2023-10-01T00-00-00-000000000Z_2023-10-01T23-59-59-999999999Z.parquet` - **Structure**: `data/{data_type}/{instrument_id}/{filename}` This format is compatible with Nautilus data catalog queries, consolidation, and management. :::note You can request data for the first day of each month without a Tardis Machine API key. Other dates require `TM_API_KEY`. ::: This process is optimized for direct output to a Nautilus Parquet data catalog. Set `NAUTILUS_PATH` to the parent directory that contains the `catalog/` subdirectory. Parquet files are written under `/catalog/data/` in subdirectories by data type and instrument. If no `output_path` is specified and `NAUTILUS_PATH` is unset, output defaults to the current working directory. ### Procedure First, ensure the `tardis-machine` docker container is running. Use the following command: ```bash docker run -p 8000:8000 -p 8001:8001 -e "TM_API_KEY=YOUR_API_KEY" -d tardisdev/tardis-machine ``` This command starts the `tardis-machine` server without a persistent local cache, which may affect performance. For better replay performance, run it with a persistent volume. ### Configuration Next, ensure you have a configuration JSON file available. **Configuration JSON fields** - `tardis_ws_url` (`str | null`): Tardis Machine WebSocket URL. Defaults to `TARDIS_MACHINE_WS_URL`. - `normalize_symbols` (`bool | null`): applies Nautilus symbol normalization. Defaults to `true`. - `output_path` (`str | null`): output directory for Parquet data. Defaults to `NAUTILUS_PATH`, then the current working directory. - `book_snapshot_output` (`"deltas" | "depth10" | null`): output format for snapshots. Defaults to `"deltas"`. - `extract_bbo_as_quotes` (`bool | null`): also writes `QuoteTick` data from best bid/offer fields in Tardis Machine `option_summary` messages. Defaults to `false`. - `compression` (`"zstd" | "snappy" | "uncompressed" | null`): Parquet compression codec. Defaults to `"zstd"` level 3. - `proxy_url` (`str | null`): proxy URL for Tardis HTTP requests. Defaults to no proxy. - `options` (`JSON[]`): required replay request option objects. An example configuration file is available at `crates/adapters/tardis/bin/example_config.json`: ```json { "tardis_ws_url": "ws://localhost:8001", "output_path": null, "options": [ { "exchange": "bitmex", "symbols": [ "xbtusd", "ethusd" ], "data_types": [ "trade" ], "from": "2019-10-01", "to": "2019-10-02" } ] } ``` ### Book snapshot output The `book_snapshot_output` configuration option controls how Tardis `book_snapshot_*` messages are converted and stored. | Value | Nautilus type | Output directory | Description | |:----------|:-------------------|:---------------------|:--------------------------------------| | `deltas` | `OrderBookDeltas` | `order_book_deltas/` | Price level updates. | | `depth10` | `OrderBookDepth10` | `order_book_depths/` | Snapshots with up to 10 price levels. | **When to use each format:** - **`deltas` (default)**: use when you need to reconstruct book state or combine snapshots with `book_change` data. Each price level becomes a separate delta record. - **`depth10`**: use when a strategy needs periodic depth snapshots. Each snapshot is a single record, and snapshots with more than 10 levels keep only the first 10. **Avoiding file overwrites:** When downloading both `book_snapshot_*` and `book_change` data for the same instrument and date range, `depth10` writes snapshots to `order_book_depths/` and avoids overwriting `order_book_deltas/`. Example configuration with explicit format: ```json { "tardis_ws_url": "ws://localhost:8001", "book_snapshot_output": "depth10", "options": [ { "exchange": "binance-futures", "symbols": ["btcusdt"], "data_types": ["book_snapshot_5_100ms", "book_change"], "from": "2024-01-01", "to": "2024-01-02" } ] } ``` ### Option summary BBO extraction Set `extract_bbo_as_quotes` to `true` when requesting Tardis Machine `option_summary` data and the backtest also needs option BBO quotes. Nautilus still writes `OptionGreeks` from every `option_summary` message. When all best bid/offer fields are present and sizes are valid, it also writes a `QuoteTick` for the same instrument and timestamps. This option only applies to Tardis Machine `option_summary` replay and stream messages. It does not change Tardis CSV loading. ```json { "tardis_ws_url": "ws://localhost:8001", "extract_bbo_as_quotes": true, "options": [ { "exchange": "deribit", "symbols": ["BTC-28JUN24-70000-C"], "data_types": ["option_summary"], "from": "2024-01-01", "to": "2024-01-02" } ] } ``` ### Python replays To run a replay in Python, create a script similar to the following: ```python import asyncio from pathlib import Path from nautilus_trader.core import nautilus_pyo3 async def run(): config_filepath = Path("YOUR_CONFIG_FILEPATH") await nautilus_pyo3.run_tardis_machine_replay(str(config_filepath.resolve())) if __name__ == "__main__": asyncio.run(run()) ``` ### Rust replays To run a replay in Rust, create a binary similar to the following: ```rust use std::path::PathBuf; use nautilus_adapters::tardis::replay::run_tardis_machine_replay_from_config; #[tokio::main] async fn main() { nautilus_common::logging::ensure_logging_initialized(); let config_filepath = PathBuf::from("YOUR_CONFIG_FILEPATH"); run_tardis_machine_replay_from_config(&config_filepath).await; } ``` Logging defaults to INFO level. To enable debug logging, export the following environment variable: ```bash export NAUTILUS_LOG=debug ``` A working example binary is available at `crates/adapters/tardis/bin/example_replay.rs`. This can also be run using cargo: ```bash cargo run --bin tardis-replay ``` ### Option-chain backtest catalog An option-chain backtest starts after the Tardis replay has written data to the Nautilus catalog. The backtest loader does not request missing Tardis data during a run, so the catalog must contain: - Option instruments from the Tardis instrument metadata API. - `QuoteTick` data from one-level option book snapshots, quote data, or `option_summary` BBO extraction. - `OptionGreeks` data from Tardis `option_summary` messages. Use both `QuoteTick` and `OptionGreeks` in the `BacktestDataConfig` list for the same option instrument IDs. The option-chain manager aggregates the replayed BBO and Greeks into `OptionChainSlice` snapshots. Use `snapshot_interval_ms=None` for raw publishing, or set an interval in milliseconds to publish thinned snapshots. Strategies can select contracts by moneyness with ATM-relative or ATM-percent strike ranges, by delta with `StrikeRange.delta(target, tolerance)`, or by fixed strike with `StrikeRange.fixed([...])`. Option order matching in backtests is quote-driven: marketable orders fill as takers against the opposing BBO, while passive limits can fill as makers when later BBO updates trade through the limit. Configure option fees explicitly on the simulated venue with structural fee models such as `CappedOptionFeeModel` or `TieredNotionalOptionFeeModel`. There is no automatic Tardis exchange to fee model mapping. ### Option-chain CSV catalog conversion For historical option chains from downloadable Tardis CSV files, use `TardisCSVDataLoader.convert_options_chain_csv(...)` to convert `options_chain` rows into Nautilus catalog data. This path does not call Tardis Machine or the instrument metadata API, so it is useful when you already have Tardis CSV files or want a no-API-key catalog bootstrap from downloaded data. The converter writes `OptionGreeks` for every selected row. With the default `extract_bbo_as_quotes=True`, complete best bid/offer rows also write `QuoteTick`. Keep this enabled for option-chain backtests: greeks-only catalogs do not provide quotes, so the chain manager cannot publish populated `OptionChainSlice` snapshots for strikes without BBO data. Instrument derivation currently supports Deribit options. For other option venues, set `write_instruments=False` before conversion and load the instruments through another source before backtesting. Leaving it enabled for a non-Deribit file can fail after data files have been written to the catalog. Pass daily `options_chain` CSV paths in chronological order. The `underlyings` filter matches symbol prefixes such as `["BTC-"]`. Set `snapshot_interval_ms` to keep the last row per instrument per interval within each input file, or use `None` to write every selected row. Rows must be ordered by `local_timestamp` within each file when thinning. Provide explicit `price_precision` and `size_precision` on the loader for deterministic quote metadata. Inferred precision can increase as later rows are read, so data written earlier in a file can keep lower precision metadata. ```python from pathlib import Path from nautilus_trader.adapters.tardis import TardisCSVDataLoader loader = TardisCSVDataLoader( price_precision=4, size_precision=1, ) loader.convert_options_chain_csv( filepaths=[Path("deribit_options_chain_2020-06-08.csv")], catalog_path=Path("catalog"), underlyings=["BTC-"], snapshot_interval_ms=60_000, ) ``` ## Loading Tardis CSV data Tardis-format CSV data can be loaded using either Python or Rust. The loader reads the CSV text data from disk and parses it into Nautilus data. Since the loader is implemented in Rust, performance remains consistent regardless of whether you run it from Python or Rust. You can also specify a `limit` parameter for the `load_*` functions and methods to control the maximum number of rows loaded. :::note Loading mixed-instrument CSV files is challenging due to precision requirements and is not recommended. Use single-instrument CSV files instead. The `load_options_chain`, `stream_options_chain`, and `convert_options_chain_csv` methods are the exception: Tardis `options_chain` files are mixed-instrument chain files, and these paths track precision per instrument. Explicit precisions are still recommended for deterministic output. ::: ### Loading CSV data in Python You can load Tardis-format CSV data in Python using the `TardisCSVDataLoader`. When loading data, you can optionally specify the instrument ID, price precision, and size precision. Providing the instrument ID improves loading performance. Price and size precision are inferred from the CSV when omitted, but explicit values are recommended for deterministic output, especially with large files. To load the data, create a script similar to the following: ```python from pathlib import Path from nautilus_trader.adapters.tardis import TardisCSVDataLoader from nautilus_trader.model import InstrumentId instrument_id = InstrumentId.from_str("BTC-PERPETUAL.DERIBIT") loader = TardisCSVDataLoader( price_precision=1, size_precision=0, instrument_id=instrument_id, ) filepath = Path("YOUR_CSV_DATA_PATH") limit = None deltas = loader.load_deltas(filepath, limit=limit) ``` ### Loading CSV data in Rust You can load Tardis-format CSV data in Rust using the loading functions in `crates/adapters/tardis/src/csv/mod.rs`. When loading data, you can optionally specify the instrument ID, price precision, and size precision. Providing the instrument ID improves loading performance. Price and size precision are inferred from the CSV when omitted, but explicit values are recommended for deterministic output. For a complete example, see `crates/adapters/tardis/bin/example_csv.rs`. To load the data, you can use code similar to the following: ```rust use std::path::Path; use nautilus_adapters::tardis; use nautilus_model::identifiers::InstrumentId; #[tokio::main] async fn main() { // Optionally specify precisions and the CSV filepath let price_precision = Some(1); let size_precision = Some(0); let filepath = Path::new("YOUR_CSV_DATA_PATH"); // Optionally specify an instrument ID and/or limit let instrument_id = InstrumentId::from("BTC-PERPETUAL.DERIBIT"); let limit = None; // Consider propagating any parsing error depending on your workflow let _deltas = tardis::csv::load_deltas( filepath, price_precision, size_precision, Some(instrument_id), limit, ) .unwrap(); } ``` ## Streaming Tardis CSV data For memory-efficient processing of large CSV files, the Tardis integration can load and process data in configurable chunks rather than loading entire files into memory at once. This is useful for processing multi-gigabyte CSV files without exhausting system memory. The Python streaming functionality is available for the high-volume CSV types: - Order book deltas (`stream_deltas`). - Quote ticks (`stream_quotes`). - Trade ticks (`stream_trades`). - Order book depth snapshots (`stream_depth10`). - Options chain rows (`stream_options_chain`). Rust also exposes streaming functions for these CSV types, plus batched deltas and funding rates. ### Streaming CSV data in Python The `TardisCSVDataLoader` provides streaming methods that yield chunks of data as iterators. Each method accepts a `chunk_size` parameter that controls how many records are read per chunk: ```python from nautilus_trader.adapters.tardis import TardisCSVDataLoader from nautilus_trader.model import InstrumentId instrument_id = InstrumentId.from_str("BTC-PERPETUAL.DERIBIT") loader = TardisCSVDataLoader( price_precision=1, size_precision=0, instrument_id=instrument_id, ) filepath = Path("large_trades_file.csv") chunk_size = 100_000 # Process 100,000 records per chunk (default) # Stream trade ticks in chunks for chunk in loader.stream_trades(filepath, chunk_size): print(f"Processing chunk with {len(chunk)} trades") # Process each chunk - only this chunk is in memory for trade in chunk: # Your processing logic here pass ``` ### Streaming order book data For order book data, streaming is available for both deltas and depth snapshots: ```python # Stream order book deltas for chunk in loader.stream_deltas(filepath): print(f"Processing {len(chunk)} deltas") # Process delta chunk # Stream depth10 snapshots (specify levels: 5 or 25) for chunk in loader.stream_depth10(filepath, levels=5): print(f"Processing {len(chunk)} depth snapshots") # Process depth chunk ``` ### Streaming quote data Quote data can be streamed similarly: ```python # Stream quote ticks for chunk in loader.stream_quotes(filepath): print(f"Processing {len(chunk)} quotes") # Process quote chunk ``` ### Memory efficiency benefits The streaming approach provides significant memory efficiency advantages: - **Controlled Memory Usage**: Only one chunk is loaded in memory at a time. - **Scalable Processing**: Can process files larger than available RAM. - **Configurable Chunk Sizes**: Tune `chunk_size` based on your system's memory and performance requirements (default 100,000). :::warning When using streaming with precision inference, the inferred precision may differ from bulk loading the entire file. Precision inference works within chunk boundaries, and different chunks may contain values with different precision requirements. For deterministic precision behavior, provide explicit `price_precision` and `size_precision` parameters. ::: ### Streaming CSV data in Rust The underlying streaming functionality is implemented in Rust and can be used directly: ```rust use std::path::Path; use nautilus_adapters::tardis::csv::stream_trades; use nautilus_model::identifiers::InstrumentId; #[tokio::main] async fn main() { let filepath = Path::new("large_trades_file.csv"); let chunk_size = 100_000; let price_precision = Some(1); let size_precision = Some(0); let instrument_id = Some(InstrumentId::from("BTC-PERPETUAL.DERIBIT")); // Stream trades in chunks let stream = stream_trades( filepath, chunk_size, price_precision, size_precision, instrument_id, ).unwrap(); for chunk_result in stream { match chunk_result { Ok(chunk) => { println!("Processing chunk with {} trades", chunk.len()); // Process chunk } Err(e) => { eprintln!("Error processing chunk: {}", e); break; } } } } ``` ## Requesting instrument definitions You can request instrument definitions in both Python and Rust using the `TardisHttpClient`. This client interacts with the [Tardis instruments metadata API](https://docs.tardis.dev/api/instruments-metadata-api) to request and parse instrument metadata into Nautilus instruments. The `TardisHttpClient` constructor accepts optional parameters for `api_key`, `base_url`, `timeout_secs`, `normalize_symbols`, and `proxy_url`. The client provides methods to retrieve either a specific `instrument`, or all `instruments` available on a particular exchange. Use Tardis lower-kebab exchange IDs such as `binance-futures`. :::note A `TARDIS_API_KEY` with access to the instruments metadata API is required. ::: ### Requesting instruments in Python To request instrument definitions in Python, create a script similar to the following: ```python import asyncio from nautilus_trader.core import nautilus_pyo3 async def run(): http_client = nautilus_pyo3.TardisHttpClient() instrument = await http_client.instrument("bitmex", "xbtusd") print(f"Received: {instrument}") instruments = await http_client.instruments("bitmex") print(f"Received: {len(instruments)} instruments") if __name__ == "__main__": asyncio.run(run()) ``` ### Requesting instruments in Rust To request instrument definitions in Rust, use code similar to the following. For a complete example, see `crates/adapters/tardis/bin/example_http.rs`. ```rust use nautilus_tardis::{ enums::TardisExchange, http::client::TardisHttpClient, }; #[tokio::main] async fn main() { nautilus_common::logging::ensure_logging_initialized(); let client = TardisHttpClient::new(None, None, None, true, None).unwrap(); // Tardis instrument definitions let resp = client .instruments_info(TardisExchange::Bitmex, Some("XBTUSD"), None) .await; println!("Received: {resp:?}"); // Nautilus instrument definitions let resp = client .instruments( TardisExchange::Bitmex, Some("XBTUSD"), None, None, None, None, None, None, ) .await; println!("Received: {resp:?}"); } ``` ## Instrument provider The `TardisInstrumentProvider` requests and parses instrument definitions from Tardis through the HTTP instrument metadata API. Since there are multiple [Tardis-supported exchanges](#venues), when loading all instruments, you must filter for the desired venues using an `InstrumentProviderConfig`: ```python from nautilus_trader.config import InstrumentProviderConfig # See supported venues https://nautilustrader.io/docs/nightly/integrations/tardis#venues venues = {"BINANCE", "BYBIT"} filters = {"venues": frozenset(venues)} instrument_provider_config = InstrumentProviderConfig(load_all=True, filters=filters) ``` You can also load specific instrument definitions in the usual way: ```python from nautilus_trader.config import InstrumentProviderConfig instrument_ids = [ InstrumentId.from_str("BTCUSDT-PERP.BINANCE"), # Uses the 'binance-futures' exchange InstrumentId.from_str("BTCUSDT.BINANCE"), # Uses the 'binance' exchange ] instrument_provider_config = InstrumentProviderConfig(load_ids=instrument_ids) ``` ### Option exchange filtering The instrument provider filters out option-specific exchanges, such as `binance-options`, `binance-european-options`, `bybit-options`, `okex-options`, and `huobi-dm-options`, when the `instrument_type` filter is not provided or does not include `"option"`. To explicitly load option instruments, include `"option"` in the `instrument_type` filter: ```python from nautilus_trader.config import InstrumentProviderConfig venues = {"BINANCE", "BYBIT"} filters = { "venues": frozenset(venues), "instrument_type": {"option"}, # Explicitly request options } instrument_provider_config = InstrumentProviderConfig(load_all=True, filters=filters) ``` This filtering prevents unnecessary API calls to option exchanges when they are not needed. :::note Instruments must be available in the cache for all subscriptions. For simplicity, it's recommended to load all instruments for the venues you intend to subscribe to. ::: ## Live data client The `TardisDataClient` integrates Tardis Machine with a running NautilusTrader system. The Python live data client translates standard subscriptions into Tardis Machine streams for: - `OrderBookDelta` (L2 granularity from Tardis, including changes or full-depth snapshots) - `QuoteTick` - `TradeTick` - `Bar` (trade bars with [Tardis-supported bar aggregations](#bars)) - `FundingRateUpdate` (from derivative_ticker messages) Configured Tardis Machine replay/stream options can also emit `OrderBookDepth10` when `book_snapshot_output` is `depth10`. `OptionGreeks` from `option_summary` is supported by the Tardis Machine replay path and catalog writer. Set `extract_bbo_as_quotes` to also emit `QuoteTick` from the best bid/offer fields in those `option_summary` messages. ### Data WebSockets The main `TardisMachineClient` data WebSocket manages all stream subscriptions received during the initial connection phase, up to the duration specified by `ws_connection_delay_secs`. For any additional subscriptions made after this period, a new `TardisMachineClient` is created. This lets the main WebSocket handle many startup subscriptions in a single stream. When an initial subscription delay is set with `ws_connection_delay_secs`, unsubscribing from any of these streams does not remove the subscription from the Tardis Machine stream because Tardis does not support selective unsubscription. The component still unsubscribes from message bus publishing. All subscriptions made after any initial delay behave normally, fully unsubscribing from the Tardis Machine stream when requested. :::tip If you anticipate frequent subscription and unsubscription of data, set `ws_connection_delay_secs` to zero. This creates a new client for each initial subscription, allowing each to close individually on unsubscription. ::: ## Trade ID derivation Trade ticks use the venue-provided trade ID from the Tardis message or CSV row as the `TradeId`. When the venue omits the trade ID (empty string or null on some exchanges), both the WebSocket parser and CSV parser fall back to a deterministic FNV-1a hash of the symbol, timestamp, price, amount, and side. The same venue event yields the same trade ID across replays, keeping downstream dedup intact. ## Limitations and considerations The following limitations and considerations are currently known: - Historical quote and trade requests are not supported by `TardisDataClient`. Historical external `Bar` requests use Tardis Machine replay and require date-based replay windows. For catalog workflows, prefer `run_tardis_machine_replay`. ## Contributing :::info For additional features or to contribute to the Tardis adapter, please see our [contributing guide](https://github.com/nautechsystems/nautilus_trader/blob/develop/CONTRIBUTING.md). ::: # Book Imbalance Backtest (Betfair) Source: https://nautilustrader.io/docs/latest/tutorials/backtest_book_imbalance_betfair/ :::note This is a **Rust-only** v2 system tutorial. It drives the Rust `BacktestEngine` directly with raw Betfair streaming data, bypassing the Python and Parquet paths. ::: This tutorial backtests a `BookImbalanceActor` on a Betfair MATCH_ODDS market. It loads a raw historical streaming `.gz` file, feeds it through the Rust `BacktestEngine`, and tracks the bid/ask quoted-volume imbalance per runner. ## Introduction Betfair is a sports betting exchange where participants back (bid) and lay (ask) outcomes at decimal odds. Each runner has its own L2 order book that behaves like a financial order book. The actor reads `OrderBookDeltas` for every runner and accumulates two running totals per side: bid volume (back orders) and ask volume (lay orders). Per-batch and cumulative imbalance are computed as: ``` imbalance = (bid_volume - ask_volume) / (bid_volume + ask_volume) ``` A positive value means the market is leaning toward backing the outcome. Sports traders use this as a starting block, often combined with price momentum or market-wide features. A release build processes about three million data points per second with full order book maintenance in the matching engine. ```mermaid flowchart LR subgraph Inputs ["Source"] F["Betfair .gz MCM file"] end subgraph Loader ["BetfairDataLoader"] I["Instrument"] DLT["Deltas (L2 snap/update)"] TR["Trade ticks"] ICL["InstrumentClose"] end subgraph Engine ["BacktestEngine"] BK["Per-runner OrderBook (L2)"] ME["Matching engine"] end subgraph Actor ["BookImbalanceActor"] AC["Aggregate batch bid_vol / ask_vol"] SUM["Cumulative volume per runner"] IMB["imbalance = (bid - ask) / (bid + ask)"] end F --> I F --> DLT F --> TR F --> ICL DLT --> BK BK --> ME BK --> AC AC --> SUM --> IMB ``` ## Prerequisites - A working Rust toolchain ([rustup.rs](https://rustup.rs)). - The NautilusTrader repository cloned and building. - A Betfair historical `.gz` file containing MCM (Market Change Message) data. Source it from [Betfair historic data](https://historicdata.betfair.com/), a third-party archive, or by recording the Exchange Streaming API yourself. Place the file at: ``` tests/test_data/local/betfair/1.253378068.gz ``` This path is gitignored and not shipped with the repository. The bundled example dataset is a football MATCH_ODDS market with 3 runners and around 82,000 MCM lines recorded over 18 days. ## Loading the data `BetfairDataLoader` reads gzip-compressed Betfair Exchange Streaming API files and parses each line into Nautilus domain objects: ```rust use nautilus_betfair::loader::{BetfairDataItem, BetfairDataLoader}; use nautilus_model::types::Currency; let mut loader = BetfairDataLoader::new(Currency::GBP(), None); let items = loader.load(&filepath)?; ``` The loader returns a `Vec`: | Variant | Description | Maps to `Data` enum? | |:--------------------|:------------------------------------------------|:---------------------------| | `Instrument` | Runner definition from market definition. | No (added separately) | | `Status` | Market status transition (PreOpen, Trading...). | No (`Data` has no variant) | | `Deltas` | Order book snapshot or delta update. | Yes, `Data::Deltas` | | `Trade` | Incremental trade tick from cumulative volumes. | Yes, `Data::Trade` | | `Ticker` | Last traded price, volume, BSP near/far. | - | | `StartingPrice` | Betfair Starting Price for a runner. | - | | `BspBookDelta` | BSP-specific book delta. | - | | `InstrumentClose` | Settlement event. | Yes, `Data::InstrumentClose` | | `SequenceCompleted` | Batch completion marker. | - | | `RaceRunnerData` | GPS tracking data (horse/greyhound racing). | - | | `RaceProgress` | Race‑level progress data. | - | The backtest engine accepts the `Data` enum, so we map the variants we need and skip the Betfair-specific types: ```rust use nautilus_model::data::{Data, OrderBookDeltas_API}; let mut instruments = AHashMap::new(); let mut data: Vec = Vec::new(); for item in items { match item { BetfairDataItem::Instrument(inst) => { instruments.insert(inst.id(), *inst); } BetfairDataItem::Deltas(d) => { data.push(Data::Deltas(OrderBookDeltas_API::new(d))); } BetfairDataItem::Trade(t) => { data.push(Data::Trade(t)); } BetfairDataItem::InstrumentClose(c) => { data.push(Data::InstrumentClose(c)); } _ => {} } } ``` `OrderBookDeltas_API` is a thin FFI wrapper around `OrderBookDeltas` required by the `Data` enum. Instruments are re-emitted on every market definition update in the stream, so the map deduplicates them by keeping the latest version. :::warning The `Status` variant carries market status transitions (PreOpen, Trading, Suspended, Closed) but the `Data` enum has no variant for it. This example does not replay status transitions. If you extend this into a strategy that places orders, the matching engine will not see market suspensions or closures from the stream. Subscribe to instrument status separately or add status routing to the engine. ::: ## The actor NautilusTrader ships `BookImbalanceActor` in the trading crate's examples module. The example wires it up with a per-runner instrument list and a log interval: ```rust use nautilus_trading::examples::actors::BookImbalanceActor; let actor = BookImbalanceActor::new(instrument_ids, 5000, None); engine.add_actor(actor)?; ``` The second argument is the log interval: print a progress line every 5,000 updates per runner. The example reads `IMBALANCE_LOG_INTERVAL` from the environment, so set it to a smaller value (`200`) when you want to capture finer-grained data for the panels at the end of this tutorial. The full source is at [`crates/trading/src/examples/actors/imbalance/actor.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/actors/imbalance/actor.rs). ### How it works A `DataActor` in Rust needs three pieces: 1. A struct holding a `DataActorCore` field plus your own state. 2. `nautilus_actor!(YourType)` to wire up the core, plus a `Debug` implementation. 3. The `DataActor` trait implementation with your callbacks. The framework provides blanket `Actor` and `Component` implementations for runtime actors. The `nautilus_actor!` macro supplies the native runtime wiring when your struct holds a `DataActorCore`, so normal actor code only implements the callbacks it needs. On start the actor subscribes to `OrderBookDeltas` for each instrument. On each update it sums per-side volume from the individual deltas and accumulates running totals. On stop it prints a per-instrument summary. Setting `managed: false` in `subscribe_book_deltas` means the data engine does not maintain a separate order book copy in the cache for the actor. The exchange-side matching engine still maintains its own book through `book.apply_delta()` on every delta. Set `managed: true` if your actor needs to read the full book state from `self.cache().order_book(&instrument_id)`. ## Backtest engine setup ### Create the engine and venue Betfair is a cash-settled betting exchange. The venue uses `AccountType::Cash`, `OmsType::Netting`, and `BookType::L2_MBP`: ```rust let mut engine = BacktestEngine::new(BacktestEngineConfig::default())?; engine.add_venue( SimulatedVenueConfig::builder() .venue(Venue::from("BETFAIR")) .oms_type(OmsType::Netting) .account_type(AccountType::Cash) .book_type(BookType::L2_MBP) .starting_balances(vec![Money::from("1_000_000 GBP")]) .build()?, )?; ``` ### Add instruments, actor, and data ```rust for instrument in instruments.values() { engine.add_instrument(instrument)?; } let actor = BookImbalanceActor::new(instrument_ids, 5000, None); engine.add_actor(actor)?; engine.add_data(data, None, true, true)?; ``` The `add_data` parameters are `(data, client_id, validate, sort)`. With `validate: true` the engine checks the first element's instrument is registered (the batch is assumed homogeneous). With `sort: true` it sorts by timestamp. ### Run ```rust engine.run(None, None, None, false)?; ``` The four parameters are `(start, end, run_config_id, streaming)`. Passing `None` for start/end uses the full time range of the loaded data. ## What happens during the run For each data point in timestamp order the engine: 1. Advances the clock to the data timestamp. 2. Routes the data to the simulated exchange, which applies each delta to the per-instrument `OrderBook` and runs the matching engine cycle. 3. Publishes the data through the data engine and message bus, triggering the actor's `on_book_deltas` callback. 4. Drains command queues and settles venues (processes any pending orders). The matching engine maintains a full order book per instrument. The example has no orders to match, so the book state is ready to use as soon as it is swapped for a `Strategy`. ## Results The bundled MATCH_ODDS dataset has three runners and 143,098 data points; a release build completes in about 48 ms: ``` --- Book imbalance summary --- 1.253378068-2426.BETFAIR updates: 53197 bid_vol: 212225339.34 ask_vol: 117422531.85 imbalance: 0.2876 1.253378068-48783.BETFAIR updates: 36475 bid_vol: 52506905.49 ask_vol: 19104694.72 imbalance: 0.4664 1.253378068-58805.BETFAIR updates: 25426 bid_vol: 24295351.82 ask_vol: 25692733.11 imbalance: -0.0280 ``` Runner `2426` (the eventual winner, settled at BSP 2.22) ends at +0.288: backing flow dominates lay flow throughout the market. Runner `48783` shows even stronger backing pressure (+0.466) over fewer updates, while `58805` ends close to neutral (-0.028). ![Cumulative imbalance per runner](./assets/backtest_book_imbalance_betfair/panel_a_imbalance_lines.png) **Figure 1.** *Cumulative `(bid - ask) / (bid + ask)` per runner across the ~143k updates of the market lifetime. Dashed lines mark each runner's final imbalance.* ![Per-batch signed flow distribution](./assets/backtest_book_imbalance_betfair/panel_b_batch_distribution.png) **Figure 2.** *Distribution of per-batch signed flow ratio `(bid - ask) / (bid + ask)` over `IMBALANCE_LOG_INTERVAL=200` batches per runner. The shape of each runner's batch distribution is a sharper signal than the cumulative imbalance.* ![Cumulative bid and ask volume](./assets/backtest_book_imbalance_betfair/panel_c_cumulative_volume.png) **Figure 3.** *Cumulative back (bid) and lay (ask) volume per runner. Both sides are non-monotonic: lay flow occasionally outpaces back flow within short bursts even when cumulative imbalance stays positive.* ### Regenerate the panels The actor logs `[runner] update #N: batch bid=B ask=A cumulative imbalance=I` on every Nth update. The renderer parses those lines and writes static PNGs using the `nautilus_dark` tearsheet theme. ```bash IMBALANCE_LOG_INTERVAL=200 cargo run -p nautilus-betfair --features examples --release \ --example betfair-backtest > /tmp/betfair.log 2>&1 uv sync --extra visualization BETFAIR_LOG=/tmp/betfair.log \ python3 docs/tutorials/assets/backtest_book_imbalance_betfair/render_panels.py ``` ## Running the example ```bash # Debug build cargo run -p nautilus-betfair --features examples --example betfair-backtest # Release build (recommended) cargo run -p nautilus-betfair --features examples --release --example betfair-backtest # Custom data file cargo run -p nautilus-betfair --features examples --release --example betfair-backtest -- path/to/file.gz ``` ## Complete source The complete example is at [`crates/adapters/betfair/examples/betfair_backtest.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/betfair/examples/betfair_backtest.rs). ## Next steps - **Add a strategy**. Replace the actor with a `Strategy` implementation that places back/lay orders based on the imbalance signal. See the `EmaCross` example in `crates/trading/src/examples/strategies/ema_cross/strategy.rs` for the pattern. - **Use managed books**. Set `managed: true` in `subscribe_book_deltas` and read the full book via `self.cache().order_book(&id)` for richer signals like top-of-book spread, depth ratios, or weighted mid-price. - **Multiple markets**. Load several `.gz` files and run them through the same engine to test cross-market signals. - **Compare with Python**. Run the same backtest from Python using the `BacktestEngine` Python API. The Rust engine processes the same data pipeline at roughly six times the throughput of the Python/Cython path. # Delta-Neutral Options Strategy (Bybit) Source: https://nautilustrader.io/docs/latest/tutorials/delta_neutral_options_bybit/ :::note This is a **Rust-only** v2 system tutorial. It runs a live delta-neutral short-volatility strategy on Bybit using the Rust `LiveNode`. ::: This tutorial runs a short OTM strangle on Bybit BTC options and delta-hedges with the BTCUSDT perpetual. The strategy selects call and put strikes at startup, enters via implied-volatility limit orders, tracks portfolio delta from venue-provided Greeks, and submits market hedge orders on the perpetual when the delta drifts beyond a threshold. :::warning This strategy trades real money on mainnet. Setting `enter_strangle: false` only disables the initial strangle entry orders. The strategy still hydrates existing positions from the cache at startup and still submits hedge orders on the perpetual when portfolio delta breaches the threshold. If the account holds option or hedge positions from a prior session, the strategy will trade. ::: ## Prerequisites - Completion of the [options data tutorial](options_data_bybit.md), which covers instrument discovery, Greeks subscriptions, and the `DataActor` pattern. - A Bybit API key with **trading permissions** for options and linear perpetuals. - Environment variables: ```bash export BYBIT_API_KEY="your-api-key" export BYBIT_API_SECRET="your-api-secret" ``` ## Strategy overview The `DeltaNeutralVol` strategy ships in the trading crate's `examples` module and runs in five stages: 1. **Strike selection**: queries the instrument cache for all BTC options, filters to the nearest expiry, selects OTM call and put strikes by percentile rank. 2. **Entry**: places SELL limit orders on both legs priced by implied volatility (via Bybit's `order_iv` parameter). Entry is optional and disabled by default in the example. 3. **Greeks tracking**: subscribes to `OptionGreeks` for both legs. Deltas and IVs come directly from Bybit's option ticker stream. 4. **Rehedging**: computes portfolio delta and submits a market order on the BTCUSDT perpetual when the threshold is breached. Triggers on every Greeks update and on a periodic safety timer. 5. **Position tracking**: tracks call, put, and hedge positions via `on_order_filled`. Hydrates existing positions from the cache at startup. ```mermaid flowchart LR subgraph Discovery ["1. Strike selection (on_start)"] L["Cache: BTC option instruments"] F["Filter by nearest expiry, sort by strike"] K["Pick CALL strike at percentile (1 - target_call_delta)
Pick PUT strike at percentile |target_put_delta|"] end subgraph Entry ["2. Entry (optional)"] EI{{"enter_strangle AND
both mark IVs available"}} SL["Submit SELL limit order_iv on each leg"] end subgraph Track ["3. Greeks track + 4. Rehedge"] G["on_option_greeks updates leg delta"] PD["portfolio_delta = call_delta * call_pos
+ put_delta * put_pos
+ hedge_position"] TH{{"|portfolio_delta|
> rehedge_delta_threshold?"}} H["Submit MARKET order on BTCUSDT-LINEAR"] end subgraph Lifecycle ["5. Position tracking"] OF["on_order_filled updates leg / hedge counters"] end L --> F --> K K --> EI EI -->|yes| SL --> OF EI -->|no| OF G --> PD --> TH TH -->|yes| H --> OF OF --> PD ``` ### Portfolio delta The strategy computes net exposure as: ``` portfolio_delta = call_delta * call_position + put_delta * put_position + hedge_position ``` A short strangle starts near delta-neutral because the call and put deltas offset. With the default `target_call_delta = 0.20` and `target_put_delta = -0.20`, the two legs cancel at entry. As the underlying moves, net delta drifts and the strategy hedges to bring it back toward zero. ## Configuration The example file at [`crates/adapters/bybit/examples/node_delta_neutral.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/bybit/examples/node_delta_neutral.rs) configures the strategy: ```rust let hedge_instrument_id = InstrumentId::from("BTCUSDT-LINEAR.BYBIT"); let strategy_config = DeltaNeutralVolConfig::builder() .option_family("BTC".to_string()) .hedge_instrument_id(hedge_instrument_id) .client_id(client_id) .contracts(1) .rehedge_delta_threshold(0.5) .rehedge_interval_secs(30) .enter_strangle(false) .iv_param_key("order_iv".to_string()) .build(); let strategy = DeltaNeutralVol::new(strategy_config); ``` Parameters (defaults shown are the struct defaults; the example overrides `enter_strangle` to `false` and `iv_param_key` to `"order_iv"`): | Parameter | Default | Example | Description | |---------------------------|------------|------------------|----------------------------------------------| | `option_family` | required | `"BTC"` | Underlying filter for instrument discovery. | | `hedge_instrument_id` | required | `BTCUSDT-LINEAR` | Perpetual used for delta hedging. | | `client_id` | required | `"BYBIT"` | Data and execution client identifier. | | `target_call_delta` | `0.20` | - | Target call delta for strike selection. | | `target_put_delta` | `-0.20` | - | Target put delta for strike selection. | | `contracts` | `1` | - | Contracts per leg. | | `rehedge_delta_threshold` | `0.5` | - | Portfolio delta that triggers a hedge. | | `rehedge_interval_secs` | `30` | - | Periodic rehedge timer interval. | | `enter_strangle` | `true` | `false` | Place entry orders when Greeks arrive. | | `entry_iv_offset` | `0.0` | - | Vol points below mark IV for entry pricing. | | `iv_param_key` | `"px_vol"` | `"order_iv"` | Adapter‑specific IV parameter key. | The `iv_param_key` is the key difference between venues. Bybit uses `order_iv`, which the adapter maps to the `orderIv` field in the place-order API. OKX uses `px_vol`. Setting this correctly is required for IV-based order placement. ## Node setup The example configures both data and execution clients with `Option` and `Linear` product types: ```rust let data_config = BybitDataClientConfig { api_key: None, api_secret: None, product_types: vec![BybitProductType::Option, BybitProductType::Linear], ..Default::default() }; let exec_config = BybitExecClientConfig { api_key: None, api_secret: None, product_types: vec![BybitProductType::Option, BybitProductType::Linear], account_id: Some(account_id), ..Default::default() }; ``` Both product types are needed: `Option` for the strangle legs, `Linear` for the BTCUSDT perpetual hedge instrument. The execution client requires `account_id` for order identity tracking. ```rust let mut node = LiveNode::builder(trader_id, environment)? .with_name("BYBIT-DELTA-NEUTRAL-001".to_string()) .add_data_client(None, Box::new(data_factory), Box::new(data_config))? .add_exec_client(None, Box::new(exec_factory), Box::new(exec_config))? .with_reconciliation(true) .with_delay_post_stop_secs(5) .build()?; node.add_strategy(strategy)?; node.run().await?; ``` `with_reconciliation(true)` queries Bybit at startup for open orders and positions, hydrating the cache before the strategy starts. The strategy then picks up any existing positions from a prior session. ## How the strategy works ### Strike selection On start the strategy queries the cache for all option instruments matching `option_family`. It discards expired options, selects the nearest expiry, separates calls and puts, and sorts each list by strike price. Strikes are chosen by percentile in the sorted list: - **Call**: index = `(1.0 - target_call_delta) * count`. With 0.20 target delta and 50 calls, this selects the 40th strike (80th percentile, OTM). - **Put**: index = `|target_put_delta| * count`. With -0.20 target delta, this selects the 10th strike (20th percentile, OTM). This is a heuristic. Strike price ordering approximates delta ordering for options at the same expiry. A production strategy would subscribe to Greeks for all strikes first, then select by actual delta. ### Entry via implied volatility When `enter_strangle` is `true` and both mark IVs have arrived, the strategy places SELL limit orders using the `order_iv` parameter: ```rust let mut call_params = Params::new(); call_params.insert("order_iv".to_string(), json!(call_entry_iv.to_string())); self.submit_order(call_order, None, Some(client_id), Some(call_params))?; ``` Bybit converts `orderIv` to a limit price server-side and gives it priority over any explicit price. The `entry_iv_offset` config subtracts vol points from mark IV: an offset of 0.02 sells two vol points below mark for faster fills. :::note Bybit's demo environment rejects orders with `order_iv`. The adapter denies them before they reach the API. Use mainnet or testnet for IV-based order placement. ::: ### Rehedging Two triggers check portfolio delta: - **Every Greeks update**: `on_option_greeks` recomputes portfolio delta after updating the leg's delta value. - **Periodic timer**: fires every `rehedge_interval_secs` as a safety net when Greeks updates stop arriving. When `|portfolio_delta| > rehedge_delta_threshold`, the strategy submits a market order on the hedge instrument. A `hedge_pending` flag prevents duplicate submissions while an order is in flight. ### Position tracking The strategy tracks positions via `on_order_filled`, not by querying the cache on every tick. Each fill updates the corresponding position counter (call, put, or hedge). At startup, existing positions are hydrated from the cache (populated by reconciliation). ### Shutdown On stop the strategy cancels open orders, unsubscribes from all data feeds, and resets the hedge-pending flag. It does not close positions. Unwinding the strangle and hedge requires manual action or a separate exit strategy. ## What the run produces A 30-second mainnet run with `enter_strangle: false` against a clean account places no orders. The strategy logs the discovered instruments and the strike selection: ``` Selected call: BTC-28APR26-81000-C-USDT-OPTION.BYBIT (strike=81000) Selected put: BTC-28APR26-75000-P-USDT-OPTION.BYBIT (strike=75000) Strangle: 1 contracts per leg, hedge on BTCUSDT-LINEAR.BYBIT ``` That is enough to reason about the strategy's structural behaviour. The panels below visualise the mechanics around the actual selected strikes (75,000 / 81,000) at the captured underlying. ![Short strangle payoff at expiry](./assets/delta_neutral_options_bybit/panel_a_strangle_payoff.png) **Figure 1.** *Pnl at expiry of the short 75,000 PUT plus short 81,000 CALL combination, assuming a 1,500 USDT total premium and zero discount. The flat top is the credit-only zone between strikes; loss grows linearly past either strike.* ![Synthetic delta drift with rehedge](./assets/delta_neutral_options_bybit/panel_c_hedge_threshold.png) **Figure 2.** *Synthetic Brownian delta drift over 150 seconds with `rehedge_delta_threshold=0.5`. The dotted curve is the un-hedged drift; the line is the strategy's portfolio delta after each market hedge fire (crosses).* ![Portfolio delta drift around entry](./assets/delta_neutral_options_bybit/panel_b_delta_drift.png) **Figure 3.** *Toy approximation of how the short call and short put leg deltas move with a 5% spot range around entry, plus the resulting portfolio delta before hedging. Negative gamma compresses the curve in the wings and steepens it across the strikes.* ![Strike selection on the IV smile](./assets/delta_neutral_options_bybit/panel_d_strike_picker.png) **Figure 4.** *The strike-selection heuristic against an illustrative IV smile. The CALL strike sits at the (1 - 0.20) percentile and the PUT at the 0.20 percentile, placing both legs OTM at roughly equal-magnitude deltas around the underlying.* ### Regenerate the panels ```bash timeout 30 ./target/release/examples/bybit-delta-neutral > /tmp/bybit_dn.log 2>&1 uv sync --extra visualization DN_LOG=/tmp/bybit_dn.log \ python3 docs/tutorials/assets/delta_neutral_options_bybit/render_panels.py ``` The renderer parses selected strikes from the log; the panels themselves are illustrative because the default config does not place orders. ## Risk considerations - **Gamma risk**: a short strangle has negative gamma. Large underlying moves increase delta exposure faster than the rehedge timer responds. Tighten `rehedge_delta_threshold` and reduce `rehedge_interval_secs` for faster response, at the cost of more hedge trades. - **Vega risk**: an IV spike increases mark-to-market loss on the short options. The strategy does not manage vega exposure. - **Liquidity**: OTM crypto options can have wide spreads. Hedge quality degrades when the underlying gaps or the perpetual trades in coarse size increments. - **Lifecycle risk**: stopping the strategy stops hedging. Positions remain open and unhedged until manually managed. ## Running the example ```bash cargo run --example bybit-delta-neutral --package nautilus-bybit --features examples ``` The example runs with `enter_strangle: false` by default, so it does not place strangle entry orders. It still hydrates existing positions and submits hedge orders if portfolio delta breaches the threshold. On a clean account with no prior positions, no orders are placed. Stop with Ctrl+C. The strategy cancels open orders and unsubscribes before shutdown. ## Complete source - Example runner: [`crates/adapters/bybit/examples/node_delta_neutral.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/bybit/examples/node_delta_neutral.rs) - Strategy implementation: [`crates/trading/src/examples/strategies/delta_neutral_vol/`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/delta_neutral_vol/) - Strategy README with full config reference: [`crates/trading/src/examples/strategies/delta_neutral_vol/README.md`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/delta_neutral_vol/README.md) ## See also - [Options data and Greeks on Bybit](options_data_bybit.md): prerequisite tutorial covering Greeks subscriptions and option chain snapshots. - [Options](../concepts/options.md): option instrument types and data architecture. - [Bybit integration](../integrations/bybit.md#options-trading): options order parameters including `order_iv` and `mmp`. # Delta-Neutral Options Strategy (Derive) Source: https://nautilustrader.io/docs/latest/tutorials/delta_neutral_options_derive/ :::note This is a **Rust-only** v2 system tutorial. It runs the live delta-neutral short-volatility strategy on Derive using the Rust `LiveNode`. ::: This tutorial runs the shared `DeltaNeutralVol` strategy with the Derive adapter. The shipped example discovers ETH options, selects an out-of-the-money call and put, subscribes to venue-provided Greeks, and delta-hedges with `ETH-PERP.DERIVE`. The Derive runner starts in hedge-only mode: it sets `enter_strangle: false`, so it does not place the initial option entry orders. It still hydrates existing positions through reconciliation and can submit market hedge orders on the perpetual when portfolio delta breaches the configured threshold. For smoke tests, set `DERIVE_DELTA_NEUTRAL_HEDGE_ENABLED=false` to keep the strategy from submitting hedge orders while it still loads instruments, reconciles the account, and subscribes to Greeks. For an entry-order smoke test, set `DERIVE_DELTA_NEUTRAL_ENTER_STRANGLE=true`; the runner submits Derive-premium option orders instead of IV-priced option orders. :::warning This strategy can trade real money on mainnet. Setting `enter_strangle: false` only disables the initial strangle entry orders. If the selected option legs or the hedge instrument already have open positions, the strategy can submit hedge orders. ::: ## Prerequisites - Completion of the [Derive integration guide](../integrations/derive.md), including wallet, subaccount, session-key, and funding setup. - A Derive testnet or mainnet subaccount with enough USDC collateral for the hedge orders you plan to allow. - A working Rust toolchain and a built NautilusTrader workspace. - Environment variables for the selected Derive environment. For testnet: ```bash export DERIVE_TESTNET_WALLET_ADDRESS="0x..." export DERIVE_TESTNET_SESSION_PRIVATE_KEY="0x..." export DERIVE_TESTNET_SUBACCOUNT_ID="12345" ``` For mainnet: ```bash export DERIVE_WALLET_ADDRESS="0x..." export DERIVE_SESSION_PRIVATE_KEY="0x..." export DERIVE_SUBACCOUNT_ID="12345" export DERIVE_ENVIRONMENT="mainnet" ``` The example defaults to testnet. Set `DERIVE_ENVIRONMENT=mainnet` only for real-funds runs. ## Strategy overview The `DeltaNeutralVol` strategy lives in the trading crate's examples module. The Derive runner uses it in five stages: 1. **Instrument load**: configures the Derive data client with `currencies: ["ETH"]`, so the adapter loads ETH perps and options into the cache. 2. **Strike selection**: filters the cache to live ETH options, selects the nearest expiry, then chooses OTM call and put strikes by percentile rank. 3. **Greeks tracking**: subscribes to `OptionGreeks` for both legs. Derive Greeks come from the shared `ticker_slim` feed and the `option_pricing` payload. 4. **Rehedging**: computes portfolio delta and submits a Derive market order on `ETH-PERP.DERIVE` when the threshold is breached. 5. **Position tracking**: tracks call, put, and hedge positions through fills. Reconciliation hydrates existing positions before the strategy starts. ```mermaid flowchart LR subgraph Derive ["Derive public + private APIs"] T["ticker_slim option payloads"] P["private positions and fills"] H["ETH-PERP order entry"] end subgraph Adapter ["nautilus-derive clients"] G["OptionGreeks from option_pricing"] R["Startup reconciliation"] M["Signed market hedge order"] end subgraph Strategy ["DeltaNeutralVol"] S["Select nearest-expiry ETH strangle"] D["portfolio_delta = call_delta * call_pos
+ put_delta * put_pos
+ hedge_pos"] C{{"|portfolio_delta| > threshold?"}} end T --> G --> S --> D --> C P --> R --> D C -->|yes| M --> H ``` ### Portfolio delta The strategy computes net exposure as: ``` portfolio_delta = call_delta * call_position + put_delta * put_position + hedge_position ``` A short strangle starts close to delta-neutral when the call and put deltas offset. As the underlying moves, net delta drifts and the strategy hedges with the perpetual to bring the portfolio back toward zero. ## Configuration The example runner at `crates/adapters/derive/examples/node_delta_neutral.rs` configures the strategy: ```rust let option_family = env_string("DERIVE_DELTA_NEUTRAL_OPTION_FAMILY", "ETH")?; let default_hedge = format!("{option_family}-PERP.DERIVE"); let hedge_instrument = env_string("DERIVE_DELTA_NEUTRAL_HEDGE_INSTRUMENT", &default_hedge)?; let enter_strangle = env_bool("DERIVE_DELTA_NEUTRAL_ENTER_STRANGLE", false)?; let hedge_enabled = env_bool("DERIVE_DELTA_NEUTRAL_HEDGE_ENABLED", true)?; let rehedge_delta_threshold = if hedge_enabled { env_f64("DERIVE_DELTA_NEUTRAL_REHEDGE_DELTA_THRESHOLD", 0.5)? } else { 1.0e12 }; let hedge_instrument_id = InstrumentId::from(hedge_instrument.as_str()); let mut strategy_config = DeltaNeutralVolConfig::builder() .option_family(option_family) .hedge_instrument_id(hedge_instrument_id) .client_id(client_id) .target_call_delta(env_f64("DERIVE_DELTA_NEUTRAL_TARGET_CALL_DELTA", 0.20)?) .target_put_delta(env_f64("DERIVE_DELTA_NEUTRAL_TARGET_PUT_DELTA", -0.20)?) .contracts(env_u64("DERIVE_DELTA_NEUTRAL_CONTRACTS", 1)?) .rehedge_delta_threshold(rehedge_delta_threshold) .rehedge_interval_secs(env_u64("DERIVE_DELTA_NEUTRAL_REHEDGE_INTERVAL_SECS", 30)?) .enter_strangle(enter_strangle) .entry_iv_offset(env_f64("DERIVE_DELTA_NEUTRAL_ENTRY_IV_OFFSET", 0.0)?) .entry_premium_offset_ticks(env_i32("DERIVE_DELTA_NEUTRAL_ENTRY_PREMIUM_OFFSET_TICKS", 1)?) .build(); if let Some(expiry) = env_optional_string("DERIVE_DELTA_NEUTRAL_EXPIRY")? { strategy_config.expiry_filter = Some(expiry); } let strategy = DeltaNeutralVol::new(strategy_config); ``` Parameters: | Parameter | Default | Derive runner | Description | |------------------------------|------------|---------------|-----------------------------------------------| | `option_family` | required | `"ETH"` | Underlying filter for instrument discovery. | | `hedge_instrument_id` | required | `ETH-PERP` | Perpetual used for delta hedging. | | `client_id` | required | `"DERIVE"` | Data and execution client identifier. | | `target_call_delta` | `0.20` | `0.20` | Target call delta for strike selection. | | `target_put_delta` | `-0.20` | `-0.20` | Target put delta for strike selection. | | `contracts` | `1` | `1` | Contracts per option leg. | | `rehedge_delta_threshold` | `0.5` | `0.5` | Portfolio delta that triggers a hedge. | | `rehedge_interval_secs` | `30` | `30` | Periodic rehedge timer interval. | | `expiry_filter` | `None` | unset | Optional expiry substring filter. | | `enter_strangle` | `true` | `false` | Place entry orders when premium data arrives. | | `entry_premium_offset_ticks` | `None` | `1` | Ticks above option ask for sell entry. | | `entry_iv_offset` | `0.0` | `0.0` | Used only when premium mode is disabled. | | `iv_param_key` | `"px_vol"` | unused | IV parameter key for IV-priced venues. | The Derive runner reads these environment variables: | Variable | Default | Description | |---------------------------------------------------|------------------------|------------------------------------| | `DERIVE_DELTA_NEUTRAL_OPTION_FAMILY` | `ETH` | Option family / Derive currency. | | `DERIVE_DELTA_NEUTRAL_HEDGE_INSTRUMENT` | `-PERP.DERIVE` | Perpetual hedge instrument. | | `DERIVE_DELTA_NEUTRAL_ENTER_STRANGLE` | `false` | Enable option entry orders. | | `DERIVE_DELTA_NEUTRAL_HEDGE_ENABLED` | `true` | Enable perpetual hedge orders. | | `DERIVE_DELTA_NEUTRAL_REHEDGE_DELTA_THRESHOLD` | `0.5` | Portfolio delta hedge threshold. | | `DERIVE_DELTA_NEUTRAL_REHEDGE_INTERVAL_SECS` | `30` | Periodic hedge check interval. | | `DERIVE_DELTA_NEUTRAL_CONTRACTS` | `1` | Contracts per option leg. | | `DERIVE_DELTA_NEUTRAL_TARGET_CALL_DELTA` | `0.20` | Call strike‑selection target. | | `DERIVE_DELTA_NEUTRAL_TARGET_PUT_DELTA` | `-0.20` | Put strike‑selection target. | | `DERIVE_DELTA_NEUTRAL_EXPIRY` | unset | Optional expiry substring filter. | | `DERIVE_DELTA_NEUTRAL_ENTRY_PREMIUM_OFFSET_TICKS` | `1` | Sell‑entry ticks above option ask. | | `DERIVE_DELTA_NEUTRAL_ENTRY_IV_OFFSET` | `0.0` | Used only outside premium mode. | | `DERIVE_DELTA_NEUTRAL_MAX_FEE_PER_CONTRACT` | `1000` | Signed per‑contract fee cap. | | `DERIVE_DELTA_NEUTRAL_MARKET_ORDER_SLIPPAGE_BPS` | adapter default | Market hedge slippage bound. | Derive signs explicit premium limit prices. The runner enables the strategy's premium-entry mode with `entry_premium_offset_ticks=1`, so entry orders use a live option ask when available and fall back to Derive IV fields when the quote side is empty. Bybit and OKX keep using the shared IV-param path. ## Node setup The Derive runner uses the live environment and selects testnet or mainnet from `DERIVE_ENVIRONMENT`: ```rust let environment = Environment::Live; let derive_environment = derive_environment_from_env(); let trader_id = TraderId::from("TESTER-001"); let account_id = AccountId::from("DERIVE-001"); let client_id = *DERIVE_CLIENT_ID; ``` The data client bulk-loads ETH instruments. This is important because the strategy selects option legs from the cache during `on_start`. ```rust let data_config = DeriveDataClientConfig { environment: derive_environment, currencies: vec![option_family.clone()], ..Default::default() }; ``` The execution client reads wallet, session key, and subaccount values from the Derive environment variables when the config fields are left unset. The example sets a fee cap and allows optional protocol-constant overrides for local testing. ```rust let exec_config = DeriveExecClientConfig { environment: derive_environment, max_fee_per_contract: Some(Decimal::from_str_exact("1000")?), domain_separator: env_override( derive_environment, "DERIVE_DOMAIN_SEPARATOR", "DERIVE_TESTNET_DOMAIN_SEPARATOR", ), action_typehash: env_override( derive_environment, "DERIVE_ACTION_TYPEHASH", "DERIVE_TESTNET_ACTION_TYPEHASH", ), trade_module_address: env_override( derive_environment, "DERIVE_TRADE_MODULE_ADDRESS", "DERIVE_TESTNET_TRADE_MODULE_ADDRESS", ), ..Default::default() }; ``` Execution clients need `DeriveExecFactoryConfig`, which carries the trader and account IDs: ```rust let exec_factory_config = DeriveExecFactoryConfig { trader_id, account_id, config: exec_config, }; ``` The node enables reconciliation so open orders, positions, balances, and reports are loaded before the strategy starts: ```rust let mut node = LiveNode::builder(trader_id, environment)? .with_name("DERIVE-DELTA-NEUTRAL-001".to_string()) .add_data_client(None, Box::new(data_factory), Box::new(data_config))? .add_exec_client(None, Box::new(exec_factory), Box::new(exec_factory_config))? .with_reconciliation(true) .with_delay_post_stop_secs(5) .build()?; node.add_strategy(strategy)?; node.run().await?; ``` ## How the strategy works ### Strike selection On start the strategy queries the cache for all option instruments matching `option_family`. For Derive, the example uses `ETH`, so matching instruments have symbols such as `ETH-20260626-3000-C.DERIVE`. It discards expired options, optionally applies `expiry_filter`, and uses the nearest expiry when no filter is set. Calls and puts are sorted by strike: - **Call**: index = `(1.0 - target_call_delta) * count`. With the default `0.20`, this selects around the 80th percentile strike. - **Put**: index = `abs(target_put_delta) * count`. With the default `-0.20`, this selects around the 20th percentile strike. This is a strike-ordering heuristic. A production strategy can subscribe to Greeks for the full chain first and then select by actual delta. ### Greeks and shared ticker feeds Derive publishes option pricing fields on the same `ticker_slim` channel used for quotes. The adapter derives `OptionGreeks` from `option_pricing`, so the strategy only needs to subscribe to the two selected option legs: ```rust self.subscribe_option_greeks(call_id, Some(client_id), None); self.subscribe_option_greeks(put_id, Some(client_id), None); ``` The adapter reference-counts the underlying ticker subscription. Quotes, mark prices, index prices, funding rates, and option Greeks for the same instrument can share a single WebSocket channel. ### Rehedging on Derive The Derive execution adapter sends market orders as signed orders with a slippage-bound limit price. Before signing, it refreshes the current ticker snapshot for the hedge instrument and writes the worst acceptable price into the EIP-712 payload. The default `market_order_slippage_bps` is `50`. The strategy submits a hedge when both selected option legs have emitted Greeks and: ``` abs(portfolio_delta) > rehedge_delta_threshold ``` A positive portfolio delta triggers a SELL on `ETH-PERP.DERIVE`; a negative portfolio delta triggers a BUY. A `hedge_pending` flag blocks duplicate submissions while an order is in flight. ### Position tracking The strategy tracks positions via `on_order_filled` rather than polling positions on every update. Reconciliation hydrates existing positions at startup; subsequent fills update the in-memory call, put, and hedge counters. ### Shutdown On stop the strategy cancels open orders for the selected option legs and the hedge instrument, unsubscribes from data feeds, and leaves positions open. Unwinding the strangle and hedge requires manual action or a separate exit strategy. ## What the run produces On testnet with `enter_strangle: false`, the strategy should discover the selected legs, subscribe to Greeks, and place no entry orders. The selected symbols depend on the live Derive chain: ``` Selected call: ETH---C.DERIVE (strike=) Selected put: ETH---P.DERIVE (strike=) Strangle: 1 contracts per leg, hedge on ETH-PERP.DERIVE Strangle entry disabled: hedging externally-held positions only. ``` If no existing positions are present, the run should remain data-only after startup. If the account already holds positions in the selected legs or hedge instrument, the periodic rehedge timer can submit hedge orders. The panels below use the same selected-strike mechanics as the Derive runner. They parse selected strikes from the smoke-test log when available and otherwise fall back to illustrative ETH strikes. ![Derive short strangle payoff at expiry](./assets/delta_neutral_options_derive/panel_a_strangle_payoff.png) **Figure 1.** *Pnl at expiry of the short ETH put plus short ETH call combination, assuming a fixed USDC premium and zero discount. The flat top is the credit-only zone between strikes; loss grows linearly past either strike.* ![Derive portfolio delta drift around entry](./assets/delta_neutral_options_derive/panel_b_delta_drift.png) **Figure 2.** *Toy approximation of how the short call and short put leg deltas move around entry, plus the resulting portfolio delta before hedging.* ![Derive synthetic delta drift with rehedge](./assets/delta_neutral_options_derive/panel_c_hedge_threshold.png) **Figure 3.** *Synthetic Brownian delta drift over 150 seconds with `rehedge_delta_threshold=0.5`. Crosses show where the strategy would submit a hedge order when hedging is enabled.* ![Derive strike selection on the IV smile](./assets/delta_neutral_options_derive/panel_d_strike_picker.png) **Figure 4.** *The strike-selection heuristic against an illustrative IV smile. The call strike sits near the `(1 - target_call_delta)` percentile and the put near `abs(target_put_delta)`.* ### Regenerate the panels ```bash export DERIVE_ENVIRONMENT=mainnet export DERIVE_DELTA_NEUTRAL_HEDGE_ENABLED=false timeout 45 cargo run --example derive-delta-neutral --package nautilus-derive --features examples \ > /tmp/derive_dn.log 2>&1 uv sync --extra visualization export DN_LOG=/tmp/derive_dn.log python3 docs/tutorials/assets/delta_neutral_options_derive/render_panels.py ``` The renderer only uses the log to pick strikes. The plots remain illustrative because the no-order smoke configuration disables entry and hedge submissions. ## Risk considerations - **Gamma risk**: a short strangle has negative gamma. Large ETH moves can increase delta exposure faster than the rehedge timer responds. - **Slippage risk**: Derive market orders sign a slippage-bound limit price before submission. Tight bounds can reject useful hedges; loose bounds can fill worse than expected. - **Entry-price risk**: Derive entry uses live option asks plus a tick offset, or computes a premium from Derive IV fields when the quote side is empty. A small or negative offset can cross the book and fill immediately. - **Collateral risk**: Derive rejects orders when the subaccount lacks initial-margin headroom. Check `private/get_subaccount` or the adapter account snapshot before enabling live hedging. - **Lifecycle risk**: stopping the strategy stops hedging. Positions remain open and unhedged until they are managed elsewhere. ## Running the example ```bash cargo run --example derive-delta-neutral --package nautilus-derive --features examples ``` Stop with Ctrl+C. The strategy cancels open orders and unsubscribes before shutdown, but it does not close positions. For a mainnet smoke test that loads the venue and account without submitting orders: ```bash export DERIVE_ENVIRONMENT=mainnet export DERIVE_DELTA_NEUTRAL_HEDGE_ENABLED=false timeout 45 cargo run --example derive-delta-neutral --package nautilus-derive --features examples ``` For a mainnet smoke test that submits Derive-premium option entry orders: ```bash export DERIVE_ENVIRONMENT=mainnet export DERIVE_DELTA_NEUTRAL_ENTER_STRANGLE=true export DERIVE_DELTA_NEUTRAL_ENTRY_PREMIUM_OFFSET_TICKS=1 timeout --signal=INT 45 cargo run --example derive-delta-neutral --package nautilus-derive \ --features examples ``` ## Complete source - Example runner: `crates/adapters/derive/examples/node_delta_neutral.rs` - Strategy implementation: `crates/trading/src/examples/strategies/delta_neutral_vol/` - Strategy README: `crates/trading/src/examples/strategies/delta_neutral_vol/README.md` ## See also - [Derive integration](../integrations/derive.md): environment setup, symbology, capabilities, and execution semantics. - [Options](../concepts/options.md): option instrument types and data architecture. - [Delta-neutral options strategy on Bybit](delta_neutral_options_bybit.md): the same shared strategy with Bybit-specific IV entry parameters. # Mean Reversion with Proxy FX Data (AX Exchange) Source: https://nautilustrader.io/docs/latest/tutorials/fx_mean_reversion_ax/ This tutorial backtests a Bollinger-band mean-reversion strategy on **EURUSD-PERP** at [AX Exchange](https://architect.exchange) using [TrueFX](https://www.truefx.com) EUR/USD spot ticks as a proxy. ## Introduction The strategy combines two indicators on 1-minute mid bars: - **Bollinger Bands** (`BBMeanReversion`'s `BB(20, 2.0sd)`): a rolling 20-bar mean and a +/-2sd envelope. The bands flag price as overextended relative to recent volatility. - **Relative Strength Index** (`RSI(14)`): a 14-bar momentum oscillator. NautilusTrader RSI is on `[0, 1]`, so the conventional 30/70 thresholds become `0.30` / `0.70`. Entry needs both signals at once: a touch of the lower band with `RSI < 0.30` opens a long; a touch of the upper band with `RSI > 0.70` opens a short. Exit is one-sided: any open position closes when the close crosses back through the BB middle. Existing positions on the opposite side are flattened before a new entry. The shipped `BBMeanReversion` strategy is intentionally simple and has no edge. ```mermaid flowchart LR subgraph Inputs ["Data"] Q["TrueFX bid/ask ticks"] end subgraph Engine ["BacktestEngine"] W["QuoteTickDataWrangler"] AGG["1-minute MID INTERNAL aggregator"] BAR["Bar close"] end subgraph Indicators BB(("BB(20, 2.0sd)")) RSI(("RSI(14)")) end subgraph Decision ["Decision"] EX{{"Net long AND close >= mid
OR
net short AND close <= mid"}} ENL{{"close <= lower
AND RSI < 0.30"}} ENS{{"close >= upper
AND RSI > 0.70"}} end subgraph Orders CL["Close all positions"] BUY["BUY market"] SELL["SELL market"] end Q --> W --> AGG --> BAR BAR --> BB BAR --> RSI BB --> ENL BB --> ENS RSI --> ENL RSI --> ENS BB --> EX EX -->|yes| CL ENL -->|yes| BUY ENS -->|yes| SELL CL --> BUY CL --> SELL ``` ### Why proxy data AX Exchange is a new venue not yet covered by historical data vendors. [TrueFX](https://www.truefx.com) publishes free, institutional-grade EUR/USD spot tick archives (Integral and Jefferies pools) that stand in cleanly for AX EURUSD-PERP backtests. ## Prerequisites - Python 3.12+ - [NautilusTrader](https://pypi.org/project/nautilus_trader/) installed. - A free TrueFX account, used to download a monthly tick archive. ## Data preparation ### Download TrueFX EUR/USD ticks 1. Go to the [TrueFX historical downloads page](https://www.truefx.com/truefx-historical-downloads/). 2. Pick **EUR/USD** and a month, for example **December 2025**. 3. Extract the ZIP. The CSV is headerless with columns `pair, timestamp, bid, ask`. ### Load into Nautilus quote ticks ```python from pathlib import Path import pandas as pd from nautilus_trader.persistence.wranglers import QuoteTickDataWrangler df = pd.read_csv( Path("EURUSD-2025-12.csv"), header=None, names=["pair", "timestamp", "bid", "ask"], ) df["timestamp"] = pd.to_datetime(df["timestamp"], format="%Y%m%d %H:%M:%S.%f") df = df.set_index("timestamp")[["bid", "ask"]] wrangler = QuoteTickDataWrangler(instrument=EURUSD_PERP) # defined below ticks = wrangler.process(df) ``` The wrangler tags every tick with the instrument ID. The strategy declares `1-MINUTE-MID-INTERNAL`, so the engine builds 1-minute MID bars from the tick stream internally. ## Instrument definition Proxy data needs a manual instrument definition. The multiplier of `1000` gives one contract a notional of 1,000 EUR. ```python from decimal import Decimal from nautilus_trader.model.currencies import USD from nautilus_trader.model.enums import AssetClass from nautilus_trader.model.identifiers import InstrumentId from nautilus_trader.model.identifiers import Symbol from nautilus_trader.model.instruments import PerpetualContract from nautilus_trader.model.objects import Price from nautilus_trader.model.objects import Quantity instrument_id = InstrumentId.from_str("EURUSD-PERP.AX") EURUSD_PERP = PerpetualContract( instrument_id=instrument_id, raw_symbol=Symbol("EURUSD-PERP"), underlying="EUR", asset_class=AssetClass.FX, quote_currency=USD, settlement_currency=USD, is_inverse=False, price_precision=5, size_precision=0, price_increment=Price.from_str("0.00001"), size_increment=Quantity.from_int(1), multiplier=Quantity.from_int(1000), lot_size=Quantity.from_int(1), margin_init=Decimal("0.05"), margin_maint=Decimal("0.025"), maker_fee=Decimal("0.0002"), taker_fee=Decimal("0.0005"), ts_event=0, ts_init=0, ) ``` Fees and margin are explicit backtest assumptions. Check the [AX Exchange documentation](https://docs.architect.exchange/) for current rates. ## Configuration | Parameter | Value | Description | | -------------------- | ------ | ---------------------------------------------------- | | `bb_period` | `20` | Rolling window for the BB mean and the standard deviation. | | `bb_std` | `2.0` | Band width in standard deviations. | | `rsi_period` | `14` | RSI lookback in bars. | | `rsi_buy_threshold` | `0.30` | Long entry confirmation (NautilusTrader RSI is `[0, 1]`). | | `rsi_sell_threshold` | `0.70` | Short entry confirmation. | | `trade_size` | `1` | One contract per trade (1,000 EUR notional). | :::tip NautilusTrader RSI returns values in `[0.0, 1.0]`, not `[0, 100]`. The `0.30` / `0.70` thresholds correspond to the textbook 30 / 70 levels. ::: ## Backtest setup ```python from nautilus_trader.backtest.config import BacktestEngineConfig from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.config import LoggingConfig from nautilus_trader.examples.strategies.bb_mean_reversion import BBMeanReversion from nautilus_trader.examples.strategies.bb_mean_reversion import BBMeanReversionConfig from nautilus_trader.model.data import BarType from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.model.identifiers import TraderId from nautilus_trader.model.identifiers import Venue from nautilus_trader.model.objects import Money engine = BacktestEngine( BacktestEngineConfig( trader_id=TraderId("BACKTESTER-001"), logging=LoggingConfig(log_level="INFO"), ), ) AX = Venue("AX") engine.add_venue( venue=AX, oms_type=OmsType.NETTING, account_type=AccountType.MARGIN, base_currency=USD, starting_balances=[Money(100_000, USD)], ) engine.add_instrument(EURUSD_PERP) engine.add_data(ticks) strategy = BBMeanReversion( BBMeanReversionConfig( instrument_id=instrument_id, bar_type=BarType.from_str("EURUSD-PERP.AX-1-MINUTE-MID-INTERNAL"), trade_size=Decimal("1"), bb_period=20, bb_std=2.0, rsi_period=14, rsi_buy_threshold=0.30, rsi_sell_threshold=0.70, ), ) engine.add_strategy(strategy) engine.run() ``` Reports come straight off `engine.trader`: ```python print(engine.trader.generate_account_report(AX)) print(engine.trader.generate_order_fills_report()) print(engine.trader.generate_positions_report()) engine.reset() engine.dispose() ``` The runnable example is at [`architect_ax_mean_reversion.py`](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/architect_ax_mean_reversion.py). ## What the run produces Replaying TrueFX EUR/USD December 2025 through `BBMeanReversion(20, 2sd, RSI 14)` prints 44,591 1-minute mid bars and closes 1,089 positions across 2,178 fills. Cumulative realised pnl ends at **-1,287 USD**: the strategy bleeds steadily through the month with no clear regime-driven recovery. Mean reversion without a regime filter pays the spread on every cycle, and EUR/USD ran a pronounced uptrend through the second half of December which the strategy fought repeatedly. ![EUR/USD 1-minute mid bars across December 2025 with BB envelope](./assets/fx_mean_reversion_ax/panel_a_overview.png) **Figure 1.** *EUR/USD 1-minute mid bars across December 2025 with the BB middle and +/-2sd envelope. Long flat patches are weekend gaps in the TrueFX feed.* ![Twelve-hour zoom on entries, exits, and RSI](./assets/fx_mean_reversion_ax/panel_b_zoom.png) **Figure 2.** *Twelve-hour zoom around the dataset midpoint. Top: mid with BB envelope, long entries (triangles up), short entries (triangles down), and closing fills (crosses). Bottom: RSI(14) with the 0.30 buy / 0.70 sell thresholds.* ![Decision space scatter](./assets/fx_mean_reversion_ax/panel_c_decision_scatter.png) **Figure 3.** *Per-bar BB z-score against RSI for the whole month. Shaded regions mark the entry-eligible quadrants: lower-left (long) and upper-right (short). The diagonal lobe is the natural co-movement of band-relative price and RSI.* ![Cumulative realised pnl per closed position](./assets/fx_mean_reversion_ax/panel_d_pnl.png) **Figure 4.** *Cumulative realised USD pnl across closed positions. The curve declines roughly linearly, dominated by spread and small adverse moves on each cycle.* ### Regenerate the panels A self-contained renderer re-runs the backtest, computes BB and RSI on the captured bars, and writes PNG panels using the shared `nautilus_dark` tearsheet theme. ```bash uv sync --extra visualization TRUEFX_CSV=tests/test_data/local/truefx/EURUSD-2025-12.csv \ python3 docs/tutorials/assets/fx_mean_reversion_ax/render_panels.py ``` Set `TRUEFX_CSV` to wherever you saved the EUR/USD archive. ## Next steps - **Add a regime filter**. The drawdown is concentrated in trending sessions. Suppress entries when realised range or a slower trend filter says the market is directional. - **Tune thresholds**. A wider band (`bb_std=2.5`) or stricter RSI cutoffs (`0.25` / `0.75`) cut entries but raise the bar for confirmation. - **Add stops**. Hard stop-loss orders cap downside per cycle and prevent carrying a losing position to the BB middle reversion. - **Go live on the AX sandbox**. Connect to the AX sandbox for paper trading once the backtest behaves. See the [AX Exchange integration guide](../integrations/architect_ax.md) for setup. ## Running live The same `BBMeanReversion` strategy runs live against AX Exchange. The launch script swaps the `BacktestEngine` for a `TradingNode` with the AX data and execution clients configured. See the live example: [`ax_mean_reversion.py`](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/architect_ax/ax_mean_reversion.py). For connection setup and API key configuration, see the [AX Exchange integration guide](../integrations/architect_ax.md). ## Further reading - [`BBMeanReversion` strategy source](https://github.com/nautechsystems/nautilus_trader/tree/develop/nautilus_trader/examples/strategies/bb_mean_reversion.py) - [Gold perpetual book imbalance tutorial](gold_book_imbalance_ax.md) - [Architect Exchange documentation](https://docs.architect.exchange/) # Gold Perpetual Book Imbalance with Proxy Futures Data (AX Exchange) Source: https://nautilustrader.io/docs/latest/tutorials/gold_book_imbalance_ax/ This tutorial backtests a top-of-book imbalance strategy on **XAU-PERP** at [AX Exchange](https://architect.exchange) using [Databento](https://databento.com) CME gold futures (`GC.v.0`) `mbp-1` quotes as a proxy. ## Introduction Top-of-book imbalance is a microstructure signal: when one side of the BBO holds significantly more resting size than the other, the book is leaning and short-term price often moves toward the thinner side as the heavier side absorbs flow. The shipped `OrderBookImbalance` strategy fires a fill-or-kill (FOK) limit order against the thicker side every time the ratio between sides clears a threshold and a cooldown has elapsed. Because the strategy only needs the BBO, it works with `mbp-1` (market by price, single best bid/ask) quote data rather than the full L2 book. That keeps source costs down for backtesting. `OrderBookImbalance` is a teaching strategy and has no edge. ```mermaid flowchart LR subgraph Inputs ["Data"] D["Databento mbp-1 quotes"] end subgraph Engine ["BacktestEngine"] L["DatabentoDataLoader"] Q["QuoteTick stream"] B["L1 OrderBook in cache"] end subgraph Strategy ["OrderBookImbalance"] R{{"larger >= trigger_min_size
AND smaller/larger < ratio
AND cooldown elapsed"}} D2{{"bid_size > ask_size?"}} BUY["Submit FOK BUY at best ask"] SELL["Submit FOK SELL at best bid"] end D --> L --> Q --> B B --> R R -->|yes| D2 D2 -->|yes| BUY D2 -->|no| SELL ``` ### Why proxy data AX Exchange is new and not yet covered by Databento. CME `GC` gold futures are the most liquid gold derivatives globally and provide representative microstructure for backtesting gold strategies. We use the **continuous contract** `GC.v.0` so the file stitches across expiries on the highest-volume contract, mirroring how a perpetual chases liquidity. The `stype_in="continuous"` parameter resolves the symbol through Databento's continuous mapping at request time. The `instrument_id` override at load time is safe because the continuous contract maps to a single underlying instrument at any moment. For a deeper read on the predictive power of book imbalance features, see Databento's [blog post on HFT signals with sklearn](https://databento.com/blog/hft-sklearn-python). ## Prerequisites - Python 3.12+ - [NautilusTrader](https://pypi.org/project/nautilus_trader/) installed. - A Databento API key: ```bash export DATABENTO_API_KEY="your-api-key" ``` - The Databento Python client: `pip install databento`. ## Data preparation ### Download CME gold futures quotes ```python import databento as db from pathlib import Path data_path = Path("gc_gold_quotes.dbn.zst") if not data_path.exists(): client = db.Historical() data = client.timeseries.get_range( dataset="GLBX.MDP3", symbols=["GC.v.0"], stype_in="continuous", schema="mbp-1", start="2024-11-15", end="2024-11-16", ) data.to_file(data_path) ``` This pulls one trading day. The file is reused on subsequent runs. ### Load into Nautilus quote ticks `DatabentoDataLoader.from_dbn_file` parses the `.dbn.zst` archive and emits `QuoteTick` objects. The `instrument_id` argument overrides the Databento symbology so every tick appears to come from `XAU-PERP.AX`. ```python from nautilus_trader.adapters.databento import DatabentoDataLoader from nautilus_trader.model.identifiers import InstrumentId instrument_id = InstrumentId.from_str("XAU-PERP.AX") loader = DatabentoDataLoader() quotes = loader.from_dbn_file( path="gc_gold_quotes.dbn.zst", instrument_id=instrument_id, ) ``` ## Instrument definition Proxy data needs a manual instrument definition. Price precision and tick size match the CME source data; margin and fee parameters reflect AX conditions. ```python from decimal import Decimal from nautilus_trader.model.currencies import USD from nautilus_trader.model.enums import AssetClass from nautilus_trader.model.identifiers import Symbol from nautilus_trader.model.instruments import PerpetualContract from nautilus_trader.model.objects import Price from nautilus_trader.model.objects import Quantity XAU_PERP = PerpetualContract( instrument_id=instrument_id, raw_symbol=Symbol("XAU-PERP"), underlying="XAU", asset_class=AssetClass.COMMODITY, quote_currency=USD, settlement_currency=USD, is_inverse=False, price_precision=2, size_precision=0, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_int(1), multiplier=Quantity.from_int(1), lot_size=Quantity.from_int(1), margin_init=Decimal("0.08"), margin_maint=Decimal("0.04"), maker_fee=Decimal("0.0002"), taker_fee=Decimal("0.0005"), ts_event=0, ts_init=0, ) ``` Fees are explicit backtest assumptions. Check [AX documentation](https://docs.architect.exchange/) for current rates. ## Strategy configuration `use_quote_ticks=True` and `book_type="L1_MBP"` together tell the strategy to consume quotes and maintain its own L1 book in cache rather than subscribing to L2 deltas. | Parameter | Value | Description | | ------------------------------ | --------- | --------------------------------------------- | | `max_trade_size` | `10` | Cap on contracts per FOK order. | | `trigger_min_size` | `1.0` | Larger side must hold at least one contract. | | `trigger_imbalance_ratio` | `0.10` | Trigger when smaller / larger < 10%. | | `min_seconds_between_triggers` | `5.0` | Cooldown between consecutive triggers. | | `book_type` | `L1_MBP` | Top of book only. | | `use_quote_ticks` | `True` | Drive the strategy from quote ticks. | ```python from nautilus_trader.examples.strategies.orderbook_imbalance import OrderBookImbalance from nautilus_trader.examples.strategies.orderbook_imbalance import OrderBookImbalanceConfig strategy = OrderBookImbalance( OrderBookImbalanceConfig( instrument_id=instrument_id, max_trade_size=Decimal(10), trigger_min_size=1.0, trigger_imbalance_ratio=0.10, min_seconds_between_triggers=5.0, book_type="L1_MBP", use_quote_ticks=True, ), ) ``` ## Backtest setup ```python from nautilus_trader.backtest.config import BacktestEngineConfig from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.config import LoggingConfig from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.model.identifiers import TraderId from nautilus_trader.model.identifiers import Venue from nautilus_trader.model.objects import Money engine = BacktestEngine( BacktestEngineConfig( trader_id=TraderId("BACKTESTER-001"), logging=LoggingConfig(log_level="INFO"), ), ) AX = Venue("AX") engine.add_venue( venue=AX, oms_type=OmsType.NETTING, account_type=AccountType.MARGIN, base_currency=USD, starting_balances=[Money(100_000, USD)], ) engine.add_instrument(XAU_PERP) engine.add_data(quotes) engine.add_strategy(strategy) engine.run() ``` Reports are on `engine.trader`: ```python print(engine.trader.generate_account_report(AX)) print(engine.trader.generate_order_fills_report()) print(engine.trader.generate_positions_report()) engine.reset() engine.dispose() ``` The runnable example is at [`architect_ax_book_imbalance.py`](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/architect_ax_book_imbalance.py). ## What the run produces Replaying 2024-11-15 GC.v.0 mbp-1 (one trading day) through `OrderBookImbalance(0.10, 1.0, 5s)` prints 2,378 FOK fills net into 5 closed position cycles. Cumulative realised pnl ends at **-4,170 USD**: the strategy bleeds steadily across the day, mostly through spread cost on incremental FOK fills that add to existing positions. ![GC.v.0 top of book around an active cycle](./assets/gold_book_imbalance_ax/panel_a_top_book.png) **Figure 1.** *GC.v.0 top of book around the cycle that opened with a short entry near 09:26 and exited near 09:31, then re-entered long until 09:35. Triangles are entries from flat, crosses are returns to flat, open circles are incremental FOK fills that grew the position.* ![Imbalance ratio distribution](./assets/gold_book_imbalance_ax/panel_b_imbalance_dist.png) **Figure 2.** *`smaller / larger` BBO size ratio across all sampled top-of-book snapshots, with the 0.10 trigger threshold marked. The mass left of the threshold is the addressable trigger region.* ![Mid and top-of-book size across the day](./assets/gold_book_imbalance_ax/panel_c_size_landscape.png) **Figure 3.** *Mid price (top) and best bid/ask size in contracts (bottom) across the trading day. Top-of-book sizes flicker between roughly two and fifty contracts; the mid traverses about a fifteen-dollar range.* ![Cumulative realised pnl per closed position](./assets/gold_book_imbalance_ax/panel_d_pnl.png) **Figure 4.** *Cumulative realised USD pnl across the five closed position cycles. The slope is consistently negative and the per-cycle pnl is dominated by spread.* ### Regenerate the panels A self-contained renderer re-runs the backtest with a quote-sampling actor and writes PNGs to the asset directory using the `nautilus_dark` tearsheet theme. ```bash uv sync --extra visualization GC_DBN=tests/test_data/local/Databento/gc_gold_quotes.dbn.zst \ python3 docs/tutorials/assets/gold_book_imbalance_ax/render_panels.py ``` ## Next steps - **Stricter trigger**. Lower `trigger_imbalance_ratio` to `0.05` or raise `trigger_min_size` to `5` to require more conviction before firing. - **Different sessions**. Replay regular trading hours (RTH) only or roll through several days to see how the strategy behaves across regimes. - **Other instruments**. AX offers FX perpetuals (`EURUSD-PERP`, `GBPUSD-PERP`) and silver (`XAG-PERP`). The same proxy approach works with the corresponding CME futures. - **Go live on the AX sandbox**. See the [AX Exchange integration guide](../integrations/architect_ax.md) once the backtest behaves. ## Running live The same `OrderBookImbalance` strategy runs live against AX Exchange. The launch script swaps the `BacktestEngine` for a `TradingNode` with the AX data and execution clients configured. See the live example: [`ax_book_imbalance.py`](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/live/architect_ax/ax_book_imbalance.py). For connection setup and API key configuration, see the [AX Exchange integration guide](../integrations/architect_ax.md). ## Further reading - [`OrderBookImbalance` strategy source](https://github.com/nautechsystems/nautilus_trader/tree/develop/nautilus_trader/examples/strategies/orderbook_imbalance.py) - [Mean Reversion with Proxy FX Data tutorial](fx_mean_reversion_ax.md) - [Architect Exchange documentation](https://docs.architect.exchange/) - [Databento: HFT signals with sklearn](https://databento.com/blog/hft-sklearn-python) # Grid Market Making with a Deadman's Switch (BitMEX) Source: https://nautilustrader.io/docs/latest/tutorials/grid_market_maker_bitmex/ This tutorial backtests the shipped `GridMarketMaker` strategy on BitMEX XBTUSD with free historical quote data from [Tardis.dev](https://tardis.dev), then runs the same configuration live through a Rust `LiveNode`. It focuses on BitMEX's **deadman's switch**: a server-side cancel-all timer that protects the strategy from stranded quotes if the client loses connectivity. ## Introduction XBTUSD is BitMEX's USD-denominated, BTC-margined inverse perpetual swap with a deep order book back to 2014. Tight spreads and predictable depth make it a natural venue for grid market making. ```mermaid flowchart LR subgraph Inputs ["Quote feed"] Q["BBO QuoteTick"] end subgraph Strategy ["GridMarketMaker"] M["mid = (bid + ask) / 2"] TH{{"|mid - last_quoted_mid|
>= requote_threshold_bps"}} CA["cancel_all_orders()"] SK["skew = skew_factor * net_position"] GR["Compute geometric grid:
buy_n = mid * (1 - bps/10000)^n - skew
sell_n = mid * (1 + bps/10000)^n - skew"] SUB["Submit GTC post-only
limit per level"] end subgraph Safety ["Deadman's switch"] T["Background task every timeout/4 seconds"] CAA["cancelAllAfter(timeout_ms)"] SRV["BitMEX server timer"] end Q --> M --> TH TH -->|yes| CA --> SK --> GR --> SUB TH -->|no| Q T --> CAA --> SRV ``` ### Why BitMEX for grid market making Two adapter features land cleanly on this strategy: 1. **Deadman's switch** (`cancelAllAfter`): BitMEX maintains a server-side cancel-all timer. The execution client refreshes it on a schedule. If the link drops and the timer expires, BitMEX cancels every open order on the account. 2. **Submit/cancel broadcaster**: the adapter can fan out submissions and cancellations across multiple HTTP connections in parallel, with the first successful response short-circuiting the rest. ### Deadman's switch mechanics When `deadmans_switch_timeout_secs` is set, a background task continuously refreshes the server-side timer at one-quarter of the timeout: ``` timeout = 60s -> refresh interval = timeout / 4 = 15s t=0s Strategy starts, cancelAllAfter(60000ms) sent t=15s Refresh: cancelAllAfter(60000ms) sent (resets timer) t=30s Refresh: cancelAllAfter(60000ms) sent t=45s Refresh: cancelAllAfter(60000ms) sent t=50s Connectivity lost (last refresh was at t=45s) t=105s Server timer fires -> BitMEX cancels all open orders ``` Stranded quotes are a serious risk for market making: a crashed client holding grid orders around mid can take an unbounded loss before manual intervention. The deadman's switch caps the exposure window at `timeout` seconds. ## Prerequisites - [NautilusTrader](https://pypi.org/project/nautilus_trader/) installed. - A Rust toolchain (`cargo`) for the live example. Install from [rustup.rs](https://rustup.rs/). - A BitMEX account: sign up at [bitmex.com](https://www.bitmex.com/) and generate an API key with order management permissions. Use the [BitMEX testnet](https://testnet.bitmex.com/) for first runs. ### Environment variables ```bash # Mainnet export BITMEX_API_KEY="your-api-key" export BITMEX_API_SECRET="your-api-secret" # Testnet export BITMEX_TESTNET_API_KEY="your-testnet-api-key" export BITMEX_TESTNET_API_SECRET="your-testnet-api-secret" ``` Alternatively place these in a `.env` file at the project root; both the Python and Rust paths load it via `dotenvy`. ## Backtesting with free Tardis quote data BitMEX does not expose historical L2 data via its own API beyond recent trades. [Tardis.dev](https://tardis.dev) captures and archives tick-level BitMEX data from March 2019 onward. The **first day of each month is free to download** without an API key. ### Download the data ```bash curl -L -o XBTUSD.csv.gz \ https://datasets.tardis.dev/v1/bitmex/quotes/2024/01/01/XBTUSD.csv.gz curl -L -o XBTUSD-trades.csv.gz \ https://datasets.tardis.dev/v1/bitmex/trades/2024/01/01/XBTUSD.csv.gz ``` The trades file is optional for the strategy but useful for the panels: the matching engine needs aggressor flow to lift maker orders, which the trades stream supplies. :::tip Full historical data (all dates) requires a paid Tardis API key. Use the [Tardis download utility](https://docs.tardis.dev/downloadable-csv-files) for bulk fetches. ::: ### Load the data `TardisCSVDataLoader` parses the `.csv.gz` files directly: ```python from nautilus_trader.adapters.tardis.loaders import TardisCSVDataLoader from nautilus_trader.model.identifiers import InstrumentId instrument_id = InstrumentId.from_str("XBTUSD.BITMEX") loader = TardisCSVDataLoader(instrument_id=instrument_id) quotes = loader.load_quotes("XBTUSD.csv.gz") trades = loader.load_trades("XBTUSD-trades.csv.gz") ``` The `instrument_id` argument tags every record as `XBTUSD.BITMEX` regardless of how it is keyed in the source CSV. ### Instrument definition XBTUSD is an **inverse perpetual**: prices are quoted in USD, but the contract is margined and settled in BTC. One contract represents 1 USD of notional exposure. ```python from decimal import Decimal from nautilus_trader.model.currencies import BTC from nautilus_trader.model.currencies import USD from nautilus_trader.model.enums import AssetClass from nautilus_trader.model.identifiers import Symbol from nautilus_trader.model.instruments import PerpetualContract from nautilus_trader.model.objects import Price from nautilus_trader.model.objects import Quantity XBTUSD = PerpetualContract( instrument_id=instrument_id, raw_symbol=Symbol("XBTUSD"), underlying="XBT", asset_class=AssetClass.CRYPTOCURRENCY, base_currency=BTC, quote_currency=USD, settlement_currency=BTC, is_inverse=True, price_precision=1, size_precision=0, price_increment=Price.from_str("0.5"), size_increment=Quantity.from_int(1), multiplier=Quantity.from_int(1), lot_size=Quantity.from_int(1), margin_init=Decimal("0.01"), margin_maint=Decimal("0.005"), maker_fee=Decimal("-0.00025"), taker_fee=Decimal("0.00075"), ts_event=0, ts_init=0, ) ``` Fee rates are explicit backtest assumptions. Check [bitmex.com/app/fees](https://www.bitmex.com/app/fees) for current rates. ### Backtest engine setup XBTUSD is BTC-margined, so the starting balance is in BTC: ```python from nautilus_trader.backtest.config import BacktestEngineConfig from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.config import LoggingConfig from nautilus_trader.model.enums import AccountType from nautilus_trader.model.enums import OmsType from nautilus_trader.model.identifiers import TraderId from nautilus_trader.model.identifiers import Venue from nautilus_trader.model.objects import Money engine = BacktestEngine( BacktestEngineConfig( trader_id=TraderId("BACKTESTER-001"), logging=LoggingConfig(log_level="INFO"), ), ) BITMEX = Venue("BITMEX") engine.add_venue( venue=BITMEX, oms_type=OmsType.NETTING, account_type=AccountType.MARGIN, base_currency=BTC, starting_balances=[Money(1, BTC)], ) engine.add_instrument(XBTUSD) engine.add_data(quotes + trades) ``` ### Strategy configuration ```python from nautilus_trader.examples.strategies.grid_market_maker import GridMarketMaker from nautilus_trader.examples.strategies.grid_market_maker import GridMarketMakerConfig strategy = GridMarketMaker( GridMarketMakerConfig( instrument_id=instrument_id, max_position=Quantity.from_int(300), trade_size=Quantity.from_int(100), num_levels=3, grid_step_bps=100, skew_factor=0.5, requote_threshold_bps=10, ), ) engine.add_strategy(strategy) ``` ### Run and review results ```python import pandas as pd engine.run() with pd.option_context("display.max_rows", 100, "display.max_columns", None, "display.width", 300): print(engine.trader.generate_account_report(BITMEX)) print(engine.trader.generate_order_fills_report()) print(engine.trader.generate_positions_report()) engine.reset() engine.dispose() ``` The complete backtest script is at [`bitmex_grid_market_maker.py`](https://github.com/nautechsystems/nautilus_trader/tree/develop/examples/backtest/bitmex_grid_market_maker.py). ### What the run produces The free 2024-01-01 sample is a quiet New Year's Day session: BTC traded in a roughly 200-USD range across most of the day. With the recommended-for-live `grid_step_bps=100` (1%) the inner buy and sell levels sit ~420 USD from mid and never get touched: the example completes with zero fills, which is the honest outcome on a calm session. The panels below use a tighter `grid_step_bps=20` configuration on the first 200,000 quotes so the maker fills are visible. With 20 bps step, two levels, and 20 bps requote threshold, the strategy fires 22 maker fills across the captured window. ![XBTUSD mid with theoretical grid bands and maker fills](./assets/grid_market_maker_bitmex/panel_a_grid_overlay.png) **Figure 1.** *XBTUSD mid (teal) with the four theoretical grid bands at `grid_step_bps=20`, `num_levels=2`. Triangles are maker fills, up = buy, down = sell.* ![Mid step distribution against requote threshold](./assets/grid_market_maker_bitmex/panel_b_requote_rate.png) **Figure 2.** *Maximum mid step (in basis points) per 5-minute bucket across the captured window. Buckets above the dashed line crossed the requote threshold at least once.* ![Net position trajectory](./assets/grid_market_maker_bitmex/panel_c_position.png) **Figure 3.** *Cumulative signed XBTUSD contracts across the maker fill sequence. Inventory skew pulls the grid back toward flat after each fill.* ![Deadman's switch timeline](./assets/grid_market_maker_bitmex/panel_d_deadman_timeline.png) **Figure 4.** *Server-side cancel-all timer with `timeout=60s`, `refresh_interval=15s`. Refreshes reset the timer to 60s. After connectivity fails at t=50s the timer drains uninterrupted; the server fires `cancelAll` at t=105s.* ### Regenerate the panels ```bash uv sync --extra visualization XBTUSD_QUOTES=XBTUSD.csv.gz XBTUSD_TRADES=XBTUSD-trades.csv.gz \ python3 docs/tutorials/assets/grid_market_maker_bitmex/render_panels.py ``` The renderer caps the dataset at 200,000 quotes and 30,000 trades to keep the run reproducible in a few minutes. ## Live trading: GridMarketMaker with deadman's switch After the backtest behaves, the same configuration runs live through the Rust `LiveNode`. The strategy is implemented natively in Rust. ### Environment setup Credentials load automatically from environment variables when not set explicitly in the config: ```bash # Testnet (recommended for first runs) export BITMEX_TESTNET_API_KEY="your-key" export BITMEX_TESTNET_API_SECRET="your-secret" ``` Or place them in a `.env` file at the project root. ### Code walkthrough The complete `main()` function is at [`node_grid_mm.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/bitmex/examples/node_grid_mm.rs): ```rust #[tokio::main] async fn main() -> Result<(), Box> { dotenvy::dotenv().ok(); let environment = Environment::Live; let trader_id = TraderId::from("TESTER-001"); let instrument_id = InstrumentId::from("XBTUSD.BITMEX"); let data_config = BitmexDataClientConfig { environment: BitmexEnvironment::Testnet, ..Default::default() }; let exec_config = BitmexExecFactoryConfig::new( trader_id, BitmexExecClientConfig { environment: BitmexEnvironment::Testnet, deadmans_switch_timeout_secs: Some(60), ..Default::default() }, ); let data_factory = BitmexDataClientFactory::new(); let exec_factory = BitmexExecutionClientFactory::new(); let log_config = LoggerConfig { stdout_level: LevelFilter::Info, ..Default::default() }; let mut node = LiveNode::builder(trader_id, environment)? .with_logging(log_config) .add_data_client(None, Box::new(data_factory), Box::new(data_config))? .add_exec_client(None, Box::new(exec_factory), Box::new(exec_config))? .with_reconciliation(true) .with_reconciliation_lookback_mins(2880) .with_delay_post_stop_secs(5) .build()?; let config = GridMarketMakerConfig::builder() .instrument_id(instrument_id) .max_position(Quantity::from("300")) .num_levels(3) .grid_step_bps(100) .skew_factor(0.5) .requote_threshold_bps(10) .build(); let strategy = GridMarketMaker::new(config); node.add_strategy(strategy)?; node.run().await?; Ok(()) } ``` Configuration points: - **`deadmans_switch_timeout_secs: Some(60)`**: arms the deadman's switch with a 60-second timeout and a 15-second refresh interval. - **`with_reconciliation(true)`**: queries the BitMEX REST API on startup to reload open orders and positions, so the strategy resumes correctly after a restart. - **`with_reconciliation_lookback_mins(2880)`**: reconciliation looks back 2880 minutes (two days) of order history. - **`with_delay_post_stop_secs(5)`**: 5-second grace period after stop for pending cancel and fill events to settle before the node exits. ### BitMEX-specific considerations #### GTC orders and post-only BitMEX grid orders are submitted as `GTC` with `ParticipateDoNotInitiate` (post-only). If the price has crossed the book by the time the order arrives, BitMEX rejects it rather than letting it take liquidity. This differs from the dYdX setup, where short-term orders provide automatic expiry every 8 seconds. On BitMEX the requote cycle is driven entirely by mid-price movement (`requote_threshold_bps`). #### Order quantization Price and size quantization for BitMEX instruments is handled automatically by the adapter. No manual rounding or conversion is needed in strategy code. #### Inverse perpetual accounting XBTUSD is BTC-margined. PnL accrues in BTC: a one-USD spread captured at a 42,000 USD price earns 1/42,000 BTC per fill. Size `max_position` and `trade_size` accordingly. ### Run the example ```bash cargo run --example bitmex-grid-mm --package nautilus-bitmex --features examples ``` ### Graceful shutdown Press **Ctrl+C** to stop the node. The shutdown sequence: 1. SIGINT received, trader stops, `on_stop()` fires. 2. Strategy cancels all orders and closes positions. 3. 5-second grace period (`delay_post_stop_secs`) processes residual events. 4. Deadman's switch background task stops. 5. Clients disconnect, node exits. ## Configuration reference ### GridMarketMaker parameters | Parameter | Type | Default | Description | | ----------------------- | -------------- | ---------- | ------------------------------------------------------------------------ | | `instrument_id` | `InstrumentId` | *required* | Instrument to trade (e.g. `XBTUSD.BITMEX`). | | `max_position` | `Quantity` | *required* | Maximum net exposure in contracts (long or short). | | `trade_size` | `Quantity` | `None` | Size per grid level. If `None`, uses instrument's `min_quantity` or 1.0. | | `num_levels` | `usize` | `3` | Number of buy and sell levels. | | `grid_step_bps` | `u32` | `10` | Grid spacing in basis points (100 = 1%). | | `skew_factor` | `f64` | `0.0` | How aggressively to shift the grid based on net inventory. | | `requote_threshold_bps` | `u32` | `5` | Minimum mid‑price move (bps) before re‑quoting. | | `expire_time_secs` | `Option` | `None` | Order expiry in seconds. Use `None` for GTC on BitMEX. | | `on_cancel_resubmit` | `bool` | `false` | Resubmit grid on next quote after an unexpected cancel. | ### Deadman's switch parameter | Parameter | Type | Description | | ------------------------------ | ------------- | ------------------------------------------------------------------------------------------------- | | `deadmans_switch_timeout_secs` | `Option` | Server‑side cancel timer in seconds. Refresh interval = `timeout / 4` (minimum 1s). `None` disables the feature. | A 60-second timeout gives a 15-second refresh interval and a 60-second window before BitMEX fires the timer. Lower values reduce the exposure window but increase API call frequency; higher values reduce overhead but extend the window. ### Choosing grid parameters - **`grid_step_bps`**: XBTUSD has tight spreads. Start at 50-100 bps to ensure fills before tightening. Each level captures half the step as spread. - **`skew_factor`**: start at `0.0` (no skew). A value of `0.5` shifts the grid by 0.5 USD per contract of net position. - **`requote_threshold_bps`**: 10 bps (0.1%) is a starting point for XBTUSD. Lower values cause cancel/replace churn; higher values leave orders stale during fast moves. ## Event flow ```mermaid flowchart TB A[LiveNode starts] --> B[connect: REST instruments + WebSocket channels] B --> C[deadman's switch task starts] B --> D[on_start subscribes to quotes] D --> E[on_quote] E --> F{should_requote?} F -->|no| E F -->|yes| G[cancel_all_orders] G --> H[compute grid with skew] H --> I[submit GTC post-only] I --> J[on_order_filled] I --> K[on_order_canceled] J --> E K --> E L[on_stop] --> M[cancel_all_orders + close positions] M --> N[deadman's switch task stops] ``` ## Monitoring and understanding output ### Key log messages | Log message | Meaning | | ----------------------------------------------------------------------- | -------------------------------------------------------- | | `Requoting grid: mid=X, last_mid=Y` | Mid moved beyond threshold, refreshing grid. | | `Starting dead man's switch: timeout=60s, refresh_interval=15s` | Deadman's switch armed at node start. | | `Dead man's switch heartbeat failed: ...` | Transient network issue; switch will retry next interval.| | `Disarming dead man's switch` | Switch stopped cleanly during shutdown. | | `benign cancel error, treating as success` | Cancel for an already‑filled or cancelled order (normal).| | `Reconciling orders from last 2880 minutes` | Startup reconciliation loading prior state. | ### Expected behaviour patterns 1. **Startup**: instruments load, reconciliation queries prior orders, WebSocket connects, first quote triggers initial grid. 2. **Steady state**: grid persists across ticks; requotes only when mid moves beyond threshold. 3. **Fills**: position updates, skew adjusts on next requote. 4. **Shutdown**: all orders cancelled, positions closed, deadman's switch stops. 5. **Restart**: reconciliation restores open order state; strategy resumes from prior grid. ## Customization tips ### High vs low volatility | Condition | Adjustment | | --------------- | ------------------------------------------------------------------------- | | High volatility | Wider `grid_step_bps` (100-200), fewer `num_levels`, lower `skew_factor`. | | Low volatility | Tighter `grid_step_bps` (20-50), more `num_levels`, higher `skew_factor`. | | Thin liquidity | Increase `requote_threshold_bps` to reduce cancel frequency. | ### Enabling the submit broadcaster For production deployments, enable the submit broadcaster to provide redundant order submission across multiple HTTP connections: ```rust let exec_config = BitmexExecFactoryConfig::new( trader_id, BitmexExecClientConfig { environment: BitmexEnvironment::Mainnet, deadmans_switch_timeout_secs: Some(60), submitter_pool_size: Some(2), canceller_pool_size: Some(2), ..Default::default() }, ); ``` With `submitter_pool_size=2`, each order submission fans out to two HTTP clients in parallel; the first successful response wins. This reduces the probability of a missed submission when one path stalls. ### Mainnet toggle Switch networks by setting the `environment` field on both configs to `BitmexEnvironment::Mainnet`. All endpoints and credential environment variables resolve automatically. ## Further reading - [BitMEX integration guide](../integrations/bitmex.md): full adapter reference. - [On-chain grid market making with short-term orders (dYdX)](./grid_market_maker_dydx.md): short-term order expiry as an alternative to the deadman's switch. - [Tardis downloadable CSV files](https://docs.tardis.dev/downloadable-csv-files): schema documentation for the Tardis archives. - [BitMEX API documentation](https://www.bitmex.com/app/apiOverview): `cancelAllAfter` endpoint and order management reference. # On-Chain Grid Market Making with Short-Term Orders (dYdX) Source: https://nautilustrader.io/docs/latest/tutorials/grid_market_maker_dydx/ This tutorial runs the shipped `GridMarketMaker` strategy on dYdX v4 through the Rust `LiveNode`. The strategy places symmetric limit orders around the mid, skews the grid to manage inventory, and lets the venue cycle short-term orders by time-to-block expiry instead of explicit cancels. ## Introduction A grid market maker maintains a ladder of resting buy and sell limits at fixed price intervals around the current mid. When an order fills, the strategy profits from the spread between the buy and sell levels. Inventory management keeps net exposure within `max_position` so the grid does not accumulate a directional position. ```mermaid flowchart LR subgraph Inputs ["Quote feed"] Q["BBO QuoteTick"] end subgraph Strategy ["GridMarketMaker"] M["mid = (bid + ask) / 2"] TH{{"|mid - last_mid|
>= requote_threshold_bps
OR no resting orders"}} CA["cancel_all_orders()"] SK["skew = skew_factor * net_position"] GR["Geometric grid:
buy_n = mid * (1 - bps/10000)^n - skew
sell_n = mid * (1 + bps/10000)^n - skew"] SUB["Submit GTD short-term limits
expire_time_secs = 8"] end subgraph Adapter ["dYdX execution adapter"] CL{{"expire_time_secs
< max_short_term_secs?"}} ST["Short-term path:
GoodTilBlock = current + N"] LT["Long-term path:
standard cancel-on-replace"] end Q --> M --> TH TH -->|yes| CA --> SK --> GR --> SUB TH -->|no| Q SUB --> CL CL -->|yes| ST CL -->|no| LT ``` ### Inventory skewing (Avellaneda-Stoikov inspired) When the position grows long the entire grid shifts down (cheaper buys, cheaper sells) to encourage the next fill on the sell side. When the position grows short the grid shifts up. This mirrors the Avellaneda-Stoikov framework adapted to a discrete grid. ### Why dYdX v4 dYdX v4 fits market-making well: - **Short-term orders** with ~20-second expiry: low-latency placement, no on-chain storage cost. - **~0.5-second block times** for fast confirmation cycles. - **No gas fees for cancellations**: short-term cancels are free under GTB replay protection. - **On-chain order book** with deterministic per-block matching. - **Batch cancel**: one `MsgBatchCancel` clears every short-term order. ## Prerequisites ### Funded dYdX account You need a dYdX account with USDC collateral. See the [Testnet setup](../integrations/dydx.md#testnet-setup) section in the integration guide for instructions on creating and funding a testnet account. The testnet wallet also needs an API trading key registered through the dYdX UI. ### Environment variables ```bash # Mainnet export DYDX_PRIVATE_KEY="0x..." export DYDX_WALLET_ADDRESS="dydx1..." # Testnet export DYDX_TESTNET_PRIVATE_KEY="0x..." export DYDX_TESTNET_WALLET_ADDRESS="dydx1..." ``` ## Strategy overview ### Geometric grid pricing Each level is a fixed percentage (basis points) away from mid: ``` Buy level N: mid * (1 - bps/10000)^N - skew Sell level N: mid * (1 + bps/10000)^N - skew ``` Where `skew = skew_factor * net_position`. For a 3-level grid with `grid_step_bps=100` (1%) around a mid of 1000.00: ``` Sell 3: 1030.30 Sell 2: 1020.10 Sell 1: 1010.00 ─── Mid: 1000.00 ─── Buy 1: 990.00 Buy 2: 980.10 Buy 3: 970.30 ``` With a long-2 position and `skew_factor=1.0`, the entire grid shifts down by 2.0: ``` Sell 3: 1028.30 Sell 2: 1018.10 Sell 1: 1008.00 ─── Mid: 1000.00 ─── Buy 1: 988.00 Buy 2: 978.10 Buy 3: 968.30 ``` ### Inventory management The strategy enforces position limits through two mechanisms: 1. **`max_position`**: a hard cap on net exposure (long or short). When the projected exposure from adding the next grid level would breach this cap, that level is skipped. 2. **Projected exposure tracking**: before placing each level the strategy tracks the worst-case per-side exposure (current position + all pending buy / sell orders) to avoid over-committing. `cancel_all_orders` is asynchronous, so pending orders may still fill between the cancel request and acknowledgement. Tracking worst-case per-side exposure prevents momentary over-exposure during cancel-requote transitions. ### Requote threshold `requote_threshold_bps` controls how much the mid must move before the strategy cancels all open orders and places a fresh grid: - **Lower threshold** (5 bps): more responsive, more cancel/place transactions. - **Higher threshold** (50 bps): fewer transactions, but orders may sit further from the current price. ## Configuration | Parameter | Type | Default | Description | | ----------------------- | -------------- | ---------- | ------------------------------------------------------------------------ | | `instrument_id` | `InstrumentId` | *required* | Instrument to trade (e.g. `ETH-USD-PERP.DYDX`). | | `max_position` | `Quantity` | *required* | Maximum net exposure (long or short). | | `trade_size` | `Quantity` | `None` | Size per grid level. If `None`, uses instrument's `min_quantity` or 1.0. | | `num_levels` | `usize` | `3` | Number of buy and sell levels. | | `grid_step_bps` | `u32` | `10` | Grid spacing in basis points (10 = 0.1%). | | `skew_factor` | `f64` | `0.0` | How aggressively to shift the grid based on inventory. | | `requote_threshold_bps` | `u32` | `5` | Minimum mid‑price move in bps before re‑quoting. | | `expire_time_secs` | `Option` | `None` | Order expiry in seconds. Uses GTD when set, GTC otherwise. | | `on_cancel_resubmit` | `bool` | `false` | Resubmit grid on next quote after an unexpected cancel. | ### Choosing parameters - **`grid_step_bps`**: 50-100 bps in volatile markets, 5-20 bps in calm conditions. Wider grids capture more spread per fill but fill less often. - **`skew_factor`**: start at `0.0`. A value of `0.5` shifts the grid by 0.5 price units per unit of net position. Too aggressive a skew can move the grid entirely above or below mid. - **`expire_time_secs`**: for dYdX short-term orders, set to `8` seconds. That fits inside the 40-block (~20 s) short-term window and keeps the orders on the fast short-term path. When `None`, orders use GTC and the long-term path. - **`on_cancel_resubmit`**: triggers a resubmission on the next quote tick after a cancel that the strategy did not initiate (short-term order expiry from the indexer, self-trade prevention, risk limits). The indexer emits a cancel event for each short-term order shortly after it expires; this flag resets the requote anchor so the next quote rebuilds the grid even if the mid has not moved beyond `requote_threshold_bps`. ## dYdX-specific considerations ### Short-term order expiry When `expire_time_secs=8`, orders are classified as short-term by the adapter: 1. The adapter checks `8s < max_short_term_secs (40 blocks * ~0.5s = ~20s)`. 2. The order is submitted as short-term with `GoodTilBlock = current_height + N`. 3. The order expires on chain after about eight seconds if not filled. Expiry costs no gas (GTB replay protection handles it on chain), but the indexer still emits an `OrderCanceled` event for each expired order shortly after the expiry block, so the strategy observes the expiry through the normal cancel event path. This is the recommended configuration for market making because: - Short-term orders have lower latency. - Expiry has no on-chain gas cost. - Continuous requoting (driven by the indexer-emitted cancel events when `on_cancel_resubmit=true`) replaces expired orders. See the [order classification](../integrations/dydx.md#order-classification) section in the integration guide for full details. ### Unexpected cancels and `on_cancel_resubmit` The `pending_self_cancels` set distinguishes self-initiated from unexpected cancels: 1. When the strategy calls `cancel_all_orders`, it records all open order IDs in `pending_self_cancels`. 2. When `on_order_canceled` fires: - If the order ID is in `pending_self_cancels`, it is a self-cancel and no action is needed. - Otherwise it was not strategy-initiated (short-term order expiry, self-trade prevention, or a risk limit). Reset `last_quoted_mid` so the next quote triggers a full grid resubmission. This stops the strategy re-quoting unnecessarily during its own cancel waves while still responding to surprises. `on_order_filled` also removes the order from `pending_self_cancels`. If an order fills before the cancel acknowledgement arrives, this prevents stale entries from accumulating. ### Order quantization Price and size quantization for dYdX markets is handled automatically by the adapter's `OrderMessageBuilder`. No manual rounding or conversion is needed. See [Price and size quantization](../integrations/dydx.md#price-and-size-quantization) for details. ### Post-only orders All grid orders are submitted with `post_only=true`. The exchange rejects any order that would cross the spread at match time, so every fill lands at the maker fee rate and the grid never inadvertently lifts its own offers during requote transitions. ## Running and stopping ### Environment setup Credentials load from environment variables or a `.env` file at the project root (loaded automatically via `dotenvy`): ```bash # Direct export export DYDX_PRIVATE_KEY="0x..." export DYDX_WALLET_ADDRESS="dydx1..." ``` ```bash # .env equivalent DYDX_PRIVATE_KEY=0x... DYDX_WALLET_ADDRESS=dydx1... ``` ### Run the example ```bash cargo run --example dydx-grid-mm --package nautilus-dydx --features examples ``` The example targets mainnet. To run against testnet, set the `DYDX_NETWORK` constant near the top of the example to `DydxNetwork::Testnet` (this needs a testnet API trading key) and rebuild. ### Graceful shutdown Press **Ctrl+C** to stop the node. The shutdown sequence: 1. SIGINT received, trader stops, `on_stop` fires. 2. Strategy cancels all orders and closes positions. 3. 5-second grace period (`delay_post_stop_secs`) processes residual events. 4. Clients disconnect, node exits. ## Code walkthrough The `main` function lives at [`crates/adapters/dydx/examples/node_grid_mm.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/dydx/examples/node_grid_mm.rs): ```rust const DYDX_NETWORK: DydxNetwork = DydxNetwork::Mainnet; #[tokio::main] async fn main() -> Result<(), Box> { dotenvy::dotenv().ok(); let network = DYDX_NETWORK; let environment = Environment::Live; let trader_id = TraderId::from("TESTER-001"); let account_id = AccountId::from("DYDX-001"); let node_name = "DYDX-GRID-MM-001".to_string(); let instrument_id = InstrumentId::from("ETH-USD-PERP.DYDX"); let data_config = DydxDataClientConfig { network, ..Default::default() }; let exec_config = DydxExecClientConfig { trader_id, account_id, network, ..Default::default() }; let data_factory = DydxDataClientFactory::new(); let exec_factory = DydxExecutionClientFactory::new(); let log_config = LoggerConfig { stdout_level: LevelFilter::Info, ..Default::default() }; let mut node = LiveNode::builder(trader_id, environment)? .with_name(node_name) .with_logging(log_config) .add_data_client(None, Box::new(data_factory), Box::new(data_config))? .add_exec_client(None, Box::new(exec_factory), Box::new(exec_config))? .with_reconciliation(false) .with_delay_post_stop_secs(5) .build()?; let config = GridMarketMakerConfig::builder() .instrument_id(instrument_id) .max_position(Quantity::from("0.10")) .num_levels(3) .grid_step_bps(100) .skew_factor(0.5) .requote_threshold_bps(10) .expire_time_secs(8) .on_cancel_resubmit(true) .build(); let strategy = GridMarketMaker::new(config); node.add_strategy(strategy)?; node.run().await?; Ok(()) } ``` Configuration points: - **`dotenvy::dotenv().ok()`**: loads `.env` from the project root if present. - **`with_reconciliation(false)`**: disabled for simplicity; enable in production to resume state across restarts. - **`with_delay_post_stop_secs(5)`**: grace period for pending cancel and close events to finalize during shutdown. ### Event flow ```mermaid flowchart TB A[LiveNode starts] --> B[connect: HTTP instruments + WebSocket channels] B --> C[on_start subscribes to quotes] C --> D[on_quote] D --> E{should_requote?} E -->|no| D E -->|yes| F[cancel_all_orders] F --> G[compute grid with skew] G --> H[submit GTD short-term limits] H --> I[on_order_filled] H --> J[on_order_canceled] I --> D J --> D K[on_stop] --> L[cancel_all_orders + close positions] ``` ## Strategy internals The key Rust snippets from `grid_mm.rs` follow. ### Trade size resolution (`on_start`) Trade size resolves from the instrument cache: config value first, then the instrument's `min_quantity`, then `1.0` as a final fallback. ```rust fn on_start(&mut self) -> anyhow::Result<()> { let instrument_id = self.config.instrument_id; let (instrument, size_precision, min_quantity) = { let cache = self.cache(); let instrument = cache .instrument(&instrument_id) .ok_or_else(|| anyhow::anyhow!("Instrument {instrument_id} not found in cache"))?; ( instrument.clone(), instrument.size_precision(), instrument.min_quantity(), ) }; self.price_precision = Some(instrument.price_precision()); self.instrument = Some(instrument); if self.trade_size.is_none() { self.trade_size = Some(min_quantity.unwrap_or_else(|| Quantity::new(1.0, size_precision))); } self.subscribe_quotes(instrument_id, None, None); Ok(()) } ``` ### Quote handler (`on_quote`, abbreviated) ```rust fn on_quote(&mut self, quote: &QuoteTick) -> anyhow::Result<()> { let mid_f64 = (quote.bid_price.as_f64() + quote.ask_price.as_f64()) / 2.0; let mid = Price::new( mid_f64, self.price_precision .expect("price_precision should be resolved in on_start"), ); if !self.should_requote(mid) { return Ok(()); // Mid hasn't moved enough, keep existing grid } self.cancel_all_orders(instrument_id, None, None, None)?; let (net_position, worst_long, worst_short) = { /* ... */ }; let grid = self.grid_orders(mid, net_position, worst_long, worst_short); if grid.is_empty() { return Ok(()); // Don't advance requote anchor when fully constrained } let (tif, expire_time) = match self.config.expire_time_secs { Some(secs) => { let now_ns = self.clock().timestamp_ns(); let expire_ns = now_ns + secs * 1_000_000_000; (Some(TimeInForce::Gtd), Some(expire_ns)) } None => (None, None), }; for (side, price) in grid { let order = self.order().limit( instrument_id, side, trade_size, price, tif, expire_time, Some(true), // post_only ); self.submit_order(order, None, None)?; } self.last_quoted_mid = Some(mid); Ok(()) } ``` ### Grid pricing (`grid_orders`) Computes geometric grid prices and enforces `max_position` per level: ```rust fn grid_orders( &self, mid: Price, net_position: f64, worst_long: Decimal, worst_short: Decimal, ) -> Vec<(OrderSide, Price)> { let instrument = self .instrument .as_ref() .expect("instrument should be resolved in on_start"); let mid_f64 = mid.as_f64(); let skew_f64 = self.config.skew_factor * net_position; let pct = self.config.grid_step_bps as f64 / 10_000.0; let trade_size = self .trade_size .expect("trade_size should be resolved in on_start") .as_decimal(); let max_pos = self.config.max_position.as_decimal(); let mut projected_long = worst_long; let mut projected_short = worst_short; let mut orders = Vec::new(); for level in 1..=self.config.num_levels { let buy_f64 = mid_f64 * (1.0 - pct).powi(level as i32) - skew_f64; let sell_f64 = mid_f64 * (1.0 + pct).powi(level as i32) - skew_f64; let buy_price = instrument.next_bid_price(buy_f64, 0); let sell_price = instrument.next_ask_price(sell_f64, 0); if let Some(buy_price) = buy_price && projected_long + trade_size <= max_pos { orders.push((OrderSide::Buy, buy_price)); projected_long += trade_size; } if let Some(sell_price) = sell_price && projected_short - trade_size >= -max_pos { orders.push((OrderSide::Sell, sell_price)); projected_short -= trade_size; } } orders } ``` ## What a 35-second mainnet run produces A 35-second mainnet run on `ETH-USD-PERP.DYDX` with the example config (`grid_step_bps=100`, `num_levels=3`, `skew_factor=0.5`, `requote_threshold_bps=10`, `expire_time_secs=8`) captures 47 requote events, 276 order submissions, 67 accepts, and 54 cancels. ETH was trading near 2,281 USD: the price never moved enough to trip the 10 bps requote threshold, so most cycles trigger from the periodic 8-second short-term order expiry rather than from price movement. ![ETH-USD-PERP mid with theoretical grid bands](./assets/grid_market_maker_dydx/panel_a_grid_overlay.png) **Figure 1.** *ETH-USD-PERP mid at every requote with the six theoretical grid bands (3 levels each side, 100 bps step). Mid sits near 2,281 USD; the inner buy and sell levels are at ~2,258 and ~2,304 USD.* ![Order lifetime distribution](./assets/grid_market_maker_dydx/panel_b_order_lifetime.png) **Figure 2.** *Time from `OrderAccepted` to `OrderCanceled` per short-term order, in seconds. The mass near 7-8 seconds matches the `expire_time_secs=8` setting; the smaller cluster below 6 seconds is strategy-initiated cancels during requote transitions.* ![Orders submitted per 250-ms bucket](./assets/grid_market_maker_dydx/panel_c_orders_per_cycle.png) **Figure 3.** *Order submission count per 250-ms bucket, split by side. Each requote cycle places six orders (3 buys + 3 sells); the spacing between bursts is the requote interval.* ![Short-term order timeline](./assets/grid_market_maker_dydx/panel_d_short_term_timeline.png) **Figure 4.** *Theoretical short-term order timeline with `expire_time_secs=8` and 0.5-second blocks. The bottom panel tracks how the chain block height advances; each order's `GoodTilBlock` target is set to ~16 blocks ahead, giving the eight-second expiry.* ### Regenerate the panels ```bash # Capture a 35-second mainnet run. timeout 35 ./target/release/examples/dydx-grid-mm > /tmp/dydx_main.log 2>&1 uv sync --extra visualization DYDX_LOG=/tmp/dydx_main.log \ python3 docs/tutorials/assets/grid_market_maker_dydx/render_panels.py ``` ## Monitoring and understanding output ### Key log messages | Log message | Meaning | | --------------------------------------------------- | -------------------------------------------------- | | `Requoting grid: mid=X, last_mid=Y` | Mid moved beyond threshold, refreshing grid. | | `Submit short‑term order N` | Order submitted via short‑term broadcast path. | | `BatchCancel N short-term orders` | Batch cancel executed for expired/stale orders. | | `benign cancel error, treating as success` | Cancel for an already‑filled or expired order (normal). | | `Sequence mismatch detected, will resync and retry` | Cosmos SDK sequence error, auto‑recovering. | ### Expected behaviour patterns 1. **Startup**: instruments load, WebSocket connects, first quote triggers initial grid. 2. **Steady state**: grid persists across ticks; requotes only when mid moves more than `requote_threshold_bps`. 3. **Fills**: position updates, skew adjusts, the next requote shifts the grid. 4. **Expiry**: short-term orders expire on chain after about eight seconds; the indexer emits a cancel event for each, and the next quote refreshes the grid. 5. **Shutdown**: all orders cancelled, positions closed, WebSocket disconnected. ## Customization tips ### High vs low volatility | Condition | Adjustment | | --------------- | ------------------------------------------------------------------------ | | High volatility | Wider `grid_step_bps` (100-200), fewer `num_levels`, lower `skew_factor`.| | Low volatility | Tighter `grid_step_bps` (10-30), more `num_levels`, higher `skew_factor`.| | Thin liquidity | Increase `requote_threshold_bps` to reduce cancel frequency. | ### Multiple instruments Run separate `GridMarketMaker` instances per instrument. Each instance manages its own grid, position, and cancel state independently: ```rust let btc_config = GridMarketMakerConfig::builder() .instrument_id(InstrumentId::from("BTC-USD-PERP.DYDX")) .max_position(Quantity::from("0.001")) .base( StrategyConfig::builder() .strategy_id(StrategyId::from("GRID_MM-BTC")) .order_id_tag("BTC".to_string()) .build(), ) .grid_step_bps(50) .build(); let eth_config = GridMarketMakerConfig::builder() .instrument_id(InstrumentId::from("ETH-USD-PERP.DYDX")) .max_position(Quantity::from("0.10")) .base( StrategyConfig::builder() .strategy_id(StrategyId::from("GRID_MM-ETH")) .order_id_tag("ETH".to_string()) .build(), ) .grid_step_bps(100) .build(); node.add_strategy(GridMarketMaker::new(btc_config))?; node.add_strategy(GridMarketMaker::new(eth_config))?; ``` ### Mainnet vs testnet toggle The example selects the network from the `DYDX_NETWORK` constant near the top of the file (`DydxNetwork::Mainnet` by default). Change it to `DydxNetwork::Testnet` and rebuild to run against testnet. ## Further reading - [dYdX v4 Integration Guide](../integrations/dydx.md): full adapter reference. - [dYdX Protocol Documentation](https://docs.dydx.xyz/): official protocol docs. - [Order types](https://docs.dydx.xyz/concepts/trading/orders): protocol-level order mechanics. - [Grid Market Making with a Deadman's Switch (BitMEX)](./grid_market_maker_bitmex.md): comparison with deadman's switch as an alternative to short-term order expiry. # Hurst/VPIN Directional Strategy (Kraken Futures) Source: https://nautilustrader.io/docs/latest/tutorials/hurst_vpin_kraken/ :::note This is a **Rust-only** tutorial. The strategy, backtest wiring, and tests all live in the compiled core. ::: This tutorial backtests a directional strategy on **PF_XBTUSD**, the USD-margined Bitcoin perpetual on [Kraken Futures](https://futures.kraken.com). The strategy combines a **Hurst-exponent regime filter** with a **VPIN** (Volume-synchronized Probability of Informed Trading) flow signal. Historical trades and quotes come from [Tardis.dev](https://tardis.dev) and replay through the Rust `BacktestEngine`. ## Introduction The strategy combines three components: a **slow regime filter** derived from bars, a **fast informed-flow signal** derived from trades, and a **quote-driven entry** that fires only when both align. - **Hurst exponent on dollar bars.** Sampled on constant-notional (value) bars, following Lopez de Prado (*Advances in Financial Machine Learning*, Chapter 2). A rescaled-range (R/S) estimate above `0.55` indicates persistent, trending behavior; below `0.50` the series is mean-reverting or noise. - **VPIN from trade aggressor flow.** Each completed dollar bar is treated as one volume bucket. The absolute imbalance between aggressive buy and aggressive sell volume, averaged over the last fifty buckets, gives the VPIN level. The signed imbalance gives the net informed direction. - **Quote-driven entry.** Once both signals agree, the strategy opens a position on the next quote tick. Entry timing stays tied to the live top of book rather than to bar closes. Exit is driven by the same ingredients: position is closed when the Hurst estimate decays back through a lower threshold, or when a holding-time cap is reached. The strategy is shipped as [`HurstVpinDirectional`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/hurst_vpin_directional) in the `nautilus_trading::examples::strategies` module. As with all shipped example strategies, it is intentionally simple and has no alpha advantage. ### Why Kraken Futures Kraken Futures lists perpetuals on Bitcoin and Ether in two forms: - **`PI_` inverse perpetuals**: quoted in USD, margined and settled in the underlying. - **`PF_` linear perpetuals**: quoted in USD, margined and settled in USD (via multi-collateral). The tutorial uses **`PF_XBTUSD`** so the account currency, quote currency, and dollar-bar sampling frame are all USD. ### Why dollar bars and VPIN together VPIN is defined on *volume* buckets rather than *time* buckets. Dollar bars (`VALUE` aggregation in NautilusTrader) close after a fixed notional has traded, so the sampling frame adapts to market activity. Defining each VPIN bucket as one dollar bar keeps both signals on the same clock, and Hurst sampled on the same bars uses the same frame. ## Prerequisites - A working Rust toolchain (see [rustup.rs](https://rustup.rs)). - The NautilusTrader repository cloned and building. - Internet access to download a free Tardis sample (no API key required for the first day of each month). ## Data preparation Kraken Futures is published by Tardis under the historical slug `cryptofacilities`. The first day of each month is available for free without an API key, which is enough for a single-day plumbing check. A full backtest needs at least two sessions to warm the 128-bar Hurst window, which requires a paid Tardis API key (see the tip below). ```bash mkdir -p /tmp/tardis_kraken curl -L -o /tmp/tardis_kraken/PF_XBTUSD_trades.csv.gz \ https://datasets.tardis.dev/v1/cryptofacilities/trades/2024/01/01/PF_XBTUSD.csv.gz curl -L -o /tmp/tardis_kraken/PF_XBTUSD_quotes.csv.gz \ https://datasets.tardis.dev/v1/cryptofacilities/quotes/2024/01/01/PF_XBTUSD.csv.gz ``` The runnable example binary shown later in this tutorial reads from `/tmp/tardis_kraken/` by default, so downloading into that directory up front means `cargo run` works without needing `KRAKEN_TRADES` or `KRAKEN_QUOTES` overrides. :::tip Full historical ranges require a paid Tardis API key. Use the [Tardis download utility](https://docs.tardis.dev/downloadable-csv-files) for bulk fetches once you move beyond single-day samples. ::: The Rust Tardis loader parses `.csv.gz` directly and tags each record with the instrument ID we supply, so no symbology mapping is needed at the strategy level: ```rust use nautilus_model::identifiers::InstrumentId; use nautilus_tardis::csv::load::{load_quotes, load_trades}; let instrument_id = InstrumentId::from("PF_XBTUSD.KRAKEN"); let trades = load_trades( "PF_XBTUSD_trades.csv.gz", Some(1), // price_precision Some(4), // size_precision Some(instrument_id), None, // limit )?; let quotes = load_quotes( "PF_XBTUSD_quotes.csv.gz", Some(1), Some(4), Some(instrument_id), None, )?; ``` Pass the instrument's `price_precision` and `size_precision` explicitly. The loader otherwise infers precision from the first few records, and a sample day without fractional prices can infer `0`, which the matching engine will reject when it sees a quote tick that does not match the instrument's declared precision. ## Instrument definition Since we are loading CSV data directly rather than through the live Kraken adapter, we define `PF_XBTUSD` manually as a [`CryptoPerpetual`](https://github.com/nautechsystems/nautilus_trader/blob/develop/crates/model/src/instruments/crypto_perpetual.rs). Linear perpetuals on Kraken Futures are quoted and margined in USD: ```rust use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::CryptoPerpetual, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; let instrument = CryptoPerpetual::builder() .instrument_id(InstrumentId::from("PF_XBTUSD.KRAKEN")) .raw_symbol(Symbol::from("PF_XBTUSD")) .base_currency(Currency::BTC()) // base .quote_currency(Currency::USD()) // quote .settlement_currency(Currency::USD()) // settlement (linear) .is_inverse(false) .price_precision(1) .size_precision(4) .price_increment(Price::from("0.5")) .size_increment(Quantity::from("0.0001")) .margin_init(dec!(0.02)) .margin_maint(dec!(0.01)) .maker_fee(dec!(0.0002)) .taker_fee(dec!(0.0005)) .ts_event(0.into()) .ts_init(0.into()) .build() .unwrap(); ``` Fees and margin are explicit backtest assumptions. Check the [Kraken Futures fee schedule](https://futures.kraken.com/features/fee-schedule) for current rates. ## Dollar-bar sampling NautilusTrader ships all the information-driven bar aggregators from AFML Chapter 2: tick, volume, value (dollar), plus imbalance and runs variants for each. We use plain `VALUE` bars here, which close after a fixed notional has traded on the tape. The bar type is expressed as a string. The `INTERNAL` suffix tells the engine to aggregate inside NautilusTrader from the underlying trade stream (price type `LAST`): ```rust use nautilus_model::data::BarType; let bar_type = BarType::from("PF_XBTUSD.KRAKEN-2000000-VALUE-LAST-INTERNAL"); ``` Each bar closes after **USD 2,000,000** of traded notional. A session typically prints under 150 bars at this size, short of the 128-bar Hurst window, so warming the defaults needs multiple sessions. For a single-day run, either shrink the bar size to **USD 500,000** or drop `hurst_window` and `vpin_window` accordingly. Full multi-day backtests should use the defaults. :::note `VALUE` bars are a *view* onto the trade tape. The backtest engine consumes the same stream that drives VPIN, so there is no double counting. ::: ## Strategy overview The `HurstVpinDirectional` strategy runs three concurrent pipelines that synchronize at bar close: trades feed a bucket accumulator, bar close triggers signal recomputation, and quotes drive entry and timeout checks. ```mermaid flowchart LR subgraph Inputs ["Data streams"] T["TradeTick"] B["Value bar close
(USD 2M notional)"] Q["QuoteTick"] end subgraph State ["Rolling state"] BV["Bucket buy / sell volume"] RET["Log-return window"] IMB["Imbalance window"] end subgraph Signals ["Signals"] H(("Hurst")) V(("VPIN + signed")) end subgraph Gates ["Decision gates"] E{"Flat AND
Hurst >= 0.55 AND
VPIN >= 0.30"} R{"Open AND
Hurst < 0.50"} X{"Open AND
held > max_holding_secs"} end subgraph Orders ["Orders"] Op["Market IOC
side = sign(signed VPIN)"] Cl["Close position"] end T -->|aggressor| BV B -->|log return| RET BV -.->|snapshot + reset
on bar close| IMB RET --> H IMB --> V H --> E V --> E Q -->|tick| E E -->|yes| Op H --> R R -->|yes| Cl Q -->|tick| X X -->|yes| Cl ``` 1. **Per trade**: accumulate aggressive buy and aggressive sell volume for the current dollar-bar bucket using `TradeTick::aggressor_side`. 2. **Per bar** (bucket close): append the bar's log return to the Hurst window, compute the bucket's signed and absolute imbalance, reset the accumulators, re-estimate rolling Hurst and VPIN, clear `exit_cooldown`, and check regime exit. 3. **Per quote**: if flat and both signals agree (Hurst trending, VPIN above threshold, signed imbalance non-zero), open a market IOC order. If already positioned, check the holding-time cap. Regime exit fires from the bar pipeline when Hurst drops below `hurst_exit`. Holding timeout fires from the quote pipeline when the position has been open longer than `max_holding_secs`. ![Signal dashboard during an active trading window](./assets/hurst_vpin_kraken/panel_b_dashboard.png) **Figure 1.** *Signal dashboard for 2024-01-16 14:09-16:15 UTC: close, Hurst, VPIN. Markers sit at actual fill price; dotted connector shows slip against the bar-close line.* ### Hurst estimator The strategy uses classical rescaled-range (R/S) regression. For each lag `k` in `(4, 8, 16, 32)`, the return window is split into non-overlapping chunks of length `k`, each chunk's rescaled range is computed, and the mean R/S is recorded. The slope of `log(R/S)` vs `log(k)` across the lag set gives the Hurst estimate. ![Hurst exponent over the full backtest](./assets/hurst_vpin_kraken/panel_e_hurst_only.png) **Figure 2.** *Rolling Hurst across 14 days of PF_XBTUSD (2024-01-15 to 2024-01-28) with enter 0.55 and exit 0.50 thresholds.* ### VPIN estimator With explicit trade aggressor side available from the venue feed, VPIN collapses to ``` VPIN = mean_k ( |V_B_k - V_S_k| / (V_B_k + V_S_k) ) ``` over the last `k` completed dollar-bar buckets. The signed variant retains the sign of `V_B - V_S` and is used to choose direction. This is more accurate than the bulk-volume classification used in the original Easley/Lopez de Prado formulation, which was only necessary when aggressor side was not directly observable. ![VPIN distribution across the backtest](./assets/hurst_vpin_kraken/panel_d_vpin_hist.png) **Figure 3.** *VPIN distribution across all bars with the 0.30 entry threshold.* ### Configuration | Parameter | Value | Description | | ------------------ | ---------------- | ---------------------------------------------------- | | `bar_type` | `2M-VALUE-LAST` | Dollar bars closing every USD 2,000,000 of notional. | | `trade_size` | `0.0100` | 0.0100 XBT per trade (matches instrument precision). | | `hurst_window` | `128` | Rolling window of dollar bar log returns. | | `hurst_lags` | `[4, 8, 16, 32]` | Lag set used in the R/S regression. | | `hurst_enter` | `0.55` | Above this, the regime is treated as trending. | | `hurst_exit` | `0.50` | Below this, open positions are flattened. | | `vpin_window` | `50` | Completed volume buckets averaged for VPIN. | | `vpin_threshold` | `0.30` | Minimum VPIN for flow to be considered informed. | | `max_holding_secs` | `1800` | Seconds a position may be held (default `3600`; overridden here). | :::tip Dollar-bar size, Hurst lags, and VPIN window are all coupled. Smaller bars give faster reaction but noisier Hurst; larger bars smooth both signals but risk too few samples in a single-day backtest. ::: ## Backtest setup Configure a `BacktestEngine` with a Kraken venue and a USD starting balance: ```rust use nautilus_backtest::{ config::{BacktestEngineConfig, SimulatedVenueConfig}, engine::BacktestEngine, }; use nautilus_model::{ data::Data, enums::{AccountType, BookType, OmsType}, identifiers::Venue, instruments::{Instrument, InstrumentAny}, types::Money, }; let mut engine = BacktestEngine::new(BacktestEngineConfig::default())?; engine.add_venue( SimulatedVenueConfig::builder() .venue(Venue::from("KRAKEN")) .oms_type(OmsType::Netting) .account_type(AccountType::Margin) .book_type(BookType::L1_MBP) .starting_balances(vec![Money::from("100_000 USD")]) .build()?, )?; engine.add_instrument(&InstrumentAny::CryptoPerpetual(instrument))?; ``` Feed the loaded trades and quotes as `Data` enum variants: ```rust let mut data: Vec = trades.into_iter().map(Data::Trade).collect(); data.extend(quotes.into_iter().map(Data::Quote)); engine.add_data(data, None, true, true)?; ``` ### Add the strategy ```rust use nautilus_model::types::Quantity; use nautilus_trading::examples::strategies::{ HurstVpinDirectional, HurstVpinDirectionalConfig, }; let config = HurstVpinDirectionalConfig::builder() .instrument_id(instrument_id) .bar_type(bar_type) .trade_size(Quantity::from("0.0100")) // match instrument size_precision .max_holding_secs(1800) .build(); engine.add_strategy(HurstVpinDirectional::new(config))?; ``` ### Run the backtest ```rust engine.run(None, None, None, false)?; ``` With the default 128/50 windows and USD 2,000,000 bars, a single-day sample stays in warmup for the whole run. The run will show the engine aggregating dollar bars and driving the strategy at trade and quote granularity, but Hurst and VPIN will not emit until the windows are full. To see signal updates and entry/exit logic fire on a single day, shrink the bar size or windows as noted above, or provide two or more sessions of data. A 14-day run (2024-01-15 to 2024-01-28) on this configuration prints 1,224 bars and fires only 2 entries, 1 partial cover, and 1 close. Entries are sparse by design: both `hurst_enter` and `vpin_threshold` must clear on the same quote. The residual size after the partial IOC cover stays open through the rest of the session, which is how a Netting OMS resolves an exit IOC that only fills part of the resting position. The wiring above is shipped as a runnable binary: ```bash cargo run -p nautilus-kraken --features examples \ --example kraken-hurst-vpin-backtest --release ``` By default it reads `PF_XBTUSD_trades.csv.gz` and `PF_XBTUSD_quotes.csv.gz` from `/tmp/tardis_kraken/`. Override with `KRAKEN_TRADES` and `KRAKEN_QUOTES` environment variables. ![Trade detail during the active window](./assets/hurst_vpin_kraken/panel_a_price_regime.png) **Figure 4.** *Close price for 2024-01-16 14:09-16:15 UTC. Teal bands mark `Hurst >= 0.55` bars; gold bands mark periods with an open position. Markers sit at actual fill price; dotted connector shows slip against the bar-close line.* ![Decision space across every bar](./assets/hurst_vpin_kraken/panel_c_decision_scatter.png) **Figure 5.** *Per-bar Hurst vs. VPIN across the backtest, colored by signed VPIN. Shaded quadrant marks the entry-eligible region.* ### Regenerate the panels The backtest strategy logs `Hurst=… VPIN=… signed=… bar_close=…` on every bar close and standard `OrderFilled` events on entries and exits, so the panels above are fully reproducible from the run's stdout: ```bash RUST_LOG=info cargo run -p nautilus-kraken --features examples \ --example kraken-hurst-vpin-backtest --release > /tmp/backtest.log 2>&1 uv sync --extra visualization BACKTEST_LOG=/tmp/backtest.log \ python3 docs/tutorials/assets/hurst_vpin_kraken/render_panels.py ``` The renderer uses the shared `nautilus_dark` tearsheet theme and writes static PNGs via Plotly's Kaleido exporter. ## Next steps - **Tune the sampling frame**. Try larger or smaller dollar-bar thresholds. The `VALUE_IMBALANCE` and `VALUE_RUNS` aggregators produce bars that close on information arrival itself, which may be an interesting substitute for constant-dollar sampling. - **Tighten the thresholds**. `hurst_enter`, `hurst_exit`, and `vpin_threshold` all interact: a higher enter threshold makes signals rarer but more specific; a tighter exit shortens average holding time. - **Add a volatility gate**. Overlay a realized-volatility estimator on the same bars to suppress entries during clearly chaotic sessions. - **Go live on Kraken Futures demo**. Once the backtest behaves, drive the same strategy through the Kraken live client factories against [demo-futures.kraken.com](https://demo-futures.kraken.com). A runnable live wiring ships as: ```bash cargo run -p nautilus-kraken --features examples \ --example kraken-hurst-vpin-live ``` Set `KRAKEN_FUTURES_API_KEY` and `KRAKEN_FUTURES_API_SECRET` in the environment before running. ## Further reading - [`HurstVpinDirectional` strategy source](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/trading/src/examples/strategies/hurst_vpin_directional) - [Data concepts: bar types and aggregation](../concepts/data.md) - [Tardis integration guide](../integrations/tardis.md) - [Kraken integration guide](../integrations/kraken.md) - [Kraken Futures documentation](https://docs.kraken.com/api/docs/futures-api) - Lopez de Prado, M. (2018). *Advances in Financial Machine Learning*, Wiley. Chapter 2 (information-driven bars) and Chapter 19 (VPIN). # Tutorials Source: https://nautilustrader.io/docs/latest/tutorials/ Step-by-step walkthroughs demonstrating specific features and workflows. :::info Most Python tutorials are Jupytext percent-format files in the docs [tutorials directory](https://github.com/nautechsystems/nautilus_trader/tree/develop/docs/tutorials). You can run those directly as scripts or open them as notebooks with Jupytext. Rust tutorials use the commands shown on their pages. ::: :::tip - **Latest**: docs built from the `master` branch for stable releases. See . - **Nightly**: docs built from the `nightly` branch for experimental features. See . ::: ## Recommended order New to NautilusTrader? Work through these in sequence: 1. [Quickstart](../getting_started/quickstart) - run your first backtest in five minutes with synthetic data 2. [Backtest (low-level API)](../getting_started/backtest_low_level) - direct `BacktestEngine` usage with real market data and execution algorithms 3. [Backtest (high-level API)](../getting_started/backtest_high_level) - config-driven backtesting with `BacktestNode` and the Parquet data catalog 4. [Loading external data][loading_external_data] - load CSV or other external data into the `ParquetDataCatalog` (how-to guide) 5. [Backtest with FX bar data][backtest_fx_bars] - FX bar backtesting with rollover interest simulation 6. Pick a topic-specific tutorial below ## Backtesting | Tutorial | Description | Data | |:------------------------------------------------------------------------------------|:-----------------------------------------------|:--------------| | [Backtest with FX Bar Data][backtest_fx_bars] | EMA cross on FX bars with rollover simulation. | Bundled | | [Backtest with Order Book Depth Data (Binance)][backtest_orderbook_binance] | Order book imbalance strategy on depth data. | User‑provided | | [Backtest with Order Book Depth Data (Bybit)][backtest_orderbook_bybit] | Order book imbalance strategy on depth data. | User‑provided | ## Data workflows For task-oriented data recipes, see the [how-to guides](../how_to/): | Guide | Description | Data | |:------------------------------------------------------------------------------------|:--------------------------------------------------|:------------------| | [Loading external data][loading_external_data] | Load external data into the `ParquetDataCatalog`. | User‑provided | | [Data catalog with Databento][data_catalog_databento] | Set up a catalog with Databento schemas. | Databento API key | ## Strategy patterns | Tutorial | Description | Data | |:------------------------------------------------------------------------------------|:--------------------------------------------------|:------------------| | [Mean Reversion with Proxy FX Data (AX Exchange)](fx_mean_reversion_ax) | Bollinger Band mean reversion on EURUSD‑PERP. | TrueFX proxy | | [Gold Perpetual Book Imbalance (AX Exchange)](gold_book_imbalance_ax) | Order book imbalance on XAU‑PERP. | Databento API key | | [Grid Market Making with a Deadman's Switch (BitMEX)](grid_market_maker_bitmex) | Grid MM with server‑side safety on XBTUSD. | Tardis.dev | | [On‑Chain Grid Market Making with Short‑Term Orders (dYdX)](grid_market_maker_dydx) | Grid MM on dYdX v4 perpetuals. | User‑provided | ## Options | Tutorial | Description | Data | |:------------------------------------------------------------------------------------|:--------------------------------------------------|:------------------| | [Options Data and Greeks (Bybit)](options_data_bybit) | Stream Greeks and option chain snapshots. | Live API | | [Delta‑Neutral Options Strategy (Bybit)](delta_neutral_options_bybit) | Short strangle with perpetual delta hedging. | Live API | | [Delta‑Neutral Options Strategy (Derive)](delta_neutral_options_derive) | Derive ETH strangle hedger with premium entry. | Live API | ## Rust | Tutorial | Description | Data | |:--------------------------------------------------------------------------------------------|:-----------------------------------------------------|:--------------------| | [Book Imbalance Backtest (Betfair)](backtest_book_imbalance_betfair) | Book imbalance actor on Betfair L2 data. | User‑provided | | [Composite Market Making on Lighter RWA with Databento EQUS NVDA](lighter_rwa_composite_mm) | Signal‑skewed MM on NVDA‑PERP. | Databento + Lighter | | [Hurst/VPIN Directional Strategy (Kraken Futures)](hurst_vpin_kraken) | Regime‑filtered informed‑flow strategy on PF_XBTUSD. | Tardis.dev | [backtest_fx_bars]: https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/tutorials/backtest_fx_bars.py [backtest_orderbook_binance]: https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/tutorials/backtest_orderbook_binance.py [backtest_orderbook_bybit]: https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/tutorials/backtest_orderbook_bybit.py [loading_external_data]: https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/how_to/loading_external_data.py [data_catalog_databento]: https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/how_to/data_catalog_databento.py # Composite Market Making on Lighter RWA with Databento US Equities NVDA Source: https://nautilustrader.io/docs/latest/tutorials/lighter_rwa_composite_mm/ This tutorial runs the shipped [`CompositeMarketMaker`][composite-market-maker] strategy on Lighter's `NVDA-PERP.LIGHTER` RWA market using Databento `NVDA.EQUS` quotes as an external signal. The strategy quotes one post-only bid and one post-only ask around the Lighter mid, then shifts both sides from a normalized Databento residual and the current Lighter inventory. The setup uses a Rust [`LiveNode`][live-node], while the strategy itself runs as the native Rust `CompositeMarketMaker` strategy. If you are new to the Lighter adapter, start with [Get started with Lighter][lighter-get-started] first. That guide isolates the Rust and Python v2 data-client paths before this tutorial adds Databento signal data and live order flow. ## Introduction Lighter lists real-world asset (RWA) perpetuals that trade continuously, including single-name equity markets. See Lighter's [RWA docs] and [market specifications] for current venue details. Databento's [US Equities][Databento US Equities] datasets provide US equity top-of-book data for `NVDA`, with `mbp-1` available through the Nautilus Databento adapter. `CompositeMarketMaker` is a small two-input market maker: - The **target instrument** is the Lighter market to quote: `NVDA-PERP.LIGHTER`. - The **signal instrument** is the Databento reference feed: `NVDA.EQUS`. - The **anchor** is the Lighter mid. - The **signal residual** is `(databento_mid / baseline) - 1.0`. - The **quote shift** is `signal_skew_factor * residual - inventory_skew_factor * net_position`. With no configured baseline, the strategy captures the first observed `NVDA.EQUS` mid as the reference price. The residual starts at zero and measures NVDA's move from that first signal mid, not the Lighter/Databento basis. Set the `SIGNAL_BASELINE` constant in the example source to pin the reference price for deterministic runs. In this setup, the Lighter BBO remains the spread anchor. Databento moves the quote center up or down through the normalized residual. ```mermaid flowchart LR subgraph Databento ["Databento data client"] DQ["NVDA.EQUS QuoteTick
dataset = EQUS.MINI
schema = mbp-1"] DS["signal_mid = (bid + ask) / 2"] DR["residual = signal_mid / baseline - 1"] end subgraph Lighter ["Lighter data + execution clients"] LQ["NVDA-PERP.LIGHTER QuoteTick"] LM["anchor = (bid + ask) / 2"] EX["Post-only limit orders"] end subgraph Strategy ["CompositeMarketMaker"] TH{{"no target orders OR anchor/signal impact
>= requote_threshold_bps"}} CA["cancel_all_orders()"] SK["shift = signal_skew - inventory_skew"] QU["bid = anchor - half_spread + shift
ask = anchor + half_spread + shift"] PO["submit post-only bid/ask"] end DQ --> DS --> DR --> SK LQ --> LM --> TH TH -->|yes| CA --> SK --> QU --> PO --> EX TH -->|no| LQ ``` The focus is the adapter wiring: one engine consumes a direct US equity feed and a crypto-native RWA venue, while order lifecycle, inventory, and quote state stay inside the same event-driven runtime. ## Prerequisites - A Rust toolchain (MSRV 1.96.0 or newer). - A Cargo project with the Nautilus, Lighter, and Databento crates as dependencies (see [Project setup](#project-setup)). - Python 3.12+ to regenerate the rendered panels. - A Databento API key with live access to Databento US Equities Mini (`EQUS.MINI`), the default dataset for the bundled `NVDA.EQUS` route. Higher tiers such as `EQUS.PLUS` need a separate Databento license; select one with `venue_dataset_map` when your account is entitled. - Lighter API credentials (numeric account index, API key index, and API secret) for the configured environment (testnet by default), required only to connect and submit orders. - The Lighter integration guide: [Lighter](../integrations/lighter.md). - The Databento integration guide: [Databento](../integrations/databento.md). The example reads credentials from environment variables and keeps the strategy parameters as editable Rust constants. It defaults to `LighterEnvironment::Testnet`, so set the testnet Lighter credentials: ```bash export DATABENTO_API_KEY="your-databento-api-key" export LIGHTER_TESTNET_ACCOUNT_INDEX="123456" export LIGHTER_TESTNET_API_KEY_INDEX="0" export LIGHTER_TESTNET_API_SECRET="your-lighter-api-secret" ``` For mainnet, change `LIGHTER_ENVIRONMENT` in the source to `LighterEnvironment::Mainnet` and use the mainnet `LIGHTER_*` credential variables described in the integration guide. Set `DATABENTO_API_KEY` before running the example. ## Project setup The strategy, node, and adapters ship as crates, so you can depend on them from your own Cargo project rather than working inside a NautilusTrader checkout. Add the following to your `Cargo.toml`, pointing every Nautilus dependency at the same `develop` git source so the crates resolve to one consistent version: ```toml [dependencies] nautilus-common = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["live"] } nautilus-core = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop" } nautilus-databento = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["high-precision", "live"] } nautilus-lighter = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["examples", "high-precision"] } nautilus-live = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["node"] } nautilus-model = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["high-precision"] } nautilus-trading = { git = "https://github.com/nautechsystems/nautilus_trader.git", branch = "develop", features = ["examples", "high-precision"] } tokio = { version = "1", features = ["full"] } ``` The `examples` feature on `nautilus-trading` exposes the `CompositeMarketMaker` strategy, and `high-precision` is required for Lighter's crypto-native pricing. For the general crate layout, feature flags, and the crates.io alternative to the git source, see the Rust [project setup guide][project-setup]. The Databento client also needs a publishers file that maps venues to datasets. Download [`publishers.json`][databento-publishers] from the Databento adapter crate and point `publishers_filepath` at your local copy. The shipped example resolves the same file relative to the checkout, so this step only applies to your own project. ## Why NVDA `NVDA` is a liquid Nasdaq-listed single-name equity, and Lighter maps its RWA perpetual to `NVDA-PERP.LIGHTER`. This pairs a licensed Databento signal with a Lighter traded market: | Role | Instrument ID | Source | Notes | | ----------------- | ------------------- | --------- | ------------------------------------------ | | Signal instrument | `NVDA.EQUS` | Databento | EQUS.MINI top‑of‑book quote updates. | | Target instrument | `NVDA-PERP.LIGHTER` | Lighter | RWA perpetual traded through Lighter. | Subscribing to `NVDA.EQUS` requests top-of-book (`mbp-1`) quotes for `NVDA` from Databento's `EQUS.MINI` dataset by default, delivered as a single `QuoteTick` stream. `EQUS.MINI` is the lowest-cost consolidated US equities tier; richer tiers such as `EQUS.PLUS` need a separate Databento license and can be selected with the client's `venue_dataset_map` (for example `{"EQUS": "EQUS.PLUS"}`) once your account is entitled. The adapter resolves the `EQUS` venue from a publishers file: the example points `DatabentoLiveClientConfig` at the `publishers.json` bundled with the Databento adapter. See [Instrument IDs and symbology][databento-symbology] for the mapping rules. The older Databento Equities Basic (`DBEQ.BASIC`) dataset name appears in some grandfathered accounts and historical examples. New Databento subscriptions use the Databento US Equities product line, so this tutorial uses the consolidated `EQUS` venue. Treat the top-of-book feed as a licensed signal proxy for the tutorial wiring, not as a full depth Nasdaq TotalView book. The example starts at `trade_size=0.05`, which aligns with the Lighter NVDA minimum base amount observed during tutorial validation. Check the [market details endpoint] before increasing size or changing instruments. ## Session constraint Lighter RWA markets trade continuously. `NVDA.EQUS` follows the US equity market data session. The first live test should run during the regular cash session (13:30-20:00 UTC, US daylight time), with special handling for holidays and half-days. `CompositeMarketMaker` does not include a built-in session gate or signal-age guard. For production use, add an actor or strategy variant that cancels quotes when the Databento signal goes stale. The tutorial example keeps this explicit instead of hiding it in a custom strategy. ## Example node There are two ways to run this: from a NautilusTrader checkout via the shipped [Lighter NVDA composite market maker example][example-script] binary, or by copying the node wiring below into a `main` in your own project that depends on the crates from [Project setup](#project-setup). A Python v2 counterpart also lives at [`python/examples/lighter/nvda_composite_mm.py`][python-example-script]; it uses the same Rust strategy through PyO3. From a checkout, with the credential variables set, the shipped binary connects the data and execution clients. It defaults to `DRY_RUN = true`, which starts the clients without adding the order-submitting strategy: ```bash cargo run --bin lighter-nvda-composite-mm --package nautilus-tutorials --features examples ``` Databento is a multi-venue data client without a fixed venue route, so the engine uses it as the default route for `NVDA.EQUS`. Lighter registers with the `LIGHTER` venue route and receives `NVDA-PERP.LIGHTER` subscriptions. The core of the setup is the three-client node plus `CompositeMarketMaker`: ```rust let lighter_environment = LIGHTER_ENVIRONMENT; let trader_id = TraderId::from(TRADER_ID); let account_id = AccountId::from(ACCOUNT_ID); let instrument_id = InstrumentId::from(INSTRUMENT_ID); let signal_instrument_id = InstrumentId::from(SIGNAL_INSTRUMENT_ID); let databento_api_key = get_env_var("DATABENTO_API_KEY")?; let databento_config = DatabentoLiveClientConfig::new(databento_api_key, publishers_filepath, true, true); let lighter_data_config = LighterDataClientConfig::builder() .environment(lighter_environment) .build(); let lighter_exec_config = LighterExecClientConfig::builder() .trader_id(trader_id) .account_id(account_id) .environment(lighter_environment) .build(); let mut strategy_config = CompositeMarketMakerConfig::builder() .instrument_id(instrument_id) .signal_instrument_id(signal_instrument_id) .max_position(max_position) .trade_size(trade_size) .half_spread_bps(HALF_SPREAD_BPS) .inventory_skew_factor(INVENTORY_SKEW_FACTOR) .signal_skew_factor(SIGNAL_SKEW_FACTOR) .requote_threshold_bps(REQUOTE_THRESHOLD_BPS) .on_cancel_resubmit(ON_CANCEL_RESUBMIT) .build(); strategy_config.base.strategy_id = Some(StrategyId::from("NVDA_COMPOSITE_MM-001")); strategy_config.base.order_id_tag = Some("001".to_string()); let mut node = LiveNode::builder(trader_id, Environment::Live)? .with_name("LIGHTER-NVDA-COMPOSITE-MM-001".to_string()) .with_reconciliation(!DRY_RUN) .add_data_client( None, Box::new(DatabentoDataClientFactory::new()), Box::new(databento_config), )? .add_data_client( None, Box::new(LighterDataClientFactory::new()), Box::new(lighter_data_config), )? .add_exec_client( None, Box::new(LighterExecutionClientFactory::new()), Box::new(lighter_exec_config), )? .build()?; if !DRY_RUN { node.add_strategy(CompositeMarketMaker::new(strategy_config))?; } ``` To allow order submission, edit the constant near the top of the example source: ```rust const DRY_RUN: bool = false; ``` Then run the same command: ```bash cargo run --bin lighter-nvda-composite-mm --package nautilus-tutorials --features examples ``` :::warning This command can submit live orders when `DRY_RUN` is `false`. Start with the smallest accepted size on a funded test account or a mainnet account sized for loss. Confirm the active instrument ID, account ID, numeric account index, and Lighter credentials before changing it. ::: For a testnet smoke run, keep `LIGHTER_ENVIRONMENT` as `LighterEnvironment::Testnet` and use the `LIGHTER_TESTNET_*` credential variables. If the run is outside the Databento US Equities cash session, it can still validate node startup, routing, Lighter data, and the order lifecycle. The Databento residual remains zero until the first `NVDA.EQUS` quote arrives. ## Strategy parameters | Parameter | Value | Description | | ----------------------- | ------------------- | -------------------------------------------------------------- | | `instrument_id` | `NVDA-PERP.LIGHTER` | Lighter RWA perpetual to quote. | | `signal_instrument_id` | `NVDA.EQUS` | Databento US Equities Mini signal feed. | | `trade_size` | `0.05` | Size per bid or ask. | | `max_position` | `0.20` | Hard cap on net Lighter exposure. | | `half_spread_bps` | `25` | Half‑spread around the Lighter anchor. | | `inventory_skew_factor` | `2.0` | Price units per unit of net position. | | `signal_skew_factor` | `55.0` | Price units per unit of normalized Databento residual. | | `signal_baseline` | First signal mid | Optional reference price for the Databento residual. | | `requote_threshold_bps` | `5` | Anchor or signal‑impact move that triggers cancel and requote. | With a Lighter mid of `207.00` and `half_spread_bps=25`, the unskewed half spread is `0.5175` USD. If Databento is 30 bps above its baseline, a `signal_skew_factor` of `55.0` shifts both sides up by `0.165` USD before inventory skew. A long position of `0.05` with `inventory_skew_factor=2.0` shifts both sides down by `0.10` USD. ## Requote behavior Signal ticks update internal state but do not submit orders by themselves. Until the first Databento quote arrives, the residual is zero. The next Lighter quote tick reads the latest signal residual and checks the quote state. A quote cycle occurs when: - no target orders are open or in-flight; - the Lighter anchor moves by at least `requote_threshold_bps`; or - the price impact of the signal residual change clears the same threshold. The strategy then cancels open orders, reads current net position and pending exposure from the cache, computes one bid and one ask, drops any side that breaches `max_position`, and submits the remaining sides as post-only limits. ## Panels The panels below use deterministic replay data. They show the quoting mechanics and the cash-session constraint. They are not a captured live Lighter fill trace. ![NVDA composite quote center against Databento and Lighter mids](./assets/lighter_rwa_composite_mm/panel_a_reference_overlay.png) **Figure 1.** *Databento `NVDA.EQUS` mid, Lighter `NVDA-PERP.LIGHTER` mid, composite bid, composite ask, and quote center.* ![Databento residual, Lighter basis, and quote-center shift](./assets/lighter_rwa_composite_mm/panel_b_signal_basis.png) **Figure 2.** *Databento residual, Lighter basis, and quote-center shift in bps.* ![Inventory skew terms for the composite market maker](./assets/lighter_rwa_composite_mm/panel_c_inventory_skew.png) **Figure 3.** *Net position, signal shift, inventory adjustment, and total shift for a `0.05` NVDA trade size and `0.20` NVDA position cap.* ![Lighter continuous trading and Databento session clock](./assets/lighter_rwa_composite_mm/panel_d_session_clock.png) **Figure 4.** *Lighter's continuous RWA market clock against the Databento US Equities cash-session signal, with signal age after the regular session.* ## Regenerate the panels ```bash uv sync --extra visualization python3 docs/tutorials/assets/lighter_rwa_composite_mm/render_panels.py ``` The renderer writes four PNGs into `docs/tutorials/assets/lighter_rwa_composite_mm/`. It uses the `nautilus_dark` Plotly theme and deterministic replay data so docs builds do not depend on vendor data licenses or live exchange access. ## Extensions The next useful improvement is a signal-age gate. For example, cancel all Lighter orders when the latest `NVDA.EQUS` quote is older than 30 seconds during the cash session, or immediately after the cash session closes. That makes the Databento signal an explicit operating dependency instead of an implicit one. For a pure fair-value strategy, use this tutorial as the client wiring and write a small variant that anchors bid/ask directly on the Databento mid, then checks the Lighter BBO only for post-only and basis limits. [composite-market-maker]: https://github.com/nautechsystems/nautilus_trader/blob/develop/crates/trading/src/examples/strategies/composite_market_maker/strategy.rs [live-node]: ../how_to/run_rust_live_trading.md [project-setup]: ../concepts/rust.md#project-setup [lighter-get-started]: ../how_to/get_started_lighter.md [databento-symbology]: ../integrations/databento.md#instrument-ids-and-symbology [databento-publishers]: https://github.com/nautechsystems/nautilus_trader/blob/develop/crates/adapters/databento/publishers.json [RWA docs]: https://docs.lighter.xyz/trading/real-world-assets-rwas [market specifications]: https://docs.lighter.xyz/trading/real-world-assets-rwas/market-specifications [market details endpoint]: https://mainnet.zklighter.elliot.ai/api/v1/orderBookDetails [Databento US Equities]: https://databento.com/blog/introducing-databento-us-equities [example-script]: https://github.com/nautechsystems/nautilus_trader/blob/develop/examples/tutorials/src/bin/lighter_nvda_composite_mm.rs [python-example-script]: https://github.com/nautechsystems/nautilus_trader/blob/develop/python/examples/lighter/nvda_composite_mm.py # Options Data and Greeks (Bybit) Source: https://nautilustrader.io/docs/latest/tutorials/options_data_bybit/ :::note This is a **Rust-only** v2 system tutorial. It uses the Rust `LiveNode` with the Bybit adapter to stream live option Greeks and aggregated chain snapshots. ::: This tutorial connects to Bybit's live options market and consumes Greeks and option chain data through two `DataActor` examples. It covers instrument discovery, venue-provided Greeks subscriptions, and periodic chain snapshots with ATM-relative strike filtering. ## Introduction Bybit publishes Greeks (delta, gamma, vega, theta) and implied volatility alongside every option ticker update. NautilusTrader exposes this data at two levels: - **Per-instrument Greeks**: subscribe to a single option contract and receive an `OptionGreeks` event on every ticker update. - **Option chain snapshots**: subscribe to an entire expiry series and receive periodic `OptionChainSlice` events that aggregate quotes and Greeks across all active strikes. Two example binaries back these patterns: the first subscribes to individual Greeks streams, the second subscribes to an aggregated chain with ATM-relative strike filtering. ```mermaid flowchart LR subgraph BybitAPI ["Bybit V5 public WebSocket"] TKR["Per-contract option ticker"] end subgraph Adapter ["nautilus-bybit data client"] Q["QuoteTick + OptionGreeks per contract"] AGG["Per-series aggregator
(ATM and strike filtering)"] end subgraph Actors ["DataActor implementations"] G["GreeksTester
on_option_greeks()"] C["OptionChainTester
on_option_chain()"] end TKR --> Q Q --> G Q --> AGG AGG -->|interval timer| C ``` ## Prerequisites - A working Rust toolchain ([rustup.rs](https://rustup.rs)). - The NautilusTrader repository cloned and building. - A Bybit API key with read permissions. No trading permissions are needed for data-only use. Create keys at [bybit.com](https://www.bybit.com/app/user/api-management). - Environment variables set for authentication: ```bash export BYBIT_API_KEY="your-api-key" export BYBIT_API_SECRET="your-api-secret" ``` A `.env` file in the repository root also works. The examples load it via `dotenvy`. :::warning Bybit demo trading uses `stream-demo.bybit.com` only for private streams. Public option market data uses the mainnet public stream `wss://stream.bybit.com/v5/public/option`. ::: ## The DataActor pattern A Rust `DataActor` needs three pieces: 1. A struct with a `core: DataActorCore` field plus your own state. 2. The `nautilus_actor!(YourType)` macro plus a `Debug` implementation. 3. A `DataActor` trait implementation with your callbacks. The macro supplies the native runtime wiring required by the blanket `Actor` and `Component` implementations, so you only implement the callbacks you need. Every callback has a default no-op implementation. ## Part 1: per-instrument Greeks The `bybit-greeks-tester` example subscribes to `OptionGreeks` for all BTC CALL options at the nearest expiry and logs each update. ### Actor structure ```rust #[derive(Debug)] struct GreeksTester { core: DataActorCore, client_id: ClientId, subscribed_instruments: Vec, } nautilus_actor!(GreeksTester); impl GreeksTester { fn new(client_id: ClientId) -> Self { Self { core: DataActorCore::new(DataActorConfig { actor_id: Some("GREEKS_TESTER-001".into()), ..Default::default() }), client_id, subscribed_instruments: Vec::new(), } } } ``` The `core` field is required by the macro. The `client_id` identifies which data client to route subscriptions to. The `subscribed_instruments` vector tracks what we subscribed to so we clean up on stop. ### Discovering instruments On start, the actor queries the cache for all option instruments, filters for BTC CALLs that have not expired, and finds the nearest expiry: ```rust fn on_start(&mut self) -> anyhow::Result<()> { let venue = Venue::new("BYBIT"); let underlying_filter = Ustr::from("BTC"); let mut options: Vec<(InstrumentId, f64, u64)> = { let cache = self.cache(); let instruments = cache.instruments(&venue, Some(&underlying_filter)); instruments .iter() .filter_map(|inst| { if inst.option_kind() == Some(OptionKind::Call) { let expiry = inst.expiration_ns()?.as_u64(); let strike = inst.strike_price()?.as_f64(); Some((inst.id(), strike, expiry)) } else { None } }) .collect() }; // cache borrow dropped here let now_ns = self.timestamp_ns().as_u64(); options.retain(|(_, _, exp)| *exp > now_ns); let nearest_expiry = options.iter().map(|(_, _, exp)| *exp).min().unwrap(); options.retain(|(_, _, exp)| *exp == nearest_expiry); options.sort_by(|(_, a, _), (_, b, _)| a.partial_cmp(b).unwrap()); // ...subscribe to each } ``` :::warning Release the cache borrow before calling any subscription methods. The cache uses `Rc>` internally, and subscription methods may need to borrow it. Collect owned data into a local `Vec`, drop the cache reference, then subscribe. ::: ### Subscribing to Greeks After discovering instruments, subscribe to each one: ```rust let client_id = self.client_id; for (instrument_id, _, _) in &options { self.subscribe_option_greeks(*instrument_id, Some(client_id), None); self.subscribed_instruments.push(*instrument_id); } ``` ### Handling updates Each ticker update from Bybit triggers `on_option_greeks` with an `OptionGreeks` event: ```rust fn on_option_greeks(&mut self, greeks: &OptionGreeks) -> anyhow::Result<()> { log::info!( "GREEKS | {} | delta={:.4} gamma={:.6} vega={:.4} theta={:.4} rho={:.6} | \ mark_iv={} bid_iv={} ask_iv={} | underlying={} oi={}", greeks.instrument_id, greeks.delta, greeks.gamma, greeks.vega, greeks.theta, greeks.rho, greeks.mark_iv.map_or("-".to_string(), |v| format!("{v:.2}")), greeks.bid_iv.map_or("-".to_string(), |v| format!("{v:.2}")), greeks.ask_iv.map_or("-".to_string(), |v| format!("{v:.2}")), greeks.underlying_price.map_or("-".to_string(), |v| format!("{v:.2}")), greeks.open_interest.map_or("-".to_string(), |v| format!("{v:.1}")), ); Ok(()) } ``` The `OptionGreeks` fields: | Field | Type | Description | |--------------------|----------------|----------------------------------------------------| | `instrument_id` | `InstrumentId` | The option contract. | | `delta` | `f64` | Price sensitivity to underlying. | | `gamma` | `f64` | Delta sensitivity to underlying. | | `vega` | `f64` | Price sensitivity to a 1% change in volatility. | | `theta` | `f64` | Daily time decay. | | `rho` | `f64` | Sensitivity to interest rate changes. | | `mark_iv` | `Option` | Mark price implied volatility. | | `bid_iv` | `Option` | Bid implied volatility. | | `ask_iv` | `Option` | Ask implied volatility. | | `underlying_price` | `Option` | Current underlying forward price for this expiry. | | `open_interest` | `Option` | Open interest for this contract. | The `delta`, `gamma`, `vega`, `theta`, and `rho` values live on a nested `greeks: OptionGreekValues` struct. `OptionGreeks` implements `Deref`, so `greeks.delta` and friends work as shown above. Bybit does not provide rho; the adapter sets it to `0.0`. ### Cleanup On stop, unsubscribe from all instruments: ```rust fn on_stop(&mut self) -> anyhow::Result<()> { let ids: Vec = self.subscribed_instruments.drain(..).collect(); let client_id = self.client_id; for instrument_id in ids { self.unsubscribe_option_greeks(instrument_id, Some(client_id), None); } log::info!("Unsubscribed from all option greeks"); Ok(()) } ``` ## Part 2: option chain snapshots The `bybit-option-chain` example subscribes to an aggregated option chain and logs periodic snapshots showing calls and puts at each strike with their quotes and Greeks. ### Why use option chains Per-instrument subscriptions give granular control, but monitoring an entire surface means managing individual streams and correlating updates across strikes. An option chain subscription handles this: the `DataEngine` aggregates quotes and Greeks across all strikes in a series and publishes a single `OptionChainSlice` on a timer. This aggregation happens inside NautilusTrader. Bybit publishes per-contract option market data and does not expose a native option chain stream in the V5 public WebSocket docs. ### Key types **`OptionSeriesId`** identifies a single expiry series: ```rust let series_id = OptionSeriesId::new( Venue::new("BYBIT"), // venue Ustr::from("BTC"), // underlying Ustr::from("USDT"), // settlement currency UnixNanos::from(expiry), // expiration timestamp ); ``` **`StrikeRange`** controls which strikes are active: | Variant | Description | |---------------|--------------------------------------------------------| | `Fixed` | A fixed set of strike prices. | | `AtmRelative` | `strikes_above` above and `strikes_below` below ATM. | | `AtmPercent` | All strikes within `pct` of the ATM price. | For ATM-based variants, subscriptions are deferred until the ATM price is determined from the venue-provided forward price. ### Subscribing ```rust let strike_range = StrikeRange::AtmRelative { strikes_above: 3, strikes_below: 3, }; let snapshot_interval_ms = Some(5_000); // snapshot every 5 seconds self.subscribe_option_chain( series_id, strike_range, snapshot_interval_ms, Some(client_id), None, // params ); ``` Pass `None` for `snapshot_interval_ms` to use raw mode, where every quote or Greeks update publishes a slice immediately. ### Handling snapshots The `on_option_chain` callback receives an `OptionChainSlice` containing all active strikes with their call and put data: ```rust fn on_option_chain(&mut self, slice: &OptionChainSlice) -> anyhow::Result<()> { log::info!( "OPTION_CHAIN | {} | atm={} | calls={} puts={} | strikes={}", slice.series_id, slice.atm_strike.map_or("-".to_string(), |p| format!("{p}")), slice.call_count(), slice.put_count(), slice.strike_count(), ); for strike in slice.strikes() { let call_info = slice.get_call(&strike).map(|d| { let greeks_str = d.greeks.as_ref().map_or("-".to_string(), |g| { format!( "d={:.3} g={:.5} v={:.2} iv={:.1}%", g.delta, g.gamma, g.vega, g.mark_iv.unwrap_or(0.0) * 100.0, ) }); format!("bid={} ask={} [{}]", d.quote.bid_price, d.quote.ask_price, greeks_str) }); let put_info = slice.get_put(&strike).map(|d| { let greeks_str = d.greeks.as_ref().map_or("-".to_string(), |g| { format!( "d={:.3} g={:.5} v={:.2} iv={:.1}%", g.delta, g.gamma, g.vega, g.mark_iv.unwrap_or(0.0) * 100.0, ) }); format!("bid={} ask={} [{}]", d.quote.bid_price, d.quote.ask_price, greeks_str) }); log::info!( " K={} | CALL: {} | PUT: {}", strike, call_info.unwrap_or_else(|| "-".to_string()), put_info.unwrap_or_else(|| "-".to_string()), ); } Ok(()) } ``` The `OptionChainSlice` fields and methods: | Name | Type / Returns | Description | |------------------|-----------------------------|--------------------------------------| | `series_id` | `OptionSeriesId` | The series this snapshot covers. | | `atm_strike` | `Option` | ATM strike from the forward price. | | `call_count()` | `usize` | Number of call strikes with data. | | `put_count()` | `usize` | Number of put strikes with data. | | `strike_count()` | `usize` | Union of all strikes. | | `strikes()` | `Vec` | Sorted list of all strike prices. | | `get_call(k)` | `Option<&OptionStrikeData>` | Call quote and Greeks at strike `k`. | | `get_put(k)` | `Option<&OptionStrikeData>` | Put quote and Greeks at strike `k`. | Each `OptionStrikeData` contains a `quote: QuoteTick` (bid/ask) and an optional `greeks: Option`. ## Node setup Both examples use the same `LiveNode` pattern. No execution client is needed for data-only use: ```rust #[tokio::main] async fn main() -> Result<(), Box> { dotenvy::dotenv().ok(); let environment = Environment::Live; let trader_id = TraderId::test_default(); let client_id = ClientId::new("BYBIT"); let bybit_config = BybitDataClientConfig { api_key: None, // loaded from BYBIT_API_KEY env var api_secret: None, // loaded from BYBIT_API_SECRET env var product_types: vec![BybitProductType::Option], ..Default::default() }; let client_factory = BybitDataClientFactory::new(); let mut node = LiveNode::builder(trader_id, environment)? .with_name("BYBIT-OPTIONS-001".to_string()) .add_data_client(None, Box::new(client_factory), Box::new(bybit_config))? .with_delay_post_stop_secs(5) .build()?; let actor = GreeksTester::new(client_id); // or OptionChainTester node.add_actor(actor)?; node.run().await?; Ok(()) } ``` Setting `product_types` to `[BybitProductType::Option]` loads only option instruments. Startup blocks while the instrument provider fetches and parses every listed option. ## Running the examples ```bash # Per-instrument Greeks cargo run --example bybit-greeks-tester --package nautilus-bybit --features examples # Option chain snapshots cargo run --example bybit-option-chain --package nautilus-bybit --features examples ``` Stop either example with Ctrl+C. The actor's `on_stop` callback unsubscribes from all streams before shutdown. ## What the examples produce A 30-second mainnet run on April 28 (BTC near 76,800 USDT, expiry 2026-04-28 08:00 UTC) captures **938 Greeks updates** across 22 BTC CALL contracts in the per-instrument tester, plus **5 chain snapshots** covering 7 strikes each in the chain tester. ### Per-instrument Greeks output ``` Found 22 BTC CALL options at nearest expiry (ts=1777359600000000000) Subscribed to option greeks for 22 instruments GREEKS | BTC-28APR26-72000-C-USDT-OPTION.BYBIT | delta=0.4733 gamma=0.000000 vega=0.0000 theta=-0.0000 rho=0.000000 | mark_iv=0.66 bid_iv=0.00 ask_iv=5.00 | underlying=76782.43 oi=0.0 GREEKS | BTC-28APR26-71000-C-USDT-OPTION.BYBIT | delta=0.4733 gamma=0.000000 vega=0.0000 theta=-0.0000 rho=0.000000 | mark_iv=0.74 bid_iv=0.00 ask_iv=5.00 | underlying=76782.43 oi=0.1 GREEKS | BTC-28APR26-73000-C-USDT-OPTION.BYBIT | delta=0.4733 gamma=0.000000 vega=0.0000 theta=-0.0000 rho=0.000000 | mark_iv=0.57 bid_iv=0.00 ask_iv=5.00 | underlying=76782.43 oi=0.0 ``` ### Option chain output ``` OPTION_CHAIN | BYBIT:BTC:USDT:2026-04-28T08:00:00Z | atm=77000 | calls=7 puts=7 | strikes=7 K=75500 | CALL: bid=1210 ask=1430 [d=0.445 g=0.00000 v=0.00 iv=36.2%] | PUT: bid=0 ask=5 [d=0.000 g=0.00000 v=0.00 iv=36.2%] K=76000 | CALL: bid=700 ask=850 [d=0.445 g=0.00000 v=0.00 iv=32.5%] | PUT: bid=0 ask=5 [d=0.000 g=0.00000 v=0.00 iv=32.5%] K=76500 | CALL: bid=265 ask=370 [d=0.442 g=0.00000 v=0.07 iv=29.9%] | PUT: bid=0 ask=5 [d=-0.003 g=0.00000 v=0.07 iv=29.9%] ``` ### Panels ![BTC CALL delta vs strike](./assets/options_data_bybit/panel_a_delta_vs_strike.png) **Figure 1.** *Last delta per BTC CALL strike at the nearest expiry, underlying ~77,000 USDT marked. Delta drops from ~0.45 below the underlying to near zero past the underlying. Bybit's delta on near-zero gamma contracts close to expiry compresses to a step-like profile around the forward.* ![IV smile per strike](./assets/options_data_bybit/panel_b_iv_smile.png) **Figure 2.** *Mark IV per strike for the latest chain snapshot (CALL and PUT overlaid). The smile is symmetric around ATM at 77,000 USDT, with IV dipping from 36% at 75,500 to 30% at 77,000 and rising back to 38% at 78,500.* ![Underlying trajectory and open interest](./assets/options_data_bybit/panel_c_underlying_oi.png) **Figure 3.** *Underlying forward price reported in each Greeks update (top) and open interest by strike at the last update (bottom). OI concentrates in the 70,000-76,000 USDT band: at-the-money to slightly out-of-the-money strikes.* ![CALL spread per chain snapshot](./assets/options_data_bybit/panel_d_call_spread.png) **Figure 4.** *Average CALL bid-ask spread per chain snapshot in USDT. Snapshots arrive every five seconds (`snapshot_interval_ms=5000`).* ### Regenerate the panels ```bash timeout 30 ./target/release/examples/bybit-greeks-tester > /tmp/bybit_greeks.log 2>&1 timeout 30 ./target/release/examples/bybit-option-chain > /tmp/bybit_chain.log 2>&1 uv sync --extra visualization GREEKS_LOG=/tmp/bybit_greeks.log CHAIN_LOG=/tmp/bybit_chain.log \ python3 docs/tutorials/assets/options_data_bybit/render_panels.py ``` ## Complete source - [`crates/adapters/bybit/examples/node_greeks.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/bybit/examples/node_greeks.rs) - [`crates/adapters/bybit/examples/node_option_chain.rs`](https://github.com/nautechsystems/nautilus_trader/tree/develop/crates/adapters/bybit/examples/node_option_chain.rs) ## Next steps - **Combine both patterns**. Use per-instrument Greeks for near-ATM contracts alongside the aggregated chain view in a single actor. Subscribe to Greeks for contracts you want to track individually, and the chain for a surface-level view. - **Add quote and depth subscriptions**. Call `subscribe_quotes` for top-of-book `QuoteTick` updates on individual option contracts. Call `subscribe_order_book_deltas` when you need the dedicated option orderbook stream. Bybit supports option depths 25 and 100. - **Options execution**. The [delta-neutral strategy tutorial](delta_neutral_options_bybit.md) walks through a short strangle with perpetual hedging, including IV-based order placement via Bybit's `order_iv` parameter. ## See also - [Options](../concepts/options.md): option instrument types, Greeks data types, and chain architecture. - [Bybit integration](../integrations/bybit.md): full Bybit adapter reference including options order parameters and limitations. # Advanced orders Source: https://nautilustrader.io/docs/latest/concepts/orders/advanced/ The following guide should be read in conjunction with the specific documentation from the broker or venue involving these order types, lists/groups and execution instructions (such as for Interactive Brokers). ## Order lists Combinations of contingent orders, or larger order bulks can be grouped together into a list with a common `order_list_id`. The orders contained in this list may or may not have a contingent relationship with each other, as this is specific to how the orders themselves are constructed, and the specific venue they are being routed to. All orders in a list must share the same venue. Orders may target different instruments at that venue (e.g. pairs, calendar spreads, multi-leg legs); whether the destination venue accepts mixed-instrument batches is venue-specific. The list's `instrument_id` is taken from the first order as a representative value; downstream consumers that need a per-order instrument resolve each order individually. Caveats for mixed-instrument lists: - Pre-trade per-order checks (price/quantity precision, GTD) use each order's own instrument. - The cumulative risk check (free balance, min/max notional, position-reducing exposure, per-order market-data lookups) uses the list's representative instrument. For mixed lists this is a single-instrument bound, not per-instrument accuracy. - Cache lookups like `cache.order_lists(instrument_id=...)` filter against the representative `instrument_id`; lists containing other instruments will not match queries for those other instruments. - The execution engine denies mixed-instrument lists when a `position_id` is supplied (a position belongs to a single instrument, regardless of OMS). - Adapter `submit_order_list` implementations vary. Some iterate orders per leg and resolve each order's own `instrument_id` against the venue API; others still build the batch request around the list's representative `instrument_id` and will misroute non-first orders. Treat mixed-instrument lists as adapter-specific; verify the target adapter's behaviour before relying on it. Backtesting and custom strategy code that handle multi-leg routing in user space remain the safest path today. ## Contingency types - **OTO (One-Triggers-Other)** – a parent order that, once executed, automatically places one or more child orders. - *Full-trigger model*: child order(s) are released **only after the parent is completely filled**. Common at most retail equity/option brokers (e.g. Schwab, Fidelity, TD Ameritrade) and many spot-crypto venues (Binance, Coinbase). - *Partial-trigger model*: child order(s) are released **pro-rata to each partial fill**. Used by professional-grade platforms such as Interactive Brokers, most futures/FX OMSs, and Kraken Pro. - **OCO (One-Cancels-Other)** – two (or more) linked live orders where executing one cancels the remainder. - **OUO (One-Updates-Other)** – two (or more) linked live orders where executing one reduces the open quantity of the remainder. :::info These contingency types relate to ContingencyType FIX tag <1385> . ::: ### One-Triggers-Other (OTO) An OTO order involves two parts: 1. **Parent order** – submitted to the matching engine immediately. 2. **Child order(s)** – held *off-book* until the trigger condition is met. #### Trigger models | Trigger model | When are child orders released? | |---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------| | **Full trigger** | When the parent order’s cumulative quantity equals its original quantity (i.e., it is *fully* filled). | | **Partial trigger** | Immediately upon each partial execution of the parent; the child’s quantity matches the executed amount and is increased as further fills occur. | :::info The default backtest venue for NautilusTrader uses a *partial-trigger model* for OTO orders. To opt-in to a *full-trigger mode*, set `oto_trigger_mode="FULL"` for the venue (e.g. via `BacktestVenueConfig`). ::: **Working with partial-trigger in production:** If your strategy requires full-trigger semantics but the venue or backtest engine uses partial-trigger: 1. Submit the parent order without contingent children. 2. Subscribe to `OrderFilled` events for the parent order. 3. Only submit child orders (stop-loss, take-profit) after confirming the parent is fully filled. 4. Use `order.is_closed` and `order.filled_qty == order.quantity` to verify complete fill. > **Why the distinction matters** > *Full trigger* leaves a risk window: any partially filled position is live without its protective exit until the remaining quantity fills. > *Partial trigger* mitigates that risk by ensuring every executed lot instantly has its linked stop/limit, at the cost of creating more order traffic and updates. An OTO order can use any supported asset type on the venue (e.g. stock entry with option hedge, futures entry with OCO bracket, crypto spot entry with TP/SL). | Venue / Adapter ID | Asset classes | Trigger rule for child | Practical notes | |----------------------------------------------|---------------------------|---------------------------------------------|-------------------------------------------------------------------| | Binance / Binance Futures (`BINANCE`) | Spot, perpetual futures | **Partial or full** – fires on first fill. | OTOCO/TP-SL children appear instantly; monitor margin usage. | | Bybit Spot (`BYBIT`) | Spot | **Full** – child placed after completion. | TP-SL preset activates only once the limit order is fully filled. | | Bybit Perps (`BYBIT`) | Perpetual futures | **Partial and full** – configurable. | “Partial‑position” mode sizes TP-SL as fills arrive. | | Kraken Futures (`KRAKEN`) | Futures & perps | **Partial and full** – automatic. | Child quantity matches every partial execution. | | OKX (`OKX`) | Spot, futures, options | **Full** – attached stop waits for fill. | Position‑level TP-SL can be added separately. | | Interactive Brokers (`INTERACTIVE_BROKERS`) | Stocks, options, FX, fut | **Configurable** – OCA can pro‑rate. | `OcaType 2/3` reduces remaining child quantities. | | dYdX v4 (`DYDX`) | Perpetual futures (DEX) | On‑chain condition (size exact). | TP-SL triggers by oracle price; partial fill not applicable. | | Polymarket (`POLYMARKET`) | Prediction market (DEX) | N/A. | Advanced contingency handled entirely at the strategy layer. | | Betfair (`BETFAIR`) | Sports betting | N/A. | Advanced contingency handled entirely at the strategy layer. | ### One-Cancels-Other (OCO) An OCO order is a set of linked orders where the execution of **any** order (full *or partial*) triggers a best-efforts cancellation of the others. Both orders are live simultaneously; once one starts filling, the venue attempts to cancel the unexecuted portion of the remainder. ### One-Updates-Other (OUO) An OUO order is a set of linked orders where execution of one order causes an immediate *reduction* of open quantity in the other order(s). Both orders are live concurrently, and each partial execution proportionally updates the remaining quantity of its peer order on a best-effort basis. ## Contingent order validation When working with contingent orders (OTO, OCO, OUO), be aware of the following validation rules and error scenarios: **Order list requirements:** - All orders in a contingent group must share the same `order_list_id`. - Parent orders must be submitted before or simultaneously with their children. - Child orders reference their parent via `parent_order_id`. **Modification rules:** - Parent orders can typically be modified while pending, but modifications may cascade to children. - Child orders can be modified independently on most venues, but check venue-specific behavior. - Canceling a parent order will cancel all associated child orders. **Common error scenarios:** | Scenario | System behavior | |----------|-----------------| | Child references non‑existent parent | Order denied with `INVALID_ORDER` error | | Parent canceled before children trigger | Children automatically canceled | | OCO sibling filled before cancel propagates | Partial fill honored, remaining quantity canceled | | Insufficient margin for bracket | Entry may execute, children rejected separately | :::warning Always handle `OrderDenied` and `OrderRejected` events in your strategy, especially for contingent orders where partial failures can leave positions unprotected. ::: ## Bracket orders Bracket orders are an advanced order type that allows traders to set both take-profit and stop-loss levels for a position simultaneously. This involves placing a parent order (entry order) and two child orders: a take-profit `LIMIT` order and a stop-loss `STOP_MARKET` order. When the parent order executes, the system places the child orders. The take-profit closes the position if the market moves favorably, and the stop-loss limits losses if it moves unfavorably. Bracket orders can be easily created using the [OrderFactory](/docs/python-api-latest/common.html#nautilus_trader.common.factories.OrderFactory), which supports various order types, parameters, and instructions. In the following example we bracket a *Market* entry to BUY 10 ETHUSDT-PERP contracts with a take-profit *Limit* at 3,300 USDT and a stop-loss *Stop-Market* triggering at 2,800 USDT. The entry defaults to `MARKET`, the take-profit to `LIMIT`, and the stop-loss to `STOP_MARKET`; the take-profit and stop-loss legs are `reduce_only` and linked with the `OUO` contingency: ```rust tab="Rust" use nautilus_model::{ enums::OrderSide, identifiers::InstrumentId, types::{Price, Quantity}, }; // `bracket()` returns a `bon` builder; finalize with `.call()`. // The result is a `Vec` ordered as [entry, stop-loss, take-profit]. let orders = self .order() .bracket() .instrument_id(InstrumentId::from("ETHUSDT-PERP.BINANCE")) .order_side(OrderSide::Buy) .quantity(Quantity::from(10)) .tp_price(Price::from("3300.00")) // take-profit LIMIT (default) .sl_trigger_price(Price::from("2800.00")) // stop-loss STOP_MARKET (default) .call(); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import OrderList bracket: OrderList = self.order_factory.bracket( instrument_id=InstrumentId.from_str("ETHUSDT-PERP.BINANCE"), order_side=OrderSide.BUY, quantity=Quantity.from_int(10), tp_price=Price.from_str("3300.00"), # <-- take-profit LIMIT (default) sl_trigger_price=Price.from_str("2800.00"), # <-- stop-loss STOP_MARKET (default) ) ``` :::warning You should be aware of the margin requirements of positions, as bracketing a position will consume more order margin. ::: ## Related guides - [Orders](index.md) - Order concepts, execution instructions, and the order factory. - [Emulated orders](emulated.md) - Emulating order types on venues without native support. - [Execution](../execution.md) - Order execution and fill handling. # Emulated orders Source: https://nautilustrader.io/docs/latest/concepts/orders/emulated/ ## Introduction Emulation lets you use order types even when your trading venue does not natively support them. Nautilus locally mimics the behavior of these order types (such as `STOP_LIMIT` or `TRAILING_STOP` orders) while using only `MARKET` and `LIMIT` orders for actual execution on the venue. When you create an emulated order, Nautilus continuously tracks a specific type of market price (specified by the `emulation_trigger` parameter) and based on the order type and conditions you've set, will automatically submit the appropriate fundamental order (`MARKET` / `LIMIT`) when the triggering condition is met. For example, if you create an emulated `STOP_LIMIT` order, Nautilus will monitor the market price until your `stop` price is reached, and then automatically submits a `LIMIT` order to the venue. To perform emulation, Nautilus needs to know which **type of market price** it should monitor. By default, it uses bid and ask prices (quotes), which is why you'll often see `emulation_trigger=TriggerType.DEFAULT` in examples (this is equivalent to using `TriggerType.BID_ASK`). However, Nautilus supports various other price types, that can guide the emulation process. ## Submitting an order for emulation The only requirement to emulate an order is to pass a `TriggerType` to the `emulation_trigger` parameter of an `Order` constructor, or `OrderFactory` creation method. The following emulation trigger types are currently supported: - `NO_TRIGGER`: disables local emulation completely and order is fully submitted to the venue. - `DEFAULT`: which is the same as `BID_ASK`. - `BID_ASK`: emulated using quotes to trigger. - `LAST_PRICE`: emulated using trades to trigger. The choice of trigger type determines how the order emulation will behave: - For `STOP` orders, the trigger price will be compared against the specified trigger type. - For `TRAILING_STOP` orders, the trailing offset will be updated based on the specified trigger type. - For `LIMIT` orders being emulated, the limit price will be compared against the specified trigger type to determine when to release the order as a `MARKET` order. Here are all the available values you can set into `emulation_trigger` parameter and their purposes: | Trigger Type | Description | Common use cases | |:------------------|:-----------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------| | `NO_TRIGGER` | Disables emulation completely. The order is sent directly to the venue without any local processing. | When you want to use the venue's native order handling, or for simple order types that don't need emulation. | | `DEFAULT` | Same as `BID_ASK`. This is the standard choice for most emulated orders. | General‑purpose emulation when you want to work with the "default" type of market prices. | | `BID_ASK` | Uses the best bid and ask prices (quotes) to guide emulation. | Stop orders, trailing stops, and other orders that should react to the current market spread. | | `LAST_PRICE` | Uses the price of the most recent trade to guide emulation. | Orders that should trigger based on actual executed trades rather than quotes. | | `DOUBLE_LAST` | Uses two consecutive last trade prices to confirm the trigger condition. | When you want additional confirmation of price movement before triggering. | | `DOUBLE_BID_ASK` | Uses two consecutive bid/ask price updates to confirm the trigger condition. | When you want extra confirmation of quote movements before triggering. | | `LAST_OR_BID_ASK` | Triggers on either last trade price or bid/ask prices. | When you want to be more responsive to any type of price movement. | | `MID_POINT` | Uses the middle point between the best bid and ask prices. | Orders that should trigger based on the theoretical fair price. | | `MARK_PRICE` | Uses the mark price (common in derivatives markets) for triggering. | Particularly useful for futures and perpetual contracts. | | `INDEX_PRICE` | Uses an underlying index price for triggering. | When trading derivatives that track an index. | ## Technical details The platform makes it possible to emulate most order types locally, regardless of whether the type is supported on a trading venue. The logic and code paths for order emulation are exactly the same for all [environment contexts](../architecture.md#environment-contexts) and use a common `OrderEmulator` component. :::note There is no limitation on the number of emulated orders you can have per running instance. ::: ## Lifecycle An emulated order will progress through the following stages: 1. Submitted by a `Strategy` through the `submit_order` method. 2. Sent to the `RiskEngine` for pre-trade risk checks (it may be denied at this point). 3. Sent to the `OrderEmulator` where it is *held* / emulated. 4. Once triggered, emulated order is transformed into a `MARKET` or `LIMIT` order and released (submitted to the venue). 5. Released order undergoes final risk checks before venue submission. :::note Emulated orders are subject to the same risk controls as *regular* orders, and can be modified and canceled by a trading strategy in the normal way. They will also be included when canceling all orders. ::: :::info An emulated order will retain its original client order ID throughout its entire life cycle, making it easy to query through the cache. ::: ### Held emulated orders The following will occur for an emulated order now *held* by the `OrderEmulator` component: - The original `SubmitOrder` command will be cached. - The emulated order will be processed inside a local `MatchingCore` component. - The `OrderEmulator` will subscribe to any needed market data (if not already) to update the matching core. - The emulated order can be modified (by the trader) and updated (by the market) until *released* or canceled. ### Released emulated orders Once data arrival triggers / matches an emulated order locally, the following *release* actions will occur: - The order will be transformed to either a `MARKET` or `LIMIT` order (see below table) through an additional `OrderInitialized` event. - The orders `emulation_trigger` will be set to `NONE` (it will no longer be treated as an emulated order by any component). - The order attached to the original `SubmitOrder` command will be sent back to the `RiskEngine` for additional checks since any modification/updates. - If not denied, then the command will continue to the `ExecutionEngine` and on to the trading venue via an `ExecutionClient` as normal. ## Order types which can be emulated The following table lists which order types are possible to emulate, and which order type they transform to when being released for submission to the trading venue. | Order type for emulation | Can emulate | Released type | |:-------------------------|:------------|:--------------| | `MARKET` | | n/a | | `MARKET_TO_LIMIT` | | n/a | | `LIMIT` | ✓ | `MARKET` | | `STOP_MARKET` | ✓ | `MARKET` | | `STOP_LIMIT` | ✓ | `LIMIT` | | `MARKET_IF_TOUCHED` | ✓ | `MARKET` | | `LIMIT_IF_TOUCHED` | ✓ | `LIMIT` | | `TRAILING_STOP_MARKET` | ✓ | `MARKET` | | `TRAILING_STOP_LIMIT` | ✓ | `LIMIT` | ## Querying When writing trading strategies, it may be necessary to know the state of emulated orders in the system. There are several ways to query emulation status: ### Through the Cache The following `Cache` methods are available: - `self.cache.orders_emulated(...)`: Returns all currently emulated orders. - `self.cache.is_order_emulated(...)`: Checks if a specific order is emulated. - `self.cache.orders_emulated_count(...)`: Returns the count of emulated orders. See the full [API reference](/docs/python-api-latest/cache.html) for additional details. ### Direct order queries You can query order objects directly using: - `order.is_emulated` If either of these return `False`, then the order has been *released* from the `OrderEmulator`, and so is no longer considered an emulated order (or was never an emulated order). :::warning Do not hold a local reference to an emulated order. The order object transforms when the emulated order is *released*. Use the `Cache` instead. ::: ## Persistence and recovery If a running system either crashes or shuts down with active emulated orders, then they will be reloaded inside the `OrderEmulator` from any configured cache database. This preserves order state across system restarts and recoveries. ## Best practices When working with emulated orders, consider the following best practices: 1. Always use the `Cache` for querying or tracking emulated orders rather than storing local references 2. Be aware that emulated orders transform to different types when released 3. Remember that emulated orders undergo risk checks both at submission and release :::note Order emulation allows you to use advanced order types even on venues that don't natively support them, making your trading strategies more portable across different venues. ::: ## Related guides - [Orders](index.md) - Order concepts, execution instructions, and the order factory. - [Advanced orders](advanced.md) - Order lists, contingency types, and bracket orders. - [Strategies](../strategies.md) - Order management from strategies. # Orders Source: https://nautilustrader.io/docs/latest/concepts/orders/ NautilusTrader supports a broad set of order types and execution instructions, exposing as much of a trading venue's functionality as possible. Traders can define instructions and contingencies for order execution and management across any trading strategy. ## Overview All order types are derived from two fundamentals: *Market* and *Limit* orders. In terms of liquidity, they are opposites. *Market* orders consume liquidity by executing immediately at the best available price, whereas *Limit* orders provide liquidity by resting in the order book at a specified price until matched. NautilusTrader supports nine order types (the `OrderType` enum values), summarized under [Order types](#order-types) with a dedicated guide for each. :::info NautilusTrader provides a unified API for many order types and execution instructions, but not all venues support every option. If an order includes an instruction or option the target venue does not support, the system does not submit it. Instead, it logs a clear, explanatory error. ::: ### Terminology - An order is **aggressive** if its type is `MARKET` or if it executes as a *marketable* order (i.e., takes liquidity). - An order is **passive** if it is not marketable (i.e., provides liquidity). - An order is **active local** if it remains within the local system boundary in one of the following three non-terminal statuses: - `INITIALIZED` - `EMULATED` - `RELEASED` - An order is **in-flight** when at one of the following statuses: - `SUBMITTED` - `PENDING_UPDATE` - `PENDING_CANCEL` - An order is **open** when at one of the following (non-terminal) statuses: - `ACCEPTED` - `TRIGGERED` - `PENDING_UPDATE` - `PENDING_CANCEL` - `PARTIALLY_FILLED` - An order is **closed** when at one of the following (terminal) statuses: - `DENIED` - `REJECTED` - `CANCELED` - `EXPIRED` - `FILLED` ### Order state flow The following diagram illustrates the order lifecycle and primary state transitions: ```mermaid flowchart TB subgraph local ["Active Local"] Initialized Emulated Released end subgraph flight ["In-Flight"] Submitted PendingUpdate PendingCancel end subgraph open ["Open (on venue)"] Accepted Triggered PartiallyFilled end subgraph closed ["Closed (terminal)"] Denied Rejected Canceled Expired Filled end Initialized -->|"Emulation trigger"| Emulated Initialized -->|"Submit"| Submitted Initialized -->|"System denied"| Denied Emulated -->|"Triggered locally"| Released Released --> Submitted Submitted -->|"Venue ACK"| Accepted Submitted --> Rejected Accepted -->|"Stop hit"| Triggered Accepted --> PartiallyFilled Triggered --> PartiallyFilled PartiallyFilled -->|"More fills"| PartiallyFilled Accepted --> PendingUpdate Accepted --> PendingCancel PartiallyFilled --> PendingUpdate PartiallyFilled --> PendingCancel PendingUpdate --> Accepted PendingCancel --> Canceled Accepted --> Filled Triggered --> Filled PartiallyFilled --> Filled PartiallyFilled --> Canceled Accepted --> Expired ``` ### Order status definitions | Status | Description | |--------------------|-------------------------------------------------------------------------------------------| | `INITIALIZED` | Order is instantiated within the Nautilus system. | | `DENIED` | Order was denied by Nautilus for being invalid, unprocessable, or exceeding a risk limit. | | `EMULATED` | Order is being emulated by the `OrderEmulator` component. | | `RELEASED` | Order was released from the `OrderEmulator` component. | | `SUBMITTED` | Order was submitted to the venue (awaiting acknowledgement). | | `ACCEPTED` | Order was acknowledged by the venue as received and valid (may now be working). | | `REJECTED` | Order was rejected by the trading venue. | | `CANCELED` | Order was canceled (terminal). | | `EXPIRED` | Order reached its GTD expiration (terminal). | | `TRIGGERED` | Order's STOP price was triggered on the venue. | | `PENDING_UPDATE` | Order is pending a modification request on the venue. | | `PENDING_CANCEL` | Order is pending a cancellation request on the venue. | | `PARTIALLY_FILLED` | Order has been partially filled on the venue. | | `FILLED` | Order has been completely filled (terminal). | ## Execution instructions Certain venues allow a trader to specify conditions and restrictions on how an order will be processed and executed. The following is a brief summary of the different execution instructions available. ### Time in force The order's time in force specifies how long the order will remain open or active before any remaining quantity is canceled. - `GTC` **(Good Till Cancel)**: The order remains active until canceled by the trader or the venue. - `IOC` **(Immediate or Cancel / Fill and Kill)**: The order executes immediately, with any unfilled portion canceled. - `FOK` **(Fill or Kill)**: The order executes immediately in full or not at all. - `GTD` **(Good Till Date)**: The order remains active until a specified expiration date and time. - `DAY` **(Good for session/day)**: The order remains active until the end of the current trading session. - `AT_THE_OPEN` **(OPG)**: The order is only active at the open of the trading session. - `AT_THE_CLOSE`: The order is only active at the close of the trading session. ### Expire time This instruction is to be used in conjunction with the `GTD` time in force to specify the time at which the order will expire and be removed from the venue's order book (or order management system). ### Post-only An order which is marked as `post_only` will only ever participate in providing liquidity to the limit order book, and never initiating a trade which takes liquidity as an aggressor. This option is important for market makers, or traders seeking to restrict the order to a liquidity *maker* fee tier. ### Reduce-only An order which is set as `reduce_only` will only ever reduce an existing position on an instrument and never open a new position (if already flat). The exact behavior of this instruction can vary between venues. However, the behavior in the Nautilus `SimulatedExchange` is typical of a real venue. - Order will be canceled if the associated position is closed (becomes flat). - Order quantity will be reduced as the associated position's size decreases. ### Display quantity The `display_qty` specifies the portion of a *Limit* order which is displayed on the limit order book. These are also known as iceberg orders as there is a visible portion to be displayed, with more quantity which is hidden. Specifying a display quantity of zero is also equivalent to setting an order as `hidden`. ### Trigger type Also known as [trigger method](https://www.interactivebrokers.com/en/software/tws/usersguidebook/configuretws/Modify%20the%20Stop%20Trigger%20Method.htm) which is applicable to conditional trigger orders, specifying the method of triggering the stop price. - `DEFAULT`: The default trigger type for the venue (typically `LAST_PRICE` or `BID_ASK`). - `LAST_PRICE`: The trigger price will be based on the last traded price. - `BID_ASK`: The trigger price will be based on the bid for buy orders and ask for sell orders. - `DOUBLE_LAST`: The trigger price will be based on the last two consecutive last prices. - `DOUBLE_BID_ASK`: The trigger price will be based on the last two consecutive bid or ask prices as applicable. - `LAST_OR_BID_ASK`: The trigger price will be based on either the last price or bid/ask. - `MID_POINT`: The trigger price will be based on the mid-point between the bid and ask. - `MARK_PRICE`: The trigger price will be based on the venue's mark price for the instrument. - `INDEX_PRICE`: The trigger price will be based on the venue's index price for the instrument. ### Trigger offset type Applicable to conditional trailing-stop trigger orders, specifies the method of triggering modification of the stop price based on the offset from the *market* (bid, ask or last price as applicable). - `DEFAULT`: The default offset type for the venue (typically `PRICE`). - `PRICE`: The offset is based on a price difference. - `BASIS_POINTS`: The offset is based on a price percentage difference expressed in basis points (100bp = 1%). - `TICKS`: The offset is based on a number of ticks. - `PRICE_TIER`: The offset is based on a venue-specific price tier. ### Contingent orders More advanced relationships can be specified between orders. For example, child orders can be assigned to trigger only when the parent is activated or filled, or orders can be linked so that one cancels or reduces the quantity of another. See the [Advanced orders](advanced.md) guide for more details. ## Order factory The easiest way to create new orders is by using the built-in `OrderFactory`, which is automatically attached to every `Strategy` class. This factory will take care of lower level details - such as ensuring the correct trader ID and strategy ID are assigned, generation of a necessary initialization ID and timestamp, and abstracts away parameters which don't necessarily apply to the order type being created, or are only needed to specify more advanced execution instructions. This leaves the factory with simpler order creation methods to work with, all the examples use an `OrderFactory` from within a `Strategy` context. See the [`OrderFactory` API Reference](/docs/python-api-latest/common.html#nautilus_trader.common.factories.OrderFactory) for further details. ## Order types NautilusTrader supports the following order types. Each links to a dedicated guide with a code example; optional parameters are marked with a comment showing the default value. | Order type | Category | Description | |----------------------------------------------------|----------------------|--------------------------------------------------------------------------| | [`MARKET`](market.md) | Aggressive | Trades the quantity immediately at the best available price. | | [`LIMIT`](limit.md) | Passive | Rests in the book and trades only at the limit price or better. | | [`STOP_MARKET`](stop_market.md) | Conditional | Once the trigger price is hit, places a *Market* order. | | [`STOP_LIMIT`](stop_limit.md) | Conditional | Once the trigger price is hit, places a *Limit* order at the set price. | | [`MARKET_TO_LIMIT`](market_to_limit.md) | Hybrid | Submits as *Market*; any remainder rests as a *Limit* at the fill price. | | [`MARKET_IF_TOUCHED`](market_if_touched.md) | Conditional | Once the trigger price is touched, places a *Market* order. | | [`LIMIT_IF_TOUCHED`](limit_if_touched.md) | Conditional | Once the trigger price is touched, places a *Limit* order at the set price. | | [`TRAILING_STOP_MARKET`](trailing_stop_market.md) | Conditional trailing | Trails the trigger by an offset, then places a *Market* order. | | [`TRAILING_STOP_LIMIT`](trailing_stop_limit.md) | Conditional trailing | Trails the trigger by an offset, then places a *Limit* order. | ### FIX OrdType mapping Each type maps to the nearest FIX 5.0 SP2 [`OrdType <40>`](https://www.onixs.biz/fix-dictionary/5.0.sp2/tagnum_40.html) value, where the protocol defines one: | Order type | FIX `OrdType <40>` | |----------------------|--------------------------------------| | Market | `1` (Market) | | Limit | `2` (Limit) | | Stop‑Market | `3` (Stop) | | Stop‑Limit | `4` (Stop Limit) | | Market‑To‑Limit | `K` (Market With Left Over as Limit) | | Market‑If‑Touched | `J` (Market If Touched) | | Limit‑If‑Touched | no dedicated value † | | Trailing‑Stop‑Market | `3` (Stop) + trailing peg | | Trailing‑Stop‑Limit | `4` (Stop Limit) + trailing peg | † FIX defines no dedicated `OrdType` for *Limit-If-Touched*; it is commonly sent as `4` (Stop Limit) with a favorable trigger. Trailing stops likewise have no dedicated value and are modeled as `3`/`4` plus trailing peg fields. ## Advanced orders Orders can be grouped into lists and linked with contingency relationships (OTO, OCO, OUO), and bracket orders attach take-profit and stop-loss children to an entry. See the [Advanced orders](advanced.md) guide for order lists, contingency types, validation rules, and brackets. ## Emulated orders NautilusTrader can locally emulate order types that a venue does not natively support, using only `MARKET` and `LIMIT` orders for actual execution. See the [Emulated orders](emulated.md) guide for the emulation lifecycle, supported types, querying, and best practices. ## Related guides - [Events](../events.md) - Order events, position events, and handler dispatch. - [Execution](../execution.md) - Order execution and fill handling. - [Positions](../positions.md) - Positions created from order fills. - [Strategies](../strategies.md) - Order management from strategies. # Limit Source: https://nautilustrader.io/docs/latest/concepts/orders/limit/ `FIX OrdType <40>=2` A *Limit* order is placed on the limit order book at a specific price, and will only execute at that price (or better). ## Use cases Use a *Limit* order when you want to control the execution price, and optionally provide liquidity: market making, scaling into or out of a position at chosen levels, or capturing maker fee tiers with `post_only`. The advantage is that it never fills worse than your price. The tradeoff is no execution guarantee: the order may rest unfilled, or only partially fill, if the market never reaches or holds your price. ## Example In the following example we create a *Limit* order on the Binance Futures Crypto exchange to SELL 20 ETHUSDT-PERP Perpetual Futures contracts at a limit price of 5000 USDT, as a market maker. ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce}, identifiers::InstrumentId, types::{Price, Quantity}, }; let order = self.order().limit( InstrumentId::from("ETHUSDT-PERP.BINANCE"), OrderSide::Sell, Quantity::from(20), Price::from("5000.00"), Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(true), // post_only (default false) Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty (default full display) None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params None, // tags None, // client_order_id ); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import LimitOrder order: LimitOrder = self.order_factory.limit( instrument_id=InstrumentId.from_str("ETHUSDT-PERP.BINANCE"), order_side=OrderSide.SELL, quantity=Quantity.from_int(20), price=Price.from_str("5_000.00"), time_in_force=TimeInForce.GTC, # <-- optional (default GTC) expire_time=None, # <-- optional (default None) post_only=True, # <-- optional (default False) reduce_only=False, # <-- optional (default False) display_qty=None, # <-- optional (default None which indicates full display) tags=None, # <-- optional (default None) ) ``` See the [`LimitOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.limit.LimitOrder) for further details. ## Related guides - [Orders](index.md) - Order concepts, execution instructions, and the order factory. - [Emulated orders](emulated.md) - Emulating *Limit* orders, released as *Market* orders on trigger. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Limit-If-Touched Source: https://nautilustrader.io/docs/latest/concepts/orders/limit_if_touched/ `FIX OrdType <40>` no dedicated value (commonly `4` Stop Limit with a favorable trigger) A *Limit-If-Touched* order is a conditional order which once triggered will immediately place a *Limit* order at the specified price. ## Use cases Use a *Limit-If-Touched* order to arm a price-protected order only once a trigger is touched, for example activating a take-profit *Limit* as price approaches a target rather than resting it early. The advantage is conditional activation combined with a capped fill price. The tradeoff, as with a *Stop-Limit*, is that the order may not fill if price moves through the limit after the trigger. ## Example In the following example we create a *Limit-If-Touched* order to BUY 5 BTCUSDT-PERP Perpetual Futures contracts on the Binance Futures exchange at a limit price of 30,100 USDT (once the market hits the trigger price of 30,150 USDT), active until midday 6th June, 2022 (UTC): ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::{OrderSide, TimeInForce, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; use ustr::Ustr; let order = self.order().limit_if_touched( InstrumentId::from("BTCUSDT-PERP.BINANCE"), OrderSide::Buy, Quantity::from(5), Price::from("30100"), Price::from("30150"), Some(TriggerType::LastPrice), // optional (default DEFAULT) Some(TimeInForce::Gtd), // optional (default GTC) Some(UnixNanos::from(1_654_516_800_000_000_000_u64)), // 2022-06-06T12:00:00 UTC Some(true), // post_only (default false) Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params Some(vec![Ustr::from("TAKE_PROFIT")]), // tags None, // client_order_id ); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import LimitIfTouchedOrder order: LimitIfTouchedOrder = self.order_factory.limit_if_touched( instrument_id=InstrumentId.from_str("BTCUSDT-PERP.BINANCE"), order_side=OrderSide.BUY, quantity=Quantity.from_int(5), price=Price.from_str("30_100"), trigger_price=Price.from_str("30_150"), trigger_type=TriggerType.LAST_PRICE, # <-- optional (default DEFAULT) time_in_force=TimeInForce.GTD, # <-- optional (default GTC) expire_time=pd.Timestamp("2022-06-06T12:00"), post_only=True, # <-- optional (default False) reduce_only=False, # <-- optional (default False) tags=["TAKE_PROFIT"], # <-- optional (default None) ) ``` See the [`LimitIfTouchedOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.limit_if_touched.LimitIfTouchedOrder) for further details. ## Related guides - [Orders](index.md#trigger-type) - Trigger types and other execution instructions. - [Emulated orders](emulated.md) - Emulating conditional orders on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Market Source: https://nautilustrader.io/docs/latest/concepts/orders/market/ `FIX OrdType <40>=1` A *Market* order is an instruction by the trader to immediately trade the given quantity at the best price available. You can also specify several time in force options, and indicate whether this order is only intended to reduce a position. ## Use cases Use a *Market* order when filling matters more than the exact price: urgent risk reduction, entering a fast-moving liquid market, or crossing a tight spread where waiting costs more than the spread. The advantage is near-certain, immediate execution. The tradeoff is no price protection: you pay the spread and risk slippage in thin or fast markets, so it suits liquid instruments far more than illiquid ones. ## Example In the following example we create a *Market* order on the Interactive Brokers [IdealPro](https://ibkr.info/node/1708) Forex ECN to BUY 100,000 AUD using USD: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce}, identifiers::InstrumentId, types::Quantity, }; use ustr::Ustr; let order = self.order().market( InstrumentId::from("AUD/USD.IDEALPRO"), OrderSide::Buy, Quantity::from(100_000), Some(TimeInForce::Ioc), // optional (default GTC) Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // exec_algorithm_id None, // exec_algorithm_params Some(vec![Ustr::from("ENTRY")]), // tags None, // client_order_id (auto-generated if None) ); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model import InstrumentId from nautilus_trader.model import Quantity from nautilus_trader.model.orders import MarketOrder order: MarketOrder = self.order_factory.market( instrument_id=InstrumentId.from_str("AUD/USD.IDEALPRO"), order_side=OrderSide.BUY, quantity=Quantity.from_int(100_000), time_in_force=TimeInForce.IOC, # <-- optional (default GTC) reduce_only=False, # <-- optional (default False) tags=["ENTRY"], # <-- optional (default None) ) ``` See the [`MarketOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.market.MarketOrder) for further details. ## Related guides - [Orders](index.md) - Order concepts, execution instructions, and the order factory. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Market-If-Touched Source: https://nautilustrader.io/docs/latest/concepts/orders/market_if_touched/ `FIX OrdType <40>=J` (Market If Touched) A *Market-If-Touched* order is a conditional order which once triggered will immediately place a *Market* order. This order type is often used to enter a new position on a stop price, or to take profits for an existing position, either as a SELL order against LONG positions, or as a BUY order against SHORT positions. ## Use cases Use a *Market-If-Touched* order to act with execution certainty when a target price is touched, such as entering on a pullback to a level or taking profit at a target. It behaves like a stop in the opposite direction (buying below or selling above the current market) and converts to a *Market* order on trigger. The tradeoff matches any market execution: the touch price is not the fill price, and the fill can slip in fast markets. ## Example In the following example we create a *Market-If-Touched* order on the Binance Futures exchange to SELL 10 ETHUSDT-PERP Perpetual Futures contracts at a trigger price of 10,000 USDT, active until further notice: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; use ustr::Ustr; let order = self.order().market_if_touched( InstrumentId::from("ETHUSDT-PERP.BINANCE"), OrderSide::Sell, Quantity::from(10), Price::from("10000.00"), Some(TriggerType::LastPrice), // optional (default DEFAULT) Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params Some(vec![Ustr::from("ENTRY")]), // tags None, // client_order_id ); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import MarketIfTouchedOrder order: MarketIfTouchedOrder = self.order_factory.market_if_touched( instrument_id=InstrumentId.from_str("ETHUSDT-PERP.BINANCE"), order_side=OrderSide.SELL, quantity=Quantity.from_int(10), trigger_price=Price.from_str("10_000.00"), trigger_type=TriggerType.LAST_PRICE, # <-- optional (default DEFAULT) time_in_force=TimeInForce.GTC, # <-- optional (default GTC) expire_time=None, # <-- optional (default None) reduce_only=False, # <-- optional (default False) tags=["ENTRY"], # <-- optional (default None) ) ``` See the [`MarketIfTouchedOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.market_if_touched.MarketIfTouchedOrder) for further details. ## Related guides - [Orders](index.md#trigger-type) - Trigger types and other execution instructions. - [Emulated orders](emulated.md) - Emulating conditional orders on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Market-To-Limit Source: https://nautilustrader.io/docs/latest/concepts/orders/market_to_limit/ `FIX OrdType <40>=K` (Market With Left Over as Limit) A *Market-To-Limit* order submits as a market order at the current best price. If the order partially fills, the system cancels the remainder and resubmits it as a *Limit* order at the executed price. ## Use cases Use a *Market-To-Limit* order to take the liquidity available at the best price immediately, without sweeping deeper levels at worse prices: helpful in thin books, or for larger orders where you want the touch price but not the market impact of walking the book. The advantage is an immediate fill at the best price with any remainder resting there as a *Limit* rather than chasing. The tradeoff is that the unfilled remainder may sit unexecuted if the market moves away. ## Example In the following example we create a *Market-To-Limit* order on the Interactive Brokers [IdealPro](https://ibkr.info/node/1708) Forex ECN to BUY 200,000 USD using JPY: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce}, identifiers::InstrumentId, types::Quantity, }; let order = self.order().market_to_limit( InstrumentId::from("USD/JPY.IDEALPRO"), OrderSide::Buy, Quantity::from(200_000), Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty (default full display) None, // exec_algorithm_id None, // exec_algorithm_params None, // tags None, // client_order_id ); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model import InstrumentId from nautilus_trader.model import Quantity from nautilus_trader.model.orders import MarketToLimitOrder order: MarketToLimitOrder = self.order_factory.market_to_limit( instrument_id=InstrumentId.from_str("USD/JPY.IDEALPRO"), order_side=OrderSide.BUY, quantity=Quantity.from_int(200_000), time_in_force=TimeInForce.GTC, # <-- optional (default GTC) reduce_only=False, # <-- optional (default False) display_qty=None, # <-- optional (default None which indicates full display) tags=None, # <-- optional (default None) ) ``` See the [`MarketToLimitOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.market_to_limit.MarketToLimitOrder) for further details. ## Related guides - [Orders](index.md) - Order concepts, execution instructions, and the order factory. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Stop-Limit Source: https://nautilustrader.io/docs/latest/concepts/orders/stop_limit/ `FIX OrdType <40>=4` (Stop Limit) A *Stop-Limit* order is a conditional order which once triggered will immediately place a *Limit* order at the specified price. ## Use cases Use a *Stop-Limit* order when you want a stop trigger but also a cap on the worst acceptable fill, such as a protective exit or breakout entry where you refuse to trade beyond a price. The advantage is price protection on the released *Limit* order. The tradeoff is the central risk versus a *Stop-Market*: if the market gaps through both the trigger and the limit, the order may not fill at all, leaving a position unprotected. ## Example In the following example we create a *Stop-Limit* order on the Currenex FX ECN to BUY 50,000 GBP at a limit price of 1.3000 USD once the market hits the trigger price of 1.30010 USD, active until midday 6th June, 2022 (UTC): ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::{OrderSide, TimeInForce, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; let order = self.order().stop_limit( InstrumentId::from("GBP/USD.CURRENEX"), OrderSide::Buy, Quantity::from(50_000), Price::from("1.30000"), Price::from("1.30010"), Some(TriggerType::BidAsk), // optional (default DEFAULT) Some(TimeInForce::Gtd), // optional (default GTC) Some(UnixNanos::from(1_654_516_800_000_000_000_u64)), // 2022-06-06T12:00:00 UTC Some(true), // post_only (default false) Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params None, // tags None, // client_order_id ); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import StopLimitOrder order: StopLimitOrder = self.order_factory.stop_limit( instrument_id=InstrumentId.from_str("GBP/USD.CURRENEX"), order_side=OrderSide.BUY, quantity=Quantity.from_int(50_000), price=Price.from_str("1.30000"), trigger_price=Price.from_str("1.30010"), trigger_type=TriggerType.BID_ASK, # <-- optional (default DEFAULT) time_in_force=TimeInForce.GTD, # <-- optional (default GTC) expire_time=pd.Timestamp("2022-06-06T12:00"), post_only=True, # <-- optional (default False) reduce_only=False, # <-- optional (default False) tags=None, # <-- optional (default None) ) ``` See the [`StopLimitOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.stop_limit.StopLimitOrder) for further details. ## Related guides - [Orders](index.md#trigger-type) - Trigger types and other execution instructions. - [Emulated orders](emulated.md) - Emulating conditional orders on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Stop-Market Source: https://nautilustrader.io/docs/latest/concepts/orders/stop_market/ `FIX OrdType <40>=3` (Stop) A *Stop-Market* order is a conditional order which once triggered, will immediately place a *Market* order. This order type is often used as a stop-loss to limit losses, either as a SELL order against LONG positions, or as a BUY order against SHORT positions. ## Use cases Use a *Stop-Market* order when you need execution certainty once a price level is breached, such as a protective stop-loss or a breakout entry. Because it converts to a *Market* order on trigger, the position is almost always opened or closed. The tradeoff is that the trigger price is not the fill price: in fast or gapping markets the fill can land well beyond the stop, so it trades price certainty for execution certainty (the opposite of a *Stop-Limit*). ## Example In the following example we create a *Stop-Market* order on the Binance Spot/Margin exchange to SELL 1 BTC at a trigger price of 100,000 USDT, active until further notice: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; let order = self.order().stop_market( InstrumentId::from("BTCUSDT.BINANCE"), OrderSide::Sell, Quantity::from(1), Price::from("100000"), Some(TriggerType::LastPrice), // optional (default DEFAULT) Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(false), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params None, // tags None, // client_order_id ); ``` ```python tab="Python" from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import StopMarketOrder order: StopMarketOrder = self.order_factory.stop_market( instrument_id=InstrumentId.from_str("BTCUSDT.BINANCE"), order_side=OrderSide.SELL, quantity=Quantity.from_int(1), trigger_price=Price.from_int(100_000), trigger_type=TriggerType.LAST_PRICE, # <-- optional (default DEFAULT) time_in_force=TimeInForce.GTC, # <-- optional (default GTC) expire_time=None, # <-- optional (default None) reduce_only=False, # <-- optional (default False) tags=None, # <-- optional (default None) ) ``` See the [`StopMarketOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.stop_market.StopMarketOrder) for further details. ## Related guides - [Orders](index.md#trigger-type) - Trigger types and other execution instructions. - [Emulated orders](emulated.md) - Emulating conditional orders on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Trailing-Stop-Limit Source: https://nautilustrader.io/docs/latest/concepts/orders/trailing_stop_limit/ `FIX OrdType <40>=4` (Stop Limit) + trailing peg A *Trailing-Stop-Limit* order is a conditional order which trails a stop trigger price a fixed offset away from the defined market price. Once triggered a *Limit* order will immediately be placed at the defined price (which is also updated as the market moves until triggered). ## Use cases Use a *Trailing-Stop-Limit* order when you want the dynamic trail of a trailing stop but also a cap on the fill price. The advantage is trailing protection combined with price control. The tradeoff is the trailing analogue of a *Stop-Limit*: in a fast reversal the released *Limit* may not fill, leaving the position open. ## Example In the following example we create a *Trailing-Stop-Limit* order on the Currenex FX ECN to BUY 1,250,000 AUD using USD at a limit price of 0.71000 USD, activating at 0.72000 USD then trailing at a stop offset of 0.00100 USD away from the current ask price, active until further notice: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce, TrailingOffsetType, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let order = self.order().trailing_stop_limit( InstrumentId::from("AUD/USD.CURRENEX"), OrderSide::Buy, Quantity::from(1_250_000), Price::from("0.71000"), // limit price dec!(0.00050), // limit_offset dec!(0.00100), // trailing_offset Some(TrailingOffsetType::Price), // optional (default PRICE) Some(Price::from("0.72000")), // activation_price None, // trigger_price (falls back to activation_price) Some(TriggerType::BidAsk), // optional (default DEFAULT) Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(false), // post_only (default false) Some(true), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params Some(vec![Ustr::from("TRAILING_STOP")]), // tags None, // client_order_id ); ``` ```python tab="Python" import pandas as pd from decimal import Decimal from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model.enums import TrailingOffsetType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import TrailingStopLimitOrder order: TrailingStopLimitOrder = self.order_factory.trailing_stop_limit( instrument_id=InstrumentId.from_str("AUD/USD.CURRENEX"), order_side=OrderSide.BUY, quantity=Quantity.from_int(1_250_000), price=Price.from_str("0.71000"), activation_price=Price.from_str("0.72000"), trigger_type=TriggerType.BID_ASK, # <-- optional (default DEFAULT) limit_offset=Decimal("0.00050"), trailing_offset=Decimal("0.00100"), trailing_offset_type=TrailingOffsetType.PRICE, time_in_force=TimeInForce.GTC, # <-- optional (default GTC) expire_time=None, # <-- optional (default None) reduce_only=True, # <-- optional (default False) tags=["TRAILING_STOP"], # <-- optional (default None) ) ``` See the [`TrailingStopLimitOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.trailing_stop_limit.TrailingStopLimitOrder) for further details. ## Related guides - [Orders](index.md#trigger-offset-type) - Trigger and trailing offset types. - [Emulated orders](emulated.md) - Emulating trailing stops on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Trailing-Stop-Market Source: https://nautilustrader.io/docs/latest/concepts/orders/trailing_stop_market/ `FIX OrdType <40>=3` (Stop) + trailing peg A *Trailing-Stop-Market* order is a conditional order which trails a stop trigger price a fixed offset away from the defined market price. Once triggered a *Market* order will immediately be placed. ## Use cases Use a *Trailing-Stop-Market* order to lock in gains while letting a position run: the trigger trails favorable moves by a fixed offset and only fires on a reversal, with no manual adjustment. The advantage is dynamic protection plus execution certainty on trigger. The tradeoff is choosing the offset, which balances whipsaw risk (too tight) against giving back profit (too wide), and the market fill can still slip on a sharp reversal. ## Example In the following example we create a *Trailing-Stop-Market* order on the Binance Futures exchange to SELL 10 ETHUSD-PERP COIN_M margined Perpetual Futures Contracts activating at a price of 5,000 USD, then trailing at an offset of 1% (in basis points) away from the current last traded price: ```rust tab="Rust" use nautilus_model::{ enums::{OrderSide, TimeInForce, TrailingOffsetType, TriggerType}, identifiers::InstrumentId, types::{Price, Quantity}, }; use rust_decimal::Decimal; use ustr::Ustr; let order = self.order().trailing_stop_market( InstrumentId::from("ETHUSD-PERP.BINANCE"), OrderSide::Sell, Quantity::from(10), Decimal::from(100), // trailing_offset Some(TrailingOffsetType::BasisPoints), // optional (default PRICE) Some(Price::from("5000")), // activation_price None, // trigger_price (falls back to activation_price) Some(TriggerType::LastPrice), // optional (default DEFAULT) Some(TimeInForce::Gtc), // optional (default GTC) None, // expire_time Some(true), // reduce_only (default false) None, // quote_quantity (default false) None, // display_qty None, // emulation_trigger None, // trigger_instrument_id None, // exec_algorithm_id None, // exec_algorithm_params Some(vec![Ustr::from("TRAILING_STOP-1")]), // tags None, // client_order_id ); ``` ```python tab="Python" import pandas as pd from decimal import Decimal from nautilus_trader.model.enums import OrderSide from nautilus_trader.model.enums import TimeInForce from nautilus_trader.model.enums import TriggerType from nautilus_trader.model.enums import TrailingOffsetType from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model.orders import TrailingStopMarketOrder order: TrailingStopMarketOrder = self.order_factory.trailing_stop_market( instrument_id=InstrumentId.from_str("ETHUSD-PERP.BINANCE"), order_side=OrderSide.SELL, quantity=Quantity.from_int(10), activation_price=Price.from_str("5_000"), trigger_type=TriggerType.LAST_PRICE, # <-- optional (default DEFAULT) trailing_offset=Decimal(100), trailing_offset_type=TrailingOffsetType.BASIS_POINTS, time_in_force=TimeInForce.GTC, # <-- optional (default GTC) expire_time=None, # <-- optional (default None) reduce_only=True, # <-- optional (default False) tags=["TRAILING_STOP-1"], # <-- optional (default None) ) ``` See the [`TrailingStopMarketOrder` API Reference](/docs/python-api-latest/model/orders.html#nautilus_trader.model.orders.trailing_stop_market.TrailingStopMarketOrder) for further details. ## Related guides - [Orders](index.md#trigger-offset-type) - Trigger and trailing offset types. - [Emulated orders](emulated.md) - Emulating trailing stops on venues without native support. - [Execution](../execution.md) - How orders reach the venue and fills are handled. # Betting Instrument Source: https://nautilustrader.io/docs/latest/concepts/instruments/betting_instrument/ `BettingInstrument` represents one selection in a sports or gaming market. It carries event, competition, market, and selection metadata so Nautilus can treat the selection as an instrument with prices, sizes, limits, margins, and fees. Examples include Betfair match-odds selections and handicap market selections. ## Fields | Field | Type | Required/default | Notes | |----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native or generated venue symbol. | | `event_type_id` | `u64` | Required | Event type identifier. | | `event_type_name` | `Ustr` | Required | Event type name, such as a sport. | | `competition_id` | `u64` | Required | Competition identifier. | | `competition_name` | `Ustr` | Required | Competition name. | | `event_id` | `u64` | Required | Event identifier. | | `event_name` | `Ustr` | Required | Event name. | | `event_country_code` | `Ustr` | Required | Event country code. | | `event_open_date` | `UnixNanos` | Required | Event open time. | | `betting_type` | `Ustr` | Required | Betting type published by the venue. | | `market_id` | `Ustr` | Required | Market identifier. | | `market_name` | `Ustr` | Required | Market name. | | `market_type` | `Ustr` | Required | Market type, such as match odds. | | `market_start_time` | `UnixNanos` | Required | Market start time. | | `selection_id` | `u64` | Required | Selection or runner identifier. | | `selection_name` | `Ustr` | Required | Selection or runner name. | | `selection_handicap` | `f64` | Required | Handicap value for handicap markets. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Price step, often set by a tick scheme. | | `size_increment` | `Quantity` | Required | Minimum size step. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `1` | Initial margin rate. | | `margin_maint` | `Option` | `1` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native or generated venue symbol. | | `event_type_id` | `int` | Required | Event type identifier. | | `event_type_name` | `str` | Required | Event type name, such as a sport. | | `competition_id` | `int` | Required | Competition identifier. | | `competition_name` | `str` | Required | Competition name. | | `event_id` | `int` | Required | Event identifier. | | `event_name` | `str` | Required | Event name. | | `event_country_code` | `str` | Required | Event country code. | | `event_open_date` | `int` | Required | Event open time. | | `betting_type` | `str` | Required | Betting type published by the venue. | | `market_id` | `str` | Required | Market identifier. | | `market_name` | `str` | Required | Market name. | | `market_type` | `str` | Required | Market type, such as match odds. | | `market_start_time` | `int` | Required | Market start time. | | `selection_id` | `int` | Required | Selection or runner identifier. | | `selection_name` | `str` | Required | Selection or runner name. | | `selection_handicap` | `float` | Required | Handicap value for handicap markets. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Price step, often set by a tick scheme. | | `size_increment` | `Quantity` | Required | Minimum size step. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `1` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `1` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python builds the instrument ID and raw symbol from the venue, market, selection, and handicap fields. Rust receives them as `instrument_id` and `raw_symbol`.* ## Behavior - `BettingInstrument` has asset class `Alternative` and instrument class `SportsBetting`. - Each selection or runner is modeled as its own instrument. - Betting instruments commonly use a registered tick scheme for valid odds steps. - Margin defaults to one because staking a bet typically reserves the full stake. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::BettingInstrument, types::{Currency, Money, Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let event_open = Utc.with_ymd_and_hms(2022, 2, 7, 23, 30, 0).unwrap(); let market_start = Utc.with_ymd_and_hms(2022, 2, 7, 23, 30, 0).unwrap(); let selection = BettingInstrument::builder() .instrument_id(InstrumentId::from("1-123456789.BETFAIR")) .raw_symbol(Symbol::from("1-123456789")) .event_type_id(6423) .event_type_name(Ustr::from("American Football")) .competition_id(12_282_733) .competition_name(Ustr::from("NFL")) .event_id(29_678_534) .event_name(Ustr::from("NFL")) .event_country_code(Ustr::from("GB")) .event_open_date(UnixNanos::from(event_open.timestamp_nanos_opt().unwrap() as u64)) .betting_type(Ustr::from("ODDS")) .market_id(Ustr::from("1-123456789")) .market_name(Ustr::from("AFC Conference Winner")) .market_type(Ustr::from("SPECIAL")) .market_start_time(UnixNanos::from(market_start.timestamp_nanos_opt().unwrap() as u64)) .selection_id(50214) .selection_name(Ustr::from("Kansas City Chiefs")) .selection_handicap(0.0) .currency(Currency::from("GBP")) .price_precision(2) .size_precision(2) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.01")) .max_quantity(Quantity::from("1000")) .min_quantity(Quantity::from("1")) .max_notional(Money::from("10000 GBP")) .min_notional(Money::from("10 GBP")) .max_price(Price::from("100.00")) .min_price(Price::from("1.00")) .margin_init(dec!(1)) .margin_maint(dec!(1)) .maker_fee(dec!(0)) .taker_fee(dec!(0)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import BettingInstrument from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Money from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol from nautilus_trader.model import Venue GBP = Currency.from_str("GBP") selection = BettingInstrument( instrument_id=InstrumentId(Symbol("1-123456789-50214"), Venue("BETFAIR")), raw_symbol=Symbol("1-123456789-50214"), event_type_id=6423, event_type_name="American Football", competition_id=12282733, competition_name="NFL", event_id=29678534, event_name="NFL", event_country_code="GB", event_open_date=pd.Timestamp("2022-02-07 23:30:00+00:00").value, betting_type="ODDS", market_id="1-123456789", market_name="AFC Conference Winner", market_type="SPECIAL", market_start_time=pd.Timestamp("2022-02-07 23:30:00+00:00").value, selection_id=50214, selection_name="Kansas City Chiefs", selection_handicap=0.0, currency=GBP, price_precision=2, size_precision=2, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.01"), min_notional=Money(1, GBP), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `BettingInstrument` instruments include: - [Betfair](../../integrations/betfair.md) for sports betting markets. - [Betfair v2](../../integrations/betfair_v2.md) for sports betting markets. ## Related guides - [Accounting](../accounting.md) covers betting account behavior. - [Data](../data.md) explains market data that references instruments. # Binary Option Source: https://nautilustrader.io/docs/latest/concepts/instruments/binary_option/ `BinaryOption` represents a binary outcome instrument that settles to a fixed payoff based on whether a condition is true. It can model prediction markets, binary options, or venue-specific yes/no contracts. Examples include prediction market outcomes and binary event contracts. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|-------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the outcome market. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `activation_ns` | `UnixNanos` | Required | Contract activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Contract expiration timestamp. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `outcome` | `Option` | `None` | Outcome label when the venue provides it. | | `description` | `Option` | `None` | Human‑readable market description. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|-------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the outcome market. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `activation_ns` | `int` | Required | Contract activation timestamp. | | `expiration_ns` | `int` | Required | Contract expiration timestamp. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `outcome` | `str \| None` | `None` | Outcome label when the venue provides it. | | `description` | `str \| None` | `None` | Human‑readable market description. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `BinaryOption` has instrument class `BinaryOption`. - It is never inverse and uses a multiplier and lot size of one. - Many venues quote binary outcomes between zero and one, but the venue defines the allowed price range and tick size. - `outcome` and `description` provide human-readable context for the contract. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol, Venue}, instruments::BinaryOption, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let raw_symbol = Symbol::from( "0x12a0cb60174abc437bf1178367c72d11f069e1a3add20b148fb0ab4279b772b2-92544998123698303655208967887569360731013655782348975589292031774495159624905", ); let expiration = Utc.with_ymd_and_hms(2024, 1, 1, 0, 0, 0).unwrap(); let yes_outcome = BinaryOption::builder() .instrument_id(InstrumentId::new(raw_symbol, Venue::from("POLYMARKET"))) .raw_symbol(raw_symbol) .asset_class(AssetClass::Alternative) .currency(Currency::from("USDC")) .activation_ns(UnixNanos::default()) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(3) .size_precision(2) .price_increment(Price::from("0.001")) .size_increment(Quantity::from("0.01")) .outcome(Ustr::from("Yes")) .description(Ustr::from("Will the outcome of this market be 'Yes'?")) .min_quantity(Quantity::from("5")) .maker_fee(dec!(0)) .taker_fee(dec!(0)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal import pandas as pd from nautilus_trader.model import AssetClass from nautilus_trader.model import BinaryOption from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol from nautilus_trader.model import Venue raw_symbol = Symbol( "0x12a0cb60174abc437bf1178367c72d11f069e1a3add20b148fb0ab4279b772b2-92544998123698303655208967887569360731013655782348975589292031774495159624905", ) price_increment = Price.from_str("0.001") size_increment = Quantity.from_str("0.01") yes_outcome = BinaryOption( instrument_id=InstrumentId(raw_symbol, Venue("POLYMARKET")), raw_symbol=raw_symbol, asset_class=AssetClass.ALTERNATIVE, currency=Currency.from_str("USDC"), activation_ns=0, expiration_ns=pd.Timestamp("2024-01-01", tz="UTC").value, price_precision=price_increment.precision, size_precision=size_increment.precision, price_increment=price_increment, size_increment=size_increment, min_quantity=Quantity.from_int(5), maker_fee=Decimal(0), taker_fee=Decimal(0), outcome="Yes", description="Will the outcome of this market be 'Yes'?", ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `BinaryOption` instruments include: - [Hyperliquid](../../integrations/hyperliquid.md) for binary and prediction-style markets. - [OKX](../../integrations/okx.md) for venue-defined binary outcome products. - [Polymarket](../../integrations/polymarket.md) for prediction market outcomes. ## Related guides - [Order Book](../order_book.md) covers binary market order book behavior. - [Data](../data.md) explains market data that references instruments. # Cfd Source: https://nautilustrader.io/docs/latest/concepts/instruments/cfd/ `Cfd` represents a contract for difference that tracks an underlying asset without transferring ownership of the underlying. The venue defines the quote currency, precision, increments, limits, margins, and fees. Examples include CFD contracts on FX, equities, indexes, and commodities. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `base_currency` | `Option` | `None` | Base currency when the CFD tracks one. | | `quote_currency` | `Currency` | Required | Currency used to quote and value prices. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `lot_size` | `Option` | `None` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `base_currency` | `Currency \| None` | `None` | Base currency when the CFD tracks one. | | `quote_currency` | `Currency` | Required | Currency used to quote and value prices. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `lot_size` | `Quantity \| None` | `None` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `Cfd` has instrument class `Cfd`. - It is never inverse and uses a multiplier of one. - It has no activation timestamp, expiration timestamp, strike, or option kind. - Use the source market type when a venue offers both cash instruments and CFDs. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::Cfd, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; let audusd = Cfd::builder() .instrument_id(InstrumentId::from("AUDUSD.OANDA")) .raw_symbol(Symbol::from("AUD/USD")) .asset_class(AssetClass::FX) .base_currency(Currency::from("AUD")) .quote_currency(Currency::from("USD")) .price_precision(5) .size_precision(0) .price_increment(Price::from("0.00001")) .size_increment(Quantity::from("1")) .lot_size(Quantity::from("1000")) .margin_init(dec!(0.03)) .margin_maint(dec!(0.03)) .maker_fee(dec!(0.00002)) .taker_fee(dec!(0.00002)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal from nautilus_trader.model import AssetClass from nautilus_trader.model import Cfd from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol audusd = Cfd( instrument_id=InstrumentId.from_str("AUDUSD.OANDA"), raw_symbol=Symbol("AUD/USD"), asset_class=AssetClass.FX, quote_currency=Currency.from_str("USD"), price_precision=5, price_increment=Price.from_str("0.00001"), size_precision=0, size_increment=Quantity.from_int(1), ts_event=0, ts_init=0, base_currency=Currency.from_str("AUD"), lot_size=Quantity.from_int(1000), margin_init=Decimal("0.03"), margin_maint=Decimal("0.03"), maker_fee=Decimal("0.00002"), taker_fee=Decimal("0.00002"), ) ``` ## Adapters Representative adapters that create or consume `Cfd` instruments include: - [Interactive Brokers](../../integrations/ib.md) for CFD contracts. ## Related guides - [Currency Pair](currency_pair.md) covers cash FX and crypto spot pairs. - [Commodity](commodity.md) covers spot commodity instruments. # Commodity Source: https://nautilustrader.io/docs/latest/concepts/instruments/commodity/ `Commodity` represents a spot commodity market such as gold, silver, oil, or another physical asset quoted in a currency. It models a spot market, not a dated futures contract. Examples include `XAUUSD.IDEALPRO` and venue-specific commodity cash symbols. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Commodity asset classification. | | `quote_currency` | `Currency` | Required | Currency used to price the commodity. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `lot_size` | `Option` | `None` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Commodity asset classification. | | `quote_currency` | `Currency` | Required | Currency used to price the commodity. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `lot_size` | `Quantity \| None` | `None` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `Commodity` has instrument class `Spot`. - It allows negative prices: spot markets such as electricity or oil can trade below zero, and the `RiskEngine` accepts negative prices on both order submission and modification. - It is never inverse, and its cost currency is the quote currency. - It has no activation timestamp, expiry, strike, option kind, or settlement currency field. - Use `FuturesContract` for dated exchange-traded commodity futures. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::Commodity, types::{Currency, Price, Quantity}, }; let gold = Commodity::builder() .instrument_id(InstrumentId::from("GOLD.COMEX")) .raw_symbol(Symbol::from("GOLD")) .asset_class(AssetClass::Commodity) .quote_currency(Currency::from("USD")) .price_precision(2) .size_precision(0) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("1")) .lot_size(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from nautilus_trader.model import AssetClass from nautilus_trader.model import Commodity from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol gold = Commodity( instrument_id=InstrumentId.from_str("GOLD.COMEX"), raw_symbol=Symbol("GOLD"), asset_class=AssetClass.COMMODITY, quote_currency=Currency.from_str("USD"), price_precision=2, price_increment=Price.from_str("0.01"), size_precision=0, size_increment=Quantity.from_int(1), ts_event=0, ts_init=0, lot_size=Quantity.from_int(1), ) ``` ## Adapters Representative adapters that create or consume `Commodity` instruments include: - [Interactive Brokers](../../integrations/ib.md) for spot commodity and metal contracts. ## Related guides - [Futures Contract](futures_contract.md) covers dated futures on commodity underlyings. - [Data](../data.md) explains market data that references instruments. # Crypto Future Source: https://nautilustrader.io/docs/latest/concepts/instruments/crypto_future/ `CryptoFuture` represents a dated crypto futures contract. It tracks a crypto underlying, quotes in a quote currency, settles in a settlement currency, and expires at a fixed timestamp. Examples include dated BTC or ETH futures on crypto derivatives venues. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the contract tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `activation_ns` | `UnixNanos` | Required | Contract activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Contract expiration timestamp. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the contract tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `activation_ns` | `int` | Required | Contract activation timestamp. | | `expiration_ns` | `int` | Required | Contract expiration timestamp. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CryptoFuture` has asset class `Cryptocurrency` and instrument class `Future`. - Linear contracts typically set `is_inverse=False` and settle in the quote currency. - Inverse contracts set `is_inverse=True` and typically settle in the underlying currency. - Quanto contracts settle in a third currency that differs from both underlying and quote. - Use `CryptoPerpetual` for crypto derivatives with no expiration. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::CryptoFuture, types::{Currency, Money, Price, Quantity}, }; let activation = Utc.with_ymd_and_hms(2024, 1, 8, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2024, 3, 29, 0, 0, 0).unwrap(); let btcusdt_future = CryptoFuture::builder() .instrument_id(InstrumentId::from("BTCUSDT-240329.BINANCE")) .raw_symbol(Symbol::from("BTCUSDT-240329")) .underlying(Currency::from("BTC")) .quote_currency(Currency::from("USDT")) .settlement_currency(Currency::from("USDT")) .is_inverse(false) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(2) .size_precision(6) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.000001")) .max_quantity(Quantity::from("9000.0")) .min_quantity(Quantity::from("0.000001")) .min_notional(Money::from("10.00 USDT")) .max_price(Price::from("1000000.00")) .min_price(Price::from("0.01")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import CryptoFuture from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Money from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol BTC = Currency.from_str("BTC") USDT = Currency.from_str("USDT") btcusdt_future = CryptoFuture( instrument_id=InstrumentId.from_str("BTCUSDT-240329.BINANCE"), raw_symbol=Symbol("BTCUSDT-240329"), underlying=BTC, quote_currency=USDT, settlement_currency=USDT, is_inverse=False, activation_ns=pd.Timestamp("2024-01-08", tz="UTC").value, expiration_ns=pd.Timestamp("2024-03-29", tz="UTC").value, price_precision=2, size_precision=6, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.000001"), max_quantity=Quantity.from_str("9000"), min_quantity=Quantity.from_str("0.000001"), min_notional=Money(10.00, USDT), max_price=Price.from_str("1000000.00"), min_price=Price.from_str("0.01"), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `CryptoFuture` instruments include: - [BitMEX](../../integrations/bitmex.md) for inverse and linear dated futures. - [Bybit](../../integrations/bybit.md) for crypto futures markets. - [Deribit](../../integrations/deribit.md) for dated crypto futures. - [OKX](../../integrations/okx.md) for dated crypto futures. - [Tardis](../../integrations/tardis.md) for crypto futures metadata. ## Related guides - [Crypto Perpetual](crypto_perpetual.md) covers perpetual crypto futures. - [Futures Contract](futures_contract.md) covers non-crypto futures contracts. # Crypto Futures Spread Source: https://nautilustrader.io/docs/latest/concepts/instruments/crypto_futures_spread/ `CryptoFuturesSpread` represents an exchange-defined spread strategy over crypto futures. The venue publishes the strategy as a single instrument with its own symbol, strategy type, precision, increments, and expiration. Examples include listed crypto futures calendar spreads. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the strategy tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `strategy_type` | `Ustr` | Required | Venue strategy type, such as calendar. | | `activation_ns` | `UnixNanos` | Required | Strategy activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Strategy expiration timestamp. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Strategy multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the strategy tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `strategy_type` | `str` | Required | Venue strategy type, such as calendar. | | `activation_ns` | `int` | Required | Strategy activation timestamp. | | `expiration_ns` | `int` | Required | Strategy expiration timestamp. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Strategy multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CryptoFuturesSpread` has asset class `Cryptocurrency` and instrument class `FuturesSpread`. - The venue publishes the spread as a single tradable instrument. - The strategy can be linear, inverse, or quanto, depending on the currency set. - Store venue-specific leg details in `info` when the adapter provides them. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::CryptoFuturesSpread, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2026, 5, 12, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2026, 5, 19, 8, 0, 0).unwrap(); let btc_spread = CryptoFuturesSpread::builder() .instrument_id(InstrumentId::from("BTC-FS-19MAY26_PERP.DERIBIT")) .raw_symbol(Symbol::from("BTC-FS-19MAY26_PERP")) .underlying(Currency::from("BTC")) .quote_currency(Currency::from("USD")) .settlement_currency(Currency::from("BTC")) .is_inverse(false) .strategy_type(Ustr::from("FS")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(1) .size_precision(0) .price_increment(Price::from("0.5")) .size_increment(Quantity::from("1")) .multiplier(Quantity::from("10")) .lot_size(Quantity::from("1")) .min_quantity(Quantity::from("1")) .maker_fee(dec!(0.0003)) .taker_fee(dec!(0.0003)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal import pandas as pd from nautilus_trader.model import CryptoFuturesSpread from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol BTC = Currency.from_str("BTC") USD = Currency.from_str("USD") btc_spread = CryptoFuturesSpread( instrument_id=InstrumentId.from_str("BTC-FS-19MAY26_PERP.DERIBIT"), raw_symbol=Symbol("BTC-FS-19MAY26_PERP"), underlying=BTC, quote_currency=USD, settlement_currency=BTC, is_inverse=False, strategy_type="FS", activation_ns=pd.Timestamp("2026-05-12T00:00:00", tz="UTC").value, expiration_ns=pd.Timestamp("2026-05-19T08:00:00", tz="UTC").value, price_precision=1, size_precision=0, price_increment=Price.from_str("0.5"), size_increment=Quantity.from_int(1), multiplier=Quantity.from_int(10), lot_size=Quantity.from_int(1), min_quantity=Quantity.from_int(1), maker_fee=Decimal("0.0003"), taker_fee=Decimal("0.0003"), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `CryptoFuturesSpread` instruments include: - [Deribit](../../integrations/deribit.md) for crypto futures combos. - [OKX](../../integrations/okx.md) for crypto futures spread markets. ## Related guides - [Crypto Future](crypto_future.md) covers single-leg dated crypto futures. - [Futures Spread](futures_spread.md) covers non-crypto futures spreads. # Crypto Option Source: https://nautilustrader.io/docs/latest/concepts/instruments/crypto_option/ `CryptoOption` represents a put or call option on a crypto underlying. It defines the option kind, strike price, activation time, expiration time, quote currency, settlement currency, and contract sizing. Examples include BTC and ETH options on crypto derivatives venues. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the option tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the premium. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `option_kind` | `OptionKind` | Required | Put or call. | | `strike_price` | `Price` | Required | Option strike price. | | `activation_ns` | `UnixNanos` | Required | Contract activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Contract expiration timestamp. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `1` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the option tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the premium. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `option_kind` | `OptionKind` | Required | Put or call. | | `strike_price` | `Price` | Required | Option strike price. | | `activation_ns` | `int` | Required | Contract activation timestamp. | | `expiration_ns` | `int` | Required | Contract expiration timestamp. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `1` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CryptoOption` has asset class `Cryptocurrency` and instrument class `Option`. - The option kind and strike price define the payoff shape. - The contract can be linear, inverse, or quanto, depending on the currency set. - Use `OptionContract` for non-crypto listed options. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::OptionKind, identifiers::{InstrumentId, Symbol}, instruments::CryptoOption, types::{Currency, Money, Price, Quantity}, }; use rust_decimal_macros::dec; let activation = Utc.with_ymd_and_hms(2022, 12, 22, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2023, 1, 13, 8, 0, 0).unwrap(); let btc_option = CryptoOption::builder() .instrument_id(InstrumentId::from("BTC-13JAN23-16000-P.DERIBIT")) .raw_symbol(Symbol::from("BTC-13JAN23-16000-P")) .underlying(Currency::from("BTC")) .quote_currency(Currency::from("USD")) .settlement_currency(Currency::from("BTC")) .is_inverse(false) .option_kind(OptionKind::Put) .strike_price(Price::from("16000.00")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(2) .size_precision(1) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.1")) .multiplier(Quantity::from("1")) .lot_size(Quantity::from("1")) .max_quantity(Quantity::from("9000")) .min_quantity(Quantity::from("0.1")) .min_notional(Money::from("10.00 USD")) .margin_init(dec!(0)) .margin_maint(dec!(0)) .maker_fee(dec!(0.0003)) .taker_fee(dec!(0.0003)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal import pandas as pd from nautilus_trader.model import CryptoOption from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Money from nautilus_trader.model import OptionKind from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol BTC = Currency.from_str("BTC") USD = Currency.from_str("USD") btc_option = CryptoOption( instrument_id=InstrumentId.from_str("BTC-13JAN23-16000-P.DERIBIT"), raw_symbol=Symbol("BTC-13JAN23-16000-P"), underlying=BTC, quote_currency=USD, settlement_currency=BTC, is_inverse=False, option_kind=OptionKind.PUT, strike_price=Price.from_str("16000.00"), activation_ns=pd.Timestamp("2022-12-22", tz="UTC").value, expiration_ns=pd.Timestamp("2023-01-13T08:00:00", tz="UTC").value, price_precision=2, size_precision=1, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.1"), max_quantity=Quantity.from_str("9000"), min_quantity=Quantity.from_str("0.1"), min_notional=Money(10.00, USD), margin_init=Decimal(0), margin_maint=Decimal(0), maker_fee=Decimal("0.0003"), taker_fee=Decimal("0.0003"), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `CryptoOption` instruments include: - [Bybit](../../integrations/bybit.md) for crypto options. - [Deribit](../../integrations/deribit.md) for crypto options. - [OKX](../../integrations/okx.md) for crypto options. - [Tardis](../../integrations/tardis.md) for crypto option metadata. ## Related guides - [Options](../options.md) covers option data, Greeks, and chain subscriptions. - [Crypto Option Spread](crypto_option_spread.md) covers exchange-defined crypto option spreads. # Crypto Option Spread Source: https://nautilustrader.io/docs/latest/concepts/instruments/crypto_option_spread/ `CryptoOptionSpread` represents an exchange-defined strategy over crypto options. The venue publishes the strategy as one instrument with its own symbol, strategy type, precision, increments, and expiration. Examples include listed BTC or ETH option combos on crypto derivatives venues. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the strategy tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the premium. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `strategy_type` | `Ustr` | Required | Venue strategy type, such as vertical. | | `activation_ns` | `UnixNanos` | Required | Strategy activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Strategy expiration timestamp. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Strategy multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Currency` | Required | Crypto asset the strategy tracks. | | `quote_currency` | `Currency` | Required | Currency used to quote the premium. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `strategy_type` | `str` | Required | Venue strategy type, such as vertical. | | `activation_ns` | `int` | Required | Strategy activation timestamp. | | `expiration_ns` | `int` | Required | Strategy expiration timestamp. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Strategy multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CryptoOptionSpread` has asset class `Cryptocurrency` and instrument class `OptionSpread`. - The venue publishes the spread as a single tradable instrument. - The strategy can be linear, inverse, or quanto, depending on the currency set. - Store venue-specific leg details in `info` when the adapter provides them. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::CryptoOptionSpread, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2026, 5, 12, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2026, 5, 19, 8, 0, 0).unwrap(); let btc_spread = CryptoOptionSpread::builder() .instrument_id(InstrumentId::from("BTC-CS-19MAY26-70000_75000.DERIBIT")) .raw_symbol(Symbol::from("BTC-CS-19MAY26-70000_75000")) .underlying(Currency::from("BTC")) .quote_currency(Currency::from("USD")) .settlement_currency(Currency::from("BTC")) .is_inverse(false) .strategy_type(Ustr::from("CS")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(4) .size_precision(1) .price_increment(Price::from("0.0001")) .size_increment(Quantity::from("0.1")) .multiplier(Quantity::from("1")) .min_quantity(Quantity::from("0.1")) .maker_fee(dec!(0.0003)) .taker_fee(dec!(0.0003)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal import pandas as pd from nautilus_trader.model import CryptoOptionSpread from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol BTC = Currency.from_str("BTC") USD = Currency.from_str("USD") btc_spread = CryptoOptionSpread( instrument_id=InstrumentId.from_str("BTC-CS-19MAY26-70000_75000.DERIBIT"), raw_symbol=Symbol("BTC-CS-19MAY26-70000_75000"), underlying=BTC, quote_currency=USD, settlement_currency=BTC, is_inverse=False, strategy_type="CS", activation_ns=pd.Timestamp("2026-05-12T00:00:00", tz="UTC").value, expiration_ns=pd.Timestamp("2026-05-19T08:00:00", tz="UTC").value, price_precision=4, size_precision=1, price_increment=Price.from_str("0.0001"), size_increment=Quantity.from_str("0.1"), min_quantity=Quantity.from_str("0.1"), maker_fee=Decimal("0.0003"), taker_fee=Decimal("0.0003"), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `CryptoOptionSpread` instruments include: - [Deribit](../../integrations/deribit.md) for crypto option combos. - [OKX](../../integrations/okx.md) for crypto option spread markets. ## Related guides - [Crypto Option](crypto_option.md) covers single-leg crypto options. - [Option Spread](option_spread.md) covers non-crypto option spreads. # Crypto Perpetual Source: https://nautilustrader.io/docs/latest/concepts/instruments/crypto_perpetual/ `CryptoPerpetual` represents a crypto perpetual futures contract, also known as a perpetual swap. It has no expiry, tracks a crypto base asset, and settles in a crypto, stablecoin, or other venue-defined settlement currency. Examples include `ETHUSDT-PERP.BINANCE`, `XBTUSD.BITMEX`, and `BTC-USD-SWAP.OKX`. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `base_currency` | `Currency` | Required | Base crypto asset. | | `quote_currency` | `Currency` | Required | Price quote currency. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `base_currency` | `Currency` | Required | Base crypto asset. | | `quote_currency` | `Currency` | Required | Price quote currency. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CryptoPerpetual` has asset class `Cryptocurrency` and instrument class `Swap`. - It has no activation or expiration timestamp. - Linear contracts typically set `is_inverse=False` and settle in the quote currency. - Inverse contracts set `is_inverse=True` and typically settle in the base currency. - Quanto contracts settle in a third currency that differs from both base and quote. - The cost currency is base for inverse contracts, settlement for quanto contracts, and quote otherwise. :::note Funding payments are not fields on the instrument. They arrive as data, such as `FundingRateUpdate`, and reference the instrument ID. ::: ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::{CryptoPerpetual, InstrumentAny}, types::{Currency, Money, Price, Quantity}, }; use rust_decimal_macros::dec; let ethusdt_perp = CryptoPerpetual::builder() .instrument_id(InstrumentId::from("ETHUSDT-PERP.BINANCE")) .raw_symbol(Symbol::from("ETHUSDT")) .base_currency(Currency::from("ETH")) .quote_currency(Currency::from("USDT")) .settlement_currency(Currency::from("USDT")) .is_inverse(false) .price_precision(2) .size_precision(3) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.001")) .max_quantity(Quantity::from("10000.000")) .min_quantity(Quantity::from("0.001")) .min_notional(Money::from("10.00 USDT")) .max_price(Price::from("15000.00")) .min_price(Price::from("1.00")) .margin_init(dec!(1.0)) .margin_maint(dec!(0.35)) .maker_fee(dec!(0.0002)) .taker_fee(dec!(0.0004)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); let instrument = InstrumentAny::CryptoPerpetual(ethusdt_perp); ``` ```python tab="Python" from decimal import Decimal from nautilus_trader.model import CryptoPerpetual from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Money from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol ETH = Currency.from_str("ETH") USDT = Currency.from_str("USDT") ethusdt_perp = CryptoPerpetual( instrument_id=InstrumentId.from_str("ETHUSDT-PERP.BINANCE"), raw_symbol=Symbol("ETHUSDT"), base_currency=ETH, quote_currency=USDT, settlement_currency=USDT, is_inverse=False, price_precision=2, size_precision=3, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.001"), ts_event=0, ts_init=0, max_quantity=Quantity.from_str("10000.000"), min_quantity=Quantity.from_str("0.001"), min_notional=Money(10.00, USDT), max_price=Price.from_str("15000.00"), min_price=Price.from_str("1.00"), margin_init=Decimal("1.0"), margin_maint=Decimal("0.35"), maker_fee=Decimal("0.0002"), taker_fee=Decimal("0.0004"), ) ``` ## Adapters Representative adapters that create or consume `CryptoPerpetual` instruments include: - [Binance](../../integrations/binance.md) for USD-M and COIN-M perpetual futures. - [BitMEX](../../integrations/bitmex.md) for inverse and linear perpetual contracts. - [Bybit](../../integrations/bybit.md) for linear and inverse perpetual products. - [dYdX](../../integrations/dydx.md) for perpetual markets. - [Hyperliquid](../../integrations/hyperliquid.md) for perpetual markets. - [Kraken](../../integrations/kraken.md) for futures venue perpetual markets. - [OKX](../../integrations/okx.md) for swap markets. - [Tardis](../../integrations/tardis.md) for crypto perpetual metadata. ## Related guides - [Data](../data.md) covers mark prices, index prices, and funding rate updates. - [Options](../options.md) covers option-specific instrument types. - [Execution](../execution.md) explains precision and notional checks before orders reach a venue. # Currency Pair Source: https://nautilustrader.io/docs/latest/concepts/instruments/currency_pair/ `CurrencyPair` represents a spot or cash market quoted as `BASE/QUOTE`. The base currency is the asset being bought or sold, and the quote currency prices one unit of the base. Nautilus uses this type for fiat FX pairs and crypto spot pairs. Examples include `EUR/USD.SIM`, `BTCUSDT.BINANCE`, and `ETH/USD.KRAKEN`. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `base_currency` | `Currency` | Required | Asset bought or sold. | | `quote_currency` | `Currency` | Required | Currency used to price the base asset. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Option` | `None` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `base_currency` | `Currency` | Required | Asset bought or sold. | | `quote_currency` | `Currency` | Required | Currency used to price the base asset. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity \| None` | `None` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `CurrencyPair` has instrument class `Spot`. - It has no expiration, strike price, option kind, or derivative underlying field. - It is never inverse. The settlement currency and cost currency are the quote currency. - Use this type for both fiat FX pairs and crypto spot pairs. :::warning Do not model dated futures, swaps, or options as `CurrencyPair` only because their symbols look like pairs. Use the specific derivative type so cost currency, settlement currency, expiration, and notional calculations match the venue. ::: ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::{CurrencyPair, InstrumentAny}, types::{Currency, Money, Price, Quantity}, }; use rust_decimal_macros::dec; let btcusdt = CurrencyPair::builder() .instrument_id(InstrumentId::from("BTCUSDT.BINANCE")) .raw_symbol(Symbol::from("BTCUSDT")) .base_currency(Currency::from("BTC")) .quote_currency(Currency::from("USDT")) .price_precision(2) .size_precision(6) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.000001")) .min_quantity(Quantity::from("0.000001")) .min_notional(Money::from("10.00 USDT")) .max_price(Price::from("1000000.00")) .min_price(Price::from("0.01")) .margin_init(dec!(0.001)) .margin_maint(dec!(0.001)) .maker_fee(dec!(0.001)) .taker_fee(dec!(0.001)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); let instrument = InstrumentAny::CurrencyPair(btcusdt); ``` ```python tab="Python" from decimal import Decimal from nautilus_trader.model import Currency from nautilus_trader.model import CurrencyPair from nautilus_trader.model import InstrumentId from nautilus_trader.model import Money from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol BTC = Currency.from_str("BTC") USDT = Currency.from_str("USDT") btcusdt = CurrencyPair( instrument_id=InstrumentId.from_str("BTCUSDT.BINANCE"), raw_symbol=Symbol("BTCUSDT"), base_currency=BTC, quote_currency=USDT, price_precision=2, size_precision=6, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.000001"), ts_event=0, ts_init=0, min_quantity=Quantity.from_str("0.000001"), min_notional=Money(10.00, USDT), max_price=Price.from_str("1000000.00"), min_price=Price.from_str("0.01"), margin_init=Decimal("0.001"), margin_maint=Decimal("0.001"), maker_fee=Decimal("0.001"), taker_fee=Decimal("0.001"), ) ``` ## Adapters Representative adapters that create or consume `CurrencyPair` instruments include: - [Binance](../../integrations/binance.md) for spot markets. - [Kraken](../../integrations/kraken.md) for spot markets. - [OKX](../../integrations/okx.md) for spot markets. - [Tardis](../../integrations/tardis.md) for spot metadata. - [Interactive Brokers](../../integrations/ib.md) for FX cash contracts. - [Hyperliquid](../../integrations/hyperliquid.md) for spot assets. ## Related guides - [Data](../data.md) explains market data that references instruments. - [Execution](../execution.md) explains order checks that use instrument precision. - [Value types](../value_types.md) explains `Price`, `Quantity`, and `Money`. # Equity Source: https://nautilustrader.io/docs/latest/concepts/instruments/equity/ `Equity` represents a listed share, ETF, or similar cash-market security. Nautilus uses this type for instruments that trade in whole units, quote in one currency, and have no contract expiry. Examples include `AAPL.XNAS`, `MSFT.XNAS`, and venue-specific ETF symbols. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `lot_size` | `Option` | `None` | Board lot or whole‑share lot size. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `isin` | `Option` | `None` | International Securities ID when known. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `lot_size` | `Quantity \| None` | `None` | Board lot or whole‑share lot size. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `isin` | `str \| None` | `None` | International Securities ID when known. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `Equity` has asset class `Equity` and instrument class `Spot`. - Quantity precision is always zero, so orders use whole-share quantities. - The multiplier and size increment are one. - It has no base currency, expiry, strike, option kind, or inverse costing flag. - Use price limits only when the venue publishes them. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::Equity, types::{Currency, Price, Quantity}, }; use ustr::Ustr; let aapl = Equity::builder() .instrument_id(InstrumentId::from("AAPL.XNAS")) .raw_symbol(Symbol::from("AAPL")) .isin(Ustr::from("US0378331005")) .currency(Currency::from("USD")) .price_precision(2) .price_increment(Price::from("0.01")) .lot_size(Quantity::from("100")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from nautilus_trader.model import Currency from nautilus_trader.model import Equity from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol aapl = Equity( instrument_id=InstrumentId.from_str("AAPL.XNAS"), raw_symbol=Symbol("AAPL"), currency=Currency.from_str("USD"), price_precision=2, price_increment=Price.from_str("0.01"), ts_event=0, ts_init=0, isin="US0378331005", lot_size=Quantity.from_int(100), ) ``` ## Adapters Representative adapters that create or consume `Equity` instruments include: - [Databento](../../integrations/databento.md) for listed US equities and ETFs. - [Interactive Brokers](../../integrations/ib.md) for listed equity contracts. ## Related guides - [Data](../data.md) explains market data that references instruments. - [Value types](../value_types.md) explains `Price`, `Quantity`, and `Money`. # Futures Contract Source: https://nautilustrader.io/docs/latest/concepts/instruments/futures_contract/ `FuturesContract` represents a dated, exchange-traded futures contract with a defined underlying, activation time, expiration time, currency, multiplier, and lot size. Examples include equity index futures, commodity futures, interest-rate futures, and currency futures. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `exchange` | `Option` | `None` | Exchange MIC or venue code when known. | | `underlying` | `Ustr` | Required | Underlying asset, index, or product. | | `activation_ns` | `UnixNanos` | Required | Contract activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Contract expiration timestamp. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `u8` | `0` | Futures trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Contract multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `1` | Minimum order quantity. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `exchange` | `str \| None` | `None` | Exchange MIC or venue code when known. | | `underlying` | `str` | Required | Underlying asset, index, or product. | | `activation_ns` | `int` | Required | Contract activation timestamp. | | `expiration_ns` | `int` | Required | Contract expiration timestamp. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `int` | `0` | Futures trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Contract multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `1` | Minimum order quantity. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `FuturesContract` has instrument class `Future`. - It is never inverse. Cost, settlement, and quote currency use `currency`. - It trades in whole contracts with size precision `0` and size increment `1`. - Use `CryptoFuture` for dated crypto futures where the underlying and settlement currencies can differ. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::FuturesContract, types::{Currency, Price, Quantity}, }; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2021, 9, 10, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2021, 12, 17, 0, 0, 0).unwrap(); let esz21 = FuturesContract::builder() .instrument_id(InstrumentId::from("ESZ21.GLBX")) .raw_symbol(Symbol::from("ESZ21")) .asset_class(AssetClass::Index) .exchange(Ustr::from("XCME")) .underlying(Ustr::from("ES")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .currency(Currency::from("USD")) .price_precision(2) .price_increment(Price::from("0.25")) .multiplier(Quantity::from("1")) .lot_size(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import FuturesContract from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol esz21 = FuturesContract( instrument_id=InstrumentId.from_str("ESZ21.GLBX"), raw_symbol=Symbol("ESZ21"), asset_class=AssetClass.INDEX, underlying="ES", activation_ns=pd.Timestamp("2021-09-10", tz="UTC").value, expiration_ns=pd.Timestamp("2021-12-17", tz="UTC").value, currency=Currency.from_str("USD"), price_precision=2, price_increment=Price.from_str("0.25"), multiplier=Quantity.from_int(1), lot_size=Quantity.from_int(1), ts_event=0, ts_init=0, exchange="XCME", ) ``` ## Adapters Representative adapters that create or consume `FuturesContract` instruments include: - [Databento](../../integrations/databento.md) for futures reference data and market data. - [Interactive Brokers](../../integrations/ib.md) for listed futures contracts. ## Related guides - [Continuous Futures](../continuous_futures.md) covers roll-adjusted futures series. - [Crypto Future](crypto_future.md) covers dated crypto futures contracts. # Futures Spread Source: https://nautilustrader.io/docs/latest/concepts/instruments/futures_spread/ `FuturesSpread` represents an exchange-defined futures strategy with more than one leg, such as a calendar spread or inter-commodity spread. The venue defines the strategy, symbol, tick size, and expiry. Examples include listed futures calendar spreads and exchange-supported spread markets. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|-------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying strategy. | | `exchange` | `Option` | `None` | Exchange MIC or venue code when known. | | `underlying` | `Ustr` | Required | Underlying product or product family. | | `strategy_type` | `Ustr` | Required | Venue strategy type, such as calendar. | | `activation_ns` | `UnixNanos` | Required | Strategy activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Strategy expiration timestamp. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `u8` | `0` | Futures spreads trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Strategy multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `1` | Minimum order quantity. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|-------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying strategy. | | `exchange` | `str \| None` | `None` | Exchange MIC or venue code when known. | | `underlying` | `str` | Required | Underlying product or product family. | | `strategy_type` | `str` | Required | Venue strategy type, such as calendar. | | `activation_ns` | `int` | Required | Strategy activation timestamp. | | `expiration_ns` | `int` | Required | Strategy expiration timestamp. | | `currency` | `Currency` | Required | Quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `int` | `0` | Futures spreads trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Strategy multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `1` | Minimum order quantity. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `FuturesSpread` has instrument class `FuturesSpread`. - The venue publishes the spread as a single tradable instrument. - It trades in whole contracts with size precision `0` and size increment `1`. - Use leg data from the adapter metadata when a strategy needs venue-specific leg details. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::FuturesSpread, types::{Currency, Price, Quantity}, }; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2022, 6, 21, 13, 30, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2024, 6, 21, 13, 30, 0).unwrap(); let es_spread = FuturesSpread::builder() .instrument_id(InstrumentId::from("ESM4-ESU4.GLBX")) .raw_symbol(Symbol::from("ESM4-ESU4")) .asset_class(AssetClass::Index) .exchange(Ustr::from("XCME")) .underlying(Ustr::from("ES")) .strategy_type(Ustr::from("EQ")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .currency(Currency::from("USD")) .price_precision(2) .price_increment(Price::from("0.01")) .multiplier(Quantity::from("1")) .lot_size(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import FuturesSpread from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol es_spread = FuturesSpread( instrument_id=InstrumentId.from_str("ESM4-ESU4.GLBX"), raw_symbol=Symbol("ESM4-ESU4"), asset_class=AssetClass.INDEX, underlying="ES", strategy_type="EQ", activation_ns=pd.Timestamp("2022-06-21T13:30:00", tz="UTC").value, expiration_ns=pd.Timestamp("2024-06-21T13:30:00", tz="UTC").value, currency=Currency.from_str("USD"), price_precision=2, price_increment=Price.from_str("0.01"), multiplier=Quantity.from_int(1), lot_size=Quantity.from_int(1), ts_event=0, ts_init=0, exchange="XCME", ) ``` ## Adapters Representative adapters that create or consume `FuturesSpread` instruments include: - [Databento](../../integrations/databento.md) for listed futures spread markets. - [Interactive Brokers](../../integrations/ib.md) for exchange-defined futures strategies. ## Related guides - [Futures Contract](futures_contract.md) covers single-leg futures. - [Continuous Futures](../continuous_futures.md) covers roll-adjusted futures series. # Instruments Source: https://nautilustrader.io/docs/latest/concepts/instruments/ An instrument represents the specification for a tradable asset, contract, or local synthetic market. Market data, orders, positions, accounting, portfolio calculations, and adapter symbology all refer back to an `InstrumentId` and its instrument definition. NautilusTrader exposes the same instrument model to Rust and Python users. Rust examples use `nautilus_model`; Python examples use `nautilus_trader.model.instruments`. ## Instrument types | Instrument type | Class | Description | Typical adapters | |----------------------------------------------------------|--------------------|------------------------------------------------------|---------------------------------| | [`Equity`](equity.md) | Spot | Listed share or ETF traded on a cash market. | Databento, Interactive Brokers. | | [`CurrencyPair`](currency_pair.md) | Spot | Fiat FX or crypto spot pair in base/quote form. | Binance, Kraken, OKX, Tardis. | | [`Commodity`](commodity.md) | Spot | Spot commodity such as gold or oil. | Interactive Brokers. | | [`Cfd`](cfd.md) | Contract for diff. | Contract for difference tracking an underlying. | Interactive Brokers. | | [`IndexInstrument`](index_instrument.md) | Spot reference | Reference index, not directly tradable. | Interactive Brokers. | | [`TokenizedAsset`](tokenized_asset.md) | Tokenized spot | Tokenized asset on a crypto venue. | Kraken. | | [`FuturesContract`](futures_contract.md) | Future | Deliverable futures contract. | Databento, Interactive Brokers. | | [`FuturesSpread`](futures_spread.md) | Futures spread | Exchange defined futures strategy with several legs. | Databento, Interactive Brokers. | | [`CryptoFuture`](crypto_future.md) | Crypto future | Dated crypto futures contract. | BitMEX, Bybit, Deribit, OKX. | | [`CryptoFuturesSpread`](crypto_futures_spread.md) | Crypto spread | Exchange defined crypto futures spread. | Deribit, OKX. | | [`CryptoPerpetual`](crypto_perpetual.md) | Swap | Crypto perpetual futures contract. | Binance, BitMEX, Bybit, dYdX. | | [`PerpetualContract`](perpetual_contract.md) | Generic swap | Perpetual futures contract across asset classes. | Architect AX. | | [`OptionContract`](option_contract.md) | Option | Exchange traded put or call option. | Databento, Interactive Brokers. | | [`OptionSpread`](option_spread.md) | Option spread | Exchange defined options strategy with several legs. | Databento, Interactive Brokers. | | [`CryptoOption`](crypto_option.md) | Crypto option | Option on a crypto underlying. | Bybit, Deribit, OKX, Tardis. | | [`CryptoOptionSpread`](crypto_option_spread.md) | Crypto spread | Exchange defined crypto option spread. | Deribit, OKX. | | [`BinaryOption`](binary_option.md) | Binary outcome | Binary instrument that settles to 0 or 1. | Hyperliquid, OKX, Polymarket. | | [`BettingInstrument`](betting_instrument.md) | Betting market | Sports or gaming market selection. | Betfair. | | [`SyntheticInstrument`](synthetic_instrument.md) | Local synthetic | Formula derived local instrument. | Local only. | ## Taxonomy NautilusTrader groups instruments by the market structure they represent: ```mermaid flowchart TD I[Instrument Types] I --> Spot I --> Derivatives I --> Other Spot --> Equity Spot --> CurrencyPair Spot --> Commodity Spot --> IndexInstrument Spot --> TokenizedAsset Derivatives --> Futures Derivatives --> Options Derivatives --> Swaps Derivatives --> Cfd Futures --> FuturesContract Futures --> FuturesSpread Futures --> CryptoFuture Futures --> CryptoFuturesSpread Options --> OptionContract Options --> OptionSpread Options --> CryptoOption Options --> CryptoOptionSpread Options --> BinaryOption Swaps --> CryptoPerpetual Swaps --> PerpetualContract Other --> BettingInstrument Other --> SyntheticInstrument ``` ## Common fields Most concrete instruments share the same core shape. Individual type pages list the complete constructor and struct fields for that type. | Field | Meaning | |---------------------|-------------------------------------------------------------------------| | `id` | Nautilus `InstrumentId`, formed from a symbol and venue. | | `raw_symbol` | Native venue symbol before Nautilus normalization. | | `price_precision` | Number of decimal places allowed for prices. | | `size_precision` | Number of decimal places allowed for quantities. | | `price_increment` | Smallest valid price step. | | `size_increment` | Smallest valid quantity step. | | `multiplier` | Contract multiplier used in notional and PnL calculations. | | `lot_size` | Rounded lot or board size when the venue publishes one. | | `margin_init` | Initial margin rate as a decimal fraction of notional value. | | `margin_maint` | Maintenance margin rate as a decimal fraction of notional value. | | `maker_fee` | Maker fee rate. Negative values represent rebates. | | `taker_fee` | Taker fee rate. Negative values represent rebates. | | `max_quantity` | Maximum order quantity when known. | | `min_quantity` | Minimum order quantity when known. | | `max_notional` | Maximum order notional value when known. | | `min_notional` | Minimum order notional value when known. | | `max_price` | Maximum valid quote or order price when known. | | `min_price` | Minimum valid quote or order price when known. | | `info` | Adapter metadata preserved from the venue or data source. | | `ts_event` | UNIX nanosecond timestamp for when the definition event occurred. | | `ts_init` | UNIX nanosecond timestamp for when Nautilus initialized the object. | | `tick_scheme` | Registered variable tick scheme name where the type supports one. | ## Symbology Every instrument has a unique `InstrumentId` made from the native symbol and venue, separated by a period. For example, Binance Futures represents the Ethereum perpetual contract as: ```text ETHUSDT-PERP.BINANCE ``` Native symbols should be unique for a venue, but this is not guaranteed by every exchange. The `{symbol}.{venue}` pair must be unique inside a Nautilus system. :::warning The instrument definition must match the market data and venue order semantics. An incorrect instrument can truncate prices or quantities, calculate notional values with the wrong currency, or make a backtest accept prices a live venue would reject. ::: ## Rust and Python surfaces Rust users work with the `nautilus_model` instrument structs and `InstrumentAny`: ```rust use nautilus_model::instruments::{CurrencyPair, InstrumentAny}; ``` Python users normally work with instrument classes from `nautilus_trader.model`: ```python from nautilus_trader.model import CurrencyPair ``` Both surfaces represent the same instrument contract: identity, precision, increments, currencies, limits, margins, fees, metadata, and timestamps. ## Loading instruments Generic test instruments can be instantiated through the `TestInstrumentProvider`: ```python from nautilus_trader.test_kit.providers import TestInstrumentProvider audusd = TestInstrumentProvider.default_fx_ccy("AUD/USD") ``` Live integration adapters expose `InstrumentProvider` objects that cache instrument definitions. Use `InstrumentProviderConfig(load_all=True)` where the integration supports it, or `load_ids` to load a known set of instruments. Subscriptions and order methods expect matching instruments to already exist in the cache. ## Finding instruments Strategies and actors retrieve instruments from the central cache: ```rust tab="Rust" use nautilus_model::identifiers::InstrumentId; let instrument_id = InstrumentId::from("ETHUSDT-PERP.BINANCE"); let instrument = cache.instrument(&instrument_id); ``` ```python tab="Python" from nautilus_trader.model import InstrumentId instrument_id = InstrumentId.from_str("ETHUSDT-PERP.BINANCE") instrument = self.cache.instrument(instrument_id) ``` It is also possible to subscribe to one instrument or all instruments for a venue: ```python self.subscribe_instrument(instrument_id) self.subscribe_instruments(venue) ``` When the `DataEngine` receives an instrument update, it passes the object to the `on_instrument()` handler. ## Precision Precision defines the canonical number of decimal places for prices and quantities on an instrument. NautilusTrader enforces the resulting price and size grids strictly because trading venues validate the same constraints, and backtests should not fill orders at prices or sizes that cannot exist in production. | Field | Constrains | Example | |-------------------|--------------------------------------|-------------------| | `price_precision` | Order prices, trigger prices, fills. | `2` -> `50000.01` | | `size_precision` | Order quantities and fill sizes. | `5` -> `1.00001` | The increment precision must match the declared precision. For example, `price_precision=2` pairs with `price_increment=Price(0.01, 2)`. Use the instrument factory methods when producing order prices and sizes: ```python instrument = self.cache.instrument(instrument_id) price = instrument.make_price(0.90500) quantity = instrument.make_qty(150) ``` :::warning The `RiskEngine` does not round values automatically. If you create a `Price` with 5 decimal places for an instrument that supports 2, the order is denied. Use `instrument.make_price()` and `instrument.make_qty()` to round explicitly. ::: ## Limits, margins, and fees Venue and adapter definitions can include optional limits: - `max_quantity` and `min_quantity`. - `max_notional` and `min_notional`. - `max_price` and `min_price`. The `MarginAccount` uses `margin_init`, `margin_maint`, and the taker fee when it calculates initial and maintenance margin. Nautilus uses one fee-rate convention across adapters and backtesting: - Positive fee rates represent commissions. - Negative fee rates represent rebates. For deeper accounting behavior, see [Accounting](../accounting.md). ## Metadata The `info` field preserves raw or adapter-specific metadata as a JSON-serializable dictionary. Use it when the venue publishes useful details that do not belong in the unified Nautilus instrument API. ## Related guides - [Data](../data.md) covers market data types that reference instruments. - [Orders](../orders/) covers order fields that reference instruments. - [Synthetics](../synthetics.md) covers local formula-derived instruments. - [Python API Reference](/docs/python-api-latest/model/instruments.html) lists Python constructors and members. # Index Instrument Source: https://nautilustrader.io/docs/latest/concepts/instruments/index_instrument/ `IndexInstrument` represents a reference index such as an equity index, volatility index, or benchmark price series. It carries precision and increment metadata so Nautilus can store and route prices consistently, but it is not a directly tradable contract. Examples include `SPX.XCBO`, `VIX.XCBO`, and venue-specific reference indexes. ## Fields | Field | Type | Required/default | Notes | |-------------------|------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `currency` | `Currency` | Required | Reference currency for quoted values. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for quantities. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-------------------|----------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `currency` | `Currency` | Required | Reference currency for quoted values. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for quantities. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `IndexInstrument` has asset class `Index` and instrument class `Spot`. - It is a reference instrument and should not be used for order submission. - It has no limits, margins, fees, contract multiplier, expiry, or settlement currency. - Use option or futures types for tradable derivatives whose underlyings are indexes. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::IndexInstrument, types::{Currency, Price, Quantity}, }; let spx = IndexInstrument::builder() .instrument_id(InstrumentId::from("SPX.XCBO")) .raw_symbol(Symbol::from("SPX")) .currency(Currency::from("USD")) .price_precision(2) .size_precision(0) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from nautilus_trader.model import Currency from nautilus_trader.model import IndexInstrument from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol spx = IndexInstrument( instrument_id=InstrumentId.from_str("SPX.XCBO"), raw_symbol=Symbol("SPX"), currency=Currency.from_str("USD"), price_precision=2, size_precision=0, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("1"), ts_event=0, ts_init=0, ) ``` ## Adapters Representative adapters that create or consume `IndexInstrument` instruments include: - [Interactive Brokers](../../integrations/ib.md) for reference indexes. - [Databento](../../integrations/databento.md) for reference data feeds. ## Related guides - [Option Contract](option_contract.md) covers listed options on index underlyings. - [Futures Contract](futures_contract.md) covers index futures. # Option Contract Source: https://nautilustrader.io/docs/latest/concepts/instruments/option_contract/ `OptionContract` represents a listed put or call option on a non-crypto underlying. It defines the option kind, strike price, activation time, expiration time, currency, multiplier, and lot size. Examples include equity options, index options, and futures options. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `exchange` | `Option` | `None` | Exchange MIC or venue code when known. | | `underlying` | `Ustr` | Required | Underlying asset, future, or index. | | `option_kind` | `OptionKind` | Required | Put or call. | | `strike_price` | `Price` | Required | Option strike price. | | `activation_ns` | `UnixNanos` | Required | Contract activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Contract expiration timestamp. | | `currency` | `Currency` | Required | Premium quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `u8` | `0` | Options trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Contract multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `1` | Minimum order quantity. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `exchange` | `str \| None` | `None` | Exchange MIC or venue code when known. | | `underlying` | `str` | Required | Underlying asset, future, or index. | | `option_kind` | `OptionKind` | Required | Put or call. | | `strike_price` | `Price` | Required | Option strike price. | | `activation_ns` | `int` | Required | Contract activation timestamp. | | `expiration_ns` | `int` | Required | Contract expiration timestamp. | | `currency` | `Currency` | Required | Premium quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `int` | `0` | Options trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Contract multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `1` | Minimum order quantity. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `OptionContract` has instrument class `Option`. - It trades in whole contracts with size precision `0` and size increment `1`. - The option kind and strike price define the payoff shape. - Use `CryptoOption` for options where the underlying and settlement are crypto currencies. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::{AssetClass, OptionKind}, identifiers::{InstrumentId, Symbol}, instruments::OptionContract, types::{Currency, Price, Quantity}, }; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2021, 9, 17, 0, 0, 0).unwrap(); let expiration = Utc.with_ymd_and_hms(2021, 12, 17, 0, 0, 0).unwrap(); let aapl_call = OptionContract::builder() .instrument_id(InstrumentId::from("AAPL211217C00150000.OPRA")) .raw_symbol(Symbol::from("AAPL211217C00150000")) .asset_class(AssetClass::Equity) .exchange(Ustr::from("GMNI")) .underlying(Ustr::from("AAPL")) .option_kind(OptionKind::Call) .strike_price(Price::from("150.00")) .currency(Currency::from("USD")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .price_precision(2) .price_increment(Price::from("0.01")) .multiplier(Quantity::from("100")) .lot_size(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import OptionContract from nautilus_trader.model import OptionKind from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol aapl_call = OptionContract( instrument_id=InstrumentId.from_str("AAPL211217C00150000.OPRA"), raw_symbol=Symbol("AAPL211217C00150000"), asset_class=AssetClass.EQUITY, underlying="AAPL", option_kind=OptionKind.CALL, strike_price=Price.from_str("150.00"), currency=Currency.from_str("USD"), activation_ns=pd.Timestamp("2021-09-17", tz="UTC").value, expiration_ns=pd.Timestamp("2021-12-17", tz="UTC").value, price_precision=2, price_increment=Price.from_str("0.01"), multiplier=Quantity.from_int(100), lot_size=Quantity.from_int(1), ts_event=0, ts_init=0, exchange="GMNI", ) ``` ## Adapters Representative adapters that create or consume `OptionContract` instruments include: - [Databento](../../integrations/databento.md) for listed options data. - [Interactive Brokers](../../integrations/ib.md) for listed option contracts. ## Related guides - [Options](../options.md) covers option data, Greeks, and chain subscriptions. - [Crypto Option](crypto_option.md) covers crypto option contracts. # Option Spread Source: https://nautilustrader.io/docs/latest/concepts/instruments/option_spread/ `OptionSpread` represents an exchange-defined options strategy with more than one leg. The venue publishes the strategy as a single instrument with its own symbol, tick size, expiration, and execution rules. Examples include listed vertical spreads, calendar spreads, and other option strategies. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying strategy. | | `exchange` | `Option` | `None` | Exchange MIC or venue code when known. | | `underlying` | `Ustr` | Required | Underlying asset, future, or index. | | `strategy_type` | `Ustr` | Required | Venue strategy type, such as vertical. | | `activation_ns` | `UnixNanos` | Required | Strategy activation timestamp. | | `expiration_ns` | `UnixNanos` | Required | Strategy expiration timestamp. | | `currency` | `Currency` | Required | Premium quote and settlement currency. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `u8` | `0` | Option spreads trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Strategy multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `1` | Minimum order quantity. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying strategy. | | `exchange` | `str \| None` | `None` | Exchange MIC or venue code when known. | | `underlying` | `str` | Required | Underlying asset, future, or index. | | `strategy_type` | `str` | Required | Venue strategy type, such as vertical. | | `activation_ns` | `int` | Required | Strategy activation timestamp. | | `expiration_ns` | `int` | Required | Strategy expiration timestamp. | | `currency` | `Currency` | Required | Premium quote and settlement currency. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_precision` | `int` | `0` | Option spreads trade in whole contracts. | | `size_increment` | `Quantity` | `1` | Minimum contract size step. | | `multiplier` | `Quantity` | Required | Strategy multiplier. | | `lot_size` | `Quantity` | Required | Rounded lot or contract lot size. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `1` | Minimum order quantity. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `OptionSpread` has instrument class `OptionSpread`. - The venue publishes the spread as a single tradable instrument. - It trades in whole contracts with size precision `0` and size increment `1`. - Store venue-specific leg details in `info` when the adapter provides them. ## Example ```rust tab="Rust" use chrono::{TimeZone, Utc}; use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::OptionSpread, types::{Currency, Price, Quantity}, }; use ustr::Ustr; let activation = Utc.with_ymd_and_hms(2023, 11, 6, 20, 54, 7).unwrap(); let expiration = Utc.with_ymd_and_hms(2024, 2, 23, 22, 59, 0).unwrap(); let sr3_spread = OptionSpread::builder() .instrument_id(InstrumentId::from("UD:U$: GN 2534559.GLBX")) .raw_symbol(Symbol::from("UD:U$: GN 2534559")) .asset_class(AssetClass::FX) .exchange(Ustr::from("XCME")) .underlying(Ustr::from("SR3")) .strategy_type(Ustr::from("GN")) .activation_ns(UnixNanos::from(activation.timestamp_nanos_opt().unwrap() as u64)) .expiration_ns(UnixNanos::from(expiration.timestamp_nanos_opt().unwrap() as u64)) .currency(Currency::from("USD")) .price_precision(2) .price_increment(Price::from("0.01")) .multiplier(Quantity::from("1")) .lot_size(Quantity::from("1")) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" import pandas as pd from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import OptionSpread from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol sr3_spread = OptionSpread( instrument_id=InstrumentId.from_str("UD:U$: GN 2534559.GLBX"), raw_symbol=Symbol("UD:U$: GN 2534559"), asset_class=AssetClass.FX, underlying="SR3", strategy_type="GN", activation_ns=pd.Timestamp("2023-11-06T20:54:07", tz="UTC").value, expiration_ns=pd.Timestamp("2024-02-23T22:59:00", tz="UTC").value, currency=Currency.from_str("USD"), price_precision=2, price_increment=Price.from_str("0.01"), multiplier=Quantity.from_int(1), lot_size=Quantity.from_int(1), ts_event=0, ts_init=0, exchange="XCME", ) ``` ## Adapters Representative adapters that create or consume `OptionSpread` instruments include: - [Databento](../../integrations/databento.md) for listed option spread markets. - [Interactive Brokers](../../integrations/ib.md) for exchange-defined option strategies. ## Related guides - [Option Contract](option_contract.md) covers single-leg option contracts. - [Options](../options.md) covers option data, Greeks, and chain subscriptions. # Perpetual Contract Source: https://nautilustrader.io/docs/latest/concepts/instruments/perpetual_contract/ `PerpetualContract` represents a generic perpetual futures contract across asset classes. Use it when a venue exposes a perpetual swap that is not specifically modeled as `CryptoPerpetual`. Examples include non-crypto perpetual contracts and venue-specific synthetic swaps. ## Fields | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `Ustr` | Required | Underlying asset or reference market. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `base_currency` | `Option` | `None` | Base currency, required for inverse. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-----------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `underlying` | `str` | Required | Underlying asset or reference market. | | `asset_class` | `AssetClass` | Required | Asset class of the underlying. | | `base_currency` | `Currency \| None` | `None` | Base currency, required for inverse. | | `quote_currency` | `Currency` | Required | Currency used to quote the price. | | `settlement_currency` | `Currency` | Required | Currency used to settle PnL and fees. | | `is_inverse` | `bool` | Required | True when sizing/costing is inverse. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity` | `1` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `PerpetualContract` has instrument class `Swap`. - It has no activation timestamp or expiration timestamp. - Inverse contracts require a base currency. - Linear contracts typically settle in the quote currency. - Use `CryptoPerpetual` for crypto perpetuals where the base asset is a currency. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::PerpetualContract, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; use ustr::Ustr; let eurusd_perp = PerpetualContract::builder() .instrument_id(InstrumentId::from("EURUSD-PERP.AX")) .raw_symbol(Symbol::from("EURUSD-PERP")) .underlying(Ustr::from("EURUSD")) .asset_class(AssetClass::FX) .base_currency(Currency::from("EUR")) .quote_currency(Currency::from("USD")) .settlement_currency(Currency::from("USD")) .is_inverse(false) .price_precision(5) .size_precision(0) .price_increment(Price::from("0.00001")) .size_increment(Quantity::from("1")) .margin_init(dec!(0.03)) .margin_maint(dec!(0.03)) .maker_fee(dec!(0.00002)) .taker_fee(dec!(0.00002)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import PerpetualContract from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol eurusd_perp = PerpetualContract( instrument_id=InstrumentId.from_str("EURUSD-PERP.AX"), raw_symbol=Symbol("EURUSD-PERP"), underlying="EURUSD", asset_class=AssetClass.FX, quote_currency=Currency.from_str("USD"), settlement_currency=Currency.from_str("USD"), is_inverse=False, price_precision=5, size_precision=0, price_increment=Price.from_str("0.00001"), size_increment=Quantity.from_int(1), ts_event=0, ts_init=0, base_currency=Currency.from_str("EUR"), margin_init=Decimal("0.03"), margin_maint=Decimal("0.03"), maker_fee=Decimal("0.00002"), taker_fee=Decimal("0.00002"), ) ``` ## Adapters Representative adapters that create or consume `PerpetualContract` instruments include: - [Architect AX](../../integrations/architect_ax.md) for venue-defined perpetual contracts. ## Related guides - [Crypto Perpetual](crypto_perpetual.md) covers crypto perpetual futures. - [Data](../data.md) covers mark prices, index prices, and funding rate updates. # Synthetic Instrument Source: https://nautilustrader.io/docs/latest/concepts/instruments/synthetic_instrument/ `SyntheticInstrument` represents a local instrument whose price comes from a formula over other instruments. It is useful for spreads, baskets, ratios, and other derived prices that should appear in the system as an instrument. Examples include `(BTC.BINANCE + LTC.BINANCE) / 2.0` and ratio-style pairs built from component instrument prices. ## Fields | Field | Type | Required/default | Notes | |-------------------|---------------------|------------------|---------------------------------------------| | `symbol` | `Symbol` | Required | Synthetic symbol used with venue `SYNTH`. | | `id` | `InstrumentId` | Derived | Instrument ID formed from `symbol.SYNTH`. | | `price_precision` | `u8` | Required | Decimal places allowed for synthetic price. | | `price_increment` | `Price` | Derived | Smallest price step from precision. | | `components` | `Vec` | Required | Component instruments used by the formula. | | `formula` | `String` | Required | Numeric expression over component IDs. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | Field | Type | Required/default | Notes | |-------------------|----------------------|------------------|---------------------------------------------| | `symbol` | `Symbol` | Required | Synthetic symbol used with venue `SYNTH`. | | `id` | `InstrumentId` | Derived | Instrument ID formed from `symbol.SYNTH`. | | `price_precision` | `int` | Required | Decimal places allowed for synthetic price. | | `price_increment` | `Price` | Derived | Smallest price step from precision. | | `components` | `list[InstrumentId]` | Required | Component instruments used by the formula. | | `formula` | `str` | Required | Numeric expression over component IDs. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | *Note: Python constructs the instrument ID from `symbol` and the `SYNTH` venue. Rust stores the same value as `id`.* ## Behavior - `SyntheticInstrument` is local to Nautilus and does not represent a venue orderable market. - It always uses the synthetic venue `SYNTH`. - Python requires at least two component instrument IDs. - The formula must compile against the supplied component identifiers before the object is valid. - It has no venue limits, margins, fees, order book, or adapter-specific metadata. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ identifiers::{InstrumentId, Symbol}, instruments::SyntheticInstrument, }; let synthetic = SyntheticInstrument::new( Symbol::from("BTC-LTC"), 2, vec![ InstrumentId::from("BTC.BINANCE"), InstrumentId::from("LTC.BINANCE"), ], "(BTC.BINANCE + LTC.BINANCE) / 2.0", UnixNanos::default(), UnixNanos::default(), ); ``` ```python tab="Python" from nautilus_trader.model import InstrumentId from nautilus_trader.model import Symbol from nautilus_trader.model import SyntheticInstrument synthetic = SyntheticInstrument( symbol=Symbol("BTC-LTC"), price_precision=2, components=[ InstrumentId.from_str("BTC.BINANCE"), InstrumentId.from_str("LTC.BINANCE"), ], formula="(BTC.BINANCE + LTC.BINANCE) / 2.0", ts_event=0, ts_init=0, ) ``` ## Adapters `SyntheticInstrument` is local only. It derives prices from component instruments that may come from any adapter already loaded into the system. ## Related guides - [Synthetics](../synthetics.md) covers formula-derived instruments and synthetic bars. - [Data](../data.md) explains market data that references instruments. # Tokenized Asset Source: https://nautilustrader.io/docs/latest/concepts/instruments/tokenized_asset/ `TokenizedAsset` represents a spot-like token that tracks another asset on a crypto venue. Use it for tokenized equities, tokenized funds, or similar instruments where the trading venue exposes a token but the economic reference is an external asset. Examples include tokenized stock or ETF symbols on crypto venues. ## Fields | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | Stored as `id` in Rust. | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Economic asset classification. | | `base_currency` | `Currency` | Required | Tokenized asset or base token. | | `quote_currency` | `Currency` | Required | Currency used to price the token. | | `price_precision` | `u8` | Required | Decimal places allowed for prices. | | `size_precision` | `u8` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `UnixNanos` | Required | Event timestamp in nanoseconds. | | `ts_init` | `UnixNanos` | Required | Initialization timestamp in nanoseconds. | | `isin` | `Option` | `None` | International Securities ID when known. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Option` | `None` | Rounded lot or board size. | | `max_quantity` | `Option` | `None` | Maximum order quantity. | | `min_quantity` | `Option` | `None` | Minimum order quantity. | | `max_notional` | `Option` | `None` | Maximum order notional value. | | `min_notional` | `Option` | `None` | Minimum order notional value. | | `max_price` | `Option` | `None` | Maximum valid quote or order price. | | `min_price` | `Option` | `None` | Minimum valid quote or order price. | | `margin_init` | `Option` | `0` | Initial margin rate. | | `margin_maint` | `Option` | `0` | Maintenance margin rate. | | `maker_fee` | `Option` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Option` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `Option` | `None` | Registered variable tick scheme name. | | `info` | `Option` | `None` | Adapter metadata. | | Field | Type | Required/default | Notes | |-------------------|--------------------|------------------|------------------------------------------| | `instrument_id` | `InstrumentId` | Required | | | `raw_symbol` | `Symbol` | Required | Native venue symbol. | | `asset_class` | `AssetClass` | Required | Economic asset classification. | | `base_currency` | `Currency` | Required | Tokenized asset or base token. | | `quote_currency` | `Currency` | Required | Currency used to price the token. | | `price_precision` | `int` | Required | Decimal places allowed for prices. | | `size_precision` | `int` | Required | Decimal places allowed for order sizes. | | `price_increment` | `Price` | Required | Smallest valid price step. | | `size_increment` | `Quantity` | Required | Smallest valid size step. | | `ts_event` | `int` | Required | Event timestamp in nanoseconds. | | `ts_init` | `int` | Required | Initialization timestamp in nanoseconds. | | `isin` | `str \| None` | `None` | International Securities ID when known. | | `multiplier` | `Quantity` | `1` | Contract multiplier. | | `lot_size` | `Quantity \| None` | `None` | Rounded lot or board size. | | `max_quantity` | `Quantity \| None` | `None` | Maximum order quantity. | | `min_quantity` | `Quantity \| None` | `None` | Minimum order quantity. | | `max_notional` | `Money \| None` | `None` | Maximum order notional value. | | `min_notional` | `Money \| None` | `None` | Minimum order notional value. | | `max_price` | `Price \| None` | `None` | Maximum valid quote or order price. | | `min_price` | `Price \| None` | `None` | Minimum valid quote or order price. | | `margin_init` | `Decimal \| None` | `0` | Initial margin rate. | | `margin_maint` | `Decimal \| None` | `0` | Maintenance margin rate. | | `maker_fee` | `Decimal \| None` | `0` | Maker fee rate. Negative values rebate. | | `taker_fee` | `Decimal \| None` | `0` | Taker fee rate. Negative values rebate. | | `tick_scheme` | `str \| None` | `None` | Registered variable tick scheme name. | | `info` | `dict \| None` | `None` | Adapter metadata. | *Note: Python constructors use `instrument_id`; Rust stores the same value as `id`.* ## Behavior - `TokenizedAsset` has instrument class `Spot`. - It is never inverse, and its cost currency is the quote currency. - It can carry an `isin` when the token references a listed security. - It has no activation timestamp, expiry, strike, or option kind. ## Example ```rust tab="Rust" use nautilus_core::UnixNanos; use nautilus_model::{ enums::AssetClass, identifiers::{InstrumentId, Symbol}, instruments::TokenizedAsset, types::{Currency, Price, Quantity}, }; use rust_decimal_macros::dec; let aaplx = TokenizedAsset::builder() .instrument_id(InstrumentId::from("AAPLx/USD.KRAKEN")) .raw_symbol(Symbol::from("AAPLxUSD")) .asset_class(AssetClass::Equity) .base_currency(Currency::get_or_create_crypto("AAPLx")) .quote_currency(Currency::from("USD")) .price_precision(2) .size_precision(4) .price_increment(Price::from("0.01")) .size_increment(Quantity::from("0.0001")) .min_quantity(Quantity::from("0.0001")) .maker_fee(dec!(-0.0002)) .taker_fee(dec!(0.001)) .ts_event(UnixNanos::default()) .ts_init(UnixNanos::default()) .build() .unwrap(); ``` ```python tab="Python" from decimal import Decimal from nautilus_trader.model import AssetClass from nautilus_trader.model import Currency from nautilus_trader.model import InstrumentId from nautilus_trader.model import Price from nautilus_trader.model import Quantity from nautilus_trader.model import Symbol from nautilus_trader.model import TokenizedAsset aaplx = TokenizedAsset( instrument_id=InstrumentId.from_str("AAPLx/USD.KRAKEN"), raw_symbol=Symbol("AAPLxUSD"), asset_class=AssetClass.EQUITY, base_currency=Currency.from_str("AAPLx"), quote_currency=Currency.from_str("USD"), price_precision=2, size_precision=4, price_increment=Price.from_str("0.01"), size_increment=Quantity.from_str("0.0001"), ts_event=0, ts_init=0, min_quantity=Quantity.from_str("0.0001"), maker_fee=Decimal("-0.0002"), taker_fee=Decimal("0.001"), ) ``` ## Adapters Representative adapters that create or consume `TokenizedAsset` instruments include: - [Kraken](../../integrations/kraken.md) for tokenized assets where the venue exposes them. ## Related guides - [Currency Pair](currency_pair.md) covers ordinary crypto spot pairs. - [Equity](equity.md) covers listed cash equities.