Live Trading

The same strategy and execution-algorithm code can run across backtest and live environments. Live execution also introduces venue, transport, timing, persistence, external-activity, and reconciliation behavior that a simulation may not reproduce.

Backtest and live differences

Backtests advance a controlled clock from historical data and execute orders on simulated venues. A live node shares strategy and execution-algorithm code with a backtest, but coordinates with systems outside the process boundary.

• Venue: Venue rules and adapter capabilities determine which order types, instructions, and events are available. See Adapters.
• Transport: A network failure can leave an order command outcome unknown. See Command outcomes.
• Timing: Independent inputs can interleave, and the runner does not define one global FIFO order. See Dispatch priority.
• Persistence: Built-in cache backends process writes independently of venue transport, and event-store capture does not gate dispatch on durable commit. See Persistence before transport.
• External activity: Venue reports can include orders created outside the node. See External order creation.
• Reconciliation: Startup and runtime checks align retained local state with venue reports. See Execution reconciliation.

Live node lifecycle

Rust LiveNode::run() prepares cached and venue state before starting trader components, then owns the event loop and coordinated shutdown.

Live node lifecycle: instruments and execution state are prepared before strategies start trading.

Cache restoration runs when a backing database is attached and cache loading is enabled. Connection, reconciliation, or trader startup failures abort startup and follow the coordinated cleanup path.

Hosted event loops

Use run_async() from Python to run a node on an event loop you already own, such as an ASGI server serving a dashboard beside the node. Use run() when the node should own the calling thread and signal handling.

This lifecycle sketch leaves node configuration and request serving to the application:

import asyncio

from nautilus_trader.live import LiveNode
from nautilus_trader.live import LiveNodeHandle


async def wait_until_running(
    handle: LiveNodeHandle,
    task: asyncio.Task[None],
) -> None:
    while not handle.is_running:
        if task.done():
            await task
            raise RuntimeError("LiveNode stopped during startup")
        await asyncio.sleep(0.01)


async def serve_with_node(node: LiveNode) -> None:
    cache, portfolio, handle = node.cache, node.portfolio, node.handle()
    run_task: asyncio.Task[None] | None = None
    service_task: asyncio.Task[None] | None = None
    try:
        run_task = asyncio.create_task(node.run_async())
        await wait_until_running(handle, run_task)

        service_task = asyncio.create_task(serve_requests(cache, portfolio, handle))
        done, _ = await asyncio.wait(
            (run_task, service_task),
            return_when=asyncio.FIRST_COMPLETED,
        )
        if run_task in done:
            await run_task
            raise RuntimeError("LiveNode stopped while the service was running")
        await service_task
    finally:
        if service_task is not None and not service_task.done():
            service_task.cancel()
            await asyncio.gather(service_task, return_exceptions=True)
        try:
            if run_task is not None:
                handle.stop()
                await run_task
        finally:
            node.dispose()

Both entry points run the same lifecycle, so a hosted node performs the same startup ordering, maintenance, reconciliation, and shutdown as an owned one. The mode decides only who owns signal handling: a hosted node installs no handlers, leaving SIGINT and SIGTERM to the host.

Access and shutdown

run_async() returns a coroutine and lends the node to it for the run's duration:

• Capture cache, portfolio, and handle() before starting. Each stays usable while the node runs, whereas reading state through the node itself raises until the run returns it.
• handle() works throughout, since it is how a host stops the node. is_running also answers throughout because it reads the same handle.
• Call dispose() after the run task finishes to release the node's resources. Calling it during the run returns without doing anything; it does not defer disposal.

LiveNodeHandle is safe to call from any thread, including a signal handler. stop() requests a graceful shutdown and returns immediately, so the awaiting task resolves only once shutdown finishes. Cancelling that task requests the same shutdown, waits for it, then re-raises the cancellation, which keeps asyncio.timeout and task groups behaving as their callers expect.

Host integration and limits

Compatibility is tested with the default asyncio loop and uvloop. An ASGI lifespan managed by Uvicorn can apply the same ownership pattern without transferring signal handling to the node. Before an ASGI lifespan reports startup complete, wait until the handle reports Running while checking whether the run task has failed. Keep supervising the task after startup, and treat unexpected completion as a service failure.

While the node is running, it yields to the host loop periodically, so a burst of events cannot starve the host's own callbacks. Startup and shutdown drain their queues without yielding, so a large instrument load or a shutdown backlog can hold the loop for the length of that drain.

A node configured with a cache database backing is rejected on a host loop. Those backings wait for their worker task by blocking the calling thread, which stalls the host loop rather than slowing it. Use run() for a database-backed node.

Configuration

For how config structs handle defaults, T vs Option<T> semantics, and builder patterns, see the Configuration concept guide.

For node and execution engine settings, strategy configuration, cache backing, and multi-venue wiring, see the Configure a live trading node how-to guide.

Execution reconciliation

For how submit, modify, and cancel commands resolve, see Command outcomes.

At startup, reconciliation aligns cached order and position state with venue reports before trader components start. Continuous checks can then monitor in-flight orders, open orders, positions, and own order books while the node runs.

When an adapter declares bounded historical reports, startup reconciliation applies their fill economics only when the report set and retained state prove a coherent position transition. Incomplete or ambiguous history can still recover exact order state without changing positions or portfolio economics.

See Execution reconciliation for configuration, recovery procedures, runtime checks, scenarios, and invariants.

Rust live runner metrics

Rust LiveNode exposes primitive runner metrics through LiveNodeHandle::metrics_snapshot(). Get the handle from the node before calling run(), then poll snapshots from another task and derive rates or utilization from deltas.

use std::time::Duration;

use nautilus_common::enums::Environment;
use nautilus_live::node::{LiveNode, RunnerMetricsDelta};

let mut node = LiveNode::builder(trader_id, Environment::Live)?
    // Add clients, actors, and strategies here.
    .build()?;

let metrics_handle = node.handle();

tokio::spawn(async move {
    let mut prev = metrics_handle.metrics_snapshot();
    let mut interval = tokio::time::interval(Duration::from_secs(1));

    loop {
        interval.tick().await;

        let next = metrics_handle.metrics_snapshot();
        let delta = RunnerMetricsDelta::from_snapshots(prev, next);
        if delta.elapsed_ns == 0 {
            prev = next;
            continue;
        }

        let elapsed_s = delta.elapsed_ns as f64 / 1_000_000_000.0;
        let data_event_rate = delta.data_events as f64 / elapsed_s;
        let data_event_staleness_ns = if next.data_events.last_dispatch_at_ns == 0 {
            0
        } else {
            next.elapsed_ns
                .saturating_sub(next.data_events.last_dispatch_at_ns)
        };

        log::info!(
            "Runner metrics: data_event_rate={data_event_rate:.0} \
             data_event_staleness_ns={data_event_staleness_ns} \
             dispatch_utilization={:.6} loop_utilization={:.6} \
             mean_dispatch_ns={} data_queue_depth={}",
            delta.dispatch_utilization(),
            delta.loop_utilization(),
            delta.mean_dispatch_ns(),
            next.data_events.queue_depth,
        );

        prev = next;
    }
});

node.run().await?;

Snapshot scope and interpretation

The snapshot covers LiveNode::run channel dispatch after startup, including residual dispatch during the shutdown grace period. It does not include startup buffering, startup flushes, or the final post-loop drain.

• dispatch_busy_ns covers the five dispatch branches. maintenance_busy_ns and external_msgbus_busy_ns cover non-dispatch loop work.
• Queue depths are point samples from the maintenance tick while the node is running, and can be stale during shutdown grace.
• Snapshots are lock-free and may not be a consistent cross-field view. Derive rates from successive snapshots with saturating deltas.
• Counters reset when LiveNode::run enters steady state.

Dispatch priority and overload behavior

The live runner's seven internal message channels are separate and unbounded. When several message channels are ready together, the runner polls in this order:

1. Time and system work.
2. Execution events.
3. Execution commands.
4. External message-bus ingress.
5. Data events.
6. Data commands.

Events precede commands within the execution and data channel pairs.

This polling order keeps a market-data backlog from taking priority over ready execution traffic. It does not define one global FIFO order across channels, adapters, or venues. Each selected branch runs to completion before the runner polls again, so a slow handler delays every channel. The runner yields to the host event loop periodically, but yielding does not change channel priority or shorten a slow handler.

Use runner metrics and queue-state events to detect this pressure. The thresholds are operational signals, not service-time guarantees. The application must decide how to alert, reduce input, halt new exposure, or stop the node when pressure persists.

Queue pressure monitoring

LiveNode converts runner queue samples into typed state transitions when LiveNodeConfig.queue_monitor is set. The monitor is disabled by default and publishes no queue-state events while the field is unset.

Configure thresholds

The following example sets the thresholds applied to every monitored runner channel:

use nautilus_live::config::{LiveNodeConfig, QueueMonitorConfig};

let config = LiveNodeConfig {
    queue_monitor: Some(
        QueueMonitorConfig::builder()
            .queue_depth_trigger(1_000)
            .queue_depth_clear(500)
            .mean_dispatch_ns_trigger(250_000)
            .mean_dispatch_ns_clear(150_000)
            .build(),
    ),
    ..Default::default()
};

The four values apply to each monitored runner channel:

• time_events
• exec_events
• exec_commands
• data_events
• data_commands

Each clear threshold must be lower than its trigger threshold. Configuration validation rejects equal or inverted thresholds.

State transitions

The live runner evaluates the monitor on its 100 ms maintenance tick, after sampling current queue depths. Queue depth is a point-in-time value. Mean dispatch time uses the messages and dispatch busy time accumulated since the previous metrics snapshot.

Each channel tracks Backlogged and Slow independently:

• A value between the clear and trigger thresholds retains the prior state, so it does not publish another event.
• If both conditions cross on one tick, the node publishes two events. Each condition clears independently.
• A sample window with no dispatches does not evaluate Slow. The condition retains its prior state until a window contains a dispatch.

Typed delivery

Each transition publishes a fresh QueueStateChanged value on events.system.QueueStateChanged.<channel>, where <channel> is the Rust variant name, such as DataEvents for Python SystemChannel.DATA_EVENTS. The event identifies the configured trader, runner channel, condition, and transition state. It also records the queue depth and mean dispatch time at the crossing, a fresh event ID, and event timestamps.

Actors subscribe with subscribe_queue_state(...) and receive events through on_queue_state(...). The Python API exposes SystemChannel, QueueCondition, QueueState, and QueueStateChanged from nautilus_trader.common. Publication stays on the in-process typed message bus, and the event has no wire representation for external message-bus streaming. See Queue pressure state for actor examples.

Socket transport state

Publication and routing

Actors can observe transport availability for adapters that opt into socket state reporting. LiveNode publishes SocketStateChanged on events.system.SocketStateChanged.<client_id>.<endpoint> with the trader ID, client ID, optional venue, stable endpoint label, state, fresh event ID, and event timestamps. The endpoint label identifies one logical adapter transport without exposing its URL. LiveNode sets both timestamps from the kernel clock when it handles the transport's neutral state notification. Adapters send the notification through the runner's system-event channel, separately from market data. The internal channel is not part of queue-pressure monitoring.

The client ID and endpoint components percent-encode bytes other than ASCII letters, digits, -, and _, so dots and wildcard characters in labels cannot change topic matching. Use subscribe_socket_state filters to select a client, endpoint, or both without constructing topic names.

State semantics

Connected reports transport availability. It does not mean that authentication, subscription replay, or adapter recovery has completed.

Dead-peer detection

A connection can stop delivering without closing: a NAT or load balancer drops it with no FIN and no RST, so writes keep succeeding into the send buffer and nothing surfaces the loss.

Heartbeat timeout

Handler-mode WebSocket and raw TCP clients reconnect on heartbeat timeout. With a configured heartbeat, the default timeout is three heartbeat intervals. An explicit heartbeat_timeout_secs overrides that window. WebSocket clients reset it on any inbound frame; raw TCP clients reset it when bytes arrive. The client expects the peer to answer heartbeats, so a transport with no heartbeat gets no default window. An explicit timeout can still enable detection without a heartbeat.

For WebSocket clients, that window counts all frames, so a keepalive reply refreshes it and a quiet market does not trip it.

Feed idle timeout

An adapter that also needs to detect a feed which stopped flowing while the transport stays healthy sets a separate idle timeout, which only Text and Binary frames refresh. That second window suits a venue which pushes data on a known cadence. Where the venue answers the keepalive with a text payload, its reply refreshes the idle timeout exactly like real data does, so the window means something only when it sits below the heartbeat interval.

Adapter and actor integration

Adapter integrations construct a SocketStateSink and set it through the network client's state_sink builder option. Publication requires the LiveNode runner; the standalone AsyncRunner does not publish these events.

Actors subscribe with subscribe_socket_state(...) and receive events through on_socket_state(...). The Python API exposes SocketState and SocketStateChanged from nautilus_trader.common. Delivery stays on the typed in-process bus; external message-bus streaming and wire formats do not expose these events.

Endpoint reconnect commands

An actor or strategy can call reconnect_socket(client_id, endpoint) with an endpoint label from a state event. The runner routes the typed command through the kernel and the engine that owns the registered endpoint. The engine invokes only that transport's reconnect handle. It does not call the containing DataClient or ExecutionClient disconnect and connect lifecycle.

An accepted request emits SocketStateChanged for the selected endpoint:

1. SocketState.DISCONNECTED as the transport enters reconnect mode.
2. SocketState.CONNECTED after transport recovery.

The transport's normal reconnect controller preserves its authentication, subscription replay, and adapter recovery behavior.

The kernel logs unknown clients, unsupported clients, unknown or ambiguous endpoints, duplicate requests, disconnecting transports, and closed transports. These rejections emit no socket state change and do not affect another endpoint. Endpoint labels use identifier characters only and never contain raw URLs.

Shutdown on error

Set LiveNodeConfig.shutdown_on_error=True so that a Rust error log requests a live node shutdown. The Rust logger records the first log::error! emitted after the kernel starts, including error logs from other threads, and the kernel publishes a ShutdownSystem command when the live event loop next checks for shutdown.

The shutdown request follows the normal live node stop path. The node stops the trader, awaits the post-stop delay, disconnects clients, and stops the engines. It does not abort the process.

from nautilus_trader.config import LiveNodeConfig

config = LiveNodeConfig(shutdown_on_error=True)

The trigger follows these rules:

• Error logs suppressed by component filters or logging bypass mode still request shutdown.
• A new kernel run clears and re-arms the trigger, so a process can restart a node without reinitializing the logging system.
• Shutdown-on-error observes Rust log records, not Python logging.error(...) calls.

Related guides

• Execution reconciliation - State recovery and runtime consistency checks.
• Execution policies - Command delivery, persistence, and recovery boundaries.
• Python - Python ownership, runtime, and public API boundaries.
• Configure a live trading node - Node and engine configuration.
• Run live trading with Rust - Rust node setup and venue connection.
• Adapters - Venue connectivity.
• Execution - Command outcomes and order execution.
• Message bus - Typed in-process publish and subscribe behavior.
• Backtesting - Testing strategies before deployment.