TestingCopy Anchor Link

Automated tests serve as executable specifications for the trading platform. They document intended behavior, support confident refactoring, and catch regressions before production. Tests also provide examples of complex flows and rapid CI feedback so issues surface early.

The suite covers these categories:

• Unit tests
• Integration tests
• Acceptance tests
• Performance tests
• Property-based tests
• Fuzzing
• Memory leak tests

Testing policyCopy Anchor Link

Tests and runtime contracts form one design system. The Design by contract ladder pushes invariants into the type system where possible. The testing ladder below escalates the remaining unknowns through larger input spaces and richer execution models. Each layer extends coverage to inputs or execution states the layer below cannot reach.

Not every module requires every technique. Use this section to decide which layers apply before adding tests or debug_assert! statements.

Mechanism ladderCopy Anchor Link

The Rust guide covers runtime contracts. The order of preference is:

1. The type system.
2. check_* from nautilus_core::correctness at API boundaries.
3. debug_assert! for internal invariants.
4. assert! for soundness-critical or always-on checks.

Test layers follow a parallel escalation. Start at the lowest layer that proves what matters. Climb only if that layer stops detecting regressions or the input space grows beyond hand-picked cases.

The formal verification rung is aspirational: no Kani or Prusti harness has landed in the workspace. The row records the escalation condition for when a verifier is adopted, not a current obligation.

Projection ruleCopy Anchor Link

Module shape determines which layers pay off. Some modules do not need the full ladder. Apply the rule to each module, not each crate. An adapter crate contains pure parsers and I/O-bound client loops. Each row applies to a different part.

When not to add coverageCopy Anchor Link

• Add debug_assert! only where a test can reach it. Release builds strip the check, so an unexercised assertion has no signal. A targeted unit test counts as a harness. A proptest or fuzz harness amplifies the signal.
• Prefer a proptest over a hand-written edge-case test when the invariant spans a whole class of inputs. Targeted unit tests remain valid for known venue pathologies and as regression reproducers for shrunk counterexamples.
• Do not duplicate a live spec acceptance card as an integration test. Link to it instead.
• Do not pad coverage with tests that assert language or framework guarantees (Option::is_some after Some(..), Vec::len after push).

DST readinessCopy Anchor Link

Deterministic simulation testing (DST) requires the runtime to be free of ambient non-determinism. Before promoting a module to run under DST, verify the following:

• Time, task, runtime, and signal primitives route through nautilus_common::live::dst rather than tokio directly. Wall-clock reads go through the seam in nautilus_core::time rather than SystemTime::now() at call sites.
• State maps with ordering-dependent iteration use IndexMap or IndexSet, not the default hash collections.
• Every tokio::select! on a control-plane path sets biased so poll order is fixed.
• No calls to Instant::now(), SystemTime::now(), tokio::signal::ctrl_c, std::thread::spawn, or tokio::task::spawn_blocking escape the seam. Blocking-thread and OS-thread primitives break madsim determinism the same way an ambient clock read does.
• Replay-sensitive IDs (trade_id, venue_order_id) are pure functions of their inputs. See crates/execution/src/reconciliation/ids.rs. Ephemeral event UUIDs on other reconciliation paths do not need to be deterministic.

The surface probe in crates/common/src/live/dst.rs only pins the re-export shape. It does not check that callers actually use the seam. Enforcement is by review. Run the audit whenever a new async module enters the workspace or an existing module gains new control-plane scheduling.

Property-based testingCopy Anchor Link

Property testing verifies that logic holds for all valid inputs, not just hand-picked examples. We use proptest in Rust to enforce invariants.

• Use cases: Core domain types (Price, Quantity, UnixNanos), accounting engines, matching engines, and state machines.
• Example invariants:
  • Round-trip serialization: parse(to_string(value)) == value
  • Inverse operations: (A + B) - B == A
  • Transitivity: If A < B and B < C, then A < C

FuzzingCopy Anchor Link

Fuzzing introduces unstructured or malicious data to the system to verify it fails gracefully.

• Use cases: Network boundaries, exchange data parsers (JSON, FIX, WebSocket feeds), and complex state machines.
• Goal: The system returns a Result::Err and never panics, hangs, or leaks memory when encountering malformed data.

Adapter fuzz binaries are registered in each adapter package behind its fuzz feature. Run all registered targets for one adapter from the repository root:

scripts/fuzz-adapter.sh derive

The workspace pins libfuzzer-sys, and nautilus-live owns the shared libFuzzer integration. A separate publish = false package is reserved only for fuzz targets whose dependencies must not enter a published adapter graph. Lighter's git-pinned Pornin differential oracle is one such dependency.

When building or modifying core types, write property tests to cover the mathematical boundaries.

Performance tests help evolve performance-critical components.

Run tests with pytest, our primary test runner. Use parametrized tests and fixtures (e.g., @pytest.mark.parametrize) to avoid repetitive code and improve clarity.

Running testsCopy Anchor Link

CI runs scripts/ci/check_test_network.py through make test-scripts to flag direct network calls with non-local literal addresses, explicit live-test switches, and fork-RPC options. It checks test directories and Rust files from their named test module onward. Loopback addresses and reserved fixture domains are allowed.

Check limits: This heuristic does not resolve variable destinations, follow calls into other code, or enforce network isolation.

Python testsCopy Anchor Link

The Python test suite lives under python/tests/ and tests the Rust-backed PyO3 package. It requires a built extension module and uses the Python project under python/. From the repository root, run:

make pytest

The Makefile target isolates certain test modules in separate pytest processes to avoid global Rust state conflicts. Use make pytest rather than invoking pytest directly.

make pytest runs tests in parallel with pytest-xdist, using one worker per CPU up to 32. Set PYTEST_XDIST_AUTO_NUM_WORKERS to change the worker count, for example make pytest PYTEST_XDIST_AUTO_NUM_WORKERS=8.

Local make pytest runs use the debug extension from make build-debug. CI tests a release wheel. Do not probe Rust panic paths in process in python/tests/ with pytest.raises(BaseException) or similar broad catches. These tests can appear to pass against the debug build and abort the interpreter against the release wheel.

For abort-prone PyO3 or FFI methods, verify the Python signature and parameter names, or isolate the call in a subprocess.

For the registered Rust benchmark set:

make cargo-ci-benches

No canonical Python performance suite is wired into CI. See the Benchmarking guide for focused Criterion and iai commands, profiling, and measurement policy. Run benchmarks separately from unit tests to avoid interference.

Rust testsCopy Anchor Link

Before running the full suite, prepare the large test fixtures.

make cargo-test
# or
cargo nextest run --workspace --features "$(bash scripts/cargo-features.bash)" --cargo-profile nextest --lib --tests

Rust doctestsCopy Anchor Link

cargo nextest cannot execute doctests, so they run through a separate target:

make cargo-test-doc
# or
cargo test --doc --workspace --features "$(bash scripts/cargo-features.bash)" --profile nextest

Doc examples are a maintained test surface. The scheduled nightly-tests workflow runs this target with Python 3.13 and 3.14. See the Rust guide for how to annotate a fence so it compiles.

Testing with optional featuresCopy Anchor Link

Use EXTRA_FEATURES to include optional features like capnp or hypersync:

# Test with capnp feature
make cargo-test EXTRA_FEATURES="capnp"

# Test with multiple features
make cargo-test EXTRA_FEATURES="capnp hypersync"

# Legacy shorthand for hypersync
make cargo-test HYPERSYNC=true

# Test specific crate with features
make cargo-test-crate-nautilus-serialization FEATURES="capnp"

IDE integrationCopy Anchor Link

• PyCharm: Right-click the tests folder or file -> "Run pytest".
• VS Code: Use the Python Test Explorer extension.

Test styleCopy Anchor Link

GeneralCopy Anchor Link

• Name test functions after what they exercise. You do not need to encode the expected assertions in the name.
• Add docstrings when they clarify setup, scenarios, or expectations.
• Group assertions when possible: perform all setup/act steps first, then assert together to avoid the act-assert-act smell.
• Use unwrap, expect, or direct panic!/assert calls inside tests. Clarity and conciseness matter more than defensive error handling here.
• For behavior tests, assert return values, state changes, or side effects rather than log messages. Log capture uses global state, depends on test execution order, and can break when wording changes.
• Capture logs only when logging itself is the contract, such as excluding secrets and preserving public diagnostic context. Serialize capture and join emitting tasks before releasing it. Assert the relevant records and secret absence; avoid whole-transcript assertions.

Python tests (python/tests/)Copy Anchor Link

Use pytest-style free functions and fixtures. Do not use test classes.

• Write each test as a standalone def test_*() function.
• Use @pytest.fixture for shared setup (instruments, engine instances, data). Prefer yield fixtures when teardown is needed (e.g., engine.dispose()).
• Use @pytest.mark.parametrize to cover multiple inputs without duplicating test bodies.
• Import model types from nautilus_trader.model, not from nautilus_trader._libnautilus.
• Test providers live in python/tests/providers.py. Use TestInstrumentProvider and TestDataProvider for common instruments and data.
• Mark tests that depend on unfinished features with @pytest.mark.skip(reason="WIP: <description>") rather than deleting them.

RustCopy Anchor Link

For Rust-specific test conventions (module structure, #[rstest], parameterization), see the Rust guide.

Waiting for asynchronous effectsCopy Anchor Link

In Rust tests, prefer a notification channel or another event owned by the test over repeated condition evaluation:

1. Subscribe before reading the authoritative state.
2. Recheck it after every notification so a transition between the read and the await cannot be missed.

When no suitable signal exists, use wait_until_async(...) from nautilus_common::testing. It stops as soon as the condition succeeds and applies a bounded timeout. Use a fixed sleep only when the time window itself is under test.

MocksCopy Anchor Link

Prefer hand-written stubs that return fixed values over mocking frameworks. Use MagicMock only to assert call counts or arguments, or to simulate complex state changes. Avoid mocking the objects you test.

Code coverageCopy Anchor Link

Aim for high coverage without sacrificing error handling or changing the architecture solely for tests. Some branches remain untestable without modifying production behavior. For example, the final condition in a defensive if-else block may trigger only for unexpected values. Leave these checks in place so future changes can exercise them if needed.

Design-time exceptions can also be impractical to test, so 100% coverage is not the target.

Excluded code coverageCopy Anchor Link

We use pragma: no cover comments to exclude code from coverage when tests would be redundant. Typical examples include:

• Asserting an abstract method raises NotImplementedError when called.
• Asserting the final condition check of an if-else block when impossible to test (as above).

Such tests are expensive to maintain because they must track refactors while providing little value. Keep concrete implementations of abstract methods fully covered. Remove pragma: no cover when it no longer applies and restrict its use to the cases above.

Debugging Rust testsCopy Anchor Link

Use the default test configuration to debug Rust tests.

To run the full suite with debug symbols for later, run make cargo-test-debug instead of make cargo-test.

In IntelliJ IDEA, adjust the run configuration for parametrised #[rstest] cases so it reads test --package nautilus-model --lib data::bar::tests::test_get_time_bar_start::case_1 (remove -- --exact and append ::case_n where n starts at 1). This workaround matches the behavior explained in rust-analyzer issue 8964.

In VS Code you can pick the specific test case to debug directly.

Debugging Python and RustCopy Anchor Link

Build the PyO3 extension with the workspace's debug-pyo3 Cargo profile when a native debugger needs Rust symbols:

make sync
(
  cd python
  CARGO_TARGET_DIR=../target \
    uv run --no-sync maturin develop --profile debug-pyo3
)

1. Start the Python program or notebook with the Python debugger.
2. Attach LLDB or GDB to that Python process for Rust breakpoints.

The repository does not generate editor launch configurations, so configure both debugger sessions in the editor you use.

Data type testingCopy Anchor Link

Each data type flows through multiple layers of the platform. The table below shows where existing types are tested, so new types can follow the same pattern.

Test layer matrixCopy Anchor Link

Coverage per data typeCopy Anchor Link

The following table shows which layers have test coverage for each data type. Use this as a checklist when adding a new type.

OptionChainSlice is assembled by the DataEngine's OptionChainManager from per-instrument greeks and quote subscriptions. It does not have its own engine subscribe command.

Adding a new data typeCopy Anchor Link

When introducing a new data type, add tests at each layer:

1. DataEngine (crates/data/tests/integration/engine/): Add test_execute_subscribe_<type> and test_execute_unsubscribe_<type> tests. Follow the pattern in existing subscribe tests: register client, build command, call engine.execute, assert subscription list.

2. DataActor Rust (crates/common/src/actor/tests.rs):

   • Add received_<type>: Vec<Type> field to TestDataActor.
   • Implement the on_<type> handler in the DataActor trait impl.
   • Add test_subscribe_and_receive_<type> and test_unsubscribe_<type> tests.
   • Use the typed publish function (msgbus::publish_<type>), not publish_any, for types that use TypedHandler routing.

3. PyO3 actor dispatch (crates/common/src/python/actor.rs):

   • Add dispatch_on_<type> method that calls py_self.call_method1("on_<type>", ...).
   • Add on_<type> in the DataActor trait impl that calls the dispatch method.
   • Add #[pyo3(name = "on_<type>")] method in the #[pymethods] block.
   • Add on_<type> to RustTestDataActor wrapper and the inline Python test class.
   • Add handler test and dispatch test.

4. Python Actor (python/tests/unit/common/test_actor.py):

   • Add test_subscribe_<type> and test_unsubscribe_<type> tests.
   • Assert actor.subscribed_<type>() returns expected entries after subscribe and is empty after unsubscribe.

5. Documentation: Add entries to actors.md callback table, strategies.md handler signatures, adapters.md subscribe method stubs, and spec_data_testing.md test cards.