nyrobrain 0.1.0__py3-none-any.whl

nyrobrain/__init__.py

@@ -0,0 +1,7 @@
+"""Nyrobrain Python libraries.
+
+This package is a namespace and deliberately exports nothing. Siblings are
+expected alongside `nyrobrain.execution` — adrs migrates in later under the same
+brand — so importing `nyrobrain` must never pull an engine, a transport or a
+research stack into the process. Import the subpackage you want.
+"""

nyrobrain/execution/CONTEXT.md

@@ -0,0 +1,109 @@
+# nyrobrain.execution
+
+The shared language of the execution library. A glossary, not a spec: it fixes what each word means so that the library, the portfolio that feeds it, and the dashboards that watch it all use it the same way.
+
+## Language
+
+### The instruction
+
+**Signal**:
+One portfolio's complete target state at one instant: a target weight for every asset it holds. The whole truth, never a change set — so applying the same signal twice leaves the portfolio where applying it once did.
+_Avoid_: Order, instruction, update, delta, rebalance
+
+**Target weight**:
+The share of a portfolio's notional budget an asset should occupy, signed for direction. Positive is long, negative is short. It says what to hold, never what to trade.
+_Avoid_: Allocation, size, position size, exposure
+
+**Gross exposure**:
+The sum of a signal's target weights ignoring their signs — the total notional a portfolio wants deployed. A signal is invalid above 1. Distinct from net exposure, which cancels longs against shorts and therefore bounds direction rather than leverage.
+_Avoid_: Total exposure, leverage, sum of weights
+
+**Flat**:
+Holding nothing in an asset. A target weight of zero and an asset's absence from a signal both mean flat; a correct publisher states the zero, and an absence is a defect worth alerting on.
+_Avoid_: Closed, exited, zeroed, neutral
+
+**Target**:
+The position an asset should hold, in base units, implied by its target weight and the portfolio's budget, leverage and the current price. What we want; never what we do about it.
+_Avoid_: Desired, goal, wanted position, ideal
+
+**Actual**:
+The position held at the venue. Read, never remembered — a figure the library keeps for itself between ticks is a figure that can go stale, and stale actuals are what made an OMS buy without end.
+_Avoid_: Current, held, exchange position, real position
+
+**Working**:
+Every order of ours that could still fill, counted toward the position we are already committed to. Includes orders sent but not yet acknowledged: a quantity in flight has been committed even though nothing has filled and the venue has not replied.
+_Avoid_: Open orders, pending, resting, outstanding, in-flight
+
+**Delta**:
+What remains to be traded for an asset: its target less its actual less its working. Derived afresh each tick, never carried forward, so an under-traded tick corrects itself and a repeated tick does nothing.
+_Avoid_: Diff, gap, remainder, adjustment, order size
+
+**Placement**:
+Deciding how to work a delta into the book — how many orders, at what prices, amended or cancelled when. Owned by the user, who may substitute their own. It is told the quantity to trade and never the position, so that position arithmetic cannot re-enter through it.
+_Avoid_: Execution, order strategy, routing, slicing
+
+**Driver**:
+The script that assembles an engine or a node, attaches the executor, runs it for a while and writes the report. Owned by the user, like placement — the library ships examples of one, never one of its own. What distinguishes it from placement is scope rather than ownership: placement decides how a single delta reaches the book, a driver decides which venue, which mode, which symbols and for how long.
+_Avoid_: Runner, entrypoint, harness, main
+
+**Committed**:
+The quantity we already stand behind for an asset — what our live orders would add to the position if every one of them filled. Placement is judged on how much it changes this, never on the size of the orders it returns: repricing an order changes its price and commits nothing, which is what makes it legal when there is nothing left to trade.
+_Avoid_: Exposure, pending, allocated, reserved
+
+**Over-delivery**:
+Placement returning orders that would trade further than the delta, or in the opposite direction. Always a defect: under-trading is bounded by doing nothing and corrects next tick, while over-trading has no ceiling and compounds.
+_Avoid_: Overshoot, overfill, excess
+
+**Base asset**:
+What a signal names — the underlying an alpha has a view on, such as `BTC`. Independent of any venue.
+_Avoid_: Ticker, symbol, instrument, coin
+
+**Symbol**:
+What a venue trades, such as `BTCUSDT`. One base asset maps to a different symbol on each venue; the mapping is configuration, never part of a signal.
+_Avoid_: Pair, instrument, market, asset
+
+**Sandbox**:
+Paper trading against live market data with simulated fills. Needs no exchange credentials, and is therefore not a venue — no order it produces exists anywhere, and its results are a simulation whose realism is a setting rather than a fact.
+_Avoid_: Paper, demo, testnet, simulation, dry run
+
+### Provenance and ordering
+
+**Publisher**:
+The process that computes a portfolio's target state and emits signals. It owns the portfolio's alphas and their aggregation; the library is only ever its consumer.
+_Avoid_: Producer, sender, portfolio process, upstream
+
+**Epoch**:
+A publisher process's identity — fixed for its lifetime and different after every restart. It exists so a restarted publisher, whose sequence has returned to zero, still outranks the stream it replaced.
+_Avoid_: Run id, generation, boot id, version
+
+**Sequence**:
+A signal's position within one epoch, increasing with each signal that publisher emits. Meaningless across epochs on its own.
+_Avoid_: Index, counter, offset, id
+
+**Supersede**:
+What a signal does to the last one applied when it is genuinely newer, judged on epoch and sequence together. A signal that does not supersede is inert rather than erroneous: redelivery is expected traffic.
+_Avoid_: Override, replace, win, update
+
+### Time and health
+
+**Expected gap**:
+How long a publisher says it will be before it speaks again, taken from its own aggregation cadence. Not the interval of the alphas it aggregates — those are usually far longer, and a deadline built on them would never fire.
+_Avoid_: Interval, period, frequency, cadence, alpha interval
+
+**Missed deadline**:
+The expected gap elapsing, plus grace, without a signal arriving. It describes the health of the feed, never the validity of a signal: one that arrives afterwards is still applied, and applying it is what clears the condition.
+_Avoid_: Timeout, stale signal, late signal, expired
+
+**Escalated**:
+The state a portfolio enters after several consecutive missed deadlines — its publisher is presumed gone rather than slow. Escalation stops new orders and cancels resting ones; it never exits positions, because a crash-looping process would then exit and rebuy in a loop, paying the spread each cycle.
+_Avoid_: Halted, stopped, paused, killed, panicked
+
+**Actionable**:
+Young enough to trade on. A signal can be the newest ever seen and still too old to act on, in which case it is not traded — a signal that stale means the publisher was broken, and guessing is worse than stopping.
+_Avoid_: Fresh, valid, current, live
+
+### Rejection
+
+**Invalid**:
+Untrustworthy, and therefore not applied at all. Rejection is always whole-signal: applying the sound part of a target state would leave a position the strategy never asked for and cannot reason about.
+_Avoid_: Malformed, bad, partial, unparseable

nyrobrain/execution/__init__.py

@@ -0,0 +1,78 @@
+"""Order execution: portfolio signal in, exchange orders out.
+
+The library's job is target-position reconciliation. A signal names a target
+weight per symbol; this package decides what orders that implies against the
+position actually held at the venue, and places them.
+
+Nothing nautilus-shaped crosses this package's public boundary. Callers work in
+plain types — symbols as `str`, quantities and weights as `Decimal` — so the
+engine underneath stays an implementation detail and hooks stay testable without
+constructing framework objects.
+
+Public so far: the input type, and the sizing and guard arithmetic the executor
+is built on. The executor itself — the nautilus Strategy that drives these on a
+timer — arrives with the hook-surface and testing-surface tickets.
+"""
+
+from nyrobrain.execution.intake import Fetch, SignalSource
+from nyrobrain.execution.ladder import LadderConfig, TouchLadder
+from nyrobrain.execution.metrics import AegisMetrics, Fanout, Metric, MetricSink
+from nyrobrain.execution.placement import (
+    Action,
+    Amend,
+    Cancel,
+    Context,
+    Place,
+    Placement,
+    Resting,
+    UnknownOrder,
+    committed_change,
+    validate,
+)
+from nyrobrain.execution.signal import (
+    MAX_GROSS_EXPOSURE,
+    SCHEMA_VERSION,
+    InvalidSignal,
+    Signal,
+    parse_signal,
+)
+from nyrobrain.execution.target import (
+    OverDelivery,
+    check_intent,
+    round_price,
+    round_quantity,
+    target_quantity,
+    trade_delta,
+)
+
+__all__ = [
+    "MAX_GROSS_EXPOSURE",
+    "SCHEMA_VERSION",
+    "Action",
+    "AegisMetrics",
+    "Amend",
+    "Cancel",
+    "Context",
+    "Fanout",
+    "Fetch",
+    "InvalidSignal",
+    "LadderConfig",
+    "Metric",
+    "MetricSink",
+    "OverDelivery",
+    "Place",
+    "Placement",
+    "Resting",
+    "Signal",
+    "SignalSource",
+    "TouchLadder",
+    "UnknownOrder",
+    "check_intent",
+    "committed_change",
+    "parse_signal",
+    "round_price",
+    "round_quantity",
+    "target_quantity",
+    "trade_delta",
+    "validate",
+]

nyrobrain/execution/aegis.py

@@ -0,0 +1,92 @@
+"""Publishing metrics to Aegis over the JetStream sidecar.
+
+Requires the ``aegis`` extra, which carries the gRPC client. The payloads — and
+every decision about what they contain — live in `nyrobrain.execution.metrics`,
+which needs nothing installed at all. A second transport replaces this file and
+inherits the rest.
+
+**This is not a NATS client.** `bq-nats-client` depends only on `grpcio` and
+`protobuf`: publishers reach JetStream through a sidecar authenticated by an API
+key and hold no NATS credentials. That is convenient here, and it is also why
+portfolio targets deliberately live in a different account — the sidecar applies
+no per-subject authorization, so anything publishable through it is publishable
+by any holder of a key.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Protocol
+
+from nyrobrain.execution.metrics import Metric
+
+
+class JetStreamClient(Protocol):
+    """The part of `nats_client.NATSClient` this uses.
+
+    Stated as a protocol so a test can drive the real sink without a sidecar,
+    and so the dependency is visible at a glance rather than implied by an
+    import.
+    """
+
+    async def jetstream(self) -> None: ...
+
+    async def js_publish(
+        self,
+        subject: str,
+        payload: bytes = b"",
+        headers: dict[str, str] | None = None,
+    ) -> object: ...
+
+
+class JetStreamSink:
+    """A `MetricSink` that publishes through the sidecar.
+
+    Deliberately thin, and deliberately not defensive: a publish that fails
+    raises. The emitter counts the failure and the report shows it, so a sink
+    that swallowed exceptions would turn a dead metrics pipeline into one that
+    looks healthy — the exact failure this map exists to prevent, and the one
+    nothing downstream detects.
+    """
+
+    def __init__(self, *, open_client: Callable[[], JetStreamClient]) -> None:
+        #: A factory, not a client, and that is the whole point. `NATSClient`
+        #: binds its gRPC channel to an event loop when it is **constructed**,
+        #: and a driver builds its publisher before the node has a loop — so a
+        #: ready-made client binds to a loop that is closed by the time the
+        #: first heartbeat fires, and every publish dies with "attached to a
+        #: different loop". Deferring construction to the first publish means
+        #: the channel is built in whichever loop is actually going to use it.
+        #:
+        #: Measured the hard way: a live run reported `published 0, failures 3`
+        #: and nothing else until the round started recording the reason.
+        self._open_client = open_client
+        self._client: JetStreamClient | None = None
+
+    async def connect(self) -> None:
+        """Build the client and configure JetStream, once.
+
+        Call this only from the loop that will publish — or not at all, since
+        `publish` does it. `js_publish` refuses with "call jetstream() first"
+        until it has run, and doing it per publish would add a round trip to
+        every row.
+        """
+        if self._client is not None:
+            return
+        client = self._open_client()
+        await client.jetstream()
+        self._client = client
+
+    async def publish(self, metric: Metric) -> None:
+        # Lazily, so the channel is built in whichever loop is publishing.
+        await self.connect()
+        assert self._client is not None
+        # `msg_id` becomes `Nats-Msg-Id` here and nowhere else. The client does
+        # `headers.setdefault("Nats-Msg-Id", uuid4())`, so omitting it would
+        # publish successfully while silently losing every cross-publish
+        # collapse — retries would still be idempotent, and nothing else would.
+        await self._client.js_publish(
+            metric.subject,
+            metric.payload,
+            headers={"Nats-Msg-Id": metric.msg_id},
+        )

nyrobrain/execution/backtest.py

@@ -0,0 +1,184 @@
+"""Assembling a backtest over a recorded signal timeseries.
+
+Requires the ``execution`` extra.
+
+`backtest_engine` hands the engine back **unrun**, so the caller decides when
+and whether to run it. That is not ceremony: this function has more callers
+inside the test suite than out of it — the property-based fuzz drives it at 500
+examples with deliberately-broken placements — and those callers need the
+assembly without the run.
+
+For a driver that uses it, see `nyrobrain.execution.examples.oms`.
+
+Signals and quotes go into `BacktestEngine` as **one timestamp-ordered stream**,
+so they interleave exactly as they would live. That is the whole reason signals
+are a `Data` subclass rather than something handed to the strategy directly —
+hand-sequencing them against the quote timeline is where an off-by-one becomes a
+backtest that looks fine and means nothing.
+
+`BacktestEngine` gives the strategy a `TestClock`, so timers fire as the data
+advances and a week of replay does not take a week.
+
+**What a run here can and cannot tell you** is set out in `candles`: with quotes
+synthesized from 1m bars, this proves the wiring, the convergence and the
+invariants. It is not evidence about fill quality, and the report withholds
+those numbers rather than inviting the mistake.
+"""
+
+from __future__ import annotations
+
+import dataclasses
+from collections.abc import Callable, Sequence
+from pathlib import Path
+
+from nautilus_trader.backtest import BacktestEngine
+from nautilus_trader.config import BacktestEngineConfig, LoggerConfig
+from nautilus_trader.model import (
+    AccountType,
+    ClientId,
+    CryptoPerpetual,
+    Currency,
+    Money,
+    OmsType,
+    TraderId,
+    Venue,
+)
+
+from nyrobrain.execution.candles import QuoteSource
+from nyrobrain.execution.data import as_custom_data
+from nyrobrain.execution.executor import TargetPositionExecutor
+from nyrobrain.execution.placement import Placement
+from nyrobrain.execution.report import (
+    RunRecorder,
+)
+from nyrobrain.execution.store import read_signals
+from nyrobrain.execution.venue import InstrumentsNotReady, VenueConfig
+
+#: The cache holds 1m bars, and the quote cadence follows from that.
+CANDLE_INTERVAL_NS = 60 * 1_000_000_000
+
+
+def stale_quote_threshold_for(quote_interval_ns: int) -> int:
+    """How old a quote may get before a candle-fed run stops believing it.
+
+    Three intervals — the same rule the signal deadline uses, so the number
+    follows the data rather than the other way round.
+
+    This exists because the live default is 30 seconds and a candle source is
+    three orders of magnitude slower. `candles._expand` stamps all four quotes
+    of a bar with the bar's own timestamp, so a 1m source emits four quotes and
+    then goes quiet for 59 seconds; against 30s that reads as a dead feed for
+    half of every minute. Measured before the override existed: **50,395 of
+    70,553 ticks skipped** across a seven-day replay, with the whole suite
+    green, because skipping a tick breaks no invariant. It only stops the run
+    meaning anything.
+
+    Named and returned rather than inlined so the rule can be tested directly,
+    without standing up an engine to observe it.
+    """
+    return 3 * quote_interval_ns
+
+
+def backtest_engine(
+    *,
+    signals: Path,
+    config: VenueConfig,
+    instruments: Sequence[CryptoPerpetual],
+    quotes: QuoteSource,
+    placement_for: Callable[[str], Placement] | None = None,
+    quote_interval_ns: int = CANDLE_INTERVAL_NS,
+) -> tuple[BacktestEngine, RunRecorder]:
+    """Assemble the engine, run it, and hand back what happened.
+
+    `instruments` is yours to supply, and there is no default. A backtest has no
+    venue to ask, so the tick size, the lot size and the fee schedule are
+    modelling inputs — they belong in your project where they can be reviewed,
+    not in a file inside this package that is silently authoritative and dated
+    to whenever someone last refreshed it. Snapshot them from a live connection
+    with `write_instruments(path, cache.instruments())`, or describe them with
+    `venue.perpetual(...)`.
+
+    `quotes` is a `QuoteSource` — a function from instrument to quotes — rather
+    than a directory, because a directory is not a format. This used to be
+    `candles: Path`, which meant "laid out the way we lay ours out": sharded
+    parquet with a date in the filename, produced by nothing outside this
+    project. A user whose candles are a CSV, a database, or nautilus's own
+    catalog had no way in. `candles.parquet_shards(dir)` is ours, named where
+    it is used; anything matching the signature is equally welcome.
+
+    The date window and the synthetic spread went with it, which is why this
+    signature is shorter than it was. Both are properties of a particular
+    source — a window matched against a filename, a spread invented because
+    candles have no book — and neither means anything to a loader reading real
+    quotes, which would have been handed them regardless.
+
+    `placement_for` is the executor's own resolver, exposed here because a
+    driver that can only run the default ladder cannot exercise the guard that
+    exists to contain a *misbehaving* placement — and that guard is the one
+    protecting users who write their own.
+    """
+    # The stale-quote threshold has to match the cadence of the quotes actually
+    # being fed, and this driver's are three orders of magnitude slower than a
+    # live ticker's. `QuoteTickDataWrangler` stamps all four quotes of a bar with
+    # the bar's own timestamp, so a 1m candle source emits four quotes and then
+    # goes quiet for 59 seconds — and the live default of 30s reads that as a
+    # dead feed for half of every minute. Measured before this existed: 50,395 of
+    # 70,553 ticks skipped across a seven-day replay, with the whole suite green,
+    # because skipping a tick breaks no invariant. It only stops the run meaning
+    # anything.
+    #
+    # Three intervals, the same rule the signal deadline uses, so the number
+    # follows the data rather than the other way round.
+    config = dataclasses.replace(
+        config,
+        freshness=dataclasses.replace(
+            config.freshness,
+            stale_quote_ns=stale_quote_threshold_for(quote_interval_ns),
+        ),
+    )
+
+    # Two lists that have to agree, so check it here rather than let it surface
+    # as a run that quietly trades nothing. A backtest has no excuse for a
+    # missing definition — there is no venue still connecting, only a driver
+    # that named a symbol and did not describe it. Live and paper keep the
+    # retry, because there "not yet" is a real answer.
+    described = {i.raw_symbol.value for i in instruments}
+    undescribed = [s for s in config.symbols if s not in described]
+    if undescribed:
+        raise InstrumentsNotReady(
+            f"no instrument described for {', '.join(undescribed)}. A backtest "
+            "has no venue to ask, so every configured symbol needs one: add it "
+            "with `venue.perpetual(...)`, or snapshot the real definitions with "
+            "`write_instruments(path, cache.instruments())` and load that. "
+            f"Described: {', '.join(sorted(described)) or '(none)'}"
+        )
+
+    engine = BacktestEngine(
+        config=BacktestEngineConfig(
+            trader_id=TraderId("NYRO-001"), logging=LoggerConfig(bypass_logging=True)
+        )
+    )
+    engine.add_venue(
+        venue=Venue(config.venue),
+        oms_type=OmsType.NETTING,
+        account_type=AccountType.MARGIN,
+        base_currency=Currency.from_str(config.currency),
+        starting_balances=[
+            Money(config.starting_balance, Currency.from_str(config.currency))  # type: ignore[arg-type]  # v2 stubs say float; the runtime takes Decimal exactly (verified) and float would lose it
+        ],
+    )
+
+    for instrument in instruments:
+        engine.add_instrument(instrument)
+        engine.add_data(list(quotes(instrument)))
+
+    engine.add_data(
+        [as_custom_data(s) for s in read_signals(signals)],
+        client_id=ClientId("NYRO"),
+    )
+
+    recorder = RunRecorder()
+    engine.add_strategy(
+        TargetPositionExecutor(config, recorder=recorder, placement_for=placement_for)
+    )
+    return engine, recorder