aether-quant 0.4.0__tar.gz → 0.5.0__tar.gz

{aether_quant-0.4.0 → aether_quant-0.5.0}/Dockerfile.retraining_worker

@@ -8,7 +8,10 @@ COPY performance/ ./performance/
 COPY regime/ ./regime/
 COPY experts/ ./experts/
 COPY topology/ ./topology/
+COPY moe/ ./moe/
+COPY inference/ ./inference/
 COPY retraining/ ./retraining/
 COPY train.py .
 COPY train_topology.py .
+COPY train_gating.py .
 CMD ["python", "-m", "retraining.worker"]

{aether_quant-0.4.0 → aether_quant-0.5.0}/PKG-INFO

@@ -1,5 +1,5 @@
 Metadata-Version: 2.4
 Name: aether-quant
-Version: 0.4.0
+Version: 0.5.0
 Summary: Aether Quant convenience CLI
 Requires-Python: >=3.10

{aether_quant-0.4.0 → aether_quant-0.5.0}/README.md

@@ -10,7 +10,7 @@
 
 <p align="center">
   <img src="https://img.shields.io/badge/python-3.10%2B-FF8C00?style=flat-square&labelColor=1A1A1A&logo=python&logoColor=white" alt="Python 3.10+">
-  <!-- AQ:TEST_BADGE_START --><img src="https://img.shields.io/badge/tests-528%2F528%20passing-brightgreen?style=flat-square&labelColor=1A1A1A" alt="528 of 528 tests passing"><!-- AQ:TEST_BADGE_END -->
+  <!-- AQ:TEST_BADGE_START --><img src="https://img.shields.io/badge/tests-583%2F583%20passing-brightgreen?style=flat-square&labelColor=1A1A1A" alt="583 of 583 tests passing"><!-- AQ:TEST_BADGE_END -->
   <img src="https://img.shields.io/pypi/v/aether-quant?style=flat-square&labelColor=1A1A1A&color=FF8C00" alt="PyPI version">
   <img src="https://img.shields.io/badge/docker-ghcr.io%2Faether--quant-2496ED?style=flat-square&labelColor=1A1A1A&logo=docker&logoColor=white" alt="Docker image on GHCR">
 </p>
@@ -391,14 +391,14 @@ and how it's wired in — this table is the index.
 
 | Metric | Value |
 |---|---|
-| Backtest window | 2014-12-01 to 2018-08-15 |
-| Sharpe Ratio | -0.765 |
-| Net Profit | -7.955% |
-| Compounding Annual Return | -2.212% |
-| Drawdown | 12.100% |
-| Total Orders | 15 |
-| Win Rate | 0% |
-| Last updated | 2026-07-05 16:59 UTC (auto-generated by `aq backtest`) |
+| Backtest window | 2018-04-01 to 2021-04-02 |
+| Sharpe Ratio | -0.71 |
+| Net Profit | -8.069% |
+| Compounding Annual Return | -2.761% |
+| Drawdown | 12.500% |
+| Total Orders | 27 |
+| Win Rate | 17% |
+| Last updated | 2026-07-06 17:20 UTC (auto-generated by `aq backtest`) |
 <!-- AQ:BACKTEST_END -->
 
 <details>
@@ -407,33 +407,33 @@ and how it's wired in — this table is the index.
 <!-- AQ:BACKTEST_FULL_STATS_START -->
 | Metric | Value |
 |---|---|
-| Total Orders | 15 |
-| Average Win | 0% |
-| Average Loss | -1.64% |
-| Compounding Annual Return | -2.212% |
-| Drawdown | 12.100% |
-| Expectancy | -1 |
-| Start Equity | 100000 |
-| End Equity | 92045.02 |
-| Net Profit | -7.955% |
-| Sharpe Ratio | -0.765 |
-| Sortino Ratio | -0.531 |
-| Probabilistic Sharpe Ratio | 0.052% |
-| Loss Rate | 100% |
-| Win Rate | 0% |
-| Profit-Loss Ratio | 0 |
-| Alpha | -0.043 |
-| Beta | 0.224 |
-| Annual Standard Deviation | 0.036 |
-| Annual Variance | 0.001 |
-| Information Ratio | -1.103 |
-| Tracking Error | 0.087 |
-| Treynor Ratio | -0.123 |
-| Total Fees | $14.51 |
-| Estimated Strategy Capacity | $310000000.00 |
+| Total Orders | 27 |
+| Average Win | 0.18% |
+| Average Loss | -1.70% |
+| Compounding Annual Return | -2.761% |
+| Drawdown | 12.500% |
+| Expectancy | -0.816 |
+| Start Equity | 100000.00 |
+| End Equity | 91930.60 |
+| Net Profit | -8.069% |
+| Sharpe Ratio | -0.71 |
+| Sortino Ratio | -0.471 |
+| Probabilistic Sharpe Ratio | 0.171% |
+| Loss Rate | 83% |
+| Win Rate | 17% |
+| Profit-Loss Ratio | 0.11 |
+| Alpha | -0.058 |
+| Beta | 0.195 |
+| Annual Standard Deviation | 0.051 |
+| Annual Variance | 0.003 |
+| Information Ratio | -0.972 |
+| Tracking Error | 0.155 |
+| Treynor Ratio | -0.184 |
+| Total Fees | $12.00 |
+| Estimated Strategy Capacity | $210000000.00 |
 | Lowest Capacity Asset | IBM R735QTJ8XC9X |
-| Portfolio Turnover | 0.06% |
-| Drawdown Recovery | 57 |
+| Portfolio Turnover | 0.07% |
+| Drawdown Recovery | 503 |
 <!-- AQ:BACKTEST_FULL_STATS_END -->
 
 </details>
@@ -463,6 +463,17 @@ experience-worker performance-trigger-worker retraining-worker`) with the
 backtest run inside that network, plus an actual trigger condition being
 met during the run.
 
+**If `phase_v2.backtest.bypass_safety_gates` is `true`:** this backtest
+also does not represent live/paper-deployable behavior. That flag (default
+`false`) disables the sticky total-drawdown lock and the regime detector's
+drawdown-driven `risk_off` override — both real, designed safety behavior
+in live/paper mode — purely to generate enough trade volume for
+statistically meaningful backtest metrics (see `development/Problems.md`
+#18). A backtest run with this flag set shows the underlying model's
+signal quality across more market conditions, not what this system would
+have actually done if deployed — in live/paper mode, both gates would have
+genuinely halted trading exactly as designed.
+
 ## Test Suite
 
 507 tests, one file per source module, run via:
@@ -495,7 +506,7 @@ pip install -e .
 Either way, `aq --help` gives the full command list:
 
 ```text
-aq train [--dataset-only|--init-only|--experts-only]
+aq train [--dataset-only|--init-only|--experts-only|--gating-only]
 aq test
 aq backtest
 aq report <backtest-folder> <result-id>
@@ -512,7 +523,7 @@ aq status
 Every command except `aq trade-lock` and `aq fetch` is a thin `subprocess`
 wrapper around a command already documented elsewhere in this README:
 
-- **`aq train`** — runs `train.py`: builds the dataset and trains the baseline + expert models.
+- **`aq train`** — runs `train.py`: builds the dataset and trains the baseline + expert models. `--gating-only` trains just the learned gating blend (`train_gating.py`) and installs it straight into active `ml/`, mirroring what `--experts-only` already does for the expert models — see `moe/README.md`.
 - **`aq test`** — runs the pytest suite and refreshes this README's test badge.
 - **`aq backtest`** — runs `lean backtest .` and refreshes this README's [Backtest Results](#backtest-results) section.
 - **`aq report <backtest-folder> <result-id>`** — generates Lean's own HTML backtest report (trade blotter, standard Lean charts) at `backtests/<backtest-folder>/report.html`.

{aether_quant-0.4.0 → aether_quant-0.5.0}/aether_quant.egg-info/PKG-INFO

@@ -1,5 +1,5 @@
 Metadata-Version: 2.4
 Name: aether-quant
-Version: 0.4.0
+Version: 0.5.0
 Summary: Aether Quant convenience CLI
 Requires-Python: >=3.10

{aether_quant-0.4.0 → aether_quant-0.5.0}/aether_quant.egg-info/SOURCES.txt

@@ -14,6 +14,7 @@ main.py
 pyproject.toml
 risk_controls.py
 train.py
+train_gating.py
 train_topology.py
 .github/workflows/ci.yml
 .github/workflows/release.yml
@@ -74,6 +75,12 @@ liquidity/README.md
 liquidity/__init__.py
 liquidity/market_liquidity.py
 ml/README.md
+ml/gating_feature_schema.json
+ml/gating_model.json
+ml/gating_training_metrics.json
+ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_feature_schema.json
+ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_model.json
+ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_training_metrics.json
 moe/README.md
 moe/__init__.py
 moe/gating.py
@@ -161,6 +168,7 @@ tests/test_status_export.py
 tests/test_telegram_alerts.py
 tests/test_telegram_client.py
 tests/test_telegram_worker.py
+tests/test_train_gating.py
 tests/test_train_pipeline.py
 tests/test_train_topology.py
 tests/test_trigger_worker.py

{aether_quant-0.4.0 → aether_quant-0.5.0}/aether_quant.egg-info/scm_file_list.json

@@ -1,6 +1,7 @@
 {
   "files": [
     "README.md",
+    "train_gating.py",
     "Dockerfile.workers",
     "aq_cli.py",
     "Dockerfile",
@@ -211,6 +212,7 @@
     "tests/test_train_pipeline.py",
     "tests/test_experience_queue.py",
     "tests/test_vault_commands.py",
+    "tests/test_train_gating.py",
     "tests/test_fetch.py",
     "data/alternative/sec/aapl/.keep",
     "data/alternative/trading-economics/.keep",
@@ -227,6 +229,12 @@
     "requirements/requirements-workers.txt",
     "requirements/requirements-retraining-worker.txt",
     "ml/README.md",
+    "ml/gating_model.json",
+    "ml/gating_feature_schema.json",
+    "ml/gating_training_metrics.json",
+    "ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_model.json",
+    "ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_feature_schema.json",
+    "ml/versions/gating-only-26218402-34d9-4a0a-add8-ee5d50dd0459/gating_training_metrics.json",
     ".github/workflows/release.yml",
     ".github/workflows/ci.yml"
   ]

aether_quant-0.5.0/aether_quant.egg-info/scm_version.json

@@ -0,0 +1,8 @@
+{
+  "tag": "0.5.0",
+  "distance": 0,
+  "node": "gb230bc437a96905925c7aa521e7006da67cfb98c",
+  "dirty": false,
+  "branch": "HEAD",
+  "node_date": "2026-07-07"
+}

{aether_quant-0.4.0 → aether_quant-0.5.0}/analyzer/README.md

@@ -38,6 +38,17 @@ risk engine, and before action categorization / Lean order placement.
   explicitly ruled out of scope for that phase as a materially riskier
   change than what its safety rule ("confidence/uncertainty only, never a
   randomized decision") allows.
+  - **Follow-up:** the learned topology overlay's confidence/disagreement
+    scores now *do* reach a real trade — but through position sizing
+    (`risk/position_sizing.py::topology_sizing_multiplier()`), not this
+    module. That integration point was chosen specifically because sizing
+    is already a continuous, shrink-only multiplier applied after this
+    analyzer has categorized the action, so it changes only *how large* an
+    already-approved trade is, never *whether* one happens — preserving
+    exactly the "confidence/uncertainty only, never a randomized decision"
+    rule above. This module's own `trade`/`simulate`/`observe`/
+    `reduce_risk` categorization remains fully deterministic and
+    unaffected. See `risk/README.md` for details.
 - `main.py` only calls `_apply_signal` when `action == "trade"`. All five
   actions are written into the per-asset `signal_payload` for
   dashboard/Grafana visibility regardless of whether a real order is

{aether_quant-0.4.0 → aether_quant-0.5.0}/aq_cli.py

@@ -32,6 +32,7 @@ import subprocess
 import sys
 import time
 import urllib.request
+import uuid
 from datetime import date
 from importlib.metadata import version as installed_version
 from pathlib import Path
@@ -176,6 +177,8 @@ def check_for_update() -> None:
 
 
 def cmd_train(args: argparse.Namespace) -> int:
+    if args.gating_only:
+        return _train_gating_only()
     cmd = [sys.executable, "train.py"]
     if args.dataset_only:
         cmd.append("--dataset-only")
@@ -186,6 +189,41 @@ def cmd_train(args: argparse.Namespace) -> int:
     return _run(cmd)
 
 
+def _train_gating_only() -> int:
+    """`aq train --gating-only`: trains the learned gating blend
+    (train_gating.py) and installs it straight into active ml/, mirroring
+    what `train.py --experts-only` already does for the expert models.
+
+    train_gating.py always writes to ml/versions/<version_id>/ (same
+    versioned-candidate convention every other trainer in this project
+    uses), so this generates a throwaway version-id, runs the trainer,
+    then copies the 3 resulting artifacts into active ml/ - the same
+    manual promotion-simulation step already documented for verifying this
+    trainer, skipping the full retraining/validate/backtest/commit/promote
+    pipeline since this is an ad-hoc, user-triggered run, not a scheduled
+    candidate."""
+    version_id = f"gating-only-{uuid.uuid4()}"
+    returncode = _run([sys.executable, "train_gating.py", "--version-id", version_id])
+    if returncode != 0:
+        return returncode
+
+    version_dir = ROOT_DIR / "ml" / "versions" / version_id
+    artifact_names = ("gating_model.json", "gating_feature_schema.json", "gating_training_metrics.json")
+    if any(not (version_dir / name).exists() for name in artifact_names):
+        print(
+            "aq train --gating-only: train_gating.py exited 0 but skipped writing artifacts "
+            "(likely insufficient validation/backtest rows) - active ml/ left unchanged.",
+            file=sys.stderr,
+        )
+        return 0
+
+    ml_dir = ROOT_DIR / "ml"
+    for name in artifact_names:
+        shutil.copy2(version_dir / name, ml_dir / name)
+    print(f"aq train --gating-only: copied {', '.join(artifact_names)} into active ml/.")
+    return 0
+
+
 def _update_readme_test_badge(passed: int, failed: int) -> None:
     """Atomically rewrites the shields.io test-count badge between the
     AQ:TEST_BADGE markers in README.md, so it never drifts from the real
@@ -392,6 +430,9 @@ def build_parser() -> argparse.ArgumentParser:
     train_group.add_argument("--dataset-only", action="store_true", help="Build dataset/scaler/manifest only")
     train_group.add_argument("--init-only", action="store_true", help="Refresh the data inventory only")
     train_group.add_argument("--experts-only", action="store_true", help="Train the 4 expert models only")
+    train_group.add_argument(
+        "--gating-only", action="store_true", help="Train the learned gating blend only (wraps python train_gating.py)"
+    )
     train_parser.set_defaults(func=cmd_train)
 
     test_parser = subparsers.add_parser("test", help="Run the test suite (wraps pytest tests/)")

{aether_quant-0.4.0 → aether_quant-0.5.0}/development/Changelog.md

@@ -854,3 +854,189 @@ Finance, formats into Lean's zip/CSV convention, writes it to the correct
   rejected by argparse (V3 not implemented yet). Test artifacts removed
   from `config.json`/`data/` after verification — not part of the real
   20-asset universe.
+
+## Trade-frequency tuning — statistical/diagnostic backtest mode
+
+A real 3-year, 20-asset backtest this session produced only 12 filled
+trades: 5 entries, 1 opportunistic exit, then a 5-symbol mass liquidation
+on 2020-03-23 that froze the algorithm for the remaining 374 days of the
+window. Two parallel research passes tracing the full per-bar decision
+pipeline found the suppression compounds across several independent gates,
+plus two structural traps where a gate that fires once effectively never
+clears for the rest of a run — the portfolio sits flat in cash and its own
+drawdown-from-peak calculation can never recover without trading.
+
+- **New opt-in flag, `phase_v2.backtest.bypass_safety_gates`** (default
+  `false`) — deliberately a standalone key, not a repurposing of
+  `aq trade-lock`'s existing `--on`/`--off`/`--auto` override (which keeps
+  its separately-documented meaning completely unchanged in every runtime
+  mode). New pure helper
+  `risk_controls.py::is_backtest_safety_bypass_active(runtime_mode, bypass_flag)`
+  returns `True` only when `runtime_mode == "backtest"` **and** the flag is
+  explicitly `true` — any non-backtest mode always returns `False`
+  regardless of the flag, so live/paper safety behavior is completely
+  untouched by this change.
+- **Bypasses the sticky total-drawdown lock** (`main.py::_refresh_risk_state()`,
+  the `trade_lock_reason != "total_drawdown_limit_breached"` exclusion) and
+  **the regime detector's `risk_off` drawdown branch**
+  (`main.py::_build_regime_payload()`, passes `float("inf")` instead of
+  `regime_risk_off_drawdown_threshold` when the bypass is active) — the
+  regime override was found to be an equally significant, earlier-firing
+  (8% vs. the lock's 12%) version of the same structural trap, independent
+  of the lock. Only these two specific mechanisms are affected; the
+  bearish-trend+high-vol and composite risk-score branches of regime
+  classification, and every liquidity/topology/cooldown/exposure gate,
+  stay fully active either way.
+- **Explicitly scoped to statistical/model-quality evaluation, not a
+  live-representative equity curve** — in live/paper mode both gates are
+  real, designed behavior that would have actually frozen trading on
+  2020-03-23. This is a deliberate, accepted tradeoff for generating enough
+  trade volume to get meaningful backtest metrics and exercise
+  performance-trigger thresholds (`trade_count_interval=100`,
+  `validation_gate.min_trade_count=30`) that never fire at ~12 trades —
+  not a claim about deployable behavior.
+- **Config-only threshold loosening** (confirmed test-safe: no existing
+  test loads the real `config.json` for any of these keys):
+  `phase6.risk.min_confidence_to_trade` 0.12→0.05,
+  `phase5.backtest.buy_threshold_offset`/`sell_threshold_offset` 0.08→0.04
+  each, `phase6.risk.trade_cooldown_bars` 3→1,
+  `phase_v2.liquidity.thin_participation_threshold` 0.002→0.01,
+  `phase_v2.liquidity.blocked_participation_threshold` 0.05→0.10,
+  `phase9.portfolio.max_active_positions` 5→10,
+  `phase9.portfolio.max_crypto_exposure` 0.25→0.35.
+- **`phase9.asset_quality.min_training_rows` 100→50**: unlocked ETHUSD,
+  XRPUSD, and ADAUSD as `training_eligible`/`trading_eligible` (they failed
+  only this one threshold — 52-54 actual training rows, since these coins
+  weren't listed on Yahoo until deep into the 2014-2017 training window;
+  they already comfortably cleared `min_total_feature_rows` and
+  `min_backtest_rows`) — confirmed via `train.py --dataset-only`: 19 of 20
+  assets now `training_eligible`, only `AAA` remains observation-only.
+  `AAA` was deliberately excluded from this fix — its usable data starts
+  2020-09-09, entirely after both the training and validation windows end,
+  so no threshold value can fix it; it would need real pre-2018 backfilled
+  history that doesn't exist for whatever instrument `AAA` actually is.
+- **Deliberately out of scope:** real short-selling (confirmed
+  `phase5.backtest.strategy_mode` is read in exactly one place in the whole
+  repo, `train.py`'s report-metadata code, and is never branched on —
+  `main.py::_apply_signal`'s `sell` branch only ever calls `self.Liquidate()`,
+  there is no code path anywhere that opens a short position; enabling real
+  shorting would be a materially bigger, riskier change than everything
+  above and isn't needed to raise trade count via long/flat cycling alone).
+  Also left alone: the topology "elevated" volatility threshold
+  (`topology/market_topology.py`'s `ELEVATED_VOLATILITY_THRESHOLD = 0.45`,
+  hardcoded, not in `config.json`) — a smaller contributor than the two
+  structural traps, left as a secondary lever for later if needed.
+- New tests: `tests/test_risk_controls.py` (4 new cases for
+  `is_backtest_safety_bypass_active`).
+- Stopping point: the real Lean backtest with `bypass_safety_gates: true`
+  to confirm the actual resulting trade count against the ~200 target was
+  left for the user to run manually, per this session's established
+  preference — the exact starting-point threshold values above may need
+  one iteration based on that real count.
+
+## Real learned gating weights + learned topology wired into position sizing
+
+Closes two gaps this project's own V3 completeness assessment had flagged
+as deferred, not rejected: `moe/gating.py`'s gating network was still
+hand-tuned arithmetic dressed as a decision layer, and the learned
+probabilistic topology overlay (V2-17.5) computed real per-symbol
+confidence/uncertainty every bar but never reached an actual trade
+decision — only the dashboard and the offline retrain-trigger pipeline.
+
+**Learned topology → position sizing** (`risk/position_sizing.py`): new
+pure function `topology_sizing_multiplier(topology_source,
+topology_confidence, topology_disagreement, min_topology_multiplier=0.5,
+max_topology_multiplier=1.0)` — a strict no-op (`1.0`) unless
+`topology_source == "learned"`, otherwise a bounded, continuous,
+**shrink-only** multiplier (`min + (max-min) * confidence * (1-disagreement)`,
+never above `1.0`). Composes into the existing `volatility_multiplier ×
+confidence_multiplier` chain in `build_dynamic_position_sizing()` as a
+third factor — this changes only *how large* an already-approved trade is,
+never *whether* it happens, so it never touches the analyzer's
+`trade`/`simulate`/`observe`/`reduce_risk` decision itself (see
+`analyzer/README.md`'s documented reason that path stays deterministic).
+Wired through `main.py::_build_dynamic_sizing_payload()`. New config keys:
+`phase_v2.dynamic_risk.topology_sizing_enabled` (default `true`),
+`min_topology_multiplier` (`0.5`), `max_topology_multiplier` (`1.0`).
+
+**Real learned gating weights** (`moe/gating.py`): additive, optional,
+always-falls-back — the existing hand-written
+quality-multiplier×performance-score×regime-alignment blend is still
+computed first and unchanged; a trained model, if present, only
+*overrides* the final probability. New `GATING_MODEL_FEATURE_KEYS`
+(26-dim: each expert's probability/quality/performance/regime-alignment ×4,
+baseline probability, one-hot trend/volatility/risk regime — already
+bounded `[0,1]`/one-hot, so no scaler needed) and
+`build_gating_model_features()`. `build_gating_decision()` gains optional
+`gating_model`/`gating_feature_schema` params; on success `decision_source`
+becomes `"learned_gating"` (new value, confirmed inert everywhere else in
+the repo); any failure/missing model silently falls back to the hardcoded
+blend, mirroring `topology/learned_topology.py`'s per-node fallback
+isolation. Wired via new `main.py::_load_gating_model()`, gated by
+`phase_v2.gating_network.learned_model_enabled` (default `true`).
+
+**New offline trainer, `train_gating.py`** (sibling of `train_topology.py`,
+same never-runs-in-Lean, exits-0-not-1-on-insufficient-data contract).
+Trains the blend on the dataset's `validation` split (avoids
+stacking-circularity: `train` already fit baseline+experts, `validation`
+is the right held-out-from-fitting size to become this model's *own*
+training data) replayed through the exported baseline+expert models via
+the existing `inference/exported_model.py::run_exported_model()`
+interpreter — zero new inference code. Evaluates once, at the end, on the
+`backtest` split (never touched by any fitting anywhere in the pipeline).
+Model is a small `AetherNet(26 → [16] → 1)`, deliberately restricted to
+`relu`/`layernorm` (never `gelu`/`silu`/`batchnorm1d`, which
+`run_exported_model()` cannot interpret). Regime reconstruction uses
+`portfolio_drawdown=0.0`/`average_correlation=0.0` (runtime-only state not
+recoverable offline — an honest, documented simplification; the two most-
+used regime keys, `trend_regime`/`volatility_regime`, are unaffected).
+Smoke-tested end-to-end against the real dataset (1,304 validation rows,
+16,184 backtest rows) — writes a valid, runtime-interpretable model in
+~90s. New config block `phase_v2.retraining.gating_training`.
+
+**Retraining pipeline wiring**: `retraining/artifacts.py` gained
+`OPTIONAL_GATING_FILES` (3 files, same best-effort contract as
+`OPTIONAL_TOPOLOGY_FILES` — never required, never blocks candidate
+promotion) plus `check_gating_artifacts()`; `retraining/orchestrator.py`
+gained `train_gating()` (mirrors `train_topology()` line-for-line) plus a
+`train_gating` CLI subparser; `retraining/worker.py` calls it right after
+`train_topology()` in `run_once()`. `config.json`'s
+`promotion.active_artifact_files` extended with the 3 new files.
+
+**New `aq train --gating-only` flag**: since `train_gating.py` always
+writes to `ml/versions/<id>/` (the versioned-candidate convention every
+trainer here uses), this generates a throwaway version-id, runs the
+trainer, then copies the 3 resulting artifacts straight into active `ml/`
+— the same manual promotion-simulation step already used to verify this
+trainer, mirroring how `train.py --experts-only` already writes directly
+to active paths without a promotion gate.
+
+**Neural-network webui tab now shows the gating network too**
+(`monitoring/neural_network_state.py`): it was previously in the page's
+hardcoded `excluded` list with a "deterministic, no weight matrix" reason
+that this change makes stale. Now reads `ml/gating_model.json` exactly
+like the baseline/experts (same optional, degrades-to-`not_trained`
+contract) and reports a `"learned"` quality-status badge once a model
+exists. `webui/src/components/neuralnet/NeuralNetworkScene3D.tsx`'s
+previously-hardcoded 5-network render order gained `'gating'` (violet),
+since the 3D scene only ever drew networks named in that list regardless
+of what the backend returned.
+
+- New tests: `tests/test_position_sizing.py` (+7), `tests/test_gating_network.py`
+  (+5), `tests/test_train_gating.py` (8, pure-function style mirroring
+  `tests/test_train_topology.py`), `tests/test_retraining_artifacts.py`/
+  `test_retraining_orchestrator.py`/`test_retraining_worker.py` (extended
+  for the new gating stage), `tests/test_aq_cli.py` (+3 for
+  `--gating-only`), `tests/test_neural_network_state.py` (+2 new, 2
+  rewritten to match gating no longer being excluded). Full suite: 581
+  passed.
+- Docs: `development/Problems.md` #14 (Redis push in backtest mode) marked
+  resolved — the project owner confirmed no downstream process reads
+  backtest-mode experience events from Postgres, so the open question this
+  entry tracked is answered; `experience/redis_queue.py::push()` itself is
+  deliberately left unchanged, since performance was never the blocker.
+- Stopping point: `python train_gating.py` / `aq train --gating-only` was
+  smoke-tested against the real dataset but its resulting model was **not**
+  installed into active `ml/` or promoted through the retraining pipeline —
+  per this session's established preference, the user runs the real
+  training/backtest themselves.

{aether_quant-0.4.0 → aether_quant-0.5.0}/development/Problems.md

@@ -440,7 +440,7 @@ the cache silently breaking hot-reload.
 ---
 
 ### 14. Redis push in backtest mode — deliberately left unoptimized
-**Severity:** n/a · **Status:** 🟠 `open` (deferred by design)
+**Severity:** n/a · **Status:** 🟢 `resolved` (confirmed no-op, no code change needed)
 
 `experience/redis_queue.py::push()` does a synchronous, blocking `XADD` per
 symbol per bar, plus one more at every session rollover (which, per #13, is
@@ -450,23 +450,28 @@ have `self.runtime_mode`/`self._experience_mode` available — and would save
 real per-bar network I/O during backtests, which never need live
 experience-stream delivery.
 
-**Why this stays open on purpose:** `development/v2_architecture.md`'s
+**Why this stayed open initially:** `development/v2_architecture.md`'s
 Redis Experience Queue section documents `"backtest"` as one of four
 normal, expected mode values flowing into Redis, and
 `tests/test_experience_queue.py`'s default fixture treats `mode="backtest"`
-as the canonical case, not an exclusion. There is at least one plausible
+as the canonical case, not an exclusion. There was at least one plausible
 downstream dependency on backtest-mode Redis events reaching Postgres via
 `experience-worker` — debugging via the observation dashboard against a
 backtest run, or later analysis of a backtest's persisted events — that
-has not been confirmed one way or the other. Skipping the push would
-contradict documented, tested behavior on the strength of an unconfirmed
-assumption, so it is recorded here instead of implemented.
-
-**Next step, when revisited:** confirm whether anything actually reads
-backtest-mode experience events out of Postgres after the fact; if nothing
-does, gate `push()` on `runtime_mode != "backtest"` and update
-`v2_architecture.md`'s Redis Experience Queue section and
-`test_experience_queue.py`'s default fixture accordingly.
+had not been confirmed one way or the other. Skipping the push would have
+contradicted documented, tested behavior on the strength of an unconfirmed
+assumption.
+
+**Resolution:** the project owner personally confirmed no downstream
+process reads backtest-mode experience events out of Postgres. Since the
+open question this entry was tracking is now answered, it's marked
+resolved on that basis alone — `experience/redis_queue.py::push()` itself
+is intentionally left unchanged (still pushes in backtest mode), since the
+performance cost was never the blocker, only the unconfirmed dependency
+was. A future optimization pass may still gate `push()` on
+`runtime_mode != "backtest"` if the per-bar I/O ever becomes a real
+bottleneck, but that is now a pure performance nice-to-have, not a
+correctness fix blocked on missing information.
 
 ---
 
@@ -571,3 +576,89 @@ runs: the first (cold cache) still showed the "building the font cache"
 message and took ~82 seconds just to import; the second (warm cache) showed
 no such message and imported in ~58 seconds, with zero isolator timeout.
 `.matplotlib_cache/` added to `.gitignore` (generated cache, not committed).
+
+---
+
+### 18. Two structural "never recovers" traps suppressed real backtest trade count to 12 over 3 years
+**Severity:** 5/10 · **Status:** 🟢 `addressed` (opt-in statistical bypass, default behavior unchanged)
+
+Found investigating why a real 3-year, 20-asset backtest produced only 12
+filled trades: a 5-symbol mass liquidation on 2020-03-23 (portfolio total
+drawdown crossed `phase6.risk.max_total_drawdown_pct`, 12%) froze the
+algorithm for the remaining 374 days of the window — no trades at all. The
+sticky lock (`main.py::_refresh_risk_state()`'s daily auto-clear
+deliberately excludes `"total_drawdown_limit_breached"`, by design, for
+live capital preservation) is only half the story: `peak_equity` is a
+running max that never decreases, so once liquidated to flat cash, the
+drawdown-from-peak percentage can never recover on its own — the lock
+cannot ever clear again regardless of the sticky-exclusion logic, purely
+because the underlying number stays permanently breached. A second,
+independent, earlier-firing (8% vs. 12%) version of the identical trap was
+found in `regime/market_regime.py::classify_risk_regime()`'s `risk_off`
+drawdown branch (`phase_v2.regime_detection.risk_off_drawdown_threshold`),
+fed from the same portfolio-wide drawdown number — this alone would
+suppress the whole universe's signals even if the 12% lock were fixed in
+isolation.
+
+**Fix:** new opt-in flag `phase_v2.backtest.bypass_safety_gates` (default
+`false`) and pure helper
+`risk_controls.py::is_backtest_safety_bypass_active(runtime_mode, bypass_flag)`
+— `True` only when `runtime_mode == "backtest"` and the flag is explicitly
+`true`; any non-backtest mode always returns `False` regardless of the
+flag. When active, bypasses only these two specific mechanisms (the sticky
+lock's exclusion, and the regime override's drawdown branch specifically —
+passing `float("inf")` in place of the configured threshold); every other
+gate (liquidity, topology, cooldown, exposure caps, the bearish-trend+
+high-vol and composite-score regime branches) stays fully active. Live/paper
+mode, and a backtest with the flag left at its default `false`, are
+completely unaffected.
+
+**Deliberately not wired into `aq trade-lock`:** `--on`/`--off`/`--auto`
+already has a separately-documented meaning (`--auto` = "return to fully
+automatic \[original, safety-preserving\] behavior") that this must not
+collide with or repurpose — a dedicated config key keeps both mechanisms
+independently meaningful in every runtime mode.
+
+**Explicitly not a claim about live-representative behavior:** in live/paper
+mode both gates are real, designed behavior that would have actually frozen
+trading on 2020-03-23. This flag is scoped to statistical/model-quality
+backtesting only (enough trade volume for meaningful metrics and to
+exercise `performance_triggers.trade_count_interval=100`/
+`retraining.validation_gate.min_trade_count=30`, neither of which ever
+fires at ~12 trades) — never to be read as "what would happen if deployed."
+
+---
+
+### 19. Neural-network webui tab's gating exclusion went stale the moment gating became learnable
+**Severity:** 2/10 · **Status:** 🟢 `fixed`
+
+`monitoring/neural_network_state.py`'s `EXCLUDED_NON_NETWORKS` listed
+`moe/gating.py`'s gating network with the reason "deterministic rule-based
+combiner ... no learned weight matrix," and `/neural-network`'s 3D scene
+(`webui/src/components/neuralnet/NeuralNetworkScene3D.tsx`) hardcoded its
+render order to exactly 5 network names, silently dropping anything else
+even if the backend did return it. Both became inaccurate the moment
+`moe/gating.py` gained an optional learned model (this session, same
+Phase E work as entry above) — the gating network now genuinely can have
+a learned weight matrix (`ml/gating_model.json`), but the webui had no way
+to show it even after one existed, and the "no learned weight matrix"
+claim was simply wrong.
+
+**Fix:** removed `gating_network` from `EXCLUDED_NON_NETWORKS` (only
+`learned_topology`'s KMeans centroids remain excluded — genuinely not a
+layered network); `build_neural_network_state()` now reads
+`ml/gating_model.json` through the exact same generic
+`_build_network_summary()` path already used for the baseline and the 4
+experts, degrading to `status="not_trained"` when no gating model exists
+yet (same graceful-optional contract as everything else in this module).
+Added `'gating'` to `NeuralNetworkScene3D.tsx`'s `NETWORK_ORDER` array so
+it actually renders in the 3D scene once returned by the backend.
+
+**Why this matters beyond gating specifically:** `NETWORK_ORDER` is a
+silent filter — any future network the backend starts reporting will not
+appear in the 3D scene (though it will still appear in the stats panel's
+list, which iterates the array directly) unless someone remembers to add
+its name here too. Left as-is rather than making it dynamic, since the
+3D layout intentionally controls left-to-right ordering for readability;
+noted here so the next network added to `build_neural_network_state()`
+doesn't quietly repeat this gap.