rl-pricing 0.5.0__tar.gz

rl_pricing-0.5.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021-2026 Bicheng Wang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

rl_pricing-0.5.0/MANIFEST.in

@@ -0,0 +1,6 @@
+include main.py
+include pytest.ini
+include requirements.in
+include requirements.txt
+include ruff.toml
+recursive-include tests *.py

rl_pricing-0.5.0/PKG-INFO

@@ -0,0 +1,86 @@
+Metadata-Version: 2.4
+Name: rl-pricing
+Version: 0.5.0
+Summary: A reproducibility lab for trading AI: causal backtests, a live forward-test track and a pre-open decision ledger for RL, LLM and forecasting policies.
+Home-page: https://github.com/BichengWang/RL-pricing
+Author: Bicheng Wang
+License-Expression: MIT
+Project-URL: Source, https://github.com/BichengWang/RL-pricing
+Project-URL: Issues, https://github.com/BichengWang/RL-pricing/issues
+Project-URL: Leaderboard, https://github.com/BichengWang/RL-pricing/blob/main/leaderboard/LEADERBOARD.md
+Project-URL: Paper, https://arxiv.org/abs/2609.31870
+Keywords: reinforcement learning,portfolio management,backtesting,benchmark,leaderboard
+Platform: any
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Topic :: Office/Business :: Financial :: Investment
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2.0
+Requires-Dist: stockstats>=0.6.2
+Requires-Dist: yfinance>=0.2.40
+Requires-Dist: matplotlib>=3.7
+Requires-Dist: gymnasium>=0.29.1
+Requires-Dist: stable-baselines3>=2.3.2
+Requires-Dist: torch>=2.0
+Provides-Extra: test
+Requires-Dist: pytest>=7.0; extra == "test"
+Provides-Extra: lint
+Requires-Dist: ruff>=0.5; extra == "lint"
+Dynamic: author
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: license-expression
+Dynamic: platform
+Dynamic: project-url
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# RL-pricing
+
+A reproducibility lab for trading AI. RL-pricing re-tests published claims
+about deep RL agents, LLM trading agents and forecasters under one
+pre-registered protocol. This package holds the lab's instruments:
+
+- `rl-pricing bench`: a causal backtesting benchmark. Policies decide at the
+  close and fill at the next open with fees; learned policies run over
+  several seeds; frozen policies are scored on a live forward-test track; and
+  a decision ledger records LLM agents' trades before each US market open.
+- `rl-pricing --mode train|backtest|ensemble|lstm`: the share-trading toolkit
+  that trains and backtests Stable-Baselines3 agents (A2C, PPO, DDPG, TD3,
+  SAC), a rolling ensemble and an LSTM forecaster on daily equity data.
+
+```bash
+pip install rl-pricing
+rl-pricing bench tasks             # list the benchmark tasks
+rl-pricing bench data us-sectors   # download a task's data and print its hash
+```
+
+The import package is `rl`, so this distribution can't share an environment
+with the unrelated `rl` (GNU Readline bindings) distribution. Leaderboard
+submissions, results and reports live in the
+[repository](https://github.com/BichengWang/RL-pricing): clone it to run the
+submissions, or pass `--submissions-root` to use your own.
+
+- [Verdicts, findings and leaderboard](https://github.com/BichengWang/RL-pricing#readme)
+- [Paper: Wang & Zhang, arXiv:2609.31870](https://arxiv.org/abs/2609.31870)
+
+The training toolkit's environment, preprocessing and training code are
+adapted from [FinRL](https://github.com/AI4Finance-Foundation/FinRL), which is
+MIT-licensed.

rl_pricing-0.5.0/README.md

@@ -0,0 +1,344 @@
+# RL-pricing: a reproducibility lab for trading AI
+
+[![tests](https://github.com/BichengWang/RL-pricing/actions/workflows/tests.yml/badge.svg)](https://github.com/BichengWang/RL-pricing/actions/workflows/tests.yml)
+[![bench](https://github.com/BichengWang/RL-pricing/actions/workflows/bench.yml/badge.svg)](https://github.com/BichengWang/RL-pricing/actions/workflows/bench.yml)
+[![arXiv](https://img.shields.io/badge/arXiv-2609.31870-b31b1b.svg)](https://arxiv.org/abs/2609.31870)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+We re-test published trading-AI claims (deep RL, LLM agents, forecasters)
+under one pre-registered protocol, and publish every verdict next to the
+authors' reply. That includes the verdicts against our own paper. The instruments are public:
+a causal backtesting benchmark, a live track scored only on market data that
+did not exist when a policy was published, and a ledger of decisions
+committed before each US market open.
+
+## Verdicts
+
+<!-- reports:start -->
+| Report | Claim | Level | Claimed | Ours (95% interval) | Verdict |
+| --- | --- | --- | --- | --- | --- |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `ddpg-annual-return` (annual_return) | L2 | 55.5% | 24.6% [17.3%, 31.9%] | ✗ does not hold |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `ddpg-sharpe` (sharpe) | L2 | 1.38 | 1.16 [0.93, 1.39] | ◐ partly holds |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `td3-sharpe` (sharpe) | L2 | 1.37 | 1.10 [0.69, 1.52] | ◐ partly holds |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `sac-sharpe` (sharpe) | L2 | 1.33 | 1.23 [1.07, 1.39] | ◐ partly holds |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `ppo-sharpe` (sharpe) | L2 | 1.09 | 1.17 [0.90, 1.44] | ◐ partly holds |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `a2c-sharpe` (sharpe) | L2 | 0.82 | 1.09 [0.91, 1.27] | ◐ partly holds |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `lstm-max-drawdown` (max_drawdown) | L2 | -9.6% | -26.5% [-29.4%, -23.7%] | ? inconclusive |
+| [r0-wang-zhang-2026](reports/r0-wang-zhang-2026/report.md) | `lstm-sharpe` (sharpe) | L2 | 1.19 | 0.75 [0.39, 1.12] | ? inconclusive |
+
+Verdicts: ✅ holds, ◐ partly holds, ✗ does not hold, ? inconclusive. [How a verdict is decided](reports/README.md#how-a-verdict-is-decided).
+<!-- reports:end -->
+
+| Next | Claim under test | Due |
+| --- | --- | --- |
+| R1 | FinRL ensemble ([Yang et al., ICAIF 2020](https://arxiv.org/abs/2511.12120)): Sharpe 1.30 vs the DJIA's 0.47, 2016–2020 | 2026-12-01 |
+| R2 | [TradingAgents](https://arxiv.org/abs/2412.20138) backtest: Sharpe 5.6–8.2 on AAPL, GOOGL, AMZN in 2024 Q1, replicated with a check for whether the models remember that period | 2027-01-05 |
+| R3 | TradingAgents run live on the same three stocks for three months, each decision committed before the open ([pre-registration](reports/r3-tradingagents-live/preregistration.md)) | 2027-02-15 |
+
+Authors get each draft seven days before publication. To nominate a claim,
+open an issue titled `Nominate: <paper>` with the quoted claim and a link to
+the code.
+
+## How the lab works
+
+- **Pre-registered.** Each report's test plan is committed before anything
+  runs; the commit time is the evidence that the plan came first.
+- **Three levels.** L1 replicates with the authors' code and setup. L2
+  re-tests under this repository's protocol: next-open fills, fees, at least
+  five seeds, survivorship-free data. L3 tries new windows and, for anything
+  that can run daily, the live ledger.
+- **Verdicts by rule:** holds, partly holds, does not hold, or inconclusive,
+  decided by thresholds fixed in the pre-registration
+  ([rules and template](reports/README.md)).
+- **LLM agents only count live.** Their historical backtests can be
+  contaminated by training data. The ledger commits each decision before the
+  open, and scoring checks the commit against GitHub's own run records.
+- **Evidence is committed.** Each report ships its per-seed results, input
+  hashes and one command that reproduces them.
+
+## R0: our own paper
+
+Full report: [`reports/r0-wang-zhang-2026/report.md`](reports/r0-wang-zhang-2026/report.md).
+
+Our preprint ([Wang & Zhang, arXiv:2609.31870](https://arxiv.org/abs/2609.31870))
+reported DDPG with a 55.5% annual return and Sharpe 1.38 on 20 large-cap
+stocks in 2019–2020, with TD3 and SAC close behind. Those are single runs on a
+universe picked in 2021 from the largest companies, which the paper itself
+lists as limitations.
+
+We re-ran it three ways, 5 seeds per agent, same 20 stocks, same 2019–2020
+window, 0.1% fees:
+
+- **Share env + crash rule**: the paper's own share-trading environment
+  (`python main.py --mode train`, now with next-open fills), including its
+  default rule that sells everything when a market-turbulence index spikes.
+- **Share env, no rule**: the same with `--no-turbulence`.
+- **Benchmark**: this repository's `paper-2021` task (portfolio weights, no
+  hand-coded exits).
+
+Sharpe ratio, mean ± sd over 5 seeds [best seed]:
+
+| Agent | Reported (1 run) | Share env + crash rule | Share env, no rule | Benchmark |
+| --- | --- | --- | --- | --- |
+| DDPG | 1.38 | 1.16 ± 0.19 [1.43] | 1.06 ± 0.18 [1.24] | 0.70 ± 0.17 [0.97] |
+| TD3 | 1.37 | 1.10 ± 0.34 [1.37] | 1.00 ± 0.30 [1.37] | 0.70 ± 0.21 [1.01] |
+| SAC | 1.33 | 1.23 ± 0.13 [1.42] | 1.23 ± 0.18 [1.49] | 0.66 ± 0.26 [0.82] |
+| PPO | 1.09 | 1.17 ± 0.21 [1.32] | 0.95 ± 0.21 [1.21] | 0.98 ± 0.04 [1.02] |
+| A2C | 0.82 | 1.09 ± 0.15 [1.27] | 0.97 ± 0.15 [1.16] | 1.03 ± 0.01 [1.05] |
+| LSTM forecast | 1.19 | – | – | 0.75 ± 0.29 [1.08] |
+| *1/N buy and hold* | – | *1.10* | *1.10* | *1.11* |
+
+What holds up and what doesn't:
+
+- **The 55.5% return does not.** The best of the 50 share-environment runs
+  earned 46% a year and the median 26%. Holding the 20 stocks in equal
+  weight earned 29.7%, so the hindsight-chosen universe and the 2019–2020
+  bull market account for most of the headline.
+- **The Sharpe ratios are best-of-five, not typical.** Values like the
+  reported 1.33–1.38 show up as the best seed for most agents.
+- **The crash rule is not learning.** With it on, all 25 runs were moved to
+  cash on the same five sessions of March 2020 by the environment, not the agent.
+  Switching it off drops every agent's mean Sharpe except SAC's below 1/N.
+- **SAC is the one survivor, but not a significant one.** Without the rule
+  its seed-averaged Sharpe is 1.32 against 1/N's 1.10, alpha 7.5% a year. Over
+  a two-year window that is not significant (bootstrap p = 0.13, alpha
+  *t* = 1.6).
+- **The benchmark environment flips the picture.** There the same agents
+  trail 1/N. Which difference drives that is open: share counts versus
+  weights, the features, or the share environment's per-trade limits.
+
+Per-seed results, equity curves, commands and the input-data hash are in
+[`reports/r0-wang-zhang-2026/evidence/`](reports/r0-wang-zhang-2026/evidence/).
+
+## Benchmark findings
+
+![Per-seed Sharpe ratios of every strategy against 1/N](leaderboard/sharpe.png)
+
+Historical track, test windows 2018–2025 (US ETFs) and 2023–2025 (crypto),
+Stable-Baselines3 agents with default and tuned hyperparameters, 5 seeds each:
+
+- **No learned policy beats 1/N buy-and-hold here.** All 30 RL and LSTM entries
+  across the four tasks have a lower seed-averaged Sharpe ratio than holding
+  every asset in equal weight. Only 5 of their 150 individual seeds do better,
+  so cherry-picking one would make a publishable-looking result.
+- **Tuning doesn't close the gap.** A declared sweep of 8 configurations each
+  for PPO and SAC, selected on validation data only, picks configurations that
+  still trail 1/N on every task. SAC's choice helps on the US tasks and
+  collapses on crypto (Sharpe 0.07 ± 0.49). The deflated Sharpe column
+  accounts for the 8 trials.
+- **The off-policy agents churn themselves to death.** SAC, TD3 and DDPG
+  turn over 46–326× their equity per year. On the US tasks their alpha
+  against 1/N is negative, with *t*-statistics down to −2.8.
+- **A2C rediscovers 1/N.** Its five seeds on US sectors land at Sharpe
+  0.67 ± 0.01 against 1/N's 0.68: the softmax policy learns to stay close to
+  equal weights.
+- **The one significant win is not RL.** In crypto, long-only minimum
+  variance beats 1/N (ΔSharpe +0.86, bootstrap p < 0.001), mostly by holding
+  the two least volatile coins, BTC (50% on average) and BNB (36%).
+
+Better configurations are what the leaderboard is for: submit one, and state
+how many you tried.
+
+## Leaderboard
+
+<!-- leaderboard:start -->
+| Track | Task | # | Strategy | Sharpe | CAGR | Max DD | ΔSharpe vs 1/N | DSR |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| historical | `crypto-majors` | 1 | Minimum variance | 1.64 | 90.9% | -33.3% | +0.86 (p<0.001) | 1.00 |
+| historical | `crypto-majors` | 2 | Inverse volatility | 0.88 | 40.7% | -46.8% | +0.10 (p=0.089) | 0.94 |
+| historical | `crypto-majors` | 3 | Momentum (top 3) | 0.84 | 37.8% | -47.9% | +0.06 (p=0.401) | 0.93 |
+| historical | `crypto-majors` | 4 | **1/N buy and hold** ◆ | 0.78 | 32.7% | -44.5% | – | 0.91 |
+| historical | `crypto-majors` | 6 | PPO (tuned) | 0.71 ± 0.22 | 26.9% | -49.3% | -0.04 (p=0.621) | 0.43 |
+| historical | `us-multi-asset` | 1 | **1/N buy and hold** ◆ | 0.73 | 8.9% | -26.6% | – | 0.98 |
+| historical | `us-multi-asset` | 2 | Momentum (top 3) | 0.72 | 9.9% | -23.7% | -0.01 (p=0.506) | 0.98 |
+| historical | `us-multi-asset` | 3 | 1/N monthly rebalance | 0.69 | 8.0% | -25.8% | -0.04 (p=0.827) | 0.97 |
+| historical | `us-multi-asset` | 4 | A2C | 0.66 ± 0.02 | 7.1% | -23.8% | -0.06 (p=0.940) | 0.97 |
+| historical | `us-sectors` | 1 | 1/N monthly rebalance | 0.69 | 11.6% | -36.7% | +0.01 (p=0.449) | 0.97 |
+| historical | `us-sectors` | 2 | **1/N buy and hold** ◆ | 0.68 | 11.5% | -35.5% | – | 0.97 |
+| historical | `us-sectors` | 3 | A2C | 0.67 ± 0.01 | 10.3% | -34.3% | -0.01 (p=0.650) | 0.97 |
+| historical | `paper-2021` (re-evaluation) | – | Momentum (top 3) | 1.31 | 48.6% | -30.4% | +0.20 (p=0.240) | 0.96 |
+| historical | `paper-2021` (re-evaluation) | – | **1/N buy and hold** ◆ | 1.11 | 29.9% | -29.4% | – | 0.94 |
+| historical | `paper-2021` (re-evaluation) | – | 1/N monthly rebalance | 1.04 | 26.7% | -29.5% | -0.07 (p=0.777) | 0.93 |
+| historical | `paper-2021` (re-evaluation) | – | A2C | 1.03 ± 0.01 | 25.1% | -28.2% | -0.08 (p=0.825) | 0.92 |
+
+Full tables: [leaderboard/LEADERBOARD.md](leaderboard/LEADERBOARD.md).
+<!-- leaderboard:end -->
+
+## Why trust these numbers
+
+| | This benchmark | Common in RL-trading papers |
+| --- | --- | --- |
+| Execution | Decide at the close, fill at the next open, proportional fees | Fill at the same close used to decide; often no fees |
+| Universe | Fixed ETFs; crypto top-10 by volume *before* the data starts, delistings included | Today's index members, back-tested into the past |
+| Randomness | 5 seeds per learned policy, mean ± sd, every seed plotted | One run |
+| Model selection | Validation window before the test; declared `trials` deflate the Sharpe | Tuned on the test period |
+| Out-of-sample | Live track: frozen policy, scored only on sessions after its public freeze | One fixed historical window |
+| Metrics | One implementation for everyone; beta and alpha against 1/N | Per-paper definitions |
+
+The live track is the part nobody can overfit. A policy is fitted on all
+data available today and its artifacts are committed with SHA-256 hashes. A
+CI job on `main` then stamps the time the freeze became public
+(`leaderboard/publications.json`, which pull requests may not edit), and
+from then on it is scored monthly on sessions that start after that stamp.
+The first live scores appear after the next monthly run.
+
+## Quick start
+
+Python 3.10 or newer:
+
+```bash
+git clone https://github.com/BichengWang/RL-pricing.git
+cd RL-pricing
+python -m venv .venv && source .venv/bin/activate
+python -m pip install -e .
+
+rl-pricing bench tasks                       # list tasks
+rl-pricing bench run --task synthetic --submission momentum --out-root /tmp/board
+rl-pricing bench run --task synthetic --submission ppo --seeds 2 --out-root /tmp/board
+rl-pricing bench leaderboard --out-root /tmp/board --no-readme
+```
+
+The `synthetic` task is offline random walks with no edge by construction.
+Real tasks download data on first use (`rl-pricing bench data us-sectors`)
+into `datasets/bench/`. PPO takes one to two minutes per seed on a laptop CPU.
+
+## Submit a policy
+
+A policy maps the history up to today's close to long-only target weights,
+which fill at the next open. Here is a complete one:
+
+```python
+# submissions/low-vol-half/policy.py
+import numpy as np
+
+from rl.bench.policy import Policy
+
+
+class LowVolHalf(Policy):
+    """Equal-weight the calmer half of the listed assets."""
+
+    def act(self, view):
+        closes = view.history.window(64).close
+        vol = np.nanstd(np.diff(np.log(closes), axis=0), axis=0)
+        vol = np.where(view.listed & np.isfinite(vol), vol, np.inf)
+        keep = np.argsort(vol)[: max(1, view.listed.sum() // 2)]
+        weights = np.zeros(len(view.tickers))
+        weights[keep] = 1 / len(keep)
+        return weights
+```
+
+```json
+{
+  "name": "Low-vol half",
+  "kind": "other",
+  "policy": "policy.py:LowVolHalf",
+  "trials": 3,
+  "authors": ["Your Name"],
+  "description": "Equal-weight the half of the assets with the lowest 63-day volatility."
+}
+```
+
+Learning policies implement `fit(history, validation_start, seed)` as well:
+train on sessions before `validation_start`, select on the rest, and set
+`stochastic = True` to be evaluated over several seeds. They also implement
+`save`/`load` so the frozen live version can be restored. See
+[`rl/bench/policies/sb3.py`](rl/bench/policies/sb3.py) for a full example.
+Set `trials` honestly to the number of configurations you tried.
+
+Then:
+
+1. `rl-pricing bench validate` and `rl-pricing bench run --task <task> --submission <id>` locally.
+2. Open a pull request with `submissions/<id>/` and your `leaderboard/historical/` records.
+   CI checks that the submission builds and the baselines reproduce.
+3. On merge, CI freezes the policy on today's data (or publishes your
+   committed freeze from `rl-pricing bench freeze`) and the monthly job scores
+   it live from then on.
+
+## Tasks
+
+| Task | Universe | Test window | Fee per side | Why it is survivorship-free |
+| --- | --- | --- | --- | --- |
+| `us-sectors` | 9 SPDR sector ETFs | 2018–2025 | 5 bps | all listed since 1998 |
+| `us-multi-asset` | SPY, QQQ, IWM, EFA, EEM, TLT, IEF, LQD, GLD, VNQ | 2018–2025 | 5 bps | all listed since 2004 |
+| `crypto-majors` | top 10 Binance USDT pairs by Q2-2018 volume | 2023–2025 | 10 bps | chosen before the data starts; BCC and EOS delist mid-sample |
+| `paper-2021` | the 2021 paper's 20 large caps | 2019–2020 | 10 bps | *it is not*: shown as a re-evaluation, not ranked |
+| `synthetic` | 5 random walks | 2018–2019 | 10 bps | offline demo |
+
+Each task trains before its validation window, selects on validation, and
+reports the test window. Definitions live in [`rl/bench/tasks.py`](rl/bench/tasks.py).
+
+## How scoring works
+
+- **Execution** ([`rl/bench/engine.py`](rl/bench/engine.py)): targets set at
+  close *t* fill at open *t+1* with fees solved exactly on traded notional.
+  Weights are long-only and sum to at most one; the rest is cash at 0%. A
+  delisted holding is sold at its last close. RL agents train in a gym
+  environment that uses the same `Account`, so training and evaluation
+  accounting are identical.
+- **Metrics** ([`rl/bench/scoring.py`](rl/bench/scoring.py)): Sharpe, CAGR,
+  max drawdown, annual turnover, and beta/alpha against 1/N (Newey-West *t*).
+  ΔSharpe against 1/N comes with a paired stationary-bootstrap p-value. The
+  deflated Sharpe ratio (Bailey & López de Prado) accounts for declared
+  `trials`. Full definitions are in [`leaderboard/LEADERBOARD.md`](leaderboard/LEADERBOARD.md).
+- **Records**: every run stores its daily equity curve per seed, data
+  SHA-256, software versions and commit under `leaderboard/`, so all
+  metrics can be recomputed.
+
+## Data
+
+US tasks use Yahoo Finance prices and dividends, downloaded to a local cache.
+Yahoo's terms prohibit redistribution, so the repository ships no prices,
+only derived results. Prices are adjusted for dividends forward from each
+task's first session rather than taken from Yahoo's adjusted close, whose
+last digits change between identical requests. A past session therefore
+keeps its prices as new sessions arrive, and replaying the same window
+reproduces its data hash; a changed hash means Yahoo revised its
+history. Crypto uses
+[Binance Vision](https://data.binance.vision) daily klines, each archive
+verified against Binance's SHA-256 checksum. Binance Vision data is licensed
+CC BY-NC-SA 4.0; the crypto results here derive from it, with attribution,
+for non-commercial research.
+
+## Limitations
+
+- Fees are proportional; spread, market impact and borrowing costs are not
+  modeled, and cash earns nothing.
+- Daily bars only; adjusted prices model reinvested distributions rather
+  than exact exchange fills.
+- Eight-year and three-year test windows are a few market regimes, which is
+  why the live track exists.
+- Reference RL agents are untuned and use one feature set
+  ([`rl/bench/features.py`](rl/bench/features.py)); they are a floor, not
+  the best RL can do.
+
+## Training toolkit
+
+The original FinRL-style training pipeline (share trading, portfolio
+allocation, LSTM forecaster, rolling ensemble via `python main.py --mode ...`)
+is still available and documented in [`docs/toolkit.md`](docs/toolkit.md).
+
+## Development
+
+```bash
+pip install -e ".[test,lint]"
+python -m pytest        # offline: synthetic data and fixtures
+ruff check .
+```
+
+Use `type(scope): title` for pull request titles. The
+[PR template](.github/pull_request_template.md) asks for a short summary and
+verification results.
+
+## Citing
+
+GitHub's "Cite this repository" uses [`CITATION.cff`](CITATION.cff). The
+original study is Wang & Zhang, *Deep Reinforcement Learning for Equity
+Trading: Benchmarking Actor-Critic Methods with Forward Retraining*,
+[arXiv:2609.31870](https://arxiv.org/abs/2609.31870)
+([local PDF](docs/Wang_Zhang_2021_Deep_Learning_Stock_Trading_arxiv.pdf)).
+
+## License
+
+MIT; see [LICENSE](LICENSE). The training toolkit's environment,
+preprocessing and training code are adapted from the MIT-licensed
+[FinRL](https://github.com/AI4Finance-Foundation/FinRL) library.

rl_pricing-0.5.0/main.py

@@ -0,0 +1,6 @@
+"""Command-line entry point; see ``python main.py --help``."""
+
+from rl.cli import main
+
+if __name__ == "__main__":
+    main()

rl_pricing-0.5.0/pyproject.toml

@@ -0,0 +1,4 @@
+[build-system]
+# 77 is the first release that writes License-Expression (PEP 639) metadata.
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"

rl_pricing-0.5.0/pytest.ini

@@ -0,0 +1,3 @@
+[pytest]
+testpaths = tests
+pythonpath = . tests

rl_pricing-0.5.0/requirements.in

@@ -0,0 +1,15 @@
+# data handling
+numpy>=1.24
+pandas>=2.0
+stockstats>=0.6.2
+yfinance>=0.2.40
+
+# plot
+matplotlib>=3.7
+
+# Model Building Requirements
+gymnasium>=0.29.1
+stable-baselines3>=2.3.2
+torch>=2.0
+
+# Test and lint tools are extras in setup.py: pip install -e ".[test]"

rl_pricing-0.5.0/requirements.txt

@@ -0,0 +1,14 @@
+# Versions the test suite was last run with (Python 3.13).
+# requirements.in lists the supported ranges; setup.py installs from it.
+#
+# For a CPU-only PyTorch (much smaller download), install it first:
+#   pip install torch --index-url https://download.pytorch.org/whl/cpu
+gymnasium==1.4.0
+matplotlib==3.11.2
+numpy==2.5.3
+pandas==3.0.5
+pytest==9.1.1
+stable-baselines3==2.9.0
+stockstats==0.6.9
+torch==2.14.1
+yfinance==1.7.0

rl_pricing-0.5.0/rl/autotrain/r1.py

@@ -0,0 +1,124 @@
+"""Driver for the R1 runs: the toolkit's ensemble with the FinRL authors' recipe.
+
+Sets what the toolkit lets us match from Yang, Liu, Zhong and Walid (ICAIF
+2020): the four indicators (``macd``, ``rsi_30``, ``cci_30``, ``dx_30``), the
+training steps per algorithm (A2C 30,000, PPO 100,000, DDPG 10,000), PPO with
+``ent_coef`` 0.005 and 128-step rollouts in minibatches of 16 (the authors'
+``nminibatches=8``), and DDPG with Ornstein-Uhlenbeck action noise of sigma
+0.5. A2C and the rest of DDPG keep the toolkit defaults, as does everything
+else (0.1% fees per side, $1,000,000, ``hmax`` 100, reward scaling 1e-4,
+63-session rebalance and validation windows, next-open fills, observation
+normalisation, refit of the winner). Each algorithm also trades its own
+account (``standalone_accounts``).
+
+Prices come from a CSV in the toolkit format (``date,tic,open,high,low,close,
+volume``), e.g. from :mod:`rl.marketdata.sharadar`::
+
+    python -m rl.autotrain.r1 --data-file prices.csv --seed 0 --run-name r1_a_s0
+"""
+
+import argparse
+import copy
+import logging
+
+from rl.autotrain.training import TrainConfig, run_ensemble
+from rl.config import config
+
+AUTHORS_INDICATORS = ("macd", "rsi_30", "cci_30", "dx_30")
+AUTHORS_TIMESTEPS = {"a2c": 30_000, "ppo": 100_000, "ddpg": 10_000}
+AUTHORS_DDPG_NOISE_SIGMA = 0.5
+
+
+def authors_model_kwargs(ddpg_noise_sigma=AUTHORS_DDPG_NOISE_SIGMA):
+    """Per-algorithm model settings (each replaces the config.py default)."""
+    return {
+        "a2c": copy.deepcopy(config.A2C_PARAMS),
+        # 128 steps in 8 minibatches; every other PPO setting is stable-baselines3's default.
+        "ppo": {"n_steps": 128, "batch_size": 16, "ent_coef": 0.005},
+        "ddpg": {
+            **copy.deepcopy(config.DDPG_PARAMS),
+            "action_noise": {"type": "ornstein_uhlenbeck", "sigma": ddpg_noise_sigma},
+        },
+    }
+
+
+def r1_config(
+    data_file,
+    *,
+    seed,
+    use_turbulence=True,
+    ticker_list=None,
+    start_date="2009-01-02",
+    start_trade_date="2016-01-04",
+    end_date="2026-01-01",
+    ddpg_noise_sigma=AUTHORS_DDPG_NOISE_SIGMA,
+    ensemble_timesteps=None,
+    standalone_accounts=True,
+    run_name=None,
+    results_dir=config.RESULTS_DIR,
+    **overrides,
+):
+    """The :class:`TrainConfig` of one R1 ensemble run.
+
+    ``end_date`` is exclusive (the default includes 2025-12-31). ``overrides``
+    set any other :class:`TrainConfig` field.
+    """
+    settings = dict(
+        agent="ensemble",
+        task="trading",
+        data_source="csv",
+        data_file=data_file,
+        ticker_list=ticker_list,
+        start_date=start_date,
+        start_trade_date=start_trade_date,
+        end_date=end_date,
+        tech_indicator_list=list(AUTHORS_INDICATORS),
+        use_turbulence=use_turbulence,
+        ensemble_timesteps=dict(AUTHORS_TIMESTEPS if ensemble_timesteps is None else ensemble_timesteps),
+        ensemble_model_kwargs=authors_model_kwargs(ddpg_noise_sigma),
+        standalone_accounts=standalone_accounts,
+        seed=seed,
+        run_name=run_name,
+        results_dir=results_dir,
+    )
+    settings.update(overrides)
+    return TrainConfig(**settings).validate()
+
+
+def run_r1(data_file, *, seed, **options):
+    """Run the ensemble for ``seed`` (see :func:`r1_config`); returns :func:`run_ensemble`'s dict."""
+    return run_ensemble(r1_config(data_file, seed=seed, **options))
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    parser.add_argument("--data-file", required=True, help="CSV of date,tic,open,high,low,close,volume")
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--tickers", nargs="+", help="default: every ticker in the file")
+    parser.add_argument("--no-turbulence", action="store_true", help="arm B: no liquidation rule")
+    parser.add_argument("--start-date", default="2009-01-02")
+    parser.add_argument("--start-trade-date", default="2016-01-04")
+    parser.add_argument("--end-date", default="2026-01-01", help="exclusive")
+    parser.add_argument("--ddpg-noise-sigma", type=float, default=AUTHORS_DDPG_NOISE_SIGMA)
+    parser.add_argument("--run-name")
+    parser.add_argument("--results-dir", default=config.RESULTS_DIR)
+    options = parser.parse_args(argv)
+    logging.basicConfig(
+        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
+    )
+    return run_r1(
+        options.data_file,
+        seed=options.seed,
+        use_turbulence=not options.no_turbulence,
+        ticker_list=options.tickers,
+        start_date=options.start_date,
+        start_trade_date=options.start_trade_date,
+        end_date=options.end_date,
+        ddpg_noise_sigma=options.ddpg_noise_sigma,
+        run_name=options.run_name,
+        results_dir=options.results_dir,
+    )
+
+
+if __name__ == "__main__":
+    main()