@bitget-ai/getagent-skill 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +28 -0
- package/.claude-plugin/plugin.json +12 -0
- package/README.md +99 -0
- package/VERSION +1 -0
- package/bin/getagent-skill.js +140 -0
- package/package.json +45 -0
- package/skills/getagent/SKILL.md +129 -0
- package/skills/getagent/examples/btc-ema-cross-demo/README.md +61 -0
- package/skills/getagent/examples/btc-ema-cross-demo/backtest.yaml +33 -0
- package/skills/getagent/examples/btc-ema-cross-demo/manifest.yaml +94 -0
- package/skills/getagent/examples/btc-ema-cross-demo/src/main.py +88 -0
- package/skills/getagent/examples/btc-ema-cross-demo/src/strategy.py +118 -0
- package/skills/getagent/references/api/enable.md +95 -0
- package/skills/getagent/references/api/error-responses.md +77 -0
- package/skills/getagent/references/api/index.md +38 -0
- package/skills/getagent/references/api/list.md +80 -0
- package/skills/getagent/references/api/my-playbooks.md +41 -0
- package/skills/getagent/references/api/publish.md +76 -0
- package/skills/getagent/references/api/run.md +149 -0
- package/skills/getagent/references/api/upload.md +76 -0
- package/skills/getagent/references/backtest-engine.md +438 -0
- package/skills/getagent/references/package-schema.md +552 -0
- package/skills/getagent/references/sandbox-runtime.md +201 -0
- package/skills/getagent/references/sdk/backtest/catalog.md +208 -0
- package/skills/getagent/references/sdk/data/arxiv.md +41 -0
- package/skills/getagent/references/sdk/data/catalog.md +56 -0
- package/skills/getagent/references/sdk/data/commodity.md +226 -0
- package/skills/getagent/references/sdk/data/coverage.md +82 -0
- package/skills/getagent/references/sdk/data/crypto.md +2906 -0
- package/skills/getagent/references/sdk/data/currency.md +123 -0
- package/skills/getagent/references/sdk/data/derivatives.md +269 -0
- package/skills/getagent/references/sdk/data/economy.md +1348 -0
- package/skills/getagent/references/sdk/data/equity.md +2120 -0
- package/skills/getagent/references/sdk/data/etf.md +372 -0
- package/skills/getagent/references/sdk/data/famafrench.md +201 -0
- package/skills/getagent/references/sdk/data/fixedincome.md +804 -0
- package/skills/getagent/references/sdk/data/imf_utils.md +225 -0
- package/skills/getagent/references/sdk/data/index.md +216 -0
- package/skills/getagent/references/sdk/data/news.md +149 -0
- package/skills/getagent/references/sdk/data/playbook-supported.md +9871 -0
- package/skills/getagent/references/sdk/data/regulators.md +299 -0
- package/skills/getagent/references/sdk/data/sentiment.md +323 -0
- package/skills/getagent/references/sdk/data/uscongress.md +126 -0
- package/skills/getagent/references/sdk/data/web_search.md +68 -0
- package/skills/getagent/references/sdk/data/wikipedia.md +97 -0
- package/skills/getagent/references/sdk/llm/catalog.md +117 -0
- package/skills/getagent/references/sdk/runtime/catalog.md +195 -0
- package/skills/getagent/references/sdk/trade/account.md +61 -0
- package/skills/getagent/references/sdk/trade/catalog.md +35 -0
- package/skills/getagent/references/sdk/trade/contract.md +331 -0
- package/skills/getagent/references/sdk/trade/helpers.md +466 -0
- package/skills/getagent/references/sdk/trade/market.md +28 -0
- package/skills/getagent/references/sdk/trade/patterns.md +102 -0
- package/skills/getagent/references/sdk/trade/spot.md +165 -0
- package/skills/getagent/references/sdk.md +198 -0
- package/skills/getagent/scripts/validate.py +965 -0
- package/skills/getagent/scripts/version_check.sh +62 -0
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# Run Playbook Backtest / Evaluation
|
|
2
|
+
|
|
3
|
+
Trigger a single evaluation run. This endpoint is asynchronous: it dispatches
|
|
4
|
+
the run and returns a `run_id`; poll `GET /api/v1/playbook/run?run_id=...` until
|
|
5
|
+
the run reaches `completed` or `failed`.
|
|
6
|
+
|
|
7
|
+
## `POST /api/v1/playbook/run`
|
|
8
|
+
|
|
9
|
+
**Auth**: `ACCESS-KEY` header from the Bitget OpenAPI credential, required on `POST /run` and `GET /run`. Missing it returns 401.
|
|
10
|
+
|
|
11
|
+
**Content-Type**: `application/json`
|
|
12
|
+
|
|
13
|
+
### Request
|
|
14
|
+
|
|
15
|
+
```json
|
|
16
|
+
{
|
|
17
|
+
"version_id": "draft-or-version-id"
|
|
18
|
+
}
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
### Supported Use Cases
|
|
22
|
+
|
|
23
|
+
- creator runs a draft or published Playbook to validate a new version
|
|
24
|
+
- any authenticated user runs a published Playbook evaluation when `backtest_support = full`
|
|
25
|
+
|
|
26
|
+
### Backtest Eligibility
|
|
27
|
+
|
|
28
|
+
The endpoint is only valid for Playbooks whose public contract says they are
|
|
29
|
+
fairly replayable on historical data.
|
|
30
|
+
|
|
31
|
+
- `backtest_support: full` -> allowed
|
|
32
|
+
- `backtest_support: none` -> reject with conflict and explain that the strategy is live-only
|
|
33
|
+
- `runtime_profile: deterministic` -> allowed today
|
|
34
|
+
- `runtime_profile: llm_bounded` -> reject here because this endpoint is historical evaluation only
|
|
35
|
+
- `runtime_profile: agentic` -> reject until that runtime exists
|
|
36
|
+
|
|
37
|
+
### Dispatch Response
|
|
38
|
+
|
|
39
|
+
```json
|
|
40
|
+
{
|
|
41
|
+
"run_id": "e5f6g7h8-...",
|
|
42
|
+
"strategy_id": "strategy-...",
|
|
43
|
+
"version_id": "draft-or-version-id",
|
|
44
|
+
"status": "pending",
|
|
45
|
+
"dispatched": true
|
|
46
|
+
}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## `GET /api/v1/playbook/run?run_id=...`
|
|
50
|
+
|
|
51
|
+
Poll the immutable run record created by `POST /run`.
|
|
52
|
+
|
|
53
|
+
### Completed Response
|
|
54
|
+
|
|
55
|
+
```json
|
|
56
|
+
{
|
|
57
|
+
"run_id": "e5f6g7h8-...",
|
|
58
|
+
"playbook_id": "a1b2c3d4-...",
|
|
59
|
+
"status": "completed",
|
|
60
|
+
"active_runtime_ms": 12340,
|
|
61
|
+
"signal_output": [
|
|
62
|
+
{
|
|
63
|
+
"type": "signal",
|
|
64
|
+
"action": "long",
|
|
65
|
+
"symbol": "BTCUSDT",
|
|
66
|
+
"confidence": 0.75,
|
|
67
|
+
"metrics": {
|
|
68
|
+
"total_return_pct": 23.5,
|
|
69
|
+
"sharpe_ratio": 1.8,
|
|
70
|
+
"win_rate": 0.62
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
],
|
|
74
|
+
"metrics_output": {
|
|
75
|
+
"total_return_pct": 23.5,
|
|
76
|
+
"sharpe_ratio": 1.8,
|
|
77
|
+
"max_drawdown_pct": 8.4,
|
|
78
|
+
"win_rate": 0.62,
|
|
79
|
+
"total_trades": 41
|
|
80
|
+
},
|
|
81
|
+
"backtest_report": {
|
|
82
|
+
"period_start": "2025-10-01T00:00:00+00:00",
|
|
83
|
+
"period_end": "2026-04-01T00:00:00+00:00"
|
|
84
|
+
},
|
|
85
|
+
"failure_reason": ""
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### Persistence Expectations
|
|
90
|
+
|
|
91
|
+
Every evaluation run should be stored as immutable history, separate from
|
|
92
|
+
subscription execution runs.
|
|
93
|
+
|
|
94
|
+
Recommended fields to persist:
|
|
95
|
+
|
|
96
|
+
- `run_kind` (`manual_backtest`)
|
|
97
|
+
- `requested_by_principal_id`
|
|
98
|
+
- `visibility` (`public` or `owner_only`)
|
|
99
|
+
- summary metrics
|
|
100
|
+
- report / chart artifact references
|
|
101
|
+
|
|
102
|
+
The Playbook's official public summary should point at one designated run,
|
|
103
|
+
not "whatever run happened last".
|
|
104
|
+
|
|
105
|
+
### Execution Environment
|
|
106
|
+
|
|
107
|
+
The Playbook runs in a dedicated `playbook_runtime` sandbox:
|
|
108
|
+
|
|
109
|
+
- **Timeout**: 180 seconds
|
|
110
|
+
- **Memory**: 2GB
|
|
111
|
+
- **Network**: Managed data/trade access, plus managed LLM access only when `runtime_profile: llm_bounded` is injected
|
|
112
|
+
- **Dependencies**: Only `getagent` + `pandas` + `numpy` (no pip install)
|
|
113
|
+
- **Entry point**: Executes the package as `python -m src.main`
|
|
114
|
+
|
|
115
|
+
For scheduled subscriber runs, the system executes the Playbook **once per enabled user**.
|
|
116
|
+
Each run gets that user's own:
|
|
117
|
+
|
|
118
|
+
- `principal_id`
|
|
119
|
+
- Telegram `chat_id`
|
|
120
|
+
- trade-proxy `bg_uid`
|
|
121
|
+
- trade-proxy base URL
|
|
122
|
+
|
|
123
|
+
That scheduled execution history should not be treated as the same thing as a
|
|
124
|
+
public backtest.
|
|
125
|
+
|
|
126
|
+
### Permissions
|
|
127
|
+
|
|
128
|
+
- Owner can run draft and published Playbooks they control
|
|
129
|
+
- Any authenticated user can run published Playbooks when `backtest_support = full`
|
|
130
|
+
- Users must not be allowed to run draft versions they do not own
|
|
131
|
+
|
|
132
|
+
### Error Responses
|
|
133
|
+
|
|
134
|
+
| Status | Description |
|
|
135
|
+
|--------|-------------|
|
|
136
|
+
| 401 | Missing `ACCESS-KEY` header |
|
|
137
|
+
| 403 | Caller cannot evaluate this Playbook version, access key revoked, or principal not found / inactive |
|
|
138
|
+
| 404 | `version_id` not found |
|
|
139
|
+
| 409 | Playbook is live-only or uses a runtime profile that cannot run historical evaluation yet |
|
|
140
|
+
| 503 | Sandbox pool not available |
|
|
141
|
+
|
|
142
|
+
### On Failure
|
|
143
|
+
|
|
144
|
+
`status` is `"failed"` and `failure_reason` contains the error message
|
|
145
|
+
(truncated to 2000 chars). Common causes:
|
|
146
|
+
|
|
147
|
+
- Data fetch timeout (exchange API unreachable)
|
|
148
|
+
- `src/main.py` runtime exception
|
|
149
|
+
- Timeout kill (exceeded 180 seconds)
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# Upload Playbook
|
|
2
|
+
|
|
3
|
+
Upload a Playbook package draft to GetAgent Cloud.
|
|
4
|
+
|
|
5
|
+
> Before dispatching this call, follow the **Endpoint Confirmation Discipline** in `SKILL.md` — echo a one-line preflight (`upload draft {name} → GetAgent prod https://api.bitget.com with ACCESS-KEY=<masked>`) and obtain the user's Bitget OpenAPI `ACCESS-KEY` if not already collected this session.
|
|
6
|
+
|
|
7
|
+
## `POST /api/v1/playbook/upload`
|
|
8
|
+
|
|
9
|
+
**Auth**: `ACCESS-KEY` header from the Bitget OpenAPI credential. Missing it returns 401.
|
|
10
|
+
|
|
11
|
+
**Content-Type**: `multipart/form-data`
|
|
12
|
+
|
|
13
|
+
**Body**: `package` — tar.gz file, max 10MB
|
|
14
|
+
|
|
15
|
+
### Request Example
|
|
16
|
+
|
|
17
|
+
Substitute `<access_key>` with the value the user provided in chat. Use the fixed GetAgent prod OpenAPI endpoint; do not parameterize it through user-provided env vars.
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
cd my-strategy/
|
|
21
|
+
tar czf ../my-strategy.tar.gz .
|
|
22
|
+
curl -X POST \
|
|
23
|
+
-H "ACCESS-KEY: <access_key>" \
|
|
24
|
+
-F "package=@../my-strategy.tar.gz" \
|
|
25
|
+
"https://api.bitget.com/api/v1/playbook/upload"
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### Success Response
|
|
29
|
+
|
|
30
|
+
```json
|
|
31
|
+
{
|
|
32
|
+
"strategy_id": "strategy-...",
|
|
33
|
+
"draft_id": "draft-...",
|
|
34
|
+
"name": "btc-ema-crossover",
|
|
35
|
+
"status": "draft",
|
|
36
|
+
"suggested_version": "1.0.1"
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Server-side Validation
|
|
41
|
+
|
|
42
|
+
The server runs these checks on upload (client-side validation is not trusted):
|
|
43
|
+
|
|
44
|
+
1. **Structure** — `manifest.yaml` and `src/main.py` must exist
|
|
45
|
+
2. **Allowed paths only** — upload may contain `manifest.yaml`, `src/**`, and optional `backtest.yaml`
|
|
46
|
+
3. **Manifest fields** — `name`, `display_name`, `description`, `market_type`, `trading_symbols`, `decision_mode`, `backtest_support`, `runtime_profile`, `execution_mode`, `follow_trade_supported` required
|
|
47
|
+
4. **Name format** — DNS label (lowercase + numbers + hyphens, 1-63 chars)
|
|
48
|
+
5. **Runtime contract consistency** — e.g. `runtime_profile: llm_bounded` requires `backtest_support: none`
|
|
49
|
+
6. **backtest.yaml shape** — only valid when `backtest_support: full`; basic type checks enforced
|
|
50
|
+
7. **Syntax** — Python files under `src/**` must compile
|
|
51
|
+
8. **Import allowlist** — Only `getagent`, `pandas`, `numpy`, `json`, `math`, `datetime`, `pathlib`, `asyncio`, etc. Imports like `requests`, `subprocess`, `os` are blocked.
|
|
52
|
+
9. **Version assignment** — upload never consumes a public version. The server returns a `draft_id`; the formal version is assigned later by `publish`.
|
|
53
|
+
10. **Local-only paths rejected** — `tests/`, `research/`, `data/`, `output/`, caches and virtualenv folders must not be included
|
|
54
|
+
|
|
55
|
+
### Error Responses
|
|
56
|
+
|
|
57
|
+
| Status | Description |
|
|
58
|
+
|--------|-------------|
|
|
59
|
+
| 401 | Missing `ACCESS-KEY` header |
|
|
60
|
+
| 403 | Access key has been revoked, or principal not found / inactive |
|
|
61
|
+
| 409 | Current state does not allow the upload operation |
|
|
62
|
+
| 413 | File exceeds 10MB |
|
|
63
|
+
| 422 | Validation failed — `detail.errors` contains specific error list |
|
|
64
|
+
|
|
65
|
+
#### 422 Example
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
{
|
|
69
|
+
"detail": {
|
|
70
|
+
"errors": [
|
|
71
|
+
"manifest.yaml: missing 'display_name'",
|
|
72
|
+
"src/main.py: blocked import 'requests' (line 3)"
|
|
73
|
+
]
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
```
|
|
@@ -0,0 +1,438 @@
|
|
|
1
|
+
# Backtest Engine
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
- [Basic Usage](#basic-usage)
|
|
6
|
+
- [Backtest Eligibility](#backtest-eligibility)
|
|
7
|
+
- [Spec Shape](#spec-shape)
|
|
8
|
+
- [Execution Model](#execution-model)
|
|
9
|
+
- [BacktestResult](#backtestresult)
|
|
10
|
+
- [Multi-Instrument Replays](#multi-instrument-replays)
|
|
11
|
+
- [Zero-Trade Diagnostics](#zero-trade-diagnostics)
|
|
12
|
+
- [Chart Generation](#chart-generation)
|
|
13
|
+
|
|
14
|
+
## Basic Usage
|
|
15
|
+
|
|
16
|
+
```python
|
|
17
|
+
from getagent import backtest, data, runtime
|
|
18
|
+
|
|
19
|
+
bars = data.crypto.futures.kline(symbol="BTCUSDT", interval="1h", exchange="binance")
|
|
20
|
+
taker_volume = data.crypto.futures.taker_volume(symbol="BTCUSDT", period="1h")
|
|
21
|
+
|
|
22
|
+
replay_frame = backtest.build_feature_frame(
|
|
23
|
+
bars,
|
|
24
|
+
base_datetime_index="date",
|
|
25
|
+
features=[
|
|
26
|
+
backtest.FeatureSource(
|
|
27
|
+
data=taker_volume,
|
|
28
|
+
datetime_index="timestamp",
|
|
29
|
+
include_columns=("buy_vol", "sell_vol"),
|
|
30
|
+
rename_columns={
|
|
31
|
+
"buy_vol": "taker_buy_volume",
|
|
32
|
+
"sell_vol": "taker_sell_volume",
|
|
33
|
+
},
|
|
34
|
+
)
|
|
35
|
+
],
|
|
36
|
+
)
|
|
37
|
+
|
|
38
|
+
result = backtest.run(
|
|
39
|
+
ohlcv_data={"BTCUSDT.BINANCE": replay_frame},
|
|
40
|
+
spec=runtime.backtest_spec,
|
|
41
|
+
)
|
|
42
|
+
chart_path = backtest.generate_chart(result)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Backtest Eligibility
|
|
46
|
+
|
|
47
|
+
Not every Playbook can produce a fair historical backtest.
|
|
48
|
+
|
|
49
|
+
### Use the engine when:
|
|
50
|
+
|
|
51
|
+
- the core trading decision is deterministic
|
|
52
|
+
- the decision can be reconstructed from historical data
|
|
53
|
+
- strategy logic can be expressed as a NautilusTrader `Strategy` subclass
|
|
54
|
+
- any LLM usage is outside the trade decision path
|
|
55
|
+
|
|
56
|
+
### Do not use the engine as public evidence when:
|
|
57
|
+
|
|
58
|
+
- open-ended LLM reasoning decides whether to trade
|
|
59
|
+
- the strategy depends on live chat, memory, or search context
|
|
60
|
+
- the decision cannot be reconstructed at the original point in time
|
|
61
|
+
|
|
62
|
+
Typical outcome:
|
|
63
|
+
|
|
64
|
+
- `backtest_support: full` -> use `backtest.run()` and publish historical metrics
|
|
65
|
+
- `backtest_support: none` -> use paper or live evidence instead
|
|
66
|
+
|
|
67
|
+
## Spec Shape
|
|
68
|
+
|
|
69
|
+
The managed engine expects an explicit Nautilus replay spec from `backtest.yaml`,
|
|
70
|
+
usually read through `runtime.backtest_spec`.
|
|
71
|
+
|
|
72
|
+
Required sections:
|
|
73
|
+
|
|
74
|
+
- `venue`
|
|
75
|
+
- `name`
|
|
76
|
+
- `account_type`
|
|
77
|
+
- `oms_type`
|
|
78
|
+
- `starting_balances`
|
|
79
|
+
- `instrument` or `instruments`
|
|
80
|
+
- `id`
|
|
81
|
+
- `kind`
|
|
82
|
+
- `raw_symbol` or `symbol`
|
|
83
|
+
- `base_currency`
|
|
84
|
+
- `quote_currency`
|
|
85
|
+
- `price_precision`
|
|
86
|
+
- `size_precision`
|
|
87
|
+
- `price_increment`
|
|
88
|
+
- `size_increment`
|
|
89
|
+
- `bar_type`
|
|
90
|
+
- perpetual instruments also need `settlement_currency`
|
|
91
|
+
- `strategy`
|
|
92
|
+
- `module`
|
|
93
|
+
- `class`
|
|
94
|
+
- optional `config_class`
|
|
95
|
+
- optional `config`
|
|
96
|
+
- `execution`
|
|
97
|
+
- optional `start`
|
|
98
|
+
- optional `end`
|
|
99
|
+
|
|
100
|
+
## Execution Model
|
|
101
|
+
|
|
102
|
+
The managed flow is:
|
|
103
|
+
|
|
104
|
+
1. author code fetches historical bars through `getagent.data`
|
|
105
|
+
2. `backtest.run()` normalizes each DataFrame, preserves extra replay columns,
|
|
106
|
+
and derives OHLCV bars for Nautilus
|
|
107
|
+
3. the engine builds Nautilus venue + instrument definitions from `backtest.yaml`
|
|
108
|
+
4. `BarDataWrangler` converts DataFrames into Nautilus bars
|
|
109
|
+
5. the author's `Strategy` subclass is imported from `src/**`
|
|
110
|
+
6. Nautilus `BacktestEngine` runs the replay
|
|
111
|
+
7. the platform normalizes summary metrics and preserves the deeper reports
|
|
112
|
+
|
|
113
|
+
For single-instrument strategies the runner auto-injects these config defaults
|
|
114
|
+
when they are absent:
|
|
115
|
+
|
|
116
|
+
- `instrument_id`
|
|
117
|
+
- `bar_type`
|
|
118
|
+
- `instrument_ids`
|
|
119
|
+
- `bar_types`
|
|
120
|
+
|
|
121
|
+
That means the common author pattern is:
|
|
122
|
+
|
|
123
|
+
- define `DemoStrategyConfig(StrategyConfig)`
|
|
124
|
+
- define `DemoStrategy(Strategy)`
|
|
125
|
+
- let `backtest.yaml` carry the wiring
|
|
126
|
+
|
|
127
|
+
If `backtest.yaml` declares `data_requirements.required_bar_fields`, the engine
|
|
128
|
+
keeps those normalized columns on the replay DataFrames and injects the per-
|
|
129
|
+
instrument frames into the strategy through `set_feature_frames(...)` or
|
|
130
|
+
`strategy.feature_frames` when either hook exists.
|
|
131
|
+
|
|
132
|
+
## BacktestResult
|
|
133
|
+
|
|
134
|
+
|
|
135
|
+
| Property | Type | Description |
|
|
136
|
+
| ------------------------- | ----- | ---------------------------- |
|
|
137
|
+
| `result.total_return_pct` | float | Account return % using the replay starting balance |
|
|
138
|
+
| `result.sharpe_ratio` | float | Sharpe ratio |
|
|
139
|
+
| `result.max_drawdown_pct` | float | Max drawdown % |
|
|
140
|
+
| `result.win_rate` | float | Win rate fraction (0.0-1.0) |
|
|
141
|
+
| `result.total_trades` | int | Total closed position count |
|
|
142
|
+
| `result.profit_factor` | float | Profit factor |
|
|
143
|
+
| `result.summary` | dict | Stable normalized account-level summary |
|
|
144
|
+
| `result.raw` | dict | Full Nautilus-derived output |
|
|
145
|
+
|
|
146
|
+
`result.summary` carries account-level numbers because the engine denominator
|
|
147
|
+
is `backtest.yaml.venue.starting_balances`. **You do not need to convert them
|
|
148
|
+
yourself.** The platform rewrites every persisted run / package metrics block
|
|
149
|
+
to the per-strategy basis (`net_pnl / manifest.strategy_config.margin_budget`)
|
|
150
|
+
before storing or surfacing them on cards / Telegram pushes / run details. The
|
|
151
|
+
account-level numbers are preserved alongside as `account_total_return_pct`
|
|
152
|
+
and `account_max_drawdown_pct` for analysts.
|
|
153
|
+
|
|
154
|
+
This is why `manifest.strategy_config.margin_budget` is **required** for
|
|
155
|
+
`backtest_support: full` and `execution_mode: follow_trade` Playbooks — it is
|
|
156
|
+
the declared denominator. Validation rejects manifests that omit it or set it
|
|
157
|
+
to zero. If you opt out of the rewrite by hand-writing strategy-level metrics,
|
|
158
|
+
also set `metrics_basis: "strategy"` in your emitted `metrics_output`; the
|
|
159
|
+
platform leaves those untouched.
|
|
160
|
+
|
|
161
|
+
`result.raw` keeps these top-level sections:
|
|
162
|
+
|
|
163
|
+
- `summary`
|
|
164
|
+
- `stats`
|
|
165
|
+
- `reports`
|
|
166
|
+
- `config`
|
|
167
|
+
|
|
168
|
+
## Multi-Instrument Replays
|
|
169
|
+
|
|
170
|
+
Use `instruments:` when the strategy needs multiple instrument definitions.
|
|
171
|
+
|
|
172
|
+
Provide one OHLCV DataFrame per declared instrument ID or symbol:
|
|
173
|
+
|
|
174
|
+
```python
|
|
175
|
+
result = backtest.run(
|
|
176
|
+
ohlcv_data={
|
|
177
|
+
"BTCUSDT.BINANCE": btc_df,
|
|
178
|
+
"ETHUSDT.BINANCE": eth_df,
|
|
179
|
+
},
|
|
180
|
+
spec=runtime.backtest_spec,
|
|
181
|
+
)
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
The engine does not synthesize cross-instrument logic for you. Multi-instrument
|
|
185
|
+
behavior belongs in the author's Nautilus strategy code.
|
|
186
|
+
|
|
187
|
+
## Enriched Replay Columns
|
|
188
|
+
|
|
189
|
+
Core OHLCV is still the minimum contract for every replay frame. Extra columns
|
|
190
|
+
such as `quote_volume`, `trade_count`, or `taker_buy_volume` can ride along in
|
|
191
|
+
the same DataFrame.
|
|
192
|
+
|
|
193
|
+
The standard author pattern is:
|
|
194
|
+
|
|
195
|
+
1. fetch the base bar series from `getagent.data`
|
|
196
|
+
2. fetch any auxiliary datasets from any supported `getagent.data` endpoints
|
|
197
|
+
3. aggregate finer-grain feeds yourself when needed (for example raw trades -> hourly features)
|
|
198
|
+
4. use `backtest.build_feature_frame(...)` to align those datasets onto the base bars
|
|
199
|
+
5. declare the resulting feature column names in `backtest.yaml`
|
|
200
|
+
|
|
201
|
+
Declare them explicitly in `backtest.yaml`:
|
|
202
|
+
|
|
203
|
+
```yaml
|
|
204
|
+
data_requirements:
|
|
205
|
+
required_bar_fields:
|
|
206
|
+
- quote_volume
|
|
207
|
+
- trade_count
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Then read them from the injected feature frames in your strategy:
|
|
211
|
+
|
|
212
|
+
```python
|
|
213
|
+
class PriceVolStrategy(Strategy):
|
|
214
|
+
def set_feature_frames(self, feature_frames):
|
|
215
|
+
self.feature_frames = feature_frames
|
|
216
|
+
|
|
217
|
+
def on_start(self):
|
|
218
|
+
primary = self.feature_frames["BTCUSDT.BINANCE"]
|
|
219
|
+
latest_quote_volume = primary.iloc[-1]["quote_volume"]
|
|
220
|
+
# ...
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
For example, if author code builds a replay frame with
|
|
224
|
+
`taker_buy_volume`, `taker_sell_volume`, and `open_interest`, those exact
|
|
225
|
+
normalized names should appear under `required_bar_fields`.
|
|
226
|
+
|
|
227
|
+
If a field declared in `data_requirements.required_bar_fields` is missing from a
|
|
228
|
+
replay DataFrame, or exists but is entirely null inside the effective execution
|
|
229
|
+
window, `backtest.run()` fails before Nautilus starts.
|
|
230
|
+
|
|
231
|
+
## Zero-Trade Diagnostics
|
|
232
|
+
|
|
233
|
+
If `result.total_trades == 0`, common causes are:
|
|
234
|
+
|
|
235
|
+
1. bars were filtered out by `execution.start` / `execution.end`
|
|
236
|
+
2. the strategy never subscribed to the declared `bar_type`
|
|
237
|
+
3. instrument IDs in strategy config do not match the spec
|
|
238
|
+
4. order sizing is invalid for the declared increment / lot size
|
|
239
|
+
|
|
240
|
+
Inspect:
|
|
241
|
+
|
|
242
|
+
- `result.raw["config"]`
|
|
243
|
+
- `result.raw["reports"]["orders"]`
|
|
244
|
+
- `result.raw["reports"]["fills"]`
|
|
245
|
+
- `result.raw["reports"]["positions"]`
|
|
246
|
+
|
|
247
|
+
## Chart Generation
|
|
248
|
+
|
|
249
|
+
```python
|
|
250
|
+
chart_path = backtest.generate_chart(result)
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
The managed chart renderer writes a PNG summary to `/workspace/output/`.
|
|
254
|
+
|
|
255
|
+
Current chart includes:
|
|
256
|
+
|
|
257
|
+
- equity curve when available
|
|
258
|
+
- normalized metrics when no curve is available
|
|
259
|
+
- key summary values
|
|
260
|
+
# Backtest Engine
|
|
261
|
+
|
|
262
|
+
## Contents
|
|
263
|
+
|
|
264
|
+
- [Basic Usage](#basic-usage)
|
|
265
|
+
- [Backtest Eligibility](#backtest-eligibility)
|
|
266
|
+
- [Spec Shape](#spec-shape)
|
|
267
|
+
- [Execution Model](#execution-model)
|
|
268
|
+
- [BacktestResult](#backtestresult)
|
|
269
|
+
- [Multi-Instrument Replays](#multi-instrument-replays)
|
|
270
|
+
- [Zero-Trade Diagnostics](#zero-trade-diagnostics)
|
|
271
|
+
- [Chart Generation](#chart-generation)
|
|
272
|
+
|
|
273
|
+
## Basic Usage
|
|
274
|
+
|
|
275
|
+
```python
|
|
276
|
+
from getagent import backtest, data, runtime
|
|
277
|
+
|
|
278
|
+
bars = data.crypto.futures.kline(symbol="BTCUSDT", interval="1h", exchange="binance")
|
|
279
|
+
ohlcv = data.to_dataframe(bars, datetime_index="date")
|
|
280
|
+
|
|
281
|
+
result = backtest.run(
|
|
282
|
+
ohlcv_data={"BTCUSDT.BINANCE": ohlcv},
|
|
283
|
+
spec=runtime.backtest_spec,
|
|
284
|
+
)
|
|
285
|
+
chart_path = backtest.generate_chart(result)
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
## Backtest Eligibility
|
|
289
|
+
|
|
290
|
+
Not every Playbook can produce a fair historical backtest.
|
|
291
|
+
|
|
292
|
+
### Use the engine when:
|
|
293
|
+
|
|
294
|
+
- the core trading decision is deterministic
|
|
295
|
+
- the decision can be reconstructed from historical data
|
|
296
|
+
- strategy logic can be expressed as a NautilusTrader `Strategy` subclass
|
|
297
|
+
- any LLM usage is outside the trade decision path
|
|
298
|
+
|
|
299
|
+
### Do not use the engine as public evidence when:
|
|
300
|
+
|
|
301
|
+
- open-ended LLM reasoning decides whether to trade
|
|
302
|
+
- the strategy depends on live chat, memory, or search context
|
|
303
|
+
- the decision cannot be reconstructed at the original point in time
|
|
304
|
+
|
|
305
|
+
Typical outcome:
|
|
306
|
+
|
|
307
|
+
- `backtest_support: full` -> use `backtest.run()` and publish historical metrics
|
|
308
|
+
- `backtest_support: none` -> use paper or live evidence instead
|
|
309
|
+
|
|
310
|
+
## Spec Shape
|
|
311
|
+
|
|
312
|
+
The managed engine expects an explicit Nautilus replay spec from `backtest.yaml`,
|
|
313
|
+
usually read through `runtime.backtest_spec`.
|
|
314
|
+
|
|
315
|
+
Required sections:
|
|
316
|
+
|
|
317
|
+
- `venue`
|
|
318
|
+
- `name`
|
|
319
|
+
- `account_type`
|
|
320
|
+
- `oms_type`
|
|
321
|
+
- `starting_balances`
|
|
322
|
+
- `instrument` or `instruments`
|
|
323
|
+
- `id`
|
|
324
|
+
- `kind`
|
|
325
|
+
- `raw_symbol` or `symbol`
|
|
326
|
+
- `base_currency`
|
|
327
|
+
- `quote_currency`
|
|
328
|
+
- `price_precision`
|
|
329
|
+
- `size_precision`
|
|
330
|
+
- `price_increment`
|
|
331
|
+
- `size_increment`
|
|
332
|
+
- `bar_type`
|
|
333
|
+
- perpetual instruments also need `settlement_currency`
|
|
334
|
+
- `strategy`
|
|
335
|
+
- `module`
|
|
336
|
+
- `class`
|
|
337
|
+
- optional `config_class`
|
|
338
|
+
- optional `config`
|
|
339
|
+
- `execution`
|
|
340
|
+
- optional `start`
|
|
341
|
+
- optional `end`
|
|
342
|
+
|
|
343
|
+
## Execution Model
|
|
344
|
+
|
|
345
|
+
The managed flow is:
|
|
346
|
+
|
|
347
|
+
1. author code fetches historical bars through `getagent.data`
|
|
348
|
+
2. `backtest.run()` normalizes each DataFrame into OHLCV bars
|
|
349
|
+
3. the engine builds Nautilus venue + instrument definitions from `backtest.yaml`
|
|
350
|
+
4. `BarDataWrangler` converts DataFrames into Nautilus bars
|
|
351
|
+
5. the author's `Strategy` subclass is imported from `src/**`
|
|
352
|
+
6. Nautilus `BacktestEngine` runs the replay
|
|
353
|
+
7. the platform normalizes summary metrics and preserves the deeper reports
|
|
354
|
+
|
|
355
|
+
For single-instrument strategies the runner auto-injects these config defaults
|
|
356
|
+
when they are absent:
|
|
357
|
+
|
|
358
|
+
- `instrument_id`
|
|
359
|
+
- `bar_type`
|
|
360
|
+
- `instrument_ids`
|
|
361
|
+
- `bar_types`
|
|
362
|
+
|
|
363
|
+
That means the common author pattern is:
|
|
364
|
+
|
|
365
|
+
- define `DemoStrategyConfig(StrategyConfig)`
|
|
366
|
+
- define `DemoStrategy(Strategy)`
|
|
367
|
+
- let `backtest.yaml` carry the wiring
|
|
368
|
+
|
|
369
|
+
## BacktestResult
|
|
370
|
+
|
|
371
|
+
|
|
372
|
+
| Property | Type | Description |
|
|
373
|
+
| ------------------------- | ----- | ---------------------------- |
|
|
374
|
+
| `result.total_return_pct` | float | Account return % using the replay starting balance |
|
|
375
|
+
| `result.sharpe_ratio` | float | Sharpe ratio |
|
|
376
|
+
| `result.max_drawdown_pct` | float | Max drawdown % |
|
|
377
|
+
| `result.win_rate` | float | Win rate fraction (0.0-1.0) |
|
|
378
|
+
| `result.total_trades` | int | Total closed position count |
|
|
379
|
+
| `result.profit_factor` | float | Profit factor |
|
|
380
|
+
| `result.summary` | dict | Stable normalized account-level summary |
|
|
381
|
+
| `result.raw` | dict | Full Nautilus-derived output |
|
|
382
|
+
|
|
383
|
+
|
|
384
|
+
`result.raw` keeps these top-level sections:
|
|
385
|
+
|
|
386
|
+
- `summary`
|
|
387
|
+
- `stats`
|
|
388
|
+
- `reports`
|
|
389
|
+
- `config`
|
|
390
|
+
|
|
391
|
+
## Multi-Instrument Replays
|
|
392
|
+
|
|
393
|
+
Use `instruments:` when the strategy needs multiple instrument definitions.
|
|
394
|
+
|
|
395
|
+
Provide one OHLCV DataFrame per declared instrument ID or symbol:
|
|
396
|
+
|
|
397
|
+
```python
|
|
398
|
+
result = backtest.run(
|
|
399
|
+
ohlcv_data={
|
|
400
|
+
"BTCUSDT.BINANCE": btc_df,
|
|
401
|
+
"ETHUSDT.BINANCE": eth_df,
|
|
402
|
+
},
|
|
403
|
+
spec=runtime.backtest_spec,
|
|
404
|
+
)
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
The engine does not synthesize cross-instrument logic for you. Multi-instrument
|
|
408
|
+
behavior belongs in the author's Nautilus strategy code.
|
|
409
|
+
|
|
410
|
+
## Zero-Trade Diagnostics
|
|
411
|
+
|
|
412
|
+
If `result.total_trades == 0`, common causes are:
|
|
413
|
+
|
|
414
|
+
1. bars were filtered out by `execution.start` / `execution.end`
|
|
415
|
+
2. the strategy never subscribed to the declared `bar_type`
|
|
416
|
+
3. instrument IDs in strategy config do not match the spec
|
|
417
|
+
4. order sizing is invalid for the declared increment / lot size
|
|
418
|
+
|
|
419
|
+
Inspect:
|
|
420
|
+
|
|
421
|
+
- `result.raw["config"]`
|
|
422
|
+
- `result.raw["reports"]["orders"]`
|
|
423
|
+
- `result.raw["reports"]["fills"]`
|
|
424
|
+
- `result.raw["reports"]["positions"]`
|
|
425
|
+
|
|
426
|
+
## Chart Generation
|
|
427
|
+
|
|
428
|
+
```python
|
|
429
|
+
chart_path = backtest.generate_chart(result)
|
|
430
|
+
```
|
|
431
|
+
|
|
432
|
+
The managed chart renderer writes a PNG summary to `/workspace/output/`.
|
|
433
|
+
|
|
434
|
+
Current chart includes:
|
|
435
|
+
|
|
436
|
+
- equity curve when available
|
|
437
|
+
- normalized metrics when no curve is available
|
|
438
|
+
- key summary values
|