opencode-pine2pyne 0.2.2__py3-none-any.whl → 0.2.3__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,343 @@
1
+ Metadata-Version: 2.4
2
+ Name: opencode-pine2pyne
3
+ Version: 0.2.3
4
+ Summary: Pine Script v6 to PyneCore Python transpiler
5
+ Author: rubycell
6
+ License-Expression: Apache-2.0
7
+ Project-URL: Homepage, https://github.com/rubycell/pine2pyne
8
+ Project-URL: Repository, https://github.com/rubycell/pine2pyne
9
+ Keywords: pinescript,tradingview,transpiler,pynecore,backtesting
10
+ Classifier: Programming Language :: Python :: 3.11
11
+ Classifier: Programming Language :: Python :: 3.12
12
+ Classifier: Programming Language :: Python :: 3.13
13
+ Classifier: Topic :: Office/Business :: Financial :: Investment
14
+ Requires-Python: >=3.11
15
+ Description-Content-Type: text/markdown
16
+ License-File: LICENSE
17
+ Provides-Extra: runtime
18
+ Requires-Dist: opencode-pyneruntime[cli]>=6.9.1; extra == "runtime"
19
+ Dynamic: license-file
20
+
21
+ # pine2pyne - Pine Script to PyneCore transpiler
22
+
23
+ Snapshot written 2026-10-19 at origin/main `dd8b5d3` (branch `card-118-local-named-like-method`, clean apart from this README.md).
24
+ The old README is kept at `/home/mike/workspace/github/pine2pyne/backup/README.before-2026-10-19.md` (git-ignored).
25
+ Each fact below names its source file. A fact marked UNVERIFIED was not checked in this snapshot.
26
+
27
+ Backlog and known bugs: [Projects board #10](https://github.com/users/rubycell/projects/10)
28
+ (Backlog = open, P0/P1/P2; Done = shipped). The board page itself was not opened in this snapshot: UNVERIFIED.
29
+
30
+ ## 1. What this repo is
31
+
32
+ - A transpiler. It reads a TradingView Pine Script file (`.pine`) and writes a PyneCore Python file (`.py`).
33
+ Source: `pine2pyne/__init__.py` (`transpile()`), `pyproject.toml` (description "Pine Script v6 to PyneCore Python transpiler").
34
+ - Users: the strategy developers of this workspace. They transpile a strategy, then run or optimize it with the `pyne` CLI
35
+ of the PyneCore runtime (repo `/home/mike/workspace/github/pynecore`).
36
+ - Where it runs: a command-line tool and a Python library. It has no service, no port and no host of its own.
37
+ Development happens on mike-PC. Full test suites run on Minh_pc (see section 7).
38
+ - Package: `opencode-pine2pyne` on PyPI, version `0.2.3` in `pine2pyne/__init__.py` and `CHANGELOG.md`.
39
+ Live PyPI state is UNVERIFIED: 0.2.3 is prepared (commit and local tag `v0.2.3`), not uploaded.
40
+ `CHANGELOG.md` lists #115 to #118 under 0.2.3; the tag `v0.2.2` is the release before them.
41
+ - The transpiler imports only the standard library (`pyproject.toml`: `dependencies = []`).
42
+ The generated script needs the runtime `opencode-pyneruntime` (`runtime` extra, `>=6.9.1`).
43
+ - License: Apache-2.0 (`pyproject.toml`, `LICENSE`).
44
+
45
+ ## 2. Architecture
46
+
47
+ The pipeline is in `transpile()` in `/home/mike/workspace/github/pine2pyne/pine2pyne/__init__.py`:
48
+
49
+ ```mermaid
50
+ flowchart LR
51
+ A[.pine source] --> B[parse_source<br/>lexer + parser + parse_repair]
52
+ B --> C[qualify_legacy_function_names<br/>v1-v4 names to v5 names]
53
+ C --> D[enforce_pine_semantics<br/>refuse what TradingView refuses]
54
+ D --> E[Transformer<br/>Pine AST to PyneCore AST]
55
+ E --> F[CodeGenerator]
56
+ F --> G[.py script]
57
+ G --> H[pyne run / pyne optimize<br/>PyneCore runtime]
58
+ ```
59
+
60
+ - Lowering passes run in the parser and in the transformer. They rewrite a Pine construct into plain
61
+ statements. Examples: `once`, `when=` of a strategy call (v1-v5), grouped history `(a + b)[1]`,
62
+ block values of `if`/`switch`, strict `and`/`or` for Pine v5 and earlier.
63
+ Sources: imports in `pine2pyne/parser.py` and `pine2pyne/transformer.py`.
64
+ - Two tables are generated from an installed runtime: `pine2pyne/namespace_members.py` and `pine2pyne/bool_builtins.py`
65
+ (both say "GENERATED ... from opencode-pyneruntime 6.9.4; do not edit by hand").
66
+ Regenerate them after a runtime upgrade with `tools/collect_namespace_members.py` and `tools/collect_bool_builtins.py`.
67
+ - The transpiler refuses a script when TradingView refuses it (`SemanticError`, card #28) and warns with
68
+ `SemanticWarning` when it only advises. `pyproject.toml` turns an unexpected `SemanticWarning` into a test failure.
69
+ - Runtime code placement rule: group a new runtime member beside its kin. Source: `CLAUDE.md`, "Runtime Code Placement".
70
+ - Three separate test systems check it (section 7). A green run of one says nothing about the others.
71
+ Source: `CLAUDE.md`.
72
+
73
+ ## 3. Modules
74
+
75
+ All in `/home/mike/workspace/github/pine2pyne/pine2pyne/` (33 `.py` files, 13,176 lines in total by `wc -l`).
76
+
77
+ Core pipeline:
78
+ - `lexer.py`, `tokens.py` - tokens, with indentation tracking.
79
+ - `parser.py`, `ast_nodes.py`, `ast_walk.py` - recursive descent parser, AST node classes, shared child walk.
80
+ - `parse_repair.py` - repairs a rejected known spelling of valid Pine (card #55).
81
+ - `legacy_function_names.py` - Pine v1-v4 call names to v5 names (card #116).
82
+ - `pine_semantics.py` - checks TradingView's compiler makes that the grammar does not (card #28).
83
+ - `transformer.py` - Pine AST to PyneCore AST. The largest file.
84
+ - `codegen.py` - emits Python source.
85
+ - `cli.py`, `__main__.py` - command line (`pine2pyne` command and `python -m pine2pyne`).
86
+ - `errors.py` - error types with source locations.
87
+
88
+ Support tables and typing:
89
+ - `pine_builtins.py` - maps Pine built-ins to PyneCore.
90
+ - `namespace_members.py`, `bool_builtins.py` - generated runtime tables.
91
+ - `symbol_table.py`, `type_inference.py`, `bool_value_typing.py` - scopes, Series or Persistent typing, bool typing.
92
+ - `import_resolver.py`, `emitted_names.py` - import lines; emitted names that must not collide.
93
+
94
+ Lowering and refusal passes:
95
+ - `scope_renamer.py` - gives a shadowing local, or a binding named like a runtime name, a name of its own.
96
+ - `library_collision.py` - a user binding named like a `pynecore.lib` name.
97
+ - `overload_resolution.py` - same-name functions and methods.
98
+ - `once_lowering.py`, `strategy_when_lowering.py`, `block_value_lowering.py`, `grouped_history_lowering.py`.
99
+ - `for_loop_end_bound.py`, `strict_boolean.py`, `udt_field_defaults.py`, `conditional_history_calls.py`.
100
+
101
+ Other top-level folders:
102
+ - `tests/` - 86 `test_*.py` files, fixtures in `tests/fixtures/`, old strategy outputs in `tests/ground_truth/`.
103
+ - `tools/` - corpus harnesses, ground-truth tools, table generators (section 7).
104
+ - `sample/pinescript/` (413 `.pine` files) and `sample_pds/` (331 `.pine` files) - the corpus. Counts by `ls | wc -l`.
105
+ - `sandbox/` - Docker image (section 6).
106
+ - `docs/` - `test_plan.md`, `instructions/` (optimizer guides), `misc/`, `code_review/`.
107
+ - `ground_truth/` - tracked baseline: `digests.json` and `manifest.json` only.
108
+ - `other_transpiler/`, `runs/`, `scripts/`, `strategy/`, `upstream-run/`, `agents/`, `.claude/agentandskill/` - strategy
109
+ experiments and agent prompts. Their use today is UNVERIFIED.
110
+ - `workdir/` - git-ignored scratch folder for runs (`.gitignore`). `backup/` and `dist/` are git-ignored too.
111
+ - `pine2pyne/README.md` is the package's own README (set as package data in `pyproject.toml`).
112
+ It may be stale (it says "transformer.py (1626 lines)"): UNVERIFIED.
113
+
114
+ ## 4. Services
115
+
116
+ None. The repo starts no long-running process: no systemd unit, no cron job, no listening port.
117
+ Source: no unit, crontab installer or server entry point found in the repo; `pyproject.toml` has one console script,
118
+ `pine2pyne = "pine2pyne.cli:main"`. Distributed optimizer runs start remote `pyne optimize` processes over SSH
119
+ from tools that live in another repo (section 7).
120
+
121
+ ## 5. Dashboard and its features
122
+
123
+ None. This repo has no web UI. Source: no web framework or HTML page in the repo.
124
+
125
+ ## 6. Docker
126
+
127
+ - One Dockerfile: `/home/mike/workspace/github/pine2pyne/sandbox/Dockerfile`. Base image `python:3.13-slim`.
128
+ Build arg `SOURCE=pypi` (default) installs `opencode-pine2pyne` and `opencode-pyneruntime[cli]>=6.9.1` from PyPI.
129
+ `SOURCE=local` installs the wheels in `sandbox/wheels/`.
130
+ The user is `pyne`, the work folder is `/work`, and the default command prints `pine2pyne --version`, then opens a shell.
131
+ - Suggested image name in `sandbox/README.md`: `pyne-sandbox`. Run with `-v "$PWD/workdir:/work"`.
132
+ `sandbox/smoke.sh <data.ohlcv>` transpiles a small strategy and runs `pyne run` on it.
133
+ - `sandbox/wheels/` is stale: it holds `opencode_pine2pyne-0.1.0` and `opencode_pyneruntime-6.6.4` (file names, dated 2026-07-31).
134
+ `git ls-files sandbox/wheels` lists nothing: the wheels are not tracked.
135
+ - No compose file, no ports, no volumes other than the `/work` mount in `sandbox/README.md`.
136
+ - Live state, read only, 2026-10-19. `ssh -o BatchMode=yes minh@minhpc 'docker ps -a; docker images'` and
137
+ `docker ps -a; docker images` on mike-PC show no `pyne-sandbox` image and no pine2pyne container.
138
+ The containers seen there belong to other projects (`vnstock-openclaw-trial-*`, `botrehearsal-trial-*`, `svb-sandbox:*` images).
139
+ Result: the sandbox image is not built on either host.
140
+
141
+ ## 7. Commands
142
+
143
+ Run all commands from the repo root with the venv active. Paths and options come from `tools/*.py` and `pine2pyne/cli.py`.
144
+
145
+ Set up:
146
+ ```bash
147
+ bash /home/mike/.claude/skills/opencode-pine2pyne-setup/setup.sh # skill script: builds .venv, seeds demo data, smoke test
148
+ # or by hand:
149
+ python3 -m venv .venv
150
+ .venv/bin/pip install -e .
151
+ .venv/bin/pip install 'opencode-pyneruntime[cli]>=6.9.1' # same floor as pyproject.toml [runtime]
152
+ .venv/bin/pip install optuna numpy # optimizer deps, not pulled in by the runtime
153
+ source .venv/bin/activate # Linux and macOS; on Windows: .venv\Scripts\activate
154
+ .venv/bin/python -c "import pynecore, pine2pyne, optuna, numpy; print('all ok')"
155
+ ```
156
+ The setup script ends with the lines `smoke test : PASS` and `READY`. A failure prints `READY : NO` and exits 1
157
+ (`setup.sh` lines 22, 31 and 111-112). It writes `.sandbox-env.sh` with `PINE_CPU_COUNT` and `PINE_MAX_WORKERS`
158
+ (source: `/home/mike/.claude/skills/opencode-pine2pyne-setup/setup.sh`; the file is git-ignored).
159
+ The `runtime` extra also works: `pip install "opencode-pine2pyne[runtime]"` (`pyproject.toml`).
160
+ Test tools (pytest, hypothesis) are in no manifest: a venv built from this README cannot run the unit suite
161
+ (project memory "Test deps undeclared"; not re-checked, UNVERIFIED).
162
+
163
+ Transpile (`pine2pyne/cli.py`, options `-o`, `--validate`, `-v`, `--version`):
164
+ ```bash
165
+ python -m pine2pyne input.pine # to stdout
166
+ python -m pine2pyne input.pine -o output.py
167
+ python -m pine2pyne "sample/pinescript/*.pine" -o workdir/scripts/ # batch into a folder
168
+ python -m pine2pyne strategy.pine --validate # parse and check only
169
+ ```
170
+
171
+ Run and optimize (the `pyne` CLI of the runtime; from `workdir/`):
172
+ ```bash
173
+ pyne run strategy.py data/data.ohlcv
174
+ pyne optimize strategy.py data/data.ohlcv optimize.json -n 20
175
+ ```
176
+ Optimizer guides: `docs/instructions/Optimizer.md` (the root `Optimizer.md` was deleted in commit `cde04c9`;
177
+ old links to it are dead).
178
+
179
+ Test. Where each run goes: `/home/mike/.claude/where-tests-run.md`. Every full suite runs on Minh_pc through the
180
+ `qa-remote-suite` skill. A batch that checks this tool across many strategies is a test, so it runs on Minh_pc too.
181
+ On mike-PC only a targeted run is allowed, through `mike-pc-test-run`.
182
+
183
+ There are three test systems. A green run of one says nothing about the others.
184
+
185
+ 1. Unit tests, pytest, `tests/` (`pyproject.toml`: `testpaths = ["tests"]`):
186
+ ```bash
187
+ .venv/bin/python -m pytest tests/ -q
188
+ .venv/bin/python -m pytest tests/ --cov=pine2pyne --cov-report=term-missing
189
+ .venv/bin/python -m pytest tests/ -m 'not corpus_runtime' -q # skip tests that run the runtime
190
+ .venv/bin/python -m pytest tests/ -m 'not release_artifact' -q # skip the wheel build test (needs uv)
191
+ ```
192
+ - No `--ignore` is needed (`CLAUDE.md`; the screener test was retired in card #106).
193
+ - Last measured numbers: 6,870 tests, 6,848 passed, 1 skipped, 21 xfailed, 93% line and 86% branch coverage,
194
+ at `a16621b` (old README). Not re-run in this snapshot: UNVERIFIED. `docs/test_plan.md` holds older numbers
195
+ (2062 passed, 93.3% line, 86.6% branch) and the mutation results.
196
+ - Strict xfail marks a pinned, unfixed bug. Do not delete the marker to get green. When the bug is fixed the
197
+ test XPASSes, strict turns that into a failure, and you then convert the test to a regression assertion.
198
+ Source: `CLAUDE.md` ("6 sites (21 test cases)" there; not re-counted, UNVERIFIED).
199
+ - Guard for the operator-grouping bug (card #6): `tests/test_codegen_operator_grouping.py`.
200
+
201
+ 2. Sample suites, end-to-end transpile and `pyne run` over the corpus (`tools/test_all_samples.py`,
202
+ `tools/test_all_sample_pds.py`, shared code in `tools/_suite_common.py`):
203
+ ```bash
204
+ python tools/test_all_samples.py # sample/pinescript
205
+ python tools/test_all_sample_pds.py # sample_pds
206
+ python tools/test_all_samples.py "ex_001*" # filter by pattern
207
+ python tools/test_all_samples.py --timeout 30 --verbose -j 0
208
+ python tools/test_all_samples.py --run-dir /tmp/run1 # private scripts/, output/ and reports (card #70)
209
+ ```
210
+ - Defaults: data `workdir/data/demo.ohlcv`, timeout 15 s, `-j` 24 (`tools/test_all_samples.py`).
211
+ Reports go to `test_results{,_pds}.{json,txt}` at the repo root (git-ignored), or into `--run-dir`.
212
+ - Latest local reports, 2026-10-02 08:45 (`test_results.json`, `test_results_pds.json`; local files, not tracked):
213
+ - `sample/pinescript`: 411 run, 2 skipped, 411 transpiled, 393 passed, 18 failed
214
+ (11 `pynecore_api`, 5 `missing_security_data`, 2 `type_error`).
215
+ - `sample_pds`: 328 run, 3 skipped, 319 transpiled, 322 passed, 6 failed (all `missing_security_data`).
216
+ Of the 322 passes, 304 ran clean and 18 are expected-error samples (`expect_error` in
217
+ `tools/sample_expectations.toml`): 9 `pine_refusal` (transpile refused, so not in the 319),
218
+ 5 `index_error` and 4 `value_error`. The report's `runtime_failed` is 15 = 6 failures + 5 + 4 expected-error passes.
219
+ (As of 2026-10-20 `expds_160` is no longer declared: TradingView does not halt on it, measured. The declared
220
+ `index_error` samples are 4 now; the report above was made before that.)
221
+ Counted from the `results` list of `test_results_pds.json` (passed, transpile, runtime, error_type per sample).
222
+ - `CLAUDE.md` holds the same numbers (table "Converting Pine Script to PyneCore Python"). Its old 2026-08-17
223
+ figures (391/411, 304/328) were replaced on 2026-10-19.
224
+ - Per-sample rules are in `tools/sample_expectations.toml` (25 sections). Keys: `expect_error` (must fail with that
225
+ category; running clean is a FAIL), `requires_feeds` (extra feeds, passed as `--security TF=name`), `runs_clean`,
226
+ `card`, `refusal`, `why`. Pytest validates the file (`tests/test_error_demo_claims.py`).
227
+ - A feed for `requires_feeds` is made with `cd workdir && pyne data aggregate data/demo.ohlcv -tf 2D`.
228
+
229
+ 3. Ground truth, numeric regression (`tools/generate_ground_truth.py`, `tools/compare_ground_truth.py`,
230
+ `tools/ground_truth_check.py`, shared code in `tools/_ground_truth_common.py`):
231
+ ```bash
232
+ python tools/generate_ground_truth.py # (re)build the baseline; only when the new output is confirmed correct
233
+ python tools/compare_ground_truth.py # compare current output with the baseline
234
+ python tools/compare_ground_truth.py --suite sample_pds -j 12
235
+ python tools/ground_truth_check.py capture # then: compare. Local check, writes only ground_truth_check/ (git-ignored)
236
+ ```
237
+ - `capture --force` moves the old `ground_truth_check/` to `backup/deleteable/ground_truth_check_<time>/`;
238
+ it deletes nothing (`tools/ground_truth_check.py` lines 27-28, 201-206).
239
+ - Baseline: `ground_truth/digests.json` (sha256 and line count per output file, plus the data sha256) and
240
+ `ground_truth/manifest.json`. The output CSVs (about 920 MB) are git-ignored (`.gitignore`, card #62).
241
+ - Current baseline (`manifest.json`, generated 2026-10-20 on pynecore `ae6f03172d6127e88c6ae4f2ced61f0c93a6a331`):
242
+ 562 tested, 551 passed, 9 transpile failures, 2 runtime failures, 0 timeouts, 1,637 output files.
243
+ `digests.json` has 551 samples. The 2026-10-03 baseline (550 passed, 1,634 files) matched no current runtime;
244
+ it was made on a pynecore side branch (`88741ade6092`, checked on `ex_001` only).
245
+ - `pynecore_commit` (in `digests.json` and `manifest.json`): the full sha of the pynecore checkout that the `pyne`
246
+ in use imports, written by `generate_ground_truth.py`. `-dirty` is added when a tracked file under that
247
+ checkout's `src/` differs; the value is `null` when the runtime is not a pynecore checkout (`src/pynecore` of a
248
+ git work tree). `compare_ground_truth.py` prints the baseline and the runtime commit and WARNS when they differ,
249
+ or when either is `-dirty` or unknown; it never refuses for this reason.
250
+ - Data: `workdir/data/VN30F1M_15m.ohlcv`, data sha256 `e9ef984b09eaa6d2...` (`digests.json`).
251
+ Capital 10,000,000,000 and timeout 120 s per sample (`tools/_ground_truth_common.py`).
252
+ Why this capital: at 1,000,000 one VN30F1M contract costs more than the equity, so nearly every strategy
253
+ records zero trades (card #57; `tools/_ground_truth_common.py` lines 14-18).
254
+ - Exit codes of compare: 0 = every recorded sample verified (skipped nondeterministic files are listed, not judged);
255
+ 1 = digests missing or empty, a regression, a new failure, an unverified recorded sample or a stale mark;
256
+ 2 = `DATA CHANGED` (the data file is not the one of the baseline).
257
+ - A `REGRESSION` line names the sample and the file (`tools/compare_ground_truth.py` line 271). To see what changed,
258
+ diff that sample's CSV in the run root against the baseline copy in `ground_truth/<suite>/<sample>/`.
259
+ - `--suite sample` or `--suite sample_pds` rebuilds one suite and keeps the other suite's digests. It refuses
260
+ (exit 1) unless a baseline made on the same data AND the same known `pynecore_commit` exists
261
+ (`tools/generate_ground_truth.py` lines 250-258): one commit must describe the whole file.
262
+ - Before a new baseline is written, the old CSV folder moves to `backup/deleteable/ground_truth_csv_<time>/`;
263
+ nothing is deleted (`tools/generate_ground_truth.py` lines 260-267).
264
+ - Each sample runs in its own folder under `backup/deleteable/ground_truth_runs/<tool>-<time>/` (about 0.9 GB each).
265
+ No tool deletes them. Move old ones to the backup disk by hand, never `rm` (card #70, old README).
266
+ - Nondeterministic plot files: 11 `sample_pds` samples call an unseeded `math.random()` in a plot, so the digest of
267
+ their plot csv `<name>.csv` changes on every run (expds_022, 037, 039, 043, 045, 081, 091, 092, 104, 114, 115).
268
+ `manifest.json` marks exactly those files with `nondeterministic_files` on the sample entry. The list of the 11
269
+ samples is one tuple, `NONDETERMINISTIC_PLOT_SAMPLES` in `tools/_ground_truth_common.py`, and
270
+ `generate_ground_truth.py` writes the marks (a full run, or `--mark-nondeterministic-only`, which changes
271
+ only the manifest and runs no sample). Do not edit the marks by hand. `compare_ground_truth.py` skips only the
272
+ marked files, prints each one under `SKIPPED NONDETERMINISTIC FILES (N)`, and still checks every other file of
273
+ those samples (`_strat.csv`, `_trade.csv`, `_plot_meta.json`). A mark that no longer holds (the file matches
274
+ its digest, or the baseline records no such file) is listed under `STALE NONDETERMINISTIC MARKS` and fails the
275
+ run: regenerate the marks. A new sample with an unseeded `math.random()` needs its key in the tuple.
276
+ Checked 2026-10-20 on pynecore `ae6f03172`: two compares in a row each exited 0 (551 OK, 0 REGRESSION, 11 files skipped).
277
+ - The old shell wrapper was retired in card #66. `tools/ground_truth_check.py capture` then `compare` replace it.
278
+
279
+ Other tools in `tools/`: `corpus_transpile_ab.py --base <git ref>` (A/B of the transpiled corpus against another revision),
280
+ `_run_layout.py`, `_suite_report_compare.py`, `probes/`.
281
+
282
+ Test data. `workdir/` is git-ignored, so a fresh clone has no data. Both data files were checked on 2026-10-19:
283
+ ```bash
284
+ mkdir -p workdir/data
285
+ git show 227f3d2^:workdir/data/demo.ohlcv > workdir/data/demo.ohlcv # sha256 starts 67e827cc1a5f71ed
286
+ cp tests/fixtures/data/demo.toml workdir/data/demo.toml
287
+ (cd workdir && pyne data aggregate data/demo.ohlcv -tf 2D && pyne data aggregate data/demo.ohlcv -tf 5D)
288
+ git show 5d63d21:workdir/data/VN30F1M_15m.ohlcv > workdir/data/VN30F1M_15m.ohlcv # sha256 starts e9ef984b09eaa6d2
289
+ cp tests/fixtures/data/VN30F1M_15m.toml workdir/data/VN30F1M_15m.toml
290
+ ```
291
+ Take the `.toml` files from `tests/fixtures/data/`, not from the old commits: the old ones number weekdays 1..7,
292
+ and pynecore rejects day 7 (0 = Monday ... 6 = Sunday). Source: old README, project memory "pyne runtime gotchas".
293
+ The `/opencode-pine2pyne-setup` skill seeds a small synthetic fixture instead. That is fine for a smoke test, but the
294
+ results do not match the recorded numbers.
295
+
296
+ Lint: no linter config found in `pyproject.toml`. UNVERIFIED for any other file.
297
+
298
+ Deploy (release to PyPI): use the skill `/home/mike/.claude/skills/pypi-publish/SKILL.md`. The repo has no CI file
299
+ and no Makefile. `CHANGELOG.md` is the release log; add the entry for #115 to #118 at the next release.
300
+
301
+ Distributed optimization. `CLAUDE.md` and the old README name `tools/pyne-dynamic.sh`, `tools/pyne-parallel.sh` and
302
+ `tools/machines.txt`. **These files are not in this repo.** They moved out in commit `227f3d2`
303
+ ("move workdir/ and tools/ out to the pinescript workbench"). They exist at
304
+ `/home/mike/workspace/github/pinescript/tools/pyne-dynamic.sh`, `.../pyne-parallel.sh` and `.../machines.txt`.
305
+ Use those paths. That they run unchanged against this repo is UNVERIFIED. The tool options are in `CLAUDE.md`
306
+ ("Distributed Optimization") and `docs/misc/DynamicJobQueue.md`. `CLAUDE.md` also holds the cluster setup, the
307
+ machine file format and the troubleshooting table (headings "Shared: Machines file format", "Adding a new node",
308
+ "Troubleshooting").
309
+
310
+ ## 8. Data and keys
311
+
312
+ - Corpus: `sample/pinescript/` and `sample_pds/` (tracked `.pine` files).
313
+ - Test data: `workdir/data/` (git-ignored; restore with the commands in section 7).
314
+ - Baseline: `ground_truth/digests.json`, `ground_truth/manifest.json` (tracked).
315
+ - Run output: `workdir/scripts/`, `workdir/output/`, `test_results*.json|txt` (git-ignored).
316
+ - Disposable output: `backup/deleteable/` (git-ignored). Disk rule: `~/.claude/CLAUDE.md`, "Never Delete on the root disk".
317
+ - Key names only. `/home/mike/workspace/github/pine2pyne/.env` holds `PYPI_TOKEN`. `.gitignore` excludes it
318
+ (`.env`, `.env.*`, `*.pem`, `*.key`, `.pypirc`, `.netrc`). `git ls-files .env` shows it is not tracked.
319
+ - Environment names: `PINE_CPU_COUNT` and `PINE_MAX_WORKERS` (written by the setup skill into `.sandbox-env.sh`).
320
+ `PYNE_OPTIMIZE_MODE=1` is set by the optimizer in
321
+ `/home/mike/workspace/github/pynecore/plugins/pyne_optimize_and_stats_tools/pynecore_optimize_and_stats_tools/optimize.py`
322
+ (lines 312, 832 and 887).
323
+ - No other key or secret file was found in the repo files that were read.
324
+
325
+ ## 9. Related repos and skills
326
+
327
+ Calls this repo:
328
+ - `/home/mike/workspace/github/pinescript` (the strategy workbench). Its `CLAUDE.md` says the transpiler and runtime
329
+ moved out to `pine2pyne` and `pynecore`, and lists two ways to get the toolchain (editable installs, or PyPI).
330
+ The workbench holds `workdir/` and the cluster scripts of section 7.
331
+ - The skills below call `pine2pyne`. They are listed in the session skill list; their code was not read here.
332
+ `opencode-pyne-run`, `opencode-pine-test-run`, `sandbox-e2e`, `opencode-strategy-optimize`.
333
+
334
+ This repo calls:
335
+ - `/home/mike/workspace/github/pynecore` (runtime `opencode-pyneruntime`, the `pyne` CLI). The `.venv` of this repo
336
+ imports it from `/home/mike/workspace/github/pynecore/src/pynecore/` (checked with `import pynecore`).
337
+ The runtime is a fork of PyneSys/pynecore. Never change original pynecore files (see that repo's rules).
338
+ - Skills it uses: `opencode-pine2pyne-setup`, `pypi-publish`, `qa-remote-suite`, `mike-pc-test-run`.
339
+ Skills live in `/home/mike/.claude/skills/`.
340
+
341
+ Rules for every repo are in `/home/mike/.claude/CLAUDE.md`. This README does not copy them. Repo-specific
342
+ instructions (runtime code placement, pinned xfail bugs, pipeline notes) are in
343
+ `/home/mike/workspace/github/pine2pyne/CLAUDE.md`.
@@ -1,8 +1,8 @@
1
- opencode_pine2pyne-0.2.2.dist-info/licenses/LICENSE,sha256=l6ZEJ0BCsllGRX5Pg-ygs6mQD03fLXJggIyKnSMwC8E,11342
1
+ opencode_pine2pyne-0.2.3.dist-info/licenses/LICENSE,sha256=l6ZEJ0BCsllGRX5Pg-ygs6mQD03fLXJggIyKnSMwC8E,11342
2
2
  pine2pyne/README.md,sha256=7O2y_Jn9EQ6yNXZio7qAvEqdfVnuiJE_C6zyTdWkUgM,6021
3
3
  pine2pyne/TRANSPILER_BEST_PRACTICES.md,sha256=I9UQYI-KlObGGODfItU0kIt-Bo6MVh9XgPfXPRvjh_4,18197
4
4
  pine2pyne/TRANSPILER_USAGE.md,sha256=7iT-dnig8t-gMxozVYHgKoxr7W--SITLoyJk41PYlrs,5569
5
- pine2pyne/__init__.py,sha256=ZU5aUIYP_ZiNbvSR0z_61_Bvfuouel6AM0WO27aGKOs,2245
5
+ pine2pyne/__init__.py,sha256=BV4O9H7xipptM5ZmeW0wBotZosKe50QrdXUaILQNuj8,2436
6
6
  pine2pyne/__main__.py,sha256=w6UY1yZijkjI0YRgq03guOunEYTp9S4DjwsAips2GOk,228
7
7
  pine2pyne/ast_nodes.py,sha256=xk0Zrjdip1sZWE6WINKFK4AK3ikxcOnZkirC9RwDl0E,11354
8
8
  pine2pyne/ast_walk.py,sha256=Qt0HUyF7nR3G9kErTB22TuarUJ0g4xQV8QBQ3YeuuIg,4037
@@ -17,24 +17,26 @@ pine2pyne/errors.py,sha256=6T2OoC0u2inr1XWMIeC3glI6aplL8dcgET7maA4jDjY,2622
17
17
  pine2pyne/for_loop_end_bound.py,sha256=y3lTjEvL2jZOVXMZ7zHG8X683mz9IQeySouhNw8uAcI,6350
18
18
  pine2pyne/grouped_history_lowering.py,sha256=hxoQcJy0HyvzA7jXQLkcYeJUJ0onDJgKs3Jm5auf5FM,16963
19
19
  pine2pyne/import_resolver.py,sha256=mWperlEZomLZ44GTFx6Cez9Wz-AexCPioboAwOJfGXE,22272
20
- pine2pyne/lexer.py,sha256=wk5O3zEtZnyWLgnr1GwYFcYLWTPA4y-reQLfZq1ew_E,21376
20
+ pine2pyne/legacy_function_names.py,sha256=zdIF4fa40A-fkFiwECa3GeGmMT7lZjvVHACs1FbjQ6g,3499
21
+ pine2pyne/lexer.py,sha256=wT-F7dptRTqaWXVduVP8xHVjZH8KfUH9pplZvR9RdE0,22810
21
22
  pine2pyne/library_collision.py,sha256=0c-t-lgNlAmmhhNO286eCp99rW0FhC9wafcGkIWkXI8,22323
22
23
  pine2pyne/namespace_members.py,sha256=CO-oAesSy6LWthD2qItUrL3cTm8ZZ2Yulg3q_mD69BA,12859
23
24
  pine2pyne/once_lowering.py,sha256=frr0g9nX612MKsPiLyeFEDvyoRcQDQTsBiUZ-qMUiq8,3481
24
25
  pine2pyne/overload_resolution.py,sha256=gPELykcmixn0LQT--nEaS85Uwn8FmrOa_fkuytoC8ag,24904
25
26
  pine2pyne/parse_repair.py,sha256=Uec959EDCvnqeHdVD1ZpSizH150rUAk55BnYF9EbNBM,18882
26
- pine2pyne/parser.py,sha256=pMiZJpYn1G-WOzSIrbdo6zzCpNZstpXG2bZoI3DSw7o,75438
27
- pine2pyne/pine_builtins.py,sha256=H6DCiG3QuUMo2GAsScdzZzzTLsSB1KucLYGqEXavO38,16181
27
+ pine2pyne/parser.py,sha256=c-JQu6LEls_EacpeSGfOHZJSJo_k25wJ9CnojZI3CqE,77634
28
+ pine2pyne/pine_builtins.py,sha256=XRds4zd7t2Twdgi7dK06iEvpw73glo863CaSz0wKG5s,16733
28
29
  pine2pyne/pine_semantics.py,sha256=e0CSoUCFp6BYzb2XNIi5_cQVgHfm--lMTKGvaTuraJw,34765
29
- pine2pyne/scope_renamer.py,sha256=ksyNG4J6xs0SsY3jyCt57pA7K_RhYYFvZYWUlSFe0Kc,43916
30
+ pine2pyne/scope_renamer.py,sha256=uM935RXVdQb6QRHd3-mTZN02343EnZE8pFibqr30H5c,44446
31
+ pine2pyne/strategy_when_lowering.py,sha256=bN5rAEKGy6bkp3Rm093GnAfpwtswExYsBbBGOdMeMbo,5334
30
32
  pine2pyne/strict_boolean.py,sha256=7Z-q2tdVk0rFgu4SecX9GVaWgBof_IFtBKjw1tBolB0,5471
31
33
  pine2pyne/symbol_table.py,sha256=SO40do-faSu-n29-5GOrkKAqEK6Ef9BGtw4sC0w9Jbo,6146
32
34
  pine2pyne/tokens.py,sha256=diG6z5_9DsDCdSN-sapdDAbdIwvklQPuC-05tkhQzDk,3792
33
35
  pine2pyne/transformer.py,sha256=mwgIMx1GRapzPsZfDWNY16NTxDPVsYVI3Y4svWlud3w,115673
34
36
  pine2pyne/type_inference.py,sha256=UFg7y9Q5Vkd_NnjQ1DD9LrqJ3ywpHSMHv28D_lQlb08,19895
35
37
  pine2pyne/udt_field_defaults.py,sha256=DpOwtiCxKdQjAK5senw2wAzT7JNfgRl3NxCrlLTfk3c,7746
36
- opencode_pine2pyne-0.2.2.dist-info/METADATA,sha256=wyP2thJ4q3QjWxhpnUOH5e5iksKYRY09YuxBHkM-B5k,14921
37
- opencode_pine2pyne-0.2.2.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
38
- opencode_pine2pyne-0.2.2.dist-info/entry_points.txt,sha256=OQmVtMqXUdYsmXqUEe09w30h54Gmn3JsPjGgVnTO4sA,49
39
- opencode_pine2pyne-0.2.2.dist-info/top_level.txt,sha256=ge_qvuxsfM_9zOv_hIwyLCMe1S8RAr5nUR6n3aBO6J0,10
40
- opencode_pine2pyne-0.2.2.dist-info/RECORD,,
38
+ opencode_pine2pyne-0.2.3.dist-info/METADATA,sha256=PFEJPwfv86uUgLoSrrUQ2Iw6hB87d1b0haOCVzZqNoI,24337
39
+ opencode_pine2pyne-0.2.3.dist-info/WHEEL,sha256=a-udL2m4V_6Yn8b-uKzWb12RjRxwwMo6ohYFM24tpaY,91
40
+ opencode_pine2pyne-0.2.3.dist-info/entry_points.txt,sha256=OQmVtMqXUdYsmXqUEe09w30h54Gmn3JsPjGgVnTO4sA,49
41
+ opencode_pine2pyne-0.2.3.dist-info/top_level.txt,sha256=ge_qvuxsfM_9zOv_hIwyLCMe1S8RAr5nUR6n3aBO6J0,10
42
+ opencode_pine2pyne-0.2.3.dist-info/RECORD,,
@@ -1,5 +1,5 @@
1
1
  Wheel-Version: 1.0
2
- Generator: setuptools (84.0.0)
2
+ Generator: setuptools (84.2.0)
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
5
5
 
pine2pyne/__init__.py CHANGED
@@ -5,12 +5,13 @@ A tool to convert TradingView Pine Script indicators and strategies
5
5
  to PyneCore Python code for local backtesting and analysis.
6
6
  """
7
7
 
8
- __version__ = '0.2.2'
8
+ __version__ = '0.2.3'
9
9
  __author__ = 'Pine2Pyne Transpiler'
10
10
 
11
11
  from .lexer import Lexer, tokenize
12
12
  from .parser import Parser, parse
13
13
  from .parse_repair import parse_source
14
+ from .legacy_function_names import qualify_legacy_function_names
14
15
  from .pine_semantics import enforce as enforce_pine_semantics
15
16
  from .transformer import Transformer, PyneTransformedScript
16
17
  from .codegen import CodeGenerator, generate_code
@@ -55,6 +56,9 @@ def transpile(source: str) -> str:
55
56
  # Tokenize and parse; a rejected known spelling of valid Pine is repaired (card #55)
56
57
  ast, _repairs = parse_source(source)
57
58
 
59
+ # Pine v1-v4 call names are spelled the way Pine v5 spells them (card #116)
60
+ ast = qualify_legacy_function_names(ast)
61
+
58
62
  # Refuse what TradingView's compiler refuses, at the Pine line (card #28)
59
63
  enforce_pine_semantics(ast)
60
64
 
@@ -0,0 +1,66 @@
1
+ """Pine v1-v4 call names, spelled the way Pine v5 spells them (card #116).
2
+
3
+ Before Pine v5 the technical-analysis, math, request and ticker functions were global: `sma(close, 5)`,
4
+ `mom(close, 2)`, `security(...)`, `heikenashi(...)`. v5 moved them into the `ta.`, `math.`, `str.`, `request.`
5
+ and `ticker.` namespaces. The transformer and the runtime know only the namespaced names, so a bare `sma(...)`
6
+ reached the generated code as a bare `sma(...)` that nothing imports: `NameError` at run.
7
+
8
+ `qualify_legacy_function_names` renames those calls in a script that declares //@version 1-4. Only a call of a
9
+ name from the table is renamed, and only when the script declares no FUNCTION of that name itself: a user's own
10
+ `sma(x, n) => ...` stays. A variable, a parameter or a loop counter of that name does not stop the rename: a call
11
+ `sma(...)` looks up a function, a variable lives apart from it (`rsi = rsi(close, 14)` is a common idiom) or is out of
12
+ scope at the call, so the builtin is the callee. A source with no //@version line, or one at v5 or later, is not
13
+ touched.
14
+
15
+ Left out on purpose (card #116 follow-ups): names a Python builtin of the same spelling already answers
16
+ (`abs`, `max`, `min`, `round`, `pow`, `range`), which run today with Python semantics; and the legacy names
17
+ that are series, not calls (a bare `tr`, `vwap`, `obv`, `tickerid`, ...).
18
+ """
19
+ from __future__ import annotations
20
+
21
+ from typing import Dict, Set
22
+
23
+ from .ast_nodes import FuncDecl, FunctionCall, Script
24
+ from .ast_walk import walk
25
+
26
+ TECHNICAL_ANALYSIS_NAMES = (
27
+ 'alma atr barssince bb bbw cci change cmo cog correlation cross crossover crossunder cum dev dmi ema falling '
28
+ 'hma highest highestbars kc kcw linreg lowest lowestbars macd mfi mom percentile_linear_interpolation '
29
+ 'percentile_nearest_rank percentrank pivothigh pivotlow rising rma roc rsi sar sma stdev stoch supertrend '
30
+ 'swma tr tsi valuewhen variance vwma wma wpr'
31
+ ).split()
32
+ MATH_NAMES = ('acos asin atan avg ceil cos exp floor log log10 random round_to_mintick sign sin sqrt sum tan '
33
+ 'todegrees toradians').split()
34
+
35
+ # call name before Pine v5 -> the name Pine v5 gives it
36
+ LEGACY_FUNCTION_NAMES: Dict[str, str] = {
37
+ **{name: f'ta.{name}' for name in TECHNICAL_ANALYSIS_NAMES},
38
+ **{name: f'math.{name}' for name in MATH_NAMES},
39
+ 'tostring': 'str.tostring',
40
+ 'tonumber': 'str.tonumber',
41
+ 'security': 'request.security',
42
+ 'heikenashi': 'ticker.heikinashi',
43
+ }
44
+
45
+
46
+ def _declared_function_names(script: Script) -> Set[str]:
47
+ """The names of the functions the script itself declares, anywhere in it."""
48
+ return {node.name for node in walk(script) if isinstance(node, FuncDecl) and isinstance(node.name, str)}
49
+
50
+
51
+ def _declared_version(script: Script) -> int:
52
+ """The //@version number, or 0 when absent or not a number."""
53
+ text = script.version.version if script.version else ''
54
+ return int(text) if str(text).isdigit() else 0
55
+
56
+
57
+ def qualify_legacy_function_names(script: Script) -> Script:
58
+ """The script with the calls of a Pine v1-v4 script spelled as Pine v5 spells them."""
59
+ if not 1 <= _declared_version(script) <= 4:
60
+ return script
61
+ own_names = _declared_function_names(script)
62
+ renames = {legacy: current for legacy, current in LEGACY_FUNCTION_NAMES.items() if legacy not in own_names}
63
+ for node in walk(script):
64
+ if isinstance(node, FunctionCall) and isinstance(node.func, str) and node.func in renames:
65
+ node.func = renames[node.func]
66
+ return script
pine2pyne/lexer.py CHANGED
@@ -225,6 +225,10 @@ class Lexer:
225
225
  while self.current_char() and (self.current_char().isalnum() or self.current_char() in ('_', '.')):
226
226
  identifier += self.advance()
227
227
 
228
+ # `type` is a keyword only at the head of a user-defined-type declaration (card #115)
229
+ if identifier == 'type' and not self._heads_type_declaration():
230
+ return Token(TokenType.IDENTIFIER, identifier, start_line, start_col, identifier)
231
+
228
232
  # Check if it's a keyword
229
233
  if identifier in KEYWORDS:
230
234
  token_type = KEYWORDS[identifier]
@@ -236,6 +240,28 @@ class Lexer:
236
240
 
237
241
  return Token(TokenType.IDENTIFIER, identifier, start_line, start_col, identifier)
238
242
 
243
+ def _heads_type_declaration(self) -> bool:
244
+ """True for `type Name`: first on its statement (or after `export`), a name following on the same line.
245
+
246
+ Anywhere else `type` is an ordinary name (the Pine v2-v4 `type=` argument of input(), a variable,
247
+ a parameter, a field). The parser accepted a TYPE token only at the head of a declaration, so a
248
+ source that parsed before is not changed by this.
249
+ """
250
+ previous = self.tokens[-1].type if self.tokens else None
251
+ if previous not in (None, TokenType.NEWLINE, TokenType.INDENT, TokenType.DEDENT, TokenType.EXPORT):
252
+ return False
253
+ offset = 0
254
+ while self.peek_char(offset) in (' ', '\t'):
255
+ offset += 1
256
+ following = self.peek_char(offset)
257
+ if following is None or not (following.isalpha() or following == '_'):
258
+ return False
259
+ name_end = offset
260
+ while (self.peek_char(name_end) or ' ').isalnum() or self.peek_char(name_end) == '_':
261
+ name_end += 1
262
+ following_word = self.source[self.pos + offset:self.pos + name_end]
263
+ return following_word not in ('and', 'or', 'in', 'to', 'by')
264
+
239
265
  def _is_continuation_line(self) -> bool:
240
266
  """Check if this line is a continuation of the previous expression.
241
267
 
pine2pyne/parser.py CHANGED
@@ -10,6 +10,7 @@ from .tokens import Token, TokenType
10
10
  from .ast_nodes import *
11
11
  from .errors import ParserError, SemanticError, UnsupportedFeatureError
12
12
  from .once_lowering import lower_once_structures
13
+ from .strategy_when_lowering import lower_strategy_when_arguments
13
14
  from .scope_renamer import fresh_name
14
15
 
15
16
 
@@ -254,7 +255,8 @@ class Parser:
254
255
 
255
256
  self.skip_newlines()
256
257
 
257
- return lower_once_structures(script)
258
+ script = lower_once_structures(script)
259
+ return lower_strategy_when_arguments(script, _pine_version(script.version))
258
260
 
259
261
  def _at_script_declaration(self, version: Optional[VersionAnnotation]) -> bool:
260
262
  """At an indicator/strategy/library call; before v5 a study() call is the indicator (card #55)."""
@@ -788,12 +790,38 @@ class Parser:
788
790
 
789
791
  # Check for = input.
790
792
  result = (self.match(TokenType.ASSIGN) and
791
- self.peek_token().type == TokenType.IDENTIFIER and
792
- self.peek_token().value.startswith('input.'))
793
+ ((self.peek_token().type == TokenType.IDENTIFIER and
794
+ self.peek_token().value.startswith('input.')) or
795
+ self._legacy_typed_input_call_ahead()))
793
796
 
794
797
  self.pos = saved_pos
795
798
  return result
796
799
 
800
+ def _legacy_typed_input_call_ahead(self) -> bool:
801
+ """At `= input(` ... `type=<kind>` ... `)` with a kind parse_function_call rewrites to an input.* call
802
+ (card #115): such a call is an input declaration, as its v5 spelling is."""
803
+ from .pine_builtins import LEGACY_INPUT_TYPE_ARGUMENT_FUNCTIONS
804
+ if self.peek_token().value != 'input' or self.peek_token(2).type != TokenType.LPAREN:
805
+ return False
806
+ depth, offset = 0, 2
807
+ while True:
808
+ token = self.peek_token(offset)
809
+ if token.type == TokenType.EOF:
810
+ return False
811
+ if token.type in (TokenType.LPAREN, TokenType.LBRACKET):
812
+ depth += 1
813
+ elif token.type in (TokenType.RPAREN, TokenType.RBRACKET):
814
+ depth -= 1
815
+ if depth == 0:
816
+ return False
817
+ elif (depth == 1 and token.type == TokenType.IDENTIFIER and token.value == 'type'
818
+ and self.peek_token(offset + 1).type == TokenType.ASSIGN):
819
+ kind = self.peek_token(offset + 2)
820
+ # a bare `float`/`bool`/`string`/`color` lexes as TYPE_IDENTIFIER; parse_function_call rewrites it too
821
+ return (kind.type in (TokenType.IDENTIFIER, TokenType.TYPE_IDENTIFIER)
822
+ and kind.value.removeprefix('input.') in LEGACY_INPUT_TYPE_ARGUMENT_FUNCTIONS)
823
+ offset += 1
824
+
797
825
  def parse_input_declaration(self) -> InputDecl:
798
826
  """Parse input declaration. Handles: name = input.*() and type name = input.*()"""
799
827
  # Skip optional type hint
@@ -1526,6 +1554,14 @@ class Parser:
1526
1554
  else:
1527
1555
  func_name = func
1528
1556
 
1557
+ # Pine v2-v4 `input(defval, title, type=input.float)` is the v5 `input.float(defval, title)` (card #115)
1558
+ if func_name == 'input' and isinstance(kwargs.get('type'), Identifier):
1559
+ from .pine_builtins import LEGACY_INPUT_TYPE_ARGUMENT_FUNCTIONS
1560
+ typed_function = LEGACY_INPUT_TYPE_ARGUMENT_FUNCTIONS.get(kwargs['type'].name.removeprefix('input.'))
1561
+ if typed_function is not None:
1562
+ func_name = typed_function
1563
+ kwargs = {key: value for key, value in kwargs.items() if key != 'type'}
1564
+
1529
1565
  # a call is placed at its callee's name, where a refusal points (card #28)
1530
1566
  return FunctionCall(func=func_name, args=args, kwargs=kwargs,
1531
1567
  line=getattr(func, 'line', 0), column=getattr(func, 'column', 0))
@@ -260,6 +260,14 @@ INPUT_POSITIONAL_PARAMS: Dict[str, list[str]] = {
260
260
  'input.enum': ['options', 'tooltip', 'inline', 'group', 'confirm', 'display', 'active'], # card #97
261
261
  }
262
262
 
263
+ # Pine v2-v4 `input(defval, title, type=input.X)`: the keyword names the kind the v5+ `input.X` functions
264
+ # carry in their names (card #115). The key is the spelling after an optional `input.` prefix.
265
+ LEGACY_INPUT_TYPE_ARGUMENT_FUNCTIONS: Dict[str, str] = {
266
+ 'integer': 'input.int', 'float': 'input.float', 'bool': 'input.bool', 'string': 'input.string',
267
+ 'symbol': 'input.symbol', 'resolution': 'input.timeframe', 'source': 'input.source',
268
+ 'session': 'input.session', 'time': 'input.time', 'color': 'input.color', 'price': 'input.price',
269
+ }
270
+
263
271
  # input.int/input.float have a second positional form after `title` (v6 reference): the options form,
264
272
  # selected by a list third argument (pynecore's _bind_numeric_tail, _NUMERIC_OPTIONS_TAIL; card #95).
265
273
  NUMERIC_INPUT_FUNCTIONS: FrozenSet[str] = frozenset({'input.int', 'input.float'})
@@ -322,12 +322,18 @@ class _ScopeRenamer:
322
322
  shadows = any(name in scope for scope in self.scopes[:-1])
323
323
  if shadows:
324
324
  self.shadowed_names.add(name)
325
- collides = name in self.library_names
326
325
  # scopes[0] is the top level; every deeper scope is a function, a loop or a block
326
+ # a variable named like a function or method of the script: Python reads the name as the variable all through
327
+ # its function, so a call of the function in that scope reads the variable (card #118). At the top level too:
328
+ # the functions are defined inside main(), where the variable is a local that hides them from every function
329
+ names_a_function = name in self.user_function_names or name in self.method_names
330
+ if names_a_function:
331
+ self.moved_bindings = True
332
+ collides = name in self.library_names
327
333
  hides_builtin_series = name in SERIES_VARIABLES and len(self.scopes) > 1
328
334
  if hides_builtin_series:
329
335
  self.moved_bindings = True
330
- if shadows or collides or hides_builtin_series:
336
+ if shadows or collides or hides_builtin_series or names_a_function:
331
337
  emitted = self.fresh_name(name)
332
338
  elif self.holds_external_spelling(name):
333
339
  self.moved_bindings = True
@@ -0,0 +1,101 @@
1
+ """`when=` of a strategy call, lowered to an `if` (card #117).
2
+
3
+ strategy.entry("l", strategy.long, when=c) if c
4
+ -> strategy.entry("l", strategy.long)
5
+
6
+ Pine v1-v5 gave the order functions of `strategy.*` a `when` argument: the call acts only on the bars where it is
7
+ true. Pine v6 removed it, and the runtime's `strategy.entry` / `strategy.close` / ... take none, so a `when=` that
8
+ reached the generated code raised `TypeError: entry() got an unexpected keyword argument 'when'` at run. The
9
+ lowering runs at the end of parsing, so no later stage sees one.
10
+
11
+ Only the keyword form, only a call that stands as a statement (the functions return nothing), and only in a script
12
+ declaring //@version 1-5. A v6 script is left as written: TradingView refuses a `when` there, so no spelling is
13
+ invented for it. A positional `when` (the 9th slot of `strategy.entry`, the 2nd of v4 `strategy.close`) is not
14
+ lowered here (card #117 follow-up).
15
+ """
16
+ import dataclasses
17
+ import itertools
18
+
19
+ from .ast_nodes import (Assignment, ExpressionStatement, FuncDecl, FunctionCall, Identifier, IfStatement, IndexAccess,
20
+ MethodCall, Script)
21
+ from .ast_walk import child_field_names, contains, is_node
22
+
23
+ STRATEGY_FUNCTIONS_WITH_WHEN = frozenset({
24
+ 'strategy.entry', 'strategy.order', 'strategy.exit', 'strategy.close', 'strategy.close_all',
25
+ 'strategy.cancel', 'strategy.cancel_all',
26
+ })
27
+
28
+
29
+ def lower_strategy_when_arguments(script: Script, pine_version: int) -> Script:
30
+ """A copy of `script` with each `strategy.*(..., when=c)` statement turned into `if c` around the call
31
+ without `when`. A script without one, or of another version, is returned unchanged."""
32
+ if not 1 <= pine_version <= 5 or not contains(script, _is_call_with_when):
33
+ return script
34
+ return _WhenLowering().lowered(script)
35
+
36
+
37
+ def _is_call_with_when(node) -> bool:
38
+ # a callee that is not a name (`na(x)`: a NaLiteral node) is unhashable and is never a strategy function
39
+ return (isinstance(node, FunctionCall) and isinstance(node.func, str) and node.func in STRATEGY_FUNCTIONS_WITH_WHEN
40
+ and 'when' in node.kwargs)
41
+
42
+
43
+ def _is_stateful_or_history(node) -> bool:
44
+ return isinstance(node, (FunctionCall, MethodCall, IndexAccess))
45
+
46
+
47
+ class _Statements(list):
48
+ """The statements one statement is replaced by (a list item that expands to several)."""
49
+
50
+
51
+ class _WhenLowering:
52
+ """Pine evaluates every argument of a `strategy.*(..., when=c)` call on every bar, whatever `c` is. An argument
53
+ holding a call (`ta.lowest(low, 10)`, a user function) or a history index (`close[1]`) has state per bar, and
54
+ inside an `if` it would update only on the bars where `c` holds. So such an argument is assigned to a temporary
55
+ before the `if`, and the temporary is what the call inside the `if` receives."""
56
+
57
+ def __init__(self):
58
+ self.temporary_numbers = itertools.count(1)
59
+
60
+ def lowered(self, node):
61
+ children = {name: self._map(getattr(node, name)) for name in child_field_names(type(node))}
62
+ rebuilt = dataclasses.replace(node, **children)
63
+ if isinstance(rebuilt, ExpressionStatement) and _is_call_with_when(rebuilt.expr):
64
+ return _Statements(self._if_around(rebuilt))
65
+ if isinstance(rebuilt, FuncDecl) and _is_call_with_when(rebuilt.body):
66
+ # `f(c) => strategy.close("l", when=c)`: the one-line body is the call, made a statement
67
+ call = rebuilt.body
68
+ statement = ExpressionStatement(expr=call, line=call.line, column=call.column)
69
+ return dataclasses.replace(rebuilt, body=self._if_around(statement))
70
+ return rebuilt
71
+
72
+ def _map(self, value):
73
+ if isinstance(value, list):
74
+ items = []
75
+ for item in value:
76
+ mapped = self._map(item)
77
+ items.extend(mapped) if isinstance(mapped, _Statements) else items.append(mapped)
78
+ return items
79
+ if isinstance(value, tuple):
80
+ return tuple(self._map(item) for item in value)
81
+ if isinstance(value, dict):
82
+ return {key: self._map(item) for key, item in value.items()}
83
+ return self.lowered(value) if is_node(value) else value
84
+
85
+ def _if_around(self, statement: ExpressionStatement) -> list:
86
+ call = statement.expr
87
+ temporaries = []
88
+ args = [self._every_bar(argument, 'argument', statement, temporaries) for argument in call.args]
89
+ kwargs = {key: self._every_bar(value, key, statement, temporaries)
90
+ for key, value in call.kwargs.items() if key != 'when'}
91
+ without_when = dataclasses.replace(call, args=args, kwargs=kwargs)
92
+ guarded = IfStatement(condition=call.kwargs['when'], body=[dataclasses.replace(statement, expr=without_when)],
93
+ line=statement.line, column=statement.column)
94
+ return [*temporaries, guarded]
95
+
96
+ def _every_bar(self, argument, label, statement, temporaries):
97
+ if not contains(argument, _is_stateful_or_history):
98
+ return argument
99
+ name = f'_when_{label}_{next(self.temporary_numbers)}'
100
+ temporaries.append(Assignment(target=name, value=argument, line=statement.line, column=statement.column))
101
+ return Identifier(name=name, line=statement.line, column=statement.column)
@@ -1,301 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: opencode-pine2pyne
3
- Version: 0.2.2
4
- Summary: Pine Script v6 to PyneCore Python transpiler
5
- Author: rubycell
6
- License-Expression: Apache-2.0
7
- Project-URL: Homepage, https://github.com/rubycell/pine2pyne
8
- Project-URL: Repository, https://github.com/rubycell/pine2pyne
9
- Keywords: pinescript,tradingview,transpiler,pynecore,backtesting
10
- Classifier: Programming Language :: Python :: 3.11
11
- Classifier: Programming Language :: Python :: 3.12
12
- Classifier: Programming Language :: Python :: 3.13
13
- Classifier: Topic :: Office/Business :: Financial :: Investment
14
- Requires-Python: >=3.11
15
- Description-Content-Type: text/markdown
16
- License-File: LICENSE
17
- Provides-Extra: runtime
18
- Requires-Dist: opencode-pyneruntime[cli]>=6.9.1; extra == "runtime"
19
- Dynamic: license-file
20
-
21
- # PyneCore - Pine Script to Python Transpiler
22
-
23
- > **Backlog & known bugs:** [Projects board #10](https://github.com/users/rubycell/projects/10)
24
- > — open work is in **Backlog** (prioritised P0/P1/P2), shipped work in **Done**.
25
- > Confirmed unfixed transpiler bugs live there too; several are pinned by strict
26
- > `xfail` tests (see [Running Tests](#running-tests)).
27
-
28
- ## Installation (New Computer)
29
-
30
- ### Fastest: the sandbox-setup skill
31
-
32
- From the repo root, run the bundled setup script (idempotent — safe to re-run):
33
-
34
- ```bash
35
- bash ~/.claude/skills/opencode-pine2pyne-setup/setup.sh # from inside this clone; the skill lives in ~/.claude/skills
36
- ```
37
-
38
- It builds `.venv`, installs everything below, detects `PINE_CPU_COUNT` /
39
- `PINE_MAX_WORKERS`, seeds a demo dataset, and runs a smoke test that ends in
40
- `smoke test : PASS` / `READY`. In Claude Code you can also invoke it as
41
- `/sandbox-setup`.
42
-
43
- ### Manual (equivalent steps)
44
-
45
- ```bash
46
- # From the repo root:
47
- python3 -m venv .venv # 1. virtualenv
48
- .venv/bin/pip install -e . # 2. this transpiler (editable)
49
- .venv/bin/pip install 'opencode-pyneruntime[cli]>=6.9.1' # 3. PyneCore runtime + `pyne` CLI
50
- .venv/bin/pip install optuna numpy # 4. optimizer deps (not pulled in by the runtime)
51
- source .venv/bin/activate # 5. activate (Linux/macOS)
52
- # .venv\Scripts\activate # (Windows)
53
- ```
54
-
55
- Verify the install:
56
-
57
- ```bash
58
- .venv/bin/python -c "import pynecore, pine2pyne, optuna, numpy; print('all ok')"
59
- ```
60
-
61
- **Note**: All commands below assume you are at the repo root with the venv activated.
62
-
63
- ## How to Convert Pine Script to Python
64
-
65
- ### Single File Conversion
66
-
67
- ```bash
68
- python -m pine2pyne path/to/script.pine -o path/to/output.py
69
- ```
70
-
71
- ### Batch Conversion
72
-
73
- ```bash
74
- python -m pine2pyne "sample/pinescript/*.pine" -o workdir/scripts/
75
- ```
76
-
77
- ## Running Tests
78
-
79
- There are **three** independent test systems. They answer different questions,
80
- so a green run of one says nothing about the others.
81
-
82
- | System | Question it answers | Runtime |
83
- |---|---|---|
84
- | **1. Unit tests** (`tests/`, pytest) | Are the transpiler's internals correct? | ~5 s |
85
- | **2. Sample suites** (`tools/test_all_*.py`) | Does every corpus sample transpile and run? | ~35 s |
86
- | **3. Ground truth** (`tools/*_ground_truth*`) | Does it produce the *same numbers* as before? | minutes |
87
-
88
- ### 1. Unit tests — pytest
89
-
90
- 6,870 tests over `pine2pyne/` (6,848 passed, 1 skipped, 21 xfailed at a16621b; 93% line / 86% branch coverage).
91
-
92
- ```bash
93
- .venv/bin/python -m pytest tests/ -q
94
-
95
- # with coverage
96
- .venv/bin/python -m pytest tests/ \
97
- --cov=pine2pyne --cov-report=term-missing
98
- ```
99
-
100
- A bare `pytest tests/` needs no `--ignore`: `tests/test_screener_regression.py`, which
101
- imported a `screener` module from a sibling repo that is absent here and aborted the
102
- whole run at collection, was retired from the repo.
103
-
104
- Some tests are `@pytest.mark.xfail(strict=True)` and pin **confirmed unfixed
105
- bugs**, each with a card on [board #10](https://github.com/users/rubycell/projects/10).
106
- Strict means fixing a bug turns its marker into a failure until the marker is
107
- removed, so nothing regresses silently in either direction.
108
-
109
- The former P0 — codegen dropping Pine grouping, so `10 - (5 - 3)` became
110
- `10 - 5 - 3` — is fixed (card #6) and guarded by
111
- `tests/test_codegen_operator_grouping.py`. See [docs/test_plan.md](docs/test_plan.md) for the full bug inventory and the
112
- measured coverage/mutation results.
113
-
114
- ### 2. Sample suites — end-to-end transpile + run
115
-
116
- Transpiles every `.pine` in the corpus and runs it through `pyne`.
117
-
118
- ```bash
119
- python tools/test_all_samples.py # sample/pinescript (~411 files)
120
- python tools/test_all_sample_pds.py # sample_pds (~328 files)
121
-
122
- python tools/test_all_samples.py "ex_001*" # filter by pattern
123
- python tools/test_all_samples.py --timeout 30 # default 15s
124
- python tools/test_all_samples.py --verbose # stderr on failure
125
- python tools/test_all_samples.py -j 0 # all CPU cores
126
- python tools/test_all_samples.py --run-dir /tmp/run1 # private scripts/, output/ and report files (card #70)
127
- ```
128
-
129
- - Source: `sample/pinescript/`, `sample_pds/` · Output: `workdir/scripts/`
130
- - `--run-dir DIR` keeps a run's transpiled scripts (`DIR/scripts/`), pyne outputs (`DIR/output/`, passed to `pyne`
131
- as explicit `--plot/--strat/--trade` paths) and `test_results*.{json,txt}` reports in `DIR`, so two runs, or a run
132
- and an old report, cannot read each other's files. Without it nothing changes: the shared `workdir/` paths and
133
- the repo-root reports. `--retry-failed` with `--run-dir` reads `DIR`'s own report.
134
- - Data: `workdir/data/demo.ohlcv` · Results: `test_results{,_pds}.{json,txt}`
135
-
136
- **Per-sample expectations** live in `tools/sample_expectations.toml`:
137
-
138
- ```toml
139
- [expds_156_Out_of_bounds_index]
140
- expect_error = "index_error" # MUST raise; running clean = FAIL
141
-
142
- [ex_300_ticker_modify]
143
- requires_feeds = { D = "demo", "2D" = "demo_2D" } # -> --security D=demo …
144
- ```
145
-
146
- `expect_error` marks Pine documentation samples that exist to *demonstrate* an
147
- error — they are reported as `XFAIL` and count as passes. `requires_feeds`
148
- supplies extra OHLCV feeds for samples calling `request.security()` at a
149
- timeframe other than the chart's; pynecore deliberately will not resample the
150
- chart feed, so the feed must be explicit. Generate one with:
151
-
152
- ```bash
153
- cd workdir && pyne data aggregate data/demo.ohlcv -tf 2D
154
- ```
155
-
156
- Shared logic for both suites lives in `tools/_suite_common.py` — edit it there,
157
- not in each harness.
158
-
159
- ### 3. Ground truth — numeric regression
160
-
161
- Checks every freshly generated output file (~1,650 across ~550 samples) against
162
- its recorded digest. This is the only system that catches *silently wrong
163
- numbers*: output that runs fine but computes something different.
164
-
165
- ```bash
166
- python tools/generate_ground_truth.py # (re)build baselines
167
- python tools/compare_ground_truth.py # check current output vs baselines
168
- python tools/compare_ground_truth.py --suite sample_pds -j 12
169
- ```
170
-
171
- **The baseline is `ground_truth/digests.json`** (card #62). It holds a sha256 and a
172
- line count for each output file, plus the sha256 of the data file. Git tracks
173
- only it and `ground_truth/manifest.json` (under 1 MB together). The output
174
- CSVs themselves (~920 MB) are gitignored: the generator still copies them to
175
- `ground_truth/{sample,sample_pds}/` on your machine for diffing. Before writing a
176
- new baseline, it moves the previous CSVs to `backup/deleteable/ground_truth_csv_<time>/`
177
- (nothing is deleted; clear old ones out by hand). `manifest.json` lists its samples sorted by
178
- suite, then filename (card #68), so a one-suite rebuild (`--suite sample`) and a full rebuild on the same data
179
- give the same samples in the same order, and a rebuild never reorders the tracked file; only `generated_at`
180
- changes by design.
181
-
182
- - **Exit codes.** The compare exits 1 when `digests.json` is missing or empty, and
183
- 2 with `DATA CHANGED` when the data file is not the one the baseline was made
184
- on (restore it, see *Test data* below, or rebaseline).
185
- - **Every sample runs in its own directory** (card #70): the generator and the compare make one fresh run
186
- root, `backup/deleteable/ground_truth_runs/<tool>-<time>/` (or an empty `--run-dir DIR`; a directory that already
187
- holds files is refused), with `<suite>/<sample>/<data file>/` inside it holding the script, its 10B settings and the three
188
- CSVs, and collect only from there. An earlier run's file of the same name cannot be read as this run's; a run
189
- that exits 0 and writes nothing is `no_output` (a failure), never `ok` or `NEW PASS`. Run roots are never
190
- deleted (see the paragraph below on where they go).
191
- - **Where run roots go, and the disk they use.** The default run root of the generator, the compare and the
192
- checker (`ground_truth_check.py capture` and `compare` each make one) is
193
- `backup/deleteable/ground_truth_runs/<tool>-<time>/`, next to the other disposable output, and each run leaves
194
- one tree of roughly 0.9 GB there (script, settings and three CSVs per sample). A generate run also copies the
195
- collected files into `ground_truth/<suite>/` (gitignored). Nothing here is deleted by a tool: when you want the
196
- space back, move old run roots (and the `gt_compare_<time>/` scratch copies) to
197
- `/mnt/data970/Backup/mike-PC-deleteable/<date>-pine2pyne/`, never `rm` them. An explicit `--run-dir DIR` is
198
- used as given (it must be empty).
199
- - **A `REGRESSION` line** names the sample and file whose digest changed. To see
200
- *what* changed, diff that sample's CSV in the run root's `<suite>/<sample>/<data file>/`
201
- against a baseline copy. That copy is in `ground_truth/<suite>/<sample>/` if you
202
- generated on the old commit, or, after a later generate, in the newest
203
- `backup/deleteable/ground_truth_csv_<time>/`.
204
- - **`--suite sample` or `--suite sample_pds`** rebuilds one suite and keeps the
205
- other suite's digests and manifest entries. It refuses (exit 1) unless a
206
- baseline made on the same data already exists.
207
- - **Expected noise:** 11 `sample_pds` samples call unseeded `math.random()` and
208
- differ on every run (expds_022, 037, 039, 043, 045, 081, 091, 092, 104, 114,
209
- 115).
210
-
211
- The old end-to-end shell wrapper is retired (card #66): it read `ground_truth/samples/`,
212
- `ground_truth/samples_pds/` and `ground_truth/strategies/`, which no tool writes, so its
213
- `--quick` mode passed with zero checks. Its default and `--samples` modes are now
214
- `python tools/compare_ground_truth.py` (`--suite sample`, `sample_pds` or `both`; it reads
215
- `workdir/data/VN30F1M_15m.ohlcv` by default, see *Test data*); its `--quick` and
216
- `--strategies` modes are `python tools/ground_truth_check.py capture`, then `compare`.
217
-
218
- `tools/ground_truth_check.py` (`capture` / `compare`) is a separate, local check of the
219
- bigtest strategies and the sample suites. It writes only to `ground_truth_check/` at the
220
- repo root (gitignored), never to `ground_truth/`, so it cannot change the tracked baseline
221
- above. `capture --force` moves the old `ground_truth_check/` to
222
- `backup/deleteable/ground_truth_check_<time>/` (nothing is deleted) and touches nothing else. Both actions run the
223
- bigtest strategies and the two suite harnesses (`--run-dir`) in one fresh run root under
224
- `backup/deleteable/ground_truth_runs/` (`--run-dir DIR` names an empty one), read only that run's files, and fail the capture when a suite wrote no report
225
- or a bigtest file did not transpile. `compare` writes its
226
- scratch copy of the current outputs to `backup/deleteable/gt_compare_<time>/` and never deletes it (card #67); each
227
- run leaves one such folder, which you move to the backup disk when you want the space back. A capture made
228
- before card #63 sits in `ground_truth/` (`transpiled/`, `strategy_runs/`, `test_suite_*`);
229
- `compare` no longer reads it and says so. Run `capture` again.
230
-
231
- > Regenerate baselines only when you have *confirmed* the new output is correct
232
- > — otherwise a bug gets frozen in as the expected result.
233
-
234
- Both tools run every sample on `workdir/data/VN30F1M_15m.ohlcv`, with a
235
- `<script>.toml` that sets `initial_capital` to 10,000,000,000, and with one
236
- per-sample timeout (120 s). Both values live in `tools/_ground_truth_common.py`.
237
- At pynecore's default capital of 1,000,000, one VN30F1M contract costs more than
238
- equity, so nearly every strategy would record zero trades (card #57).
239
-
240
- ### Test data
241
-
242
- `workdir/data/` is gitignored, so a fresh clone has none and the sample suites
243
- abort with `Data file not found`. Restore the exact dataset the recorded results
244
- were measured against:
245
-
246
- ```bash
247
- mkdir -p workdir/data
248
- # Sample suites: CCXT BTCUSDT 1D, plus the 2D/5D feeds `requires_feeds` names
249
- git show 227f3d2^:workdir/data/demo.ohlcv > workdir/data/demo.ohlcv
250
- cp tests/fixtures/data/demo.toml workdir/data/demo.toml
251
- (cd workdir && pyne data aggregate data/demo.ohlcv -tf 2D && pyne data aggregate data/demo.ohlcv -tf 5D)
252
- # Ground truth: VN30F1M 15m
253
- git show 5d63d21:workdir/data/VN30F1M_15m.ohlcv > workdir/data/VN30F1M_15m.ohlcv
254
- cp tests/fixtures/data/VN30F1M_15m.toml workdir/data/VN30F1M_15m.toml
255
- ```
256
-
257
- The `.ohlcv` files come from those commits (sha256 `67e827cc1a5f71ed…` for
258
- demo, `e9ef984b09eaa6d2…` for VN30F1M). Take the `.toml` files from
259
- `tests/fixtures/data/`, not from the commits. The committed ones number weekdays
260
- 1..7, and pynecore rejects day 7 (it numbers 0 = Monday … 6 = Sunday). They also
261
- declare one session per week, which leaves session-anchored values such as
262
- `ta.vwap` NaN or weekly (card #57).
263
-
264
- (`/sandbox-setup` instead seeds a small *synthetic* fixture — fine for a smoke
265
- test, but results will not match the recorded rates.)
266
-
267
- ## For Claude Code / LLM Development
268
-
269
- See `CLAUDE.md` in this directory for transpiler architecture, output format specs, optimizer documentation, and CSV rounding rules. See `pine2pyne/README.md` for transpiler internals and transformation rules.
270
-
271
- ## Optimizing Strategies
272
-
273
- Quick start: `pyne optimize script.py data.ohlcv params.json -n 20`
274
-
275
- See [Optimizer.md](./Optimizer.md) for full documentation (parameter JSON format, parallel execution, output files).
276
-
277
- ### Distributed Optimization (multi-machine)
278
-
279
- Two tools distribute `pyne optimize` across SSH clusters:
280
-
281
- | Tool | Model | Best for |
282
- |------|-------|----------|
283
- | `pyne-dynamic.sh` | Flat-queue (on-demand dispatch) | Multi-variant runs, heterogeneous clusters, long jobs |
284
- | `pyne-parallel.sh` | Static pre-assignment | Quick jobs on similar-speed machines |
285
-
286
- ```bash
287
- # Dynamic (recommended): flat-queue, pre-syncs once, auto-adapts to machine speed
288
- # Run from workdir/ directory:
289
- ../tools/pyne-dynamic.sh scripts/strategy.py data/data.ohlcv optimize_variants/ \
290
- -H ../tools/machines.txt -C 24 --name my_run --output-dir runs/output/
291
-
292
- # Static: pre-assigns chunks by core count
293
- ./tools/pyne-parallel.sh scripts/strategy.py data/data.ohlcv optimize.json \
294
- -H tools/machines.txt --sync
295
-
296
- # Check progress / collect results
297
- ../tools/pyne-dynamic.sh --status
298
- ../tools/pyne-dynamic.sh --collect
299
- ```
300
-
301
- Both use the same `machines.txt` format. See `CLAUDE.md` for cluster setup, machine file format, and troubleshooting.