tidy3syntax 0.1.1__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- tidy3syntax-0.2.0/CHANGELOG.md +70 -0
- {tidy3syntax-0.1.1/src/tidy3syntax.egg-info → tidy3syntax-0.2.0}/PKG-INFO +38 -12
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/README.md +36 -11
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/docs/design.md +15 -10
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/pyproject.toml +2 -1
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/__init__.py +8 -0
- tidy3syntax-0.2.0/src/tidy3syntax/__version__.py +1 -0
- tidy3syntax-0.2.0/src/tidy3syntax/continuation.py +434 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/dialect.py +17 -4
- tidy3syntax-0.2.0/src/tidy3syntax/ipython.py +285 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/pipeline.py +9 -1
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/repairs.py +2 -2
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0/src/tidy3syntax.egg-info}/PKG-INFO +38 -12
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax.egg-info/SOURCES.txt +5 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax.egg-info/requires.txt +1 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/tests/dialects.py +2 -0
- tidy3syntax-0.2.0/tests/test_continuation.py +124 -0
- tidy3syntax-0.2.0/tests/test_integration.py +96 -0
- tidy3syntax-0.2.0/tests/test_ipython.py +157 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/tests/test_reinterpretations.py +2 -2
- tidy3syntax-0.1.1/CHANGELOG.md +0 -29
- tidy3syntax-0.1.1/src/tidy3syntax/__version__.py +0 -1
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/LICENSE +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/MANIFEST.in +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/setup.cfg +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/ownership.py +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/placeholders.py +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/py.typed +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax/scanner.py +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax.egg-info/dependency_links.txt +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/src/tidy3syntax.egg-info/top_level.txt +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/tests/test_properties.py +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/tests/test_repairs.py +0 -0
- {tidy3syntax-0.1.1 → tidy3syntax-0.2.0}/tests/test_scanner.py +0 -0
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.2.0 — 2026-11-02
|
|
4
|
+
|
|
5
|
+
Phase 7 of [the design](docs/design.md).
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- `Dialect.mask` takes the notebook's imports as a third argument,
|
|
10
|
+
`mask(source, known, context)`, so a package's export masking can tell its
|
|
11
|
+
own calls from another package's; `Dialect.export_imports` lists the lines
|
|
12
|
+
an exported script needs once the mask changed something (`import tidy3a`).
|
|
13
|
+
- A formula being quoted turns only `__r_name__` back into a backtick name;
|
|
14
|
+
tidy3's `__tidy3_bt__` and plot3's `__plot3_bt__`, which no release of
|
|
15
|
+
either writes since tidy3 0.17 and plot3 0.10, are no longer read.
|
|
16
|
+
|
|
17
|
+
## 0.1.2 — 2026-11-01
|
|
18
|
+
|
|
19
|
+
Phase 6a of [the design](docs/design.md): continuation lines and the coordinator.
|
|
20
|
+
|
|
21
|
+
- Continuation lines. A line starting with one of a dialect's
|
|
22
|
+
`pipe_operators` (tidy3's `>>`) continues the statement above, a repair; a
|
|
23
|
+
pipe that still does not parse is joined with backslashes, so Python
|
|
24
|
+
reports its real error. A `+ name(...)` line, for a dialect's
|
|
25
|
+
`layer_calls` (plot3's layers), joins by name when indented and, when not,
|
|
26
|
+
only if the call is the package's own and the statement above is one of
|
|
27
|
+
its plots (`ggplot(...)`, a pipe ending in one, or a variable not known to
|
|
28
|
+
hold something else). Joined lines keep their line numbers.
|
|
29
|
+
- `tidy3syntax.ipython`, the coordinator: `register(shell, dialect,
|
|
30
|
+
ast_transformer=...)` puts every package's syntax in one input
|
|
31
|
+
transformer, whatever the load order, kept first before each cell;
|
|
32
|
+
`unregister`, `enable` and `registered` go with it. A cell that imports a
|
|
33
|
+
package with a `tidy3syntax.dialects` entry point gets that package's
|
|
34
|
+
syntax in the same cell. `%%tidy3syntax <package>` reinterprets one cell
|
|
35
|
+
whoever owns its calls, and `reinterpret="always"` does so for the session.
|
|
36
|
+
- The integration test: tidy3, tidy3a, plot3 and stats3 in one notebook in
|
|
37
|
+
all 24 load orders, in CI on pull requests, weekly, and before a release.
|
|
38
|
+
- A `tidy3syntax.dialects` entry point names a function returning the
|
|
39
|
+
package's `Dialect`, which says where its notebook loader is
|
|
40
|
+
(`extension`) and how to apply its AST pass to source text for an export
|
|
41
|
+
(`mask`). `tidy3syntax.installed()` returns every installed package's
|
|
42
|
+
dialect, for rewriting without a notebook.
|
|
43
|
+
|
|
44
|
+
## 0.1.1 — 2026-11-01
|
|
45
|
+
|
|
46
|
+
The first release as `tidy3syntax`, the tidy3 family's name; tidy3 0.15.1,
|
|
47
|
+
tidy3a 0.16.1, plot3 0.8.1 and stats3 0.6.1 use it. The code is 0.1.0's.
|
|
48
|
+
0.1.0 was published as `tidysyntax`, which this name supersedes.
|
|
49
|
+
|
|
50
|
+
## 0.1.0 — 2026-11-01
|
|
51
|
+
|
|
52
|
+
First version, published as `tidysyntax`: phases 1 to 5 of
|
|
53
|
+
[the design](docs/design.md).
|
|
54
|
+
|
|
55
|
+
- `Dialect` describes one package's R syntax as data: which calls take `!`,
|
|
56
|
+
`name = value` before a positional argument, `left ~ right` rules (tidy3's
|
|
57
|
+
tuples), `y ~ x` formulas (plot3's and stats3's strings), `^` as power and
|
|
58
|
+
one-sided formulas, and a package's own repair stages.
|
|
59
|
+
- `repair()` rewrites text that is not Python, by the call's name: formulas,
|
|
60
|
+
rules, negation, backtick assignments, keywords before positional
|
|
61
|
+
arguments, and backtick names, which become the neutral placeholders
|
|
62
|
+
`__r_name__` and `__r_assign__`.
|
|
63
|
+
- `rewrite()` adds the reinterpretations, `^` → `**` and `~x` → `"~x"`, only in
|
|
64
|
+
calls the dialect's package owns, by tidy3's rule (the latest binding in the
|
|
65
|
+
cell, then the context), or in packages listed in `always=`.
|
|
66
|
+
- `imported_names()` builds a context from earlier cells' imports, for exports
|
|
67
|
+
that have no live namespace.
|
|
68
|
+
- `Dialect.prefixes` says which module prefixes make `prefix.name(...)` one of
|
|
69
|
+
a dialect's calls: any (the default), none (bare calls only), or a set of
|
|
70
|
+
names, such as stats3's `{"stats3"}`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: tidy3syntax
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.2.0
|
|
4
4
|
Summary: R syntax in notebook cells, rewritten to Python: shared by tidy3, tidy3a, plot3 and stats3
|
|
5
5
|
Author: Rigoberto Leyva Salmeron
|
|
6
6
|
License-Expression: MIT
|
|
@@ -15,6 +15,7 @@ Description-Content-Type: text/markdown
|
|
|
15
15
|
License-File: LICENSE
|
|
16
16
|
Provides-Extra: dev
|
|
17
17
|
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
18
|
+
Requires-Dist: ipython>=8.0; extra == "dev"
|
|
18
19
|
Dynamic: license-file
|
|
19
20
|
|
|
20
21
|
# tidy3syntax
|
|
@@ -39,6 +40,29 @@ ts.rewrite('df >> filter(!is_na(x)) >> select(Cus = NUM, Row)', [stats3, tidy3])
|
|
|
39
40
|
# "df >> filter(~is_na(x)) >> select(__r_assign__('Cus', (NUM)), Row)"
|
|
40
41
|
```
|
|
41
42
|
|
|
43
|
+
## In a notebook
|
|
44
|
+
|
|
45
|
+
A package's notebook extension registers its dialect with the coordinator,
|
|
46
|
+
which installs one input transformer for every package:
|
|
47
|
+
|
|
48
|
+
```python
|
|
49
|
+
import tidy3syntax.ipython
|
|
50
|
+
|
|
51
|
+
def load_ipython_extension(shell):
|
|
52
|
+
tidy3syntax.ipython.register(shell, DIALECT, ast_transformer=MyMaskTransformer(shell.user_ns))
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
A package can also declare an entry point, so a cell that imports it gets
|
|
56
|
+
its syntax in that same cell:
|
|
57
|
+
|
|
58
|
+
```toml
|
|
59
|
+
[project.entry-points."tidy3syntax.dialects"]
|
|
60
|
+
plot3 = "plot3.jupyter:load_ipython_extension"
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`%%tidy3syntax plot3` as a cell's first line reinterprets that cell's calls
|
|
64
|
+
whoever owns them.
|
|
65
|
+
|
|
42
66
|
## Two kinds of rewrite
|
|
43
67
|
|
|
44
68
|
**Repairs** change only text that is a syntax error in Python, so no valid
|
|
@@ -87,15 +111,16 @@ turns them into its column references inside its calls.
|
|
|
87
111
|
|
|
88
112
|
Phases 1 to 5 of [the design](docs/design.md): the scanner, every repair,
|
|
89
113
|
the ownership rule and the two reinterpretations, used by tidy3 0.15.1, tidy3a
|
|
90
|
-
0.16.1, plot3 0.8.1 and stats3 0.6.1.
|
|
91
|
-
|
|
92
|
-
|
|
114
|
+
0.16.1, plot3 0.8.1 and stats3 0.6.1. Phase 6: continuation lines (`>>` for
|
|
115
|
+
tidy3, `+` for plot3) and the coordinator, in 0.1.2, which tidy3 0.16, tidy3a
|
|
116
|
+
0.17, plot3 0.9 and stats3 0.7 register with.
|
|
93
117
|
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
`
|
|
118
|
+
Phase 7, in 0.2.0: `__r_name__` and `__r_assign__` are the only
|
|
119
|
+
placeholders any package writes or reads; the packages' own names from before
|
|
120
|
+
(`__tidy3_bt__`, `__plot3_bt__`) are gone. Inside its own calls each
|
|
121
|
+
package's AST pass renames `__r_assign__` to a private name of its own
|
|
122
|
+
(`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in a
|
|
123
|
+
namespace can belong to no single package.
|
|
99
124
|
|
|
100
125
|
Where the output differs from a package's own rewrite before the switch:
|
|
101
126
|
|
|
@@ -105,9 +130,10 @@ Where the output differs from a package's own rewrite before the switch:
|
|
|
105
130
|
(`df.select(a = x, b)`) is rewritten like the verb, as `!` in one already
|
|
106
131
|
was; backticks inside strings and comments stay; a backtick assignment is
|
|
107
132
|
rewritten only as a call argument.
|
|
108
|
-
- tidy3a: its placeholder is `__r_assign__
|
|
109
|
-
- plot3: a
|
|
110
|
-
(`facet_wrap(~
|
|
133
|
+
- tidy3a: its placeholder is `__r_assign__`.
|
|
134
|
+
- plot3: a backtick name inside a one-sided facet formula that another
|
|
135
|
+
package's rewrite reached first (`facet_wrap(~__r_name__('a b'))`) becomes
|
|
136
|
+
the name again: `'~a b'`.
|
|
111
137
|
|
|
112
138
|
`scripts/family/` runs all five suites together on another machine: `sync.sh`
|
|
113
139
|
copies the working trees and starts `run.sh` in a tmux session there, and
|
|
@@ -20,6 +20,29 @@ ts.rewrite('df >> filter(!is_na(x)) >> select(Cus = NUM, Row)', [stats3, tidy3])
|
|
|
20
20
|
# "df >> filter(~is_na(x)) >> select(__r_assign__('Cus', (NUM)), Row)"
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
+
## In a notebook
|
|
24
|
+
|
|
25
|
+
A package's notebook extension registers its dialect with the coordinator,
|
|
26
|
+
which installs one input transformer for every package:
|
|
27
|
+
|
|
28
|
+
```python
|
|
29
|
+
import tidy3syntax.ipython
|
|
30
|
+
|
|
31
|
+
def load_ipython_extension(shell):
|
|
32
|
+
tidy3syntax.ipython.register(shell, DIALECT, ast_transformer=MyMaskTransformer(shell.user_ns))
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
A package can also declare an entry point, so a cell that imports it gets
|
|
36
|
+
its syntax in that same cell:
|
|
37
|
+
|
|
38
|
+
```toml
|
|
39
|
+
[project.entry-points."tidy3syntax.dialects"]
|
|
40
|
+
plot3 = "plot3.jupyter:load_ipython_extension"
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
`%%tidy3syntax plot3` as a cell's first line reinterprets that cell's calls
|
|
44
|
+
whoever owns them.
|
|
45
|
+
|
|
23
46
|
## Two kinds of rewrite
|
|
24
47
|
|
|
25
48
|
**Repairs** change only text that is a syntax error in Python, so no valid
|
|
@@ -68,15 +91,16 @@ turns them into its column references inside its calls.
|
|
|
68
91
|
|
|
69
92
|
Phases 1 to 5 of [the design](docs/design.md): the scanner, every repair,
|
|
70
93
|
the ownership rule and the two reinterpretations, used by tidy3 0.15.1, tidy3a
|
|
71
|
-
0.16.1, plot3 0.8.1 and stats3 0.6.1.
|
|
72
|
-
|
|
73
|
-
|
|
94
|
+
0.16.1, plot3 0.8.1 and stats3 0.6.1. Phase 6: continuation lines (`>>` for
|
|
95
|
+
tidy3, `+` for plot3) and the coordinator, in 0.1.2, which tidy3 0.16, tidy3a
|
|
96
|
+
0.17, plot3 0.9 and stats3 0.7 register with.
|
|
74
97
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
`
|
|
98
|
+
Phase 7, in 0.2.0: `__r_name__` and `__r_assign__` are the only
|
|
99
|
+
placeholders any package writes or reads; the packages' own names from before
|
|
100
|
+
(`__tidy3_bt__`, `__plot3_bt__`) are gone. Inside its own calls each
|
|
101
|
+
package's AST pass renames `__r_assign__` to a private name of its own
|
|
102
|
+
(`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in a
|
|
103
|
+
namespace can belong to no single package.
|
|
80
104
|
|
|
81
105
|
Where the output differs from a package's own rewrite before the switch:
|
|
82
106
|
|
|
@@ -86,9 +110,10 @@ Where the output differs from a package's own rewrite before the switch:
|
|
|
86
110
|
(`df.select(a = x, b)`) is rewritten like the verb, as `!` in one already
|
|
87
111
|
was; backticks inside strings and comments stay; a backtick assignment is
|
|
88
112
|
rewritten only as a call argument.
|
|
89
|
-
- tidy3a: its placeholder is `__r_assign__
|
|
90
|
-
- plot3: a
|
|
91
|
-
(`facet_wrap(~
|
|
113
|
+
- tidy3a: its placeholder is `__r_assign__`.
|
|
114
|
+
- plot3: a backtick name inside a one-sided facet formula that another
|
|
115
|
+
package's rewrite reached first (`facet_wrap(~__r_name__('a b'))`) becomes
|
|
116
|
+
the name again: `'~a b'`.
|
|
92
117
|
|
|
93
118
|
`scripts/family/` runs all five suites together on another machine: `sync.sh`
|
|
94
119
|
copies the working trees and starts `run.sh` in a tmux session there, and
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# A shared package for R syntax in notebooks
|
|
2
2
|
|
|
3
|
-
Status: accepted 2026-11-01; phases 1 to
|
|
3
|
+
Status: accepted 2026-11-01; phases 1 to 6 (§8) implemented and released (tidy3syntax 0.1.2, tidy3 0.16.0, tidy3a 0.17.0, plot3 0.9.0, stats3 0.7.0); phase 7 released in tidy3syntax 0.2.0, tidy3 0.17.0, tidy3a 0.18.0, plot3 0.10.0 and stats3 0.8.0.
|
|
4
4
|
|
|
5
5
|
tidy3, tidy3a, plot3 and stats3 each let notebook cells use R syntax that Python
|
|
6
6
|
cannot parse: bare column names, `` `odd names` ``, `!` for negation, `name =
|
|
@@ -397,17 +397,22 @@ bound raised in each package, with no ported code.
|
|
|
397
397
|
|
|
398
398
|
Each phase ends with every package's suite passing, and nothing is released before its exit check.
|
|
399
399
|
|
|
400
|
-
1. **Create `tidy3syntax` from tidy3's code.** tidy3's copy is the most complete, so it is the base. Move the scanner, `rewrite_backticks`, `rewrite_bang_not`, `rewrite_keywords_before_positional` and `rewrite_formula_rules`, generalised over dialects, plus stats3's and plot3's formula quoting, the ownership rule and the `^` and unary `~` reinterpretations from plot3's fix, and seed the corpus. Exit check: the corpus passes, including every tidy3 rewrite case unchanged. *Done 2026-11-01,
|
|
401
|
-
2. **Switch tidy3**, keeping its old placeholders as aliases (§5). Exit check: tidy3's suite passes with no assertion changed except the placeholder names in rewritten text. *Done 2026-11-01,
|
|
402
|
-
3. **Switch tidy3a.** *Done 2026-11-01,
|
|
403
|
-
4. **Switch stats3.** Replace `quote_formulas` with a dialect of formula calls; its bare-name pass stays. Exit check: stats3's `test_masking.py` passes. *Done 2026-11-01 in stats3
|
|
400
|
+
1. **Create `tidy3syntax` from tidy3's code.** tidy3's copy is the most complete, so it is the base. Move the scanner, `rewrite_backticks`, `rewrite_bang_not`, `rewrite_keywords_before_positional` and `rewrite_formula_rules`, generalised over dialects, plus stats3's and plot3's formula quoting, the ownership rule and the `^` and unary `~` reinterpretations from plot3's fix, and seed the corpus. Exit check: the corpus passes, including every tidy3 rewrite case unchanged. *Done 2026-11-01, released as tidy3syntax 0.1.1 (first as tidysyntax 0.1.0):* `src/tidy3syntax/`, 272 tests on Python 3.10 to 3.14. The cases whose text differs from a package's current output are listed in the README.
|
|
401
|
+
2. **Switch tidy3**, keeping its old placeholders as aliases (§5). Exit check: tidy3's suite passes with no assertion changed except the placeholder names in rewritten text. *Done 2026-11-01, released in tidy3 0.15.0 (0.15.1 on the new name):* tidy3 runs tidy3syntax's repairs and turns the neutral placeholders back into `__tidy3_bt__` and `__tidy3_assign__`, which plot3, stats3 and tidy3a read today and `nb_export` writes; its AST pass also reads the neutral names. One assertion changed: `df.select(a=NUM, b)` is now rewritten like the verb, as `!` in a method call already was, instead of being left as a syntax error.
|
|
402
|
+
3. **Switch tidy3a.** *Done 2026-11-01, released in tidy3a 0.16.0 (0.16.1 on the new name):* it writes `__r_assign__`, still reads `__tidy3a_assign__`, and hands the placeholder to tidy3 as `__tidy3_assign__` in tidy3's calls. Delete its scanner, `rewrite_bang_not`, `rewrite_keywords_before_positional` and the carrier translation in `visit_Call`, and move `[x, y, z] = …` to a dialect stage. Exit check: `test_notebooks.py` passes, and the two-package tests pass in both load orders.
|
|
403
|
+
4. **Switch stats3.** Replace `quote_formulas` with a dialect of formula calls; its bare-name pass stays. Exit check: stats3's `test_masking.py` passes. *Done 2026-11-01, released in stats3 0.6.0 (0.6.1 on the new name):* every existing case passes unchanged, and backtick names in formulas now work (`lm` reads them; `t_test`'s own formula parsing does not yet).
|
|
404
404
|
|
|
405
405
|
Until every package's AST pass reads the neutral placeholders, a package that switches early calls only the repairs that leave none: stats3 calls `tidy3syntax.repairs.quote_formulas`, not `repair`. Otherwise a stats3 pass running first would leave `__r_name__` in a tidy3 call, which tidy3 0.14 does not read. Each package widens to `repair` once tidy3 (phase 2) reads them.
|
|
406
406
|
|
|
407
407
|
tidy3 0.14's backtick pass rewrites backticks inside string literals too. When it runs after another package's formula quoting, it breaks the quoted formula (`'`body mass` ~ sex'` becomes `'__tidy3_bt__('body mass') ~ sex'`). tidy3syntax's backtick repair skips strings, so phase 2 fixes it; until then a backtick name in a stats3 formula works only when tidy3's notebook layer is not loaded.
|
|
408
|
-
5. **Switch plot3.** *Done 2026-11-01 in plot3
|
|
409
|
-
6. **Coordinator, continuation lines and ownership in the AST passes.** Install the single transformer and the entry points; move tidy3's `>>` and plot3's `+` continuation lines; let each package's AST pass use the shared `Owners`. Delete tidy3's plot3 hook and plot3 imports from `export.py`. Exit check: the four-package notebook passes in all 24 load orders.
|
|
410
|
-
|
|
408
|
+
5. **Switch plot3.** *Done 2026-11-01, released in plot3 0.8.0 (0.8.1 on the new name), with its facet-formula work:* its scanner, facet formulas, `^` and ownership check are gone; it turns `__r_name__` back into `__plot3_bt__`, and joining `+` lines uses tidy3syntax's `Owners`. Move backticks and binary `~` facet formulas onto `tidy3syntax` as repairs, and `^`, unary `~` and unindented `+` continuation lines as reinterpretations gated by ownership. Delete its handling of tidy3's placeholders and its detection of tidy3's preparser. Exit check: the §6 regression cases pass in plot3's own suite.
|
|
409
|
+
6. **Coordinator, continuation lines and ownership in the AST passes.** Install the single transformer and the entry points; move tidy3's `>>` and plot3's `+` continuation lines; let each package's AST pass use the shared `Owners`. Delete tidy3's plot3 hook and plot3 imports from `export.py`. Exit check: the four-package notebook passes in all 24 load orders. In three steps, each released on its own (planned 2026-11-01):
|
|
410
|
+
- **6a, tidy3syntax only (0.1.2, additive).** Continuation lines: a line starting with `>>` continues the statement above, a repair (tidy3's `_wrap_pipe_statements`, and `join_broken_pipe`, which joins a broken pipe with backslashes so Python names the real error); an indented `+` line is a repair; an unindented `+` line is a reinterpretation, joined only when the call after `+` is owned by the dialect that lists it and the statement above starts with an owned call or ends a pipe whose last step is one. tidy3 today joins a `+` line after any pipe by name, the §1.3 bug for `+`; this replaces it. The coordinator, `tidy3syntax.ipython`: one input transformer for every registered dialect, kept ahead of CRAFT's router before each cell (tidy3's `pre_run_cell` hook does this today), the importing cell (§4.7), and entry points. The integration test: a notebook with the four packages in all 24 load orders, run in this repository's CI on pull requests and weekly.
|
|
411
|
+
- **6b, each package (one release each).** Register with the coordinator instead of installing a transformer; delete its continuation code, its handling of the other packages' placeholders and hooks (tidy3's plot3 hook and `export.py`'s plot3 imports); write the neutral names; use `Owners` in its AST pass. Packages on 6b and packages not yet on it must still share a notebook: the coordinator leaves a package's own transformer in place until that package registers.
|
|
412
|
+
- **6c.** The integration test passes against the four packages' releases. *Done 2026-11-01:* tidy3syntax's CI installed tidy3 0.16.0, tidy3a 0.17.0, plot3 0.9.0 and stats3 0.7.0 from PyPI with tidy3syntax 0.1.2, and all 24 load orders passed.
|
|
413
|
+
|
|
414
|
+
Progress (2026-11-02): 6a is committed (continuation lines, the coordinator, the integration test). 6b is done in the four packages' working trees, not released: each registers with the coordinator and declares an entry point; stats3, tidy3a and plot3 read `__r_name__` as a column; each AST pass renames `__r_assign__` to its own private name inside its own calls (`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in the namespace can belong to no single package; plot3's AST pass checks ownership with `Owners`; tidy3a takes `>>` continuation lines; tidy3's `pre_run_cell` hook and plot3 hook are gone, and plot3 no longer looks for tidy3's preparser. tidy3's `nb_export`, partial runs and CRAFT path rewrite through tidy3syntax as well (2026-11-02): an entry point names a function returning the package's dialect, whose `extension` is its notebook loader and whose `mask` applies its AST pass to source text; `tidy3syntax.installed()` lists them, so `export.py` finds plot3 without importing it. tidy3a's and stats3's dialects have no `mask` yet: their AST passes write internal names an export cannot turn into public code, and tidy3's export did not handle them before either. At release, tidy3 should require the plot3 release that has the entry point.
|
|
415
|
+
7. **Drop the old placeholder aliases** two minor releases after phase 2. *Done 2026-11-01, one minor release after phase 2 at the owner's choice, and released 2026-11-02:* the source passes write only `__r_name__` and `__r_assign__`, and no AST pass reads `__tidy3_bt__` or `__plot3_bt__`, or another package's assignment name; tidy3a no longer hands `__r_assign__` to tidy3. What stays are the private runtime names each AST pass renames `__r_assign__` to inside its own calls (`__tidy3_assign__`, `__tidy3a_assign__`), which no source text holds; scripts `nb_export` wrote define `__tidy3_assign__` themselves, so they keep running. A notebook mixing these releases with tidy3 0.15, tidy3a 0.16, plot3 0.8 or stats3 0.6 or older can lose backtick names in the other package's calls.
|
|
411
416
|
|
|
412
417
|
Phases 2 to 5 are independent once phase 1 is released. Phase 6 needs all four.
|
|
413
418
|
|
|
@@ -452,5 +457,5 @@ than writing it.
|
|
|
452
457
|
1. ~~Name~~: `tidy3syntax`, named for the tidy3 family it serves. It was first published as `tidysyntax` 0.1.0 on 2026-11-01 and renamed the same day; that release is superseded, and the packages' next releases depend on `tidy3syntax`.
|
|
453
458
|
2. **Dependency or vendoring** (§9). Phase 1 assumes a dependency.
|
|
454
459
|
3. **When to rename placeholders**: in phase 2, as proposed, or after phase 5, so each package's first switch changes no visible names.
|
|
455
|
-
4.
|
|
456
|
-
5.
|
|
460
|
+
4. ~~New syntax per package~~ (decided 2026-11-01): each package takes the syntax tidy3syntax already offers that has a reading in its calls. tidy3a takes backtick names and backtick assignments now, and `>>` continuation lines when phase 6 moves them here. plot3 does not take `!`: its aesthetics read names as column-name strings, so `!flag` has no meaning there.
|
|
461
|
+
5. ~~Where the four-package integration test runs~~ (decided 2026-11-01): the four-package integration test lives in tidy3syntax and runs in its CI. A separate compatibility repository can be introduced if release coordination later requires it. tidy3syntax owns the mixed-package notebook and the tests of interactions, including the packages' load orders; each package owns its detailed behaviour tests; the integration test runs on tidy3syntax's pull requests and on a schedule, so a change in a package also shows up there.
|
|
@@ -22,7 +22,8 @@ classifiers = [
|
|
|
22
22
|
dependencies = []
|
|
23
23
|
|
|
24
24
|
[project.optional-dependencies]
|
|
25
|
-
|
|
25
|
+
# IPython: the coordinator's tests run real cells.
|
|
26
|
+
dev = ["pytest>=7.0", "ipython>=8.0"]
|
|
26
27
|
|
|
27
28
|
[project.urls]
|
|
28
29
|
Homepage = "https://github.com/rleyvasal/tidy3syntax"
|
|
@@ -20,6 +20,13 @@ from tidy3syntax.pipeline import reinterpret, rewrite
|
|
|
20
20
|
from tidy3syntax.placeholders import ASSIGN, NAME, RAssign, RName, runtime_values
|
|
21
21
|
from tidy3syntax.repairs import repair
|
|
22
22
|
|
|
23
|
+
|
|
24
|
+
def installed() -> list[Dialect]:
|
|
25
|
+
"""The dialects of every installed package that declares a ``tidy3syntax.dialects`` entry point."""
|
|
26
|
+
from tidy3syntax.ipython import installed as found
|
|
27
|
+
|
|
28
|
+
return found()
|
|
29
|
+
|
|
23
30
|
__all__ = [
|
|
24
31
|
"ASSIGN",
|
|
25
32
|
"NAME",
|
|
@@ -30,6 +37,7 @@ __all__ = [
|
|
|
30
37
|
"RName",
|
|
31
38
|
"__version__",
|
|
32
39
|
"imported_names",
|
|
40
|
+
"installed",
|
|
33
41
|
"module_of",
|
|
34
42
|
"owned_calls",
|
|
35
43
|
"reinterpret",
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
__version__ = "0.2.0"
|