tidy3syntax 0.1.2__tar.gz → 0.2.0__tar.gz

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.2.0 — 2026-11-02
+
+Phase 7 of [the design](docs/design.md).
+
+### Changed
+
+- `Dialect.mask` takes the notebook's imports as a third argument,
+  `mask(source, known, context)`, so a package's export masking can tell its
+  own calls from another package's; `Dialect.export_imports` lists the lines
+  an exported script needs once the mask changed something (`import tidy3a`).
+- A formula being quoted turns only `__r_name__` back into a backtick name;
+  tidy3's `__tidy3_bt__` and plot3's `__plot3_bt__`, which no release of
+  either writes since tidy3 0.17 and plot3 0.10, are no longer read.
+
 ## 0.1.2 — 2026-11-01
 
 Phase 6a of [the design](docs/design.md): continuation lines and the coordinator.

{tidy3syntax-0.1.2/src/tidy3syntax.egg-info → tidy3syntax-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tidy3syntax
-Version: 0.1.2
+Version: 0.2.0
 Summary: R syntax in notebook cells, rewritten to Python: shared by tidy3, tidy3a, plot3 and stats3
 Author: Rigoberto Leyva Salmeron
 License-Expression: MIT
@@ -115,11 +115,12 @@ the ownership rule and the two reinterpretations, used by tidy3 0.15.1, tidy3a
 tidy3, `+` for plot3) and the coordinator, in 0.1.2, which tidy3 0.16, tidy3a
 0.17, plot3 0.9 and stats3 0.7 register with.
 
-Until every package's AST pass reads the neutral placeholders, each package
-calls only the repairs whose output the others already read, and tidy3 and
-plot3 turn `__r_name__` and `__r_assign__` back into their own names
-(`__tidy3_bt__`, `__tidy3_assign__`, `__plot3_bt__`). tidy3a writes
-`__r_assign__` and hands it to tidy3 as `__tidy3_assign__` in tidy3's calls.
+Phase 7, in 0.2.0: `__r_name__` and `__r_assign__` are the only
+placeholders any package writes or reads; the packages' own names from before
+(`__tidy3_bt__`, `__plot3_bt__`) are gone. Inside its own calls each
+package's AST pass renames `__r_assign__` to a private name of its own
+(`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in a
+namespace can belong to no single package.
 
 Where the output differs from a package's own rewrite before the switch:
 
@@ -129,9 +130,10 @@ Where the output differs from a package's own rewrite before the switch:
   (`df.select(a = x, b)`) is rewritten like the verb, as `!` in one already
   was; backticks inside strings and comments stay; a backtick assignment is
   rewritten only as a call argument.
-- tidy3a: its placeholder is `__r_assign__` (it still reads `__tidy3a_assign__`).
-- plot3: a placeholder another package left inside a one-sided facet formula
-  (`facet_wrap(~__tidy3_bt__('a b'))`) becomes the name again: `'~a b'`.
+- tidy3a: its placeholder is `__r_assign__`.
+- plot3: a backtick name inside a one-sided facet formula that another
+  package's rewrite reached first (`facet_wrap(~__r_name__('a b'))`) becomes
+  the name again: `'~a b'`.
 
 `scripts/family/` runs all five suites together on another machine: `sync.sh`
 copies the working trees and starts `run.sh` in a tmux session there, and

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/README.md

@@ -95,11 +95,12 @@ the ownership rule and the two reinterpretations, used by tidy3 0.15.1, tidy3a
 tidy3, `+` for plot3) and the coordinator, in 0.1.2, which tidy3 0.16, tidy3a
 0.17, plot3 0.9 and stats3 0.7 register with.
 
-Until every package's AST pass reads the neutral placeholders, each package
-calls only the repairs whose output the others already read, and tidy3 and
-plot3 turn `__r_name__` and `__r_assign__` back into their own names
-(`__tidy3_bt__`, `__tidy3_assign__`, `__plot3_bt__`). tidy3a writes
-`__r_assign__` and hands it to tidy3 as `__tidy3_assign__` in tidy3's calls.
+Phase 7, in 0.2.0: `__r_name__` and `__r_assign__` are the only
+placeholders any package writes or reads; the packages' own names from before
+(`__tidy3_bt__`, `__plot3_bt__`) are gone. Inside its own calls each
+package's AST pass renames `__r_assign__` to a private name of its own
+(`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in a
+namespace can belong to no single package.
 
 Where the output differs from a package's own rewrite before the switch:
 
@@ -109,9 +110,10 @@ Where the output differs from a package's own rewrite before the switch:
   (`df.select(a = x, b)`) is rewritten like the verb, as `!` in one already
   was; backticks inside strings and comments stay; a backtick assignment is
   rewritten only as a call argument.
-- tidy3a: its placeholder is `__r_assign__` (it still reads `__tidy3a_assign__`).
-- plot3: a placeholder another package left inside a one-sided facet formula
-  (`facet_wrap(~__tidy3_bt__('a b'))`) becomes the name again: `'~a b'`.
+- tidy3a: its placeholder is `__r_assign__`.
+- plot3: a backtick name inside a one-sided facet formula that another
+  package's rewrite reached first (`facet_wrap(~__r_name__('a b'))`) becomes
+  the name again: `'~a b'`.
 
 `scripts/family/` runs all five suites together on another machine: `sync.sh`
 copies the working trees and starts `run.sh` in a tmux session there, and

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/docs/design.md

@@ -1,6 +1,6 @@
 # A shared package for R syntax in notebooks
 
-Status: accepted 2026-11-01; phases 1 to 5 (§8) implemented and released (tidy3syntax 0.1.1, tidy3 0.15.1, tidy3a 0.16.1, plot3 0.8.1, stats3 0.6.1); phases 6 and 7 to come.
+Status: accepted 2026-11-01; phases 1 to 6 (§8) implemented and released (tidy3syntax 0.1.2, tidy3 0.16.0, tidy3a 0.17.0, plot3 0.9.0, stats3 0.7.0); phase 7 released in tidy3syntax 0.2.0, tidy3 0.17.0, tidy3a 0.18.0, plot3 0.10.0 and stats3 0.8.0.
 
 tidy3, tidy3a, plot3 and stats3 each let notebook cells use R syntax that Python
 cannot parse: bare column names, `` `odd names` ``, `!` for negation, `name =
@@ -409,10 +409,10 @@ tidy3 0.14's backtick pass rewrites backticks inside string literals too. When i
 6. **Coordinator, continuation lines and ownership in the AST passes.** Install the single transformer and the entry points; move tidy3's `>>` and plot3's `+` continuation lines; let each package's AST pass use the shared `Owners`. Delete tidy3's plot3 hook and plot3 imports from `export.py`. Exit check: the four-package notebook passes in all 24 load orders. In three steps, each released on its own (planned 2026-11-01):
    - **6a, tidy3syntax only (0.1.2, additive).** Continuation lines: a line starting with `>>` continues the statement above, a repair (tidy3's `_wrap_pipe_statements`, and `join_broken_pipe`, which joins a broken pipe with backslashes so Python names the real error); an indented `+` line is a repair; an unindented `+` line is a reinterpretation, joined only when the call after `+` is owned by the dialect that lists it and the statement above starts with an owned call or ends a pipe whose last step is one. tidy3 today joins a `+` line after any pipe by name, the §1.3 bug for `+`; this replaces it. The coordinator, `tidy3syntax.ipython`: one input transformer for every registered dialect, kept ahead of CRAFT's router before each cell (tidy3's `pre_run_cell` hook does this today), the importing cell (§4.7), and entry points. The integration test: a notebook with the four packages in all 24 load orders, run in this repository's CI on pull requests and weekly.
    - **6b, each package (one release each).** Register with the coordinator instead of installing a transformer; delete its continuation code, its handling of the other packages' placeholders and hooks (tidy3's plot3 hook and `export.py`'s plot3 imports); write the neutral names; use `Owners` in its AST pass. Packages on 6b and packages not yet on it must still share a notebook: the coordinator leaves a package's own transformer in place until that package registers.
-   - **6c.** The integration test passes against the four packages' releases.
+   - **6c.** The integration test passes against the four packages' releases. *Done 2026-11-01:* tidy3syntax's CI installed tidy3 0.16.0, tidy3a 0.17.0, plot3 0.9.0 and stats3 0.7.0 from PyPI with tidy3syntax 0.1.2, and all 24 load orders passed.
 
    Progress (2026-11-02): 6a is committed (continuation lines, the coordinator, the integration test). 6b is done in the four packages' working trees, not released: each registers with the coordinator and declares an entry point; stats3, tidy3a and plot3 read `__r_name__` as a column; each AST pass renames `__r_assign__` to its own private name inside its own calls (`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in the namespace can belong to no single package; plot3's AST pass checks ownership with `Owners`; tidy3a takes `>>` continuation lines; tidy3's `pre_run_cell` hook and plot3 hook are gone, and plot3 no longer looks for tidy3's preparser. tidy3's `nb_export`, partial runs and CRAFT path rewrite through tidy3syntax as well (2026-11-02): an entry point names a function returning the package's dialect, whose `extension` is its notebook loader and whose `mask` applies its AST pass to source text; `tidy3syntax.installed()` lists them, so `export.py` finds plot3 without importing it. tidy3a's and stats3's dialects have no `mask` yet: their AST passes write internal names an export cannot turn into public code, and tidy3's export did not handle them before either. At release, tidy3 should require the plot3 release that has the entry point.
-7. **Drop the old placeholder aliases** two minor releases after phase 2.
+7. **Drop the old placeholder aliases** two minor releases after phase 2. *Done 2026-11-01, one minor release after phase 2 at the owner's choice, and released 2026-11-02:* the source passes write only `__r_name__` and `__r_assign__`, and no AST pass reads `__tidy3_bt__` or `__plot3_bt__`, or another package's assignment name; tidy3a no longer hands `__r_assign__` to tidy3. What stays are the private runtime names each AST pass renames `__r_assign__` to inside its own calls (`__tidy3_assign__`, `__tidy3a_assign__`), which no source text holds; scripts `nb_export` wrote define `__tidy3_assign__` themselves, so they keep running. A notebook mixing these releases with tidy3 0.15, tidy3a 0.16, plot3 0.8 or stats3 0.6 or older can lose backtick names in the other package's calls.
 
 Phases 2 to 5 are independent once phase 1 is released. Phase 6 needs all four.
 

tidy3syntax-0.2.0/src/tidy3syntax/__version__.py

@@ -0,0 +1 @@
+__version__ = "0.2.0"

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/src/tidy3syntax/dialect.py

@@ -3,9 +3,9 @@
 from __future__ import annotations
 
 import importlib
-from collections.abc import Callable, Iterable
+from collections.abc import Callable, Iterable, Mapping
 from dataclasses import dataclass, field
-from typing import Literal
+from typing import Any, Literal
 
 
 def _names(values: Iterable[str]) -> frozenset[str]:
@@ -51,9 +51,13 @@ class Dialect:
     # The package's notebook loader, "module:function" taking the shell (its load_ipython_extension):
     # the coordinator calls it for a cell that imports the package.
     extension: str | None = field(default=None, compare=False)
-    # The package's AST pass on source text, given the names that are not columns (an export's
-    # imports and assignments): mask(source, known) -> source. None for a package without one.
-    mask: Callable[[str, frozenset[str]], str] | None = field(default=None, compare=False)
+    # The package's AST pass on source text, for an export: mask(source, known, context) -> source,
+    # where known holds the names that are not columns (the notebook's imports and assignments) and
+    # context maps the names the notebook imports to their modules (ownership.imported_names). The
+    # result is public code a plain script can run, given the dialect's export_imports.
+    mask: Callable[[str, frozenset[str], Mapping[str, Any]], str] | None = field(default=None, compare=False)
+    # Lines an exported script needs once the mask has changed a cell ("import tidy3a").
+    export_imports: tuple[str, ...] = field(default=(), compare=False)
 
     def __post_init__(self) -> None:
         for name in (

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/src/tidy3syntax/repairs.py

@@ -32,8 +32,8 @@ from tidy3syntax.scanner import (
 )
 
 _BACKTICK = re.compile(r"`([^`\n]+)`")
-# Placeholders a package's own pass may have left before tidy3syntax ran (tidy3, plot3), and tidy3syntax's own.
-_NAME_CALL = re.compile(r"__(?:r_name|tidy3_bt|plot3_bt)__\((['\"])(.*?)\1\)")
+# A backtick name another rewrite already turned into the placeholder, inside a formula being quoted.
+_NAME_CALL = re.compile(r"__r_name__\((['\"])(.*?)\1\)")
 
 
 def _union(dialects: Iterable[Dialect], field: str) -> frozenset[str]:

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0/src/tidy3syntax.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tidy3syntax
-Version: 0.1.2
+Version: 0.2.0
 Summary: R syntax in notebook cells, rewritten to Python: shared by tidy3, tidy3a, plot3 and stats3
 Author: Rigoberto Leyva Salmeron
 License-Expression: MIT
@@ -115,11 +115,12 @@ the ownership rule and the two reinterpretations, used by tidy3 0.15.1, tidy3a
 tidy3, `+` for plot3) and the coordinator, in 0.1.2, which tidy3 0.16, tidy3a
 0.17, plot3 0.9 and stats3 0.7 register with.
 
-Until every package's AST pass reads the neutral placeholders, each package
-calls only the repairs whose output the others already read, and tidy3 and
-plot3 turn `__r_name__` and `__r_assign__` back into their own names
-(`__tidy3_bt__`, `__tidy3_assign__`, `__plot3_bt__`). tidy3a writes
-`__r_assign__` and hands it to tidy3 as `__tidy3_assign__` in tidy3's calls.
+Phase 7, in 0.2.0: `__r_name__` and `__r_assign__` are the only
+placeholders any package writes or reads; the packages' own names from before
+(`__tidy3_bt__`, `__plot3_bt__`) are gone. Inside its own calls each
+package's AST pass renames `__r_assign__` to a private name of its own
+(`__tidy3_assign__`, `__tidy3a_assign__`), since the shared name's value in a
+namespace can belong to no single package.
 
 Where the output differs from a package's own rewrite before the switch:
 
@@ -129,9 +130,10 @@ Where the output differs from a package's own rewrite before the switch:
   (`df.select(a = x, b)`) is rewritten like the verb, as `!` in one already
   was; backticks inside strings and comments stay; a backtick assignment is
   rewritten only as a call argument.
-- tidy3a: its placeholder is `__r_assign__` (it still reads `__tidy3a_assign__`).
-- plot3: a placeholder another package left inside a one-sided facet formula
-  (`facet_wrap(~__tidy3_bt__('a b'))`) becomes the name again: `'~a b'`.
+- tidy3a: its placeholder is `__r_assign__`.
+- plot3: a backtick name inside a one-sided facet formula that another
+  package's rewrite reached first (`facet_wrap(~__r_name__('a b'))`) becomes
+  the name again: `'~a b'`.
 
 `scripts/family/` runs all five suites together on another machine: `sync.sh`
 copies the working trees and starts `run.sh` in a tmux session there, and

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/tests/test_ipython.py

@@ -144,13 +144,13 @@ def test_entry_points_name_a_function_returning_the_dialect(shell, monkeypatch):
     module.loaded = []
     module.dialect = lambda: Dialect(
         "fakeplot", power_calls={"geom_function"}, exports={"geom_function"},
-        extension="fakeplot:load", mask=lambda source, known: source.replace("MASKED", "masked"),
+        extension="fakeplot:load", mask=lambda source, known, context: source.replace("MASKED", "masked"),
     )
     module.load = lambda shell: (module.loaded.append(shell), coordinator.register(shell, module.dialect()))
     _entry_points(monkeypatch, ("fakeplot", "fakeplot:dialect"), ("broken", "nowhere_at_all:dialect"))
 
     assert [d.package for d in coordinator.installed()] == ["fakeplot"]
-    assert coordinator.installed()[0].mask("MASKED", frozenset()) == "masked"
+    assert coordinator.installed()[0].mask("MASKED", frozenset(), {}) == "masked"
 
     coordinator.register(shell, VERBS)
     ns = run(shell, "from fakeplot import geom_function\nr = geom_function(lambda x: x ^ 2)")

{tidy3syntax-0.1.2 → tidy3syntax-0.2.0}/tests/test_reinterpretations.py

@@ -38,8 +38,8 @@ USER_GEOM_FUNCTION = "def geom_function(fun):\n    return fun(3)\n"
         ("facet_wrap( ~ cyl , ncol=2)", "facet_wrap( '~ cyl' , ncol=2)"),
         ("p + facet_wrap(\n    ~ gear,\n    ncol=2)", "p + facet_wrap(\n    '~ gear',\n    ncol=2)"),
         ('facet_wrap("~cyl")', None),
-        # tidy3's backtick placeholder, left by a pass that ran first, keeps its name inside the formula.
-        ("facet_wrap(~__tidy3_bt__('my col'))", "facet_wrap('~my col')"),
+        # A backtick name the backtick repair already rewrote keeps its name inside the formula.
+        ("facet_wrap(~__r_name__('my col'))", "facet_wrap('~my col')"),
     ],
 )
 def test_only_plot3_calls_are_reinterpreted(cell, expected):

tidy3syntax-0.1.2/src/tidy3syntax/__version__.py

@@ -1 +0,0 @@
-__version__ = "0.1.2"