redroot 0.2.0__tar.gz → 0.4.0__tar.gz

redroot-0.4.0/CHANGELOG.md

@@ -0,0 +1,219 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.4.0] - 2026-10-07
+
+### Changed
+
+- `@traced` functions defined in code run through `exec()` (which have no
+  module) are named `<exec>.<name>` instead of `<name>`, and `@traced`
+  refuses the names of built-in operations. Traces saved by 0.3.0 that used
+  such helpers report them as not replayable.
+- `explain()` is written for reviewers: other outputs appear by name
+  (`out:summary.total_matched - out:summary.total_expected`) instead of being
+  expanded, containers with more than three items are summarised
+  (`deposits_by_day([5 items], [11 items], 'SQUARE')`), long constants are
+  shortened and the formula stays within `max_length` (200) characters.
+  `explain(..., full=True)` expands everything. Reasons use the same
+  rendering, capped at 120 characters.
+- Graphs stay small: `visualizer.graph_data()` groups inputs (and repeated
+  outputs) that differ only by list index (`ext:deposits[*].amount · 11
+  inputs`), hides list-length checks, focuses on one output's lineage up to
+  other outputs (`focus=`, with drill-down), and returns at most `max_nodes`
+  nodes. The web viewer and `to_graphviz()` use it; the viewer's
+  `/api/graph?focus=<key>` serves focused graphs.
+- An edit naming an input the run did not have, under one of its input roots
+  (e.g. an added row), is reported as the new run-level reason `NEW_INPUT`
+  instead of raising `KeyError`, so callers can handle it like any other
+  re-execution case. Keys under other roots still raise.
+- `exec_source()` gives each source a unique default filename
+  (`<workflow 1a2b3c...>`) instead of `<workflow>`, so concurrent workflows
+  cannot be confused.
+- Recalculation scales with the edit, not the run: `propagate()` follows an
+  index from each step to the steps that use it (built once per trace)
+  instead of scanning everything recorded after the edited input, and output
+  statuses are computed when first read. On a run with 10,000 invoices
+  (120,000 steps, 70,000 outputs), editing one invoice went from 103 ms to
+  under 2 ms. `descendants()` and `dependents()` use the same index.
+
+### Added
+- Overriding calculated values: `propagate(edits, overrides={"out:summary.total":
+  Decimal("9000.00")})` pins that step's value (by output key, node or traced
+  value) and recalculates everything after it, re-checking decisions as
+  usual. Outputs bound to it get the new status `OVERRIDDEN`;
+  `Propagation.overridden` lists them, and `verify()` skips what overrides
+  reach, since a re-execution cannot apply them.
+- `propagate(..., changed_only=True)` reports only the outputs whose status
+  is `UPDATED`, `STALE` or `OVERRIDDEN`.
+- Traces record a fingerprint of every `@traced` function they use (a hash
+  of its source, or `@traced(version=...)`), serialized as `"functions"`.
+  When a loaded trace is recalculated and the registered function differs
+  (or is not registered), its results are reported stale with the new reason
+  `FUNCTION_CHANGED` instead of being recomputed with different code.
+  `Trace.changed_functions()` lists such functions up front.
+- `redroot.limits`: bounds on recalculation for traces from untrusted places.
+  The riskiest operations (powers, shifts, repetition, padding and format
+  widths, `str.replace` growth, `decimal` precision) are refused before they
+  run, every result's size is checked, and `Limits(max_seconds=...,
+  max_steps=...)` adds time and step budgets. Size limits are on by default
+  for `propagate()` and `check_identity()`; exceeding one raises
+  `LimitExceeded`.
+- Inputs can carry their source from the start: `track(data, root,
+  envelope=Envelope())` recognises values wrapped as `{"value": ...,
+  "source": {...}}`, keys each by its field's path (`ext:bank.balance`), traces
+  the value in place and stores the source on the input
+  (`node.meta["source"]`); `meta=fn` attaches any metadata. `run()`,
+  `apply_edits()`, `check_perturbation()` and `check_fidelity()` accept
+  `envelope` too.
+- In instrumented code, plain `json.dumps(...)` (no `default=` or `cls=`) is
+  one repeatable step, so its output stays traced and updates exactly,
+  however deeply the payload nests traced values (named tuples and dict
+  subclasses such as `OrderedDict` included); a custom encoder remains an
+  opaque, guarded call. `json.dump` writes real values and is guarded:
+  recalculation cannot rewrite the file, and what was written may be read
+  back, so editing what it wrote requires re-execution.
+- In instrumented code, `x if cond else y` whose branches are literals or
+  variables is recorded as a choice step: when the condition flips, the
+  other value is picked exactly instead of requiring re-execution
+  (checkboxes such as `"Yes" if married else "Off"`, picking one of two
+  rates). A single comparison used as the condition is recorded as data even
+  without `bools=True`. Falls back to a guarded branch when a branch cannot
+  be evaluated yet or is not a scalar.
+- `benchmarks/propagate.py` measures recalculation on large runs.
+
+### Fixed
+
+- In instrumented code, a call whose arguments hold traced values more than
+  two levels deep, or inside a dict or list subclass, is now recorded (or
+  guarded) instead of losing its lineage silently.
+- In class bodies, conditional expressions in method defaults and decorator
+  arguments read class-level names again.
+- An unassigned local read in a conditional expression raises
+  `UnboundLocalError`, as in plain Python, not `NameError`.
+- `apply_edits(..., envelope=...)` accepts edits to an envelope's other
+  fields (`ext:balance.source.page`).
+- `explain()` puts parentheses around a conditional expression nested in
+  another one, and rejects `max_depth`/`max_length` below 1.
+- Focused graphs show an input that is also an output as the input, and
+  `max_nodes` counts every node, after grouping.
+- Function fingerprints include the constant values a helper reads from its
+  globals, closure and defaults: two helpers with the same source but
+  different rates were swapped after a trace was loaded.
+- A named tuple passed to a `@traced` function is saved with its class
+  name, and a loaded trace reports such steps as not replayable, instead of
+  replaying them with a plain tuple.
+- Overriding an output bound to an input without a key crashed.
+- Reading a `Propagation` after its trace recorded more raises
+  `RuntimeError`; it used to mix the old results with the new steps.
+- Limits: modular `pow` and converting or formatting `Decimal`s with huge
+  exponents are checked before they run (a tampered trace could stall a
+  recalculation for hours); `%(key)s` templates whose key holds digits and
+  `0 << n` are no longer refused.
+- `apply_edits()` can add a field or append a row, and explains edits it
+  cannot apply instead of raising `IndexError`/`TypeError`.
+
+
+## [0.3.0] - 2026-10-05
+
+### Added
+- Boolean tracing, opt-in with `Trace(bools=True)` (or `run(..., bools=True)`):
+  comparisons return `TracedBool` values that are data rather than guards, so
+  flags that are stored or computed with propagate exactly; a guard is
+  recorded only where a flag's truth value is used. Boolean inputs become
+  editable inputs. Instrumented code also routes `not`, `x is True/False`,
+  `isinstance(x, bool)` and `x in traced_str`, and passes real `bool`s to
+  external code.
+- `validation.check_fidelity()`: run a workflow plainly and traced and
+  compare their outputs, catching differences tracing itself introduces.
+- `redroot.unwrap_deep()` is public.
+
+### Fixed
+- `explain()` no longer doubles parentheses around operator expressions
+  passed to calls, e.g. `round(x * 1.1, 2)` instead of `round((x * 1.1), 2)`.
+
+## [0.2.0] - 2026-10-02
+
+Re-evaluable traces: RedRoot now answers "what does every output become if
+this input changes?" without re-running the code, and says when it cannot.
+
+### Added
+- `Trace`: a per-run recording, used as a context manager, with inputs
+  (`track`, `leaf`) and outputs (`collect`, `output`) keyed by semantic
+  paths such as `ext:bank.line_items[3].amount`; `run()` helper.
+- Navigation: `sources()`, `ancestors()`, `descendants()`, `dependents()`,
+  `explain()`.
+- Forward propagation: `Trace.propagate()` returns per-output
+  `UNCHANGED`/`UPDATED`/`STALE`/`UNLINKED` statuses with reasons, with early
+  cutoff, and `Propagation.verify()` compares against a re-execution.
+- Guards: comparisons, truth tests, `int()`/`float()`/`hash()`/`len()` and
+  container shapes are recorded and re-checked during propagation.
+- `TracedDecimal`; `round()`, `math.floor/ceil/trunc`, `format()`, `str()`,
+  `divmod()`, `str` and `Decimal` methods stay traced; results that are
+  lists, tuples or dicts are traced item by item.
+- `@traced` (one re-callable node per call), `@traced_llm`, `annotate()`,
+  `derive()`.
+- `redroot.instrument`: source rewriting that traces what operator
+  overloading cannot see (`0.8 * traced_int`, `float(text)`, f-strings,
+  `", ".join(...)`, indexing with traced ints).
+- `redroot.validation`: `coverage()`, `check_identity()`,
+  `check_perturbation()`.
+- Versioned JSON serialization (`to_json`/`from_json`); CLI commands
+  `explain` and `coverage`; web viewer highlights an output's lineage.
+- `docs/design.md`, `docs/integration.md`, `AGENTS.md`, `SECURITY.md`,
+  a tracing-overhead benchmark.
+
+### Changed
+- **Renamed from RedThread to RedRoot.** The name `redthread` on PyPI belongs
+  to an unrelated project, so the package is now `pip install redroot` and
+  `import redroot`; the command-line tool is `redroot`, and the trace format
+  is identified as `redroot.trace`.
+- **Breaking:** nodes are lightweight objects with integer ids, owned by a
+  `Trace`; operations are recorded only inside an active trace (outside one,
+  traced values behave like plain values).
+- **Breaking:** every operand is recorded in order, constants included.
+- Pydantic is now an optional extra (`redroot[pydantic]`); the core has no
+  dependencies.
+- About 8x faster per traced operation than 0.1.0.
+
+### Removed
+- **Breaking:** `LineageGraph`, `TracedValue`, `get_context()`,
+  `lineage_context()`, `transform()` and `trace()`, superseded by `Trace`,
+  `annotate()` and `explain()`.
+
+### Fixed
+- Mixed-type arithmetic returned wrong values (`TracedInt(5) + 1.5 == 6`).
+- `TracedInt / 2` dropped tracing.
+- `transform()` and `@traced_llm` raised `TypeError`.
+- `lineage_context()` did not isolate recorded nodes; the global registry
+  grew without bound.
+- Passing a traced value through a Pydantic model severed its lineage.
+- The web viewer inserted trace values into the page as HTML.
+- Soundness gaps found by adversarial review, where propagation reported an
+  exact value that re-execution contradicted: exceptions caught by
+  `try`/`except` (now recorded as guards), `decimal.localcontext()` (each
+  `Decimal` node keeps its context), intermediate values changing type,
+  `@traced` helpers sharing a name, inputs reaching `@traced` functions
+  through sets, dict views or closures, `hash(-1) == hash(-2)`, and, under
+  instrumentation, `bisect`/`sorted`/`heapq` with plain floats, `min`/`max`
+  with closure keys, `map(float, ...)`, `sum(gen, start)`, unbound
+  `str` methods, `repr`, f-strings of containers, and opaque library calls
+  such as `date.fromisoformat` (now guarded).
+- `apply_edits()` failed on tuple inputs.
+
+### Security
+- The web viewer renders trace content as text only.
+
+## [0.1.0] - 2026-01-24
+
+Prototype, not published to PyPI.
+
+### Added
+- `TracedInt`, `TracedFloat`, `TracedStr`, a lineage graph
+  with JSON export, LLM transformation tracking, Pydantic integration,
+  GraphViz/NetworkX export and a web visualizer.

{redroot-0.2.0 → redroot-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: redroot
-Version: 0.2.0
+Version: 0.4.0
 Summary: Value-level lineage for Python: trace where every output came from, and propagate input edits to every output without re-running
 Project-URL: Homepage, https://github.com/gagan-gaurav/redthread
 Project-URL: Documentation, https://github.com/gagan-gaurav/redthread#readme
@@ -101,7 +101,10 @@ for key, change in result.outputs.items():
 - **Guards.** Comparisons, branches, conversions and lookups are recorded. If an edit would change one, every output is reported `stale` rather than shown with a value that may be wrong.
 - **Semantic keys.** Inputs and outputs are named by their path, e.g. `ext:bank.line_items[3].amount`.
 - **No cooperation needed from the code.** Tracing happens through operator overloading. For code you run from source, such as generated code, `redroot.instrument` closes the remaining gaps.
+- **Fast on large runs.** Recalculation visits only what an edit reaches: one edit in a run of 120,000 steps takes about 2 ms.
+- **Fix calculated values.** Override any calculated step and everything after it is recalculated.
 - **Re-execution as the arbiter.** Built-in tools compare the graph's answers with a real re-run, and measure coverage on your workloads.
+- **Safe to load.** Traces record which version of each helper they used, and recalculation is bounded against tampered traces.
 - **Opaque steps.** `@traced` turns a function (a tax-table lookup, a geocoder) into one re-callable node. `@traced_llm` marks LLM calls as not replayable.
 - **Persistable.** Traces serialize to versioned JSON, ship with a CLI, and come with a web viewer plus GraphViz and NetworkX exports.
 - **Zero dependencies** in the core. Pydantic, visualization and the web viewer are optional extras.
@@ -145,6 +148,18 @@ print(trace.outputs["out:summary.year"].linked)  # False
 
 `rr.run(workflow, inputs, input_root=..., output_root=...)` does the `track`, call and `collect` steps for you.
 
+Extracted values often arrive wrapped with their provenance. `envelope=rr.Envelope()` keys each wrapped value by its field, traces it in place (the workflow sees the same shape) and stores the source on the input:
+
+```python
+extracted = {"balance": {"value": Decimal("12450.00"), "source": {"file": "stmt.pdf", "page": 2}}}
+with rr.Trace() as trace:
+    ext = trace.track(extracted, root="ext", envelope=rr.Envelope())
+print(list(trace.inputs))  # ['ext:balance']
+print(trace.inputs["ext:balance"].meta)  # {'source': {'file': 'stmt.pdf', 'page': 2}}
+```
+
+`track(..., meta=lambda key, value: {...})` attaches any other metadata.
+
 ### Navigating backwards and forwards
 
 | Question | Call |
@@ -186,7 +201,31 @@ print(result.outputs["out:other"].status.value)  # stale
 
 When a guard flips, *every* output is reported stale, not just those computed under the branch. A trace only records the path that ran: writes on the other path are invisible to it, so no output can be vouched for. See [docs/design.md](docs/design.md#why-a-flipped-guard-makes-every-output-stale).
 
-Exceptions count as branches too: an operation that raised (and was caught) is recorded with its outcome, so an edit that makes it stop raising flips it. Other reasons for re-execution include an edit to an untracked input, a value that changes type, an operation that would now raise, and an input change reaching a non-replayable step. Each is reported in `result.reasons`.
+Exceptions count as branches too: an operation that raised (and was caught) is recorded with its outcome, so an edit that makes it stop raising flips it. Other reasons for re-execution include an edit to an untracked input or to an input the run did not have (a new row), a value that changes type, an operation that would now raise, and an input change reaching a non-replayable step. Each is reported in `result.reasons`.
+
+On large runs, `trace.propagate(edits, changed_only=True)` reports only the outputs that changed: recalculation itself already visits only what the edit reaches.
+
+### Fixing a calculated value
+
+A reviewer may find that a *calculated* value is wrong rather than an input. Override it, and everything after it is recalculated, with decisions re-checked as usual:
+
+```python
+def taxes(ext):
+    total = ext["wages"] + ext["interest"]
+    return {"total_income": total, "tax": (total * Decimal("0.2")).quantize(Decimal("0.01"))}
+
+
+trace, _ = rr.run(
+    taxes, {"wages": Decimal("5000.00"), "interest": Decimal("120.00")}, input_root="ext"
+)
+result = trace.propagate({}, overrides={"out:total_income": Decimal("9000.00")})
+for key, change in result.outputs.items():
+    print(key, change.status.value, change.new)
+# out:total_income overridden 9000.00
+# out:tax updated 1800.00
+```
+
+The overridden step keeps its value even if its own inputs are edited. A re-execution cannot apply an override, so `verify()` skips what overrides reach.
 
 ### Verifying with re-execution
 
@@ -251,6 +290,8 @@ trace, _ = rr.run(namespace["main"], {"gross": "5,200.50"}, input_root="ext")
 print(trace.explain("out:net"))  # 0.8 * float(ext:gross.replace(',', ''))
 ```
 
+Instrumentation also records plain `json.dumps(...)` as one repeatable step, and turns `x if cond else y` with literal or variable branches (`"Yes" if married else "Off"`, picking one of two rates) into a choice step, so a flipped condition updates outputs exactly instead of requiring re-execution.
+
 Instrumented code behaves exactly like the original. `exec_source` executes the code it is given: only use it on code you would run anyway.
 
 ### Saving, inspecting and viewing traces
@@ -267,7 +308,11 @@ redroot coverage trace.json                  # coverage report + re-evaluation c
 redroot visualize trace.json                 # web viewer (needs redroot[web])
 ```
 
-To re-evaluate a loaded trace, the modules defining its `@traced` functions must be imported, since they register those operations.
+To re-evaluate a loaded trace, the modules defining its `@traced` functions must be imported, since they register those operations. A trace records a fingerprint of each one (a hash of its source, or `@traced(version="...")`). If the registered function differs from the one the run used, its results are reported stale (reason `FUNCTION_CHANGED`) instead of being recomputed with new code; `trace.changed_functions()` lists them up front.
+
+Recalculating a trace runs the operations it names, so a tampered trace could ask for something huge (`10 ** 10**9`). Size limits are on by default; for traces from untrusted places add a budget, e.g. `trace.propagate(edits, limits=rr.Limits(max_seconds=2))`. Exceeding a limit raises `rr.LimitExceeded`.
+
+The viewer and `to_graphviz()` stay readable on large runs: inputs that differ only by list index are grouped (`ext:deposits[*].amount · 11 inputs`), and clicking an output shows its lineage up to other outputs, which you can open in turn (`redroot.visualizer.graph_data(trace, focus=...)`).
 
 ### Measuring whether tracing covers your workload
 
@@ -283,6 +328,27 @@ report = check_perturbation(guarded, {"A": 1200, "B": 300}, {"ext:A": 1100}, inp
 assert report.consistent  # graph agrees with re-execution wherever it claimed to know
 ```
 
+### Booleans
+
+Python does not allow subclassing `bool`, so by default comparisons return plain `bool`s (recorded as guards) and boolean inputs are untracked. Pass `bools=True` to trace them too:
+
+```python
+def form(ext):
+    eligible = ext["income"] > 30000
+    return {"eligible": eligible, "credit": 500 * eligible + 100 * ext["married"]}
+
+
+trace, _ = rr.run(form, {"income": 25000, "married": True}, input_root="ext", bools=True)
+print(trace.explain("out:eligible"))  # ext:income > 30000
+
+result = trace.propagate({"ext:income": 40000})  # the comparison flips...
+print(result.exact, result.outputs["out:credit"].new)  # True 600
+```
+
+A comparison then returns a `TracedBool`, which is data: a flag that is stored, counted or multiplied propagates exactly when it changes. A guard is recorded only where its truth value is used (`if`, `while`, `and`/`or`), and boolean inputs such as checkboxes become editable inputs.
+
+A `TracedBool` is an `int` subclass that prints, formats, hashes and compares like `True`/`False`. In code you do not instrument, `isinstance(x, bool)` and `x is True` are false and `json.dumps` writes `1`/`0`. Instrumented code gets all three right: external calls receive real `bool`s. Use `redroot.validation.check_fidelity(workflow, inputs, bools=True)` to confirm that tracing leaves a workflow's outputs unchanged.
+
 ### Pydantic
 
 With `redroot[pydantic]`, traced types work as model fields. Traced values pass through validation with their lineage, plain values become inputs, and models serialize to plain values.
@@ -295,7 +361,7 @@ Overhead is about 0.7 µs per traced operation (0.9 µs for `Decimal`) and about
 
 ## Limitations
 
-- Only `int`, `float`, `Decimal` and `str` are traced. `bool` and `None` inputs are recorded as untracked: editing them requires re-execution.
+- `int`, `float`, `Decimal` and `str` are traced, and `bool` with `bools=True`. `None` inputs (and `bool` inputs otherwise) are recorded as untracked: editing them requires re-execution.
 - Without instrumentation, a few C-level operations lose lineage (see above). `redroot.validation` finds them. [docs/design.md](docs/design.md#what-remains-invisible) lists what remains invisible even with it.
 - Lineage does not cross threads unless the context is propagated (`contextvars.copy_context()`), nor processes, nor `pickle`.
 - `type(x) is float` is `False` for traced values (instrumented code is unaffected).

{redroot-0.2.0 → redroot-0.4.0}/README.md

@@ -54,7 +54,10 @@ for key, change in result.outputs.items():
 - **Guards.** Comparisons, branches, conversions and lookups are recorded. If an edit would change one, every output is reported `stale` rather than shown with a value that may be wrong.
 - **Semantic keys.** Inputs and outputs are named by their path, e.g. `ext:bank.line_items[3].amount`.
 - **No cooperation needed from the code.** Tracing happens through operator overloading. For code you run from source, such as generated code, `redroot.instrument` closes the remaining gaps.
+- **Fast on large runs.** Recalculation visits only what an edit reaches: one edit in a run of 120,000 steps takes about 2 ms.
+- **Fix calculated values.** Override any calculated step and everything after it is recalculated.
 - **Re-execution as the arbiter.** Built-in tools compare the graph's answers with a real re-run, and measure coverage on your workloads.
+- **Safe to load.** Traces record which version of each helper they used, and recalculation is bounded against tampered traces.
 - **Opaque steps.** `@traced` turns a function (a tax-table lookup, a geocoder) into one re-callable node. `@traced_llm` marks LLM calls as not replayable.
 - **Persistable.** Traces serialize to versioned JSON, ship with a CLI, and come with a web viewer plus GraphViz and NetworkX exports.
 - **Zero dependencies** in the core. Pydantic, visualization and the web viewer are optional extras.
@@ -98,6 +101,18 @@ print(trace.outputs["out:summary.year"].linked)  # False
 
 `rr.run(workflow, inputs, input_root=..., output_root=...)` does the `track`, call and `collect` steps for you.
 
+Extracted values often arrive wrapped with their provenance. `envelope=rr.Envelope()` keys each wrapped value by its field, traces it in place (the workflow sees the same shape) and stores the source on the input:
+
+```python
+extracted = {"balance": {"value": Decimal("12450.00"), "source": {"file": "stmt.pdf", "page": 2}}}
+with rr.Trace() as trace:
+    ext = trace.track(extracted, root="ext", envelope=rr.Envelope())
+print(list(trace.inputs))  # ['ext:balance']
+print(trace.inputs["ext:balance"].meta)  # {'source': {'file': 'stmt.pdf', 'page': 2}}
+```
+
+`track(..., meta=lambda key, value: {...})` attaches any other metadata.
+
 ### Navigating backwards and forwards
 
 | Question | Call |
@@ -139,7 +154,31 @@ print(result.outputs["out:other"].status.value)  # stale
 
 When a guard flips, *every* output is reported stale, not just those computed under the branch. A trace only records the path that ran: writes on the other path are invisible to it, so no output can be vouched for. See [docs/design.md](docs/design.md#why-a-flipped-guard-makes-every-output-stale).
 
-Exceptions count as branches too: an operation that raised (and was caught) is recorded with its outcome, so an edit that makes it stop raising flips it. Other reasons for re-execution include an edit to an untracked input, a value that changes type, an operation that would now raise, and an input change reaching a non-replayable step. Each is reported in `result.reasons`.
+Exceptions count as branches too: an operation that raised (and was caught) is recorded with its outcome, so an edit that makes it stop raising flips it. Other reasons for re-execution include an edit to an untracked input or to an input the run did not have (a new row), a value that changes type, an operation that would now raise, and an input change reaching a non-replayable step. Each is reported in `result.reasons`.
+
+On large runs, `trace.propagate(edits, changed_only=True)` reports only the outputs that changed: recalculation itself already visits only what the edit reaches.
+
+### Fixing a calculated value
+
+A reviewer may find that a *calculated* value is wrong rather than an input. Override it, and everything after it is recalculated, with decisions re-checked as usual:
+
+```python
+def taxes(ext):
+    total = ext["wages"] + ext["interest"]
+    return {"total_income": total, "tax": (total * Decimal("0.2")).quantize(Decimal("0.01"))}
+
+
+trace, _ = rr.run(
+    taxes, {"wages": Decimal("5000.00"), "interest": Decimal("120.00")}, input_root="ext"
+)
+result = trace.propagate({}, overrides={"out:total_income": Decimal("9000.00")})
+for key, change in result.outputs.items():
+    print(key, change.status.value, change.new)
+# out:total_income overridden 9000.00
+# out:tax updated 1800.00
+```
+
+The overridden step keeps its value even if its own inputs are edited. A re-execution cannot apply an override, so `verify()` skips what overrides reach.
 
 ### Verifying with re-execution
 
@@ -204,6 +243,8 @@ trace, _ = rr.run(namespace["main"], {"gross": "5,200.50"}, input_root="ext")
 print(trace.explain("out:net"))  # 0.8 * float(ext:gross.replace(',', ''))
 ```
 
+Instrumentation also records plain `json.dumps(...)` as one repeatable step, and turns `x if cond else y` with literal or variable branches (`"Yes" if married else "Off"`, picking one of two rates) into a choice step, so a flipped condition updates outputs exactly instead of requiring re-execution.
+
 Instrumented code behaves exactly like the original. `exec_source` executes the code it is given: only use it on code you would run anyway.
 
 ### Saving, inspecting and viewing traces
@@ -220,7 +261,11 @@ redroot coverage trace.json                  # coverage report + re-evaluation c
 redroot visualize trace.json                 # web viewer (needs redroot[web])
 ```
 
-To re-evaluate a loaded trace, the modules defining its `@traced` functions must be imported, since they register those operations.
+To re-evaluate a loaded trace, the modules defining its `@traced` functions must be imported, since they register those operations. A trace records a fingerprint of each one (a hash of its source, or `@traced(version="...")`). If the registered function differs from the one the run used, its results are reported stale (reason `FUNCTION_CHANGED`) instead of being recomputed with new code; `trace.changed_functions()` lists them up front.
+
+Recalculating a trace runs the operations it names, so a tampered trace could ask for something huge (`10 ** 10**9`). Size limits are on by default; for traces from untrusted places add a budget, e.g. `trace.propagate(edits, limits=rr.Limits(max_seconds=2))`. Exceeding a limit raises `rr.LimitExceeded`.
+
+The viewer and `to_graphviz()` stay readable on large runs: inputs that differ only by list index are grouped (`ext:deposits[*].amount · 11 inputs`), and clicking an output shows its lineage up to other outputs, which you can open in turn (`redroot.visualizer.graph_data(trace, focus=...)`).
 
 ### Measuring whether tracing covers your workload
 
@@ -236,6 +281,27 @@ report = check_perturbation(guarded, {"A": 1200, "B": 300}, {"ext:A": 1100}, inp
 assert report.consistent  # graph agrees with re-execution wherever it claimed to know
 ```
 
+### Booleans
+
+Python does not allow subclassing `bool`, so by default comparisons return plain `bool`s (recorded as guards) and boolean inputs are untracked. Pass `bools=True` to trace them too:
+
+```python
+def form(ext):
+    eligible = ext["income"] > 30000
+    return {"eligible": eligible, "credit": 500 * eligible + 100 * ext["married"]}
+
+
+trace, _ = rr.run(form, {"income": 25000, "married": True}, input_root="ext", bools=True)
+print(trace.explain("out:eligible"))  # ext:income > 30000
+
+result = trace.propagate({"ext:income": 40000})  # the comparison flips...
+print(result.exact, result.outputs["out:credit"].new)  # True 600
+```
+
+A comparison then returns a `TracedBool`, which is data: a flag that is stored, counted or multiplied propagates exactly when it changes. A guard is recorded only where its truth value is used (`if`, `while`, `and`/`or`), and boolean inputs such as checkboxes become editable inputs.
+
+A `TracedBool` is an `int` subclass that prints, formats, hashes and compares like `True`/`False`. In code you do not instrument, `isinstance(x, bool)` and `x is True` are false and `json.dumps` writes `1`/`0`. Instrumented code gets all three right: external calls receive real `bool`s. Use `redroot.validation.check_fidelity(workflow, inputs, bools=True)` to confirm that tracing leaves a workflow's outputs unchanged.
+
 ### Pydantic
 
 With `redroot[pydantic]`, traced types work as model fields. Traced values pass through validation with their lineage, plain values become inputs, and models serialize to plain values.
@@ -248,7 +314,7 @@ Overhead is about 0.7 µs per traced operation (0.9 µs for `Decimal`) and about
 
 ## Limitations
 
-- Only `int`, `float`, `Decimal` and `str` are traced. `bool` and `None` inputs are recorded as untracked: editing them requires re-execution.
+- `int`, `float`, `Decimal` and `str` are traced, and `bool` with `bools=True`. `None` inputs (and `bool` inputs otherwise) are recorded as untracked: editing them requires re-execution.
 - Without instrumentation, a few C-level operations lose lineage (see above). `redroot.validation` finds them. [docs/design.md](docs/design.md#what-remains-invisible) lists what remains invisible even with it.
 - Lineage does not cross threads unless the context is propagated (`contextvars.copy_context()`), nor processes, nor `pickle`.
 - `type(x) is float` is `False` for traced values (instrumented code is unaffected).

{redroot-0.2.0 → redroot-0.4.0}/src/redroot/__init__.py

@@ -15,9 +15,10 @@ every output becomes when an input changes, without running it again::
 See the README for the full guide.
 """
 
-from redroot._core import Node, Traced, active_trace, node_of, unwrap
+from redroot._core import Node, Traced, active_trace, node_of, unwrap, unwrap_deep
 from redroot.functions import annotate, derive, traced, traced_llm
-from redroot.paths import apply_edits, format_key, parse_key
+from redroot.limits import LimitExceeded, Limits
+from redroot.paths import Envelope, apply_edits, format_key, parse_key
 from redroot.propagation import (
     Mismatch,
     OutputChange,
@@ -28,9 +29,9 @@ from redroot.propagation import (
     Verification,
 )
 from redroot.trace import Output, Trace, run, track
-from redroot.types import TracedDecimal, TracedFloat, TracedInt, TracedStr
+from redroot.types import TracedBool, TracedDecimal, TracedFloat, TracedInt, TracedStr
 
-__version__ = "0.2.0"
+__version__ = "0.4.0"
 
 
 def is_traced(value: object) -> bool:
@@ -39,6 +40,9 @@ def is_traced(value: object) -> bool:
 
 
 __all__ = [
+    "Envelope",
+    "LimitExceeded",
+    "Limits",
     "Mismatch",
     "Node",
     "Output",
@@ -49,6 +53,7 @@ __all__ = [
     "Status",
     "Trace",
     "Traced",
+    "TracedBool",
     "TracedDecimal",
     "TracedFloat",
     "TracedInt",
@@ -68,4 +73,5 @@ __all__ = [
     "traced_llm",
     "track",
     "unwrap",
+    "unwrap_deep",
 ]

{redroot-0.2.0 → redroot-0.4.0}/src/redroot/_core.py

@@ -124,6 +124,25 @@ def rebuild_tuple(template: tuple[Any, ...], items: Iterator[Any] | list[Any]) -
     return type(template)._make(items)  # type: ignore[attr-defined, no-any-return]
 
 
+class LoadedTuple(tuple):  # type: ignore[type-arg]
+    """A named tuple (or other tuple subclass) read back from a saved trace.
+
+    Its class is not restored (loading a trace imports nothing), so a node
+    using it cannot be re-run: recalculation reports it, rather than passing
+    a plain tuple to code that may check the type.
+    """
+
+    type_name: str
+
+    def __new__(cls, items: Any, type_name: str) -> LoadedTuple:
+        self = super().__new__(cls, items)
+        self.type_name = type_name
+        return self
+
+    def __repr__(self) -> str:
+        return f"{self.type_name}{tuple.__repr__(self)}"
+
+
 class Raised:
     """The recorded outcome of an operation that raised an exception.
 
@@ -349,7 +368,7 @@ def record_result(
     if isinstance(value, Traced) or kind is list or kind is dict or isinstance(value, tuple):
         value = unwrap_deep(value)  # e.g. a function returned one of its traced arguments
     factory = _WRAPPERS.get(type(value))
-    if factory is not None:
+    if factory is not None and (type(value) is not bool or trace._bools):
         return factory(
             value, add_node(trace, op, args, kwargs, value, OP, meta=meta, ctx=ctx, fn=fn)
         )
@@ -554,3 +573,54 @@ def observe_value(op: str, fn: Callable[..., Any], *operands: Any) -> Any:
         raise
     add_node(trace, op, tuple(refs), None, result, GUARD)
     return result
+
+
+def compare(op: str, fn: Callable[[Any, Any], Any], left: Any, right: Any) -> Any:
+    """Record a comparison of traced values.
+
+    In a trace that traces booleans the outcome is a traced boolean (data),
+    and a guard is recorded only where its truth value is used. Otherwise the
+    plain ``bool`` is returned and recorded as a guard right away.
+    """
+    trace = _active.get()
+    if trace is not None and not isinstance(trace, Observer) and trace._bools:
+        return binary(op, fn, left, right)
+    return observe_value(op, fn, left, right)
+
+
+def compare_data(op: str, fn: Callable[[Any, Any], Any], left: Any, right: Any) -> Any:
+    """Record a comparison as data, a traced boolean, whatever ``Trace.bools`` says.
+
+    For internal consumers that use the outcome as data and never let the
+    traced boolean escape (e.g. instrumented conditional expressions).
+    """
+    trace = _active.get()
+    if trace is None or isinstance(trace, Observer):
+        return observe_value(op, fn, left, right)
+    refs: list[Any] = []
+    plains: list[Any] = []
+    linked = False
+    for operand in (left, right):
+        if isinstance(operand, Traced):
+            node = operand._rt_node
+            plains.append(node.value)
+            if node.trace is trace:
+                refs.append(node)
+                linked = True
+            else:
+                refs.append(node.value)
+        else:
+            plains.append(operand)
+            refs.append(operand)
+    if not linked:
+        return fn(*plains)
+    try:
+        result = fn(*plains)
+    except Exception as exc:
+        record_raise(trace, op, tuple(refs), None, exc)
+        raise
+    if type(result) is not bool:
+        add_node(trace, op, tuple(refs), None, result, GUARD)
+        return result
+    node = add_node(trace, op, tuple(refs), None, result, OP)
+    return _WRAPPERS[bool](result, node)