query-cascade 0.2.2__tar.gz → 0.2.4__tar.gz

query_cascade-0.2.4/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: query-cascade
+Version: 0.2.4
+Summary: Minimal demand-driven query framework for incremental computation.
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+Requires-Dist: hypothesis; extra == "dev"
+Requires-Dist: mutmut; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+
+# Cascade Query
+
+Cascade Query is a Python library for incremental dependency tracking. It caches function results and re-executes them only when their specific inputs or upstream dependencies change.
+
+[![PyPI version](https://img.shields.io/pypi/v/query-cascade?color=3775A9&label=PyPI)](https://pypi.org/project/query-cascade/)
+[![Python versions](https://img.shields.io/pypi/pyversions/query-cascade?color=5A9)](https://pypi.org/project/query-cascade/)
+
+---
+
+## Core Principles
+
+1. **Automatic Caching:** Results are stored. If dependencies are unchanged, the function body does not execute.
+2. **Dependency Tracking:** Cascade records every `@engine.input` or `@engine.query` accessed during execution.
+3. **Targeted Updates:** When an input changes, Cascade identifies and invalidates only the affected downstream functions.
+4. **Early Bail-out:** If a function's output remains identical after its dependencies change, re-computation stops for that branch.
+
+---
+
+## Quickstart
+
+```python
+import time
+from cascade import Engine
+
+engine = Engine()
+
+@engine.input
+def user_id():
+    return "user_1"
+
+@engine.query
+def fetch_data():
+    time.sleep(2) 
+    return {"id": user_id(), "data": "value"}
+
+@engine.query
+def get_result():
+    data = fetch_data()
+    return f"Result for {data['id']}"
+
+# First run: Executes for 2 seconds.
+print(get_result())
+
+# Second run: Returns immediately from cache.
+print(get_result())
+
+# Update input:
+user_id.set("user_2")
+
+# Third run: Executes for 2 seconds to refresh.
+print(get_result())
+```
+
+---
+
+## Engine API
+
+### Core Methods
+*   **`Engine(max_entries=10000, stats=False)`**: Initializes the engine. `max_entries` sets the limit for the Least Recently Used (LRU) cache.
+*   **`@engine.input`**: Decorator for mutable data roots.
+    *   `input.set(value)`: Updates the value and increments the global revision.
+    *   `input.set(*args, value=value)`: Updates a keyed input.
+*   **`@engine.query`**: Decorator for cached computations.
+*   **`engine.snapshot()`**: Returns a `Snapshot` object pinning the current global revision. Use `query(snapshot=s)` to read data as it existed at that revision.
+*   **`engine.save(path)` / `engine.load(path)`**: Persists all inputs and cached results to a SQLite database.
+
+### Parallel & Background Execution
+*   **`engine.compute_many(calls, workers=None)`**: Executes a list of queries in parallel using a thread pool.
+*   **`engine.submit(query, *args, executor=None)`**: Schedules a query for background execution. Returns a `concurrent.futures.Future`.
+*   **`QueryCancelled`**: Exception raised if a background query's dependencies change before it completes.
+
+### Graph Utilities
+*   **`engine.inspect_graph()`**: Returns a dictionary of all nodes and edges in the dependency graph.
+*   **`engine.subgraph(roots, direction="deps")`**: Filters the graph to the dependency chain of the specified root nodes.
+*   **`engine.prune(roots)`**: Removes cached query results that are not reachable from the specified roots.
+
+---
+
+## Advanced Features
+
+### Side-Effect Accumulators
+Queries must be pure functions. Use `Accumulator` to record side-effects (like logs or warnings) that must be replayed when a result is served from the cache.
+
+```python
+warnings = engine.accumulator("warnings")
+
+@engine.query
+def validate_data():
+    data = fetch_data()
+    if not data:
+        warnings.push("No data found")
+    return data
+
+# On cache hit, 'warnings' are re-populated into the effects dictionary.
+effects = {}
+validate_data(effects=effects)
+print(effects["warnings"])
+```
+
+### Performance Metrics
+Set `stats=True` in the `Engine` constructor to track execution timing.
+*   **`engine.stats_summary()`**: Returns wall-clock time spent in function bodies and cache eviction counts.
+*   **`engine.reset_stats()`**: Clears accumulated timing data.
+
+---
+
+## Visualization
+Cascade provides renderers for the dependency graph.
+
+```python
+from cascade import export_dot, export_mermaid
+
+graph = engine.inspect_graph()
+# Generate Graphviz DOT format
+dot_text = export_dot(graph)
+# Generate Mermaid flowchart format
+mermaid_text = export_mermaid(graph)
+```
+
+---
+
+## Limitations
+
+1. **Cycle Detection:** Cascade detects and rejects recursive function calls (cycles) with a `CycleError`.
+2. **Thread Safety:** While Cascade supports parallel query execution, the `Engine` object itself should be modified (`.set()`, `@engine.query`) from a single thread or with external synchronization.
+3. **Persistence Security:** `engine.load()` uses JSON to resolve types via `importlib`. Only load databases from trusted sources.
+4. **Python Version:** Optimization for parallel CPU-bound work requires **CPython 3.14+ free-threaded** builds with `PYTHON_GIL=0`.
+
+---
+
+## Installation
+
+```bash
+pip install query-cascade
+```

query_cascade-0.2.4/README.md

@@ -0,0 +1,135 @@
+# Cascade Query
+
+Cascade Query is a Python library for incremental dependency tracking. It caches function results and re-executes them only when their specific inputs or upstream dependencies change.
+
+[![PyPI version](https://img.shields.io/pypi/v/query-cascade?color=3775A9&label=PyPI)](https://pypi.org/project/query-cascade/)
+[![Python versions](https://img.shields.io/pypi/pyversions/query-cascade?color=5A9)](https://pypi.org/project/query-cascade/)
+
+---
+
+## Core Principles
+
+1. **Automatic Caching:** Results are stored. If dependencies are unchanged, the function body does not execute.
+2. **Dependency Tracking:** Cascade records every `@engine.input` or `@engine.query` accessed during execution.
+3. **Targeted Updates:** When an input changes, Cascade identifies and invalidates only the affected downstream functions.
+4. **Early Bail-out:** If a function's output remains identical after its dependencies change, re-computation stops for that branch.
+
+---
+
+## Quickstart
+
+```python
+import time
+from cascade import Engine
+
+engine = Engine()
+
+@engine.input
+def user_id():
+    return "user_1"
+
+@engine.query
+def fetch_data():
+    time.sleep(2) 
+    return {"id": user_id(), "data": "value"}
+
+@engine.query
+def get_result():
+    data = fetch_data()
+    return f"Result for {data['id']}"
+
+# First run: Executes for 2 seconds.
+print(get_result())
+
+# Second run: Returns immediately from cache.
+print(get_result())
+
+# Update input:
+user_id.set("user_2")
+
+# Third run: Executes for 2 seconds to refresh.
+print(get_result())
+```
+
+---
+
+## Engine API
+
+### Core Methods
+*   **`Engine(max_entries=10000, stats=False)`**: Initializes the engine. `max_entries` sets the limit for the Least Recently Used (LRU) cache.
+*   **`@engine.input`**: Decorator for mutable data roots.
+    *   `input.set(value)`: Updates the value and increments the global revision.
+    *   `input.set(*args, value=value)`: Updates a keyed input.
+*   **`@engine.query`**: Decorator for cached computations.
+*   **`engine.snapshot()`**: Returns a `Snapshot` object pinning the current global revision. Use `query(snapshot=s)` to read data as it existed at that revision.
+*   **`engine.save(path)` / `engine.load(path)`**: Persists all inputs and cached results to a SQLite database.
+
+### Parallel & Background Execution
+*   **`engine.compute_many(calls, workers=None)`**: Executes a list of queries in parallel using a thread pool.
+*   **`engine.submit(query, *args, executor=None)`**: Schedules a query for background execution. Returns a `concurrent.futures.Future`.
+*   **`QueryCancelled`**: Exception raised if a background query's dependencies change before it completes.
+
+### Graph Utilities
+*   **`engine.inspect_graph()`**: Returns a dictionary of all nodes and edges in the dependency graph.
+*   **`engine.subgraph(roots, direction="deps")`**: Filters the graph to the dependency chain of the specified root nodes.
+*   **`engine.prune(roots)`**: Removes cached query results that are not reachable from the specified roots.
+
+---
+
+## Advanced Features
+
+### Side-Effect Accumulators
+Queries must be pure functions. Use `Accumulator` to record side-effects (like logs or warnings) that must be replayed when a result is served from the cache.
+
+```python
+warnings = engine.accumulator("warnings")
+
+@engine.query
+def validate_data():
+    data = fetch_data()
+    if not data:
+        warnings.push("No data found")
+    return data
+
+# On cache hit, 'warnings' are re-populated into the effects dictionary.
+effects = {}
+validate_data(effects=effects)
+print(effects["warnings"])
+```
+
+### Performance Metrics
+Set `stats=True` in the `Engine` constructor to track execution timing.
+*   **`engine.stats_summary()`**: Returns wall-clock time spent in function bodies and cache eviction counts.
+*   **`engine.reset_stats()`**: Clears accumulated timing data.
+
+---
+
+## Visualization
+Cascade provides renderers for the dependency graph.
+
+```python
+from cascade import export_dot, export_mermaid
+
+graph = engine.inspect_graph()
+# Generate Graphviz DOT format
+dot_text = export_dot(graph)
+# Generate Mermaid flowchart format
+mermaid_text = export_mermaid(graph)
+```
+
+---
+
+## Limitations
+
+1. **Cycle Detection:** Cascade detects and rejects recursive function calls (cycles) with a `CycleError`.
+2. **Thread Safety:** While Cascade supports parallel query execution, the `Engine` object itself should be modified (`.set()`, `@engine.query`) from a single thread or with external synchronization.
+3. **Persistence Security:** `engine.load()` uses JSON to resolve types via `importlib`. Only load databases from trusted sources.
+4. **Python Version:** Optimization for parallel CPU-bound work requires **CPython 3.14+ free-threaded** builds with `PYTHON_GIL=0`.
+
+---
+
+## Installation
+
+```bash
+pip install query-cascade
+```

{query_cascade-0.2.2 → query_cascade-0.2.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "query-cascade"
-version = "0.2.2"
+version = "0.2.4"
 description = "Minimal demand-driven query framework for incremental computation."
 requires-python = ">=3.12"
 readme = "README.md"

{query_cascade-0.2.2 → query_cascade-0.2.4}/src/cascade/engine.py

@@ -3,7 +3,7 @@ from __future__ import annotations
 import concurrent.futures
 import os
 import threading
-from typing import Any, Callable, Iterable, Literal, Mapping, Sequence
+from typing import Any, Callable, Iterable, Iterator, Literal, Mapping, Sequence
 
 from ._errors import CancellationError, CycleError, QueryCancelled
 from ._evaluator import Evaluator
@@ -262,18 +262,95 @@ class Engine:
         *,
         workers: int | None = None,
         snapshot: Snapshot | None = None,
+        effects: dict[str, list[Any]] | None = None,
     ) -> list[Any]:
         if not calls:
             return []
         run_snapshot = snapshot or self.snapshot()
         worker_count = workers or min(32, max(1, len(calls)))
         scheduler = WorkStealingExecutor(worker_count)
+        per_call_effects: list[dict[str, list[Any]] | None]
+        if effects is None:
+            per_call_effects = [None] * len(calls)
+        else:
+            per_call_effects = [{} for _ in range(len(calls))]
         for idx, (query, args) in enumerate(calls):
             scheduler.submit_indexed(
                 idx,
-                lambda q=query, a=args: self._query_call(q.id, q.raw, a, snapshot=run_snapshot),
+                lambda q=query, a=args, e=per_call_effects[idx]: self._query_call(
+                    q.id,
+                    q.raw,
+                    a,
+                    snapshot=run_snapshot,
+                    effects=e,
+                ),
             )
-        return scheduler.run(len(calls))
+        results = scheduler.run(len(calls))
+        if effects is not None:
+            for call_effects in per_call_effects:
+                if not call_effects:
+                    continue
+                for name, items in call_effects.items():
+                    if not items:
+                        continue
+                    effects.setdefault(name, []).extend(items)
+        return results
+
+    def compute_many_stream(
+        self,
+        calls: Sequence[tuple[_QueryHandle, tuple[Any, ...]]],
+        *,
+        workers: int | None = None,
+        snapshot: Snapshot | None = None,
+        effects: dict[str, list[Any]] | None = None,
+    ) -> Iterator[tuple[int, Any, dict[str, list[Any]]]]:
+        """Yield completed results as each call finishes.
+
+        Yields ``(index, value, call_effects)`` where ``index`` matches the position in
+        ``calls`` and ``call_effects`` contains accumulator output for that call when
+        ``effects`` was provided (otherwise an empty dict).
+
+        Unlike :meth:`compute_many`, this API yields items in *completion order*.
+        If ``effects`` is provided, it is populated after all calls complete, merged
+        deterministically in call order (matching :meth:`compute_many`).
+        """
+        if not calls:
+            return
+
+        run_snapshot = snapshot or self.snapshot()
+        worker_count = workers or min(32, max(1, len(calls)))
+
+        per_call_effects: list[dict[str, list[Any]]]
+        per_call_effects = [{} for _ in range(len(calls))]
+
+        with concurrent.futures.ThreadPoolExecutor(max_workers=worker_count) as pool:
+            future_to_index: dict[concurrent.futures.Future[Any], int] = {}
+            for idx, (query, args) in enumerate(calls):
+                fut = pool.submit(
+                    self._query_call,
+                    query.id,
+                    query.raw,
+                    args,
+                    snapshot=run_snapshot,
+                    effects=per_call_effects[idx] if effects is not None else None,
+                )
+                future_to_index[fut] = idx
+
+            for fut in concurrent.futures.as_completed(future_to_index):
+                idx = future_to_index[fut]
+                value = fut.result()
+                call_effects = per_call_effects[idx] if effects is not None else {}
+                item = (idx, value, call_effects)
+                yield item
+
+        if effects is not None:
+            for call_effects in per_call_effects:
+                if not call_effects:
+                    continue
+                for name, items in call_effects.items():
+                    if not items:
+                        continue
+                    effects.setdefault(name, []).extend(items)
 
     def traces(self) -> list[TraceEvent]:
         return self._store.traces()

query_cascade-0.2.4/src/query_cascade.egg-info/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: query-cascade
+Version: 0.2.4
+Summary: Minimal demand-driven query framework for incremental computation.
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+Requires-Dist: hypothesis; extra == "dev"
+Requires-Dist: mutmut; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+
+# Cascade Query
+
+Cascade Query is a Python library for incremental dependency tracking. It caches function results and re-executes them only when their specific inputs or upstream dependencies change.
+
+[![PyPI version](https://img.shields.io/pypi/v/query-cascade?color=3775A9&label=PyPI)](https://pypi.org/project/query-cascade/)
+[![Python versions](https://img.shields.io/pypi/pyversions/query-cascade?color=5A9)](https://pypi.org/project/query-cascade/)
+
+---
+
+## Core Principles
+
+1. **Automatic Caching:** Results are stored. If dependencies are unchanged, the function body does not execute.
+2. **Dependency Tracking:** Cascade records every `@engine.input` or `@engine.query` accessed during execution.
+3. **Targeted Updates:** When an input changes, Cascade identifies and invalidates only the affected downstream functions.
+4. **Early Bail-out:** If a function's output remains identical after its dependencies change, re-computation stops for that branch.
+
+---
+
+## Quickstart
+
+```python
+import time
+from cascade import Engine
+
+engine = Engine()
+
+@engine.input
+def user_id():
+    return "user_1"
+
+@engine.query
+def fetch_data():
+    time.sleep(2) 
+    return {"id": user_id(), "data": "value"}
+
+@engine.query
+def get_result():
+    data = fetch_data()
+    return f"Result for {data['id']}"
+
+# First run: Executes for 2 seconds.
+print(get_result())
+
+# Second run: Returns immediately from cache.
+print(get_result())
+
+# Update input:
+user_id.set("user_2")
+
+# Third run: Executes for 2 seconds to refresh.
+print(get_result())
+```
+
+---
+
+## Engine API
+
+### Core Methods
+*   **`Engine(max_entries=10000, stats=False)`**: Initializes the engine. `max_entries` sets the limit for the Least Recently Used (LRU) cache.
+*   **`@engine.input`**: Decorator for mutable data roots.
+    *   `input.set(value)`: Updates the value and increments the global revision.
+    *   `input.set(*args, value=value)`: Updates a keyed input.
+*   **`@engine.query`**: Decorator for cached computations.
+*   **`engine.snapshot()`**: Returns a `Snapshot` object pinning the current global revision. Use `query(snapshot=s)` to read data as it existed at that revision.
+*   **`engine.save(path)` / `engine.load(path)`**: Persists all inputs and cached results to a SQLite database.
+
+### Parallel & Background Execution
+*   **`engine.compute_many(calls, workers=None)`**: Executes a list of queries in parallel using a thread pool.
+*   **`engine.submit(query, *args, executor=None)`**: Schedules a query for background execution. Returns a `concurrent.futures.Future`.
+*   **`QueryCancelled`**: Exception raised if a background query's dependencies change before it completes.
+
+### Graph Utilities
+*   **`engine.inspect_graph()`**: Returns a dictionary of all nodes and edges in the dependency graph.
+*   **`engine.subgraph(roots, direction="deps")`**: Filters the graph to the dependency chain of the specified root nodes.
+*   **`engine.prune(roots)`**: Removes cached query results that are not reachable from the specified roots.
+
+---
+
+## Advanced Features
+
+### Side-Effect Accumulators
+Queries must be pure functions. Use `Accumulator` to record side-effects (like logs or warnings) that must be replayed when a result is served from the cache.
+
+```python
+warnings = engine.accumulator("warnings")
+
+@engine.query
+def validate_data():
+    data = fetch_data()
+    if not data:
+        warnings.push("No data found")
+    return data
+
+# On cache hit, 'warnings' are re-populated into the effects dictionary.
+effects = {}
+validate_data(effects=effects)
+print(effects["warnings"])
+```
+
+### Performance Metrics
+Set `stats=True` in the `Engine` constructor to track execution timing.
+*   **`engine.stats_summary()`**: Returns wall-clock time spent in function bodies and cache eviction counts.
+*   **`engine.reset_stats()`**: Clears accumulated timing data.
+
+---
+
+## Visualization
+Cascade provides renderers for the dependency graph.
+
+```python
+from cascade import export_dot, export_mermaid
+
+graph = engine.inspect_graph()
+# Generate Graphviz DOT format
+dot_text = export_dot(graph)
+# Generate Mermaid flowchart format
+mermaid_text = export_mermaid(graph)
+```
+
+---
+
+## Limitations
+
+1. **Cycle Detection:** Cascade detects and rejects recursive function calls (cycles) with a `CycleError`.
+2. **Thread Safety:** While Cascade supports parallel query execution, the `Engine` object itself should be modified (`.set()`, `@engine.query`) from a single thread or with external synchronization.
+3. **Persistence Security:** `engine.load()` uses JSON to resolve types via `importlib`. Only load databases from trusted sources.
+4. **Python Version:** Optimization for parallel CPU-bound work requires **CPython 3.14+ free-threaded** builds with `PYTHON_GIL=0`.
+
+---
+
+## Installation
+
+```bash
+pip install query-cascade
+```

{query_cascade-0.2.2 → query_cascade-0.2.4}/tests/test_engine_compute_many.py

@@ -83,6 +83,79 @@ def test_compute_many_with_zero_workers_falls_back_to_default_worker_selection()
     assert result == [i + 1 for i in range(6)]
 
 
+def test_compute_many_can_collect_accumulator_effects() -> None:
+    engine = Engine()
+    progress = engine.accumulator("progress")
+
+    @engine.query
+    def job(name: str) -> str:
+        progress.push(("start", name))
+        progress.push(("done", name))
+        return name.upper()
+
+    effects: dict[str, list[object]] = {}
+    result = engine.compute_many([(job, ("a",)), (job, ("b",)), (job, ("c",))], workers=3, effects=effects)
+    assert result == ["A", "B", "C"]
+    assert effects == {
+        "progress": [
+            ("start", "a"),
+            ("done", "a"),
+            ("start", "b"),
+            ("done", "b"),
+            ("start", "c"),
+            ("done", "c"),
+        ]
+    }
+
+
+def test_compute_many_stream_yields_in_completion_order() -> None:
+    engine = Engine()
+
+    @engine.query
+    def delayed(i: int) -> int:
+        time.sleep(0.004 * (4 - i))
+        return i
+
+    calls = [(delayed, (i,)) for i in range(4)]
+    items = list(engine.compute_many_stream(calls, workers=4))
+    # i=3 sleeps least, so it should complete first.
+    assert [idx for idx, _value, _effects in items] == [3, 2, 1, 0]
+    assert sorted(value for _idx, value, _effects in items) == [0, 1, 2, 3]
+
+
+def test_compute_many_stream_can_yield_and_collect_accumulator_effects() -> None:
+    engine = Engine()
+    progress = engine.accumulator("progress")
+
+    @engine.query
+    def job(name: str, seconds: float) -> str:
+        progress.push(("start", name))
+        time.sleep(seconds)
+        progress.push(("done", name))
+        return name.upper()
+
+    calls = [(job, ("a", 0.03)), (job, ("b", 0.01)), (job, ("c", 0.02))]
+    effects: dict[str, list[object]] = {}
+    items = list(engine.compute_many_stream(calls, workers=3, effects=effects))
+
+    # Stream yields in completion order (b, c, a).
+    assert [value for _idx, value, _call_effects in items] == ["B", "C", "A"]
+    # Each yielded item includes just that call's effects.
+    by_value = {value: call_effects["progress"] for _idx, value, call_effects in items}
+    assert by_value["A"] == [("start", "a"), ("done", "a")]
+    assert by_value["B"] == [("start", "b"), ("done", "b")]
+    assert by_value["C"] == [("start", "c"), ("done", "c")]
+    # Aggregate effects dict is merged in call order, matching compute_many.
+    assert effects["progress"] == [
+        ("start", "a"),
+        ("done", "a"),
+        ("start", "b"),
+        ("done", "b"),
+        ("start", "c"),
+        ("done", "c"),
+    ]
+
+
 def test_work_stealing_run_raises_first_recorded_task_error() -> None:
     scheduler = WorkStealingExecutor(workers=2)