zu-shadow 0.1.13__tar.gz

zu_shadow-0.1.13/.gitignore

@@ -0,0 +1,66 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+
+# uv / venv
+.venv/
+uv.lock.bak
+
+# Test / type caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Zu runtime artifacts
+*.db
+zu.db
+zu.yaml.local
+zu_review.jsonl
+*.review.jsonl
+# Per-agent cost telemetry ledger — machine-local run history, not source.
+cost.jsonl
+# A recorded replay path is learned per-run and machine-local — regenerated on
+# every successful run, not source. The agent ships; its track does not.
+track.json
+# …except the flagship example ships its track on purpose, as a demo of the
+# record/replay convergence (committed; re-runs show as ordinary modifications).
+!examples/agents/vet-appointment/track.json
+
+# Editor / OS
+.idea/
+.vscode/
+.DS_Store
+
+# Claude Code local session state
+.claude/
+
+# Secrets
+.env
+.env.*
+!.env.example
+
+# Microsoft Office temp/lock files
+~$*
+
+# Internal design / strategy docs — kept local, never in the public repo
+*.docx
+*.pdf
+# BUILD.md is the internal build-sequence / deferred-gaps ledger — kept local.
+# (ARCHITECTURE.md is public: an onboarding agent needs the structural map.)
+docs/BUILD.md
+
+# Local secret — API key for live validation, never commit
+zu_demo_key.md
+*_key.md
+
+# Local PyPI publish token — never commit
+/pypi
+
+# Local Discord credentials (bot token / app secrets) — never commit
+/discord

zu_shadow-0.1.13/PKG-INFO

@@ -0,0 +1,81 @@
+Metadata-Version: 2.4
+Name: zu-shadow
+Version: 0.1.13
+Summary: Zu Shadow: author a production agent by demonstration — record a human session, redact at capture, synthesize an agent + rail (§2.8)
+Project-URL: Homepage, https://github.com/k3-mt/zu
+Project-URL: Repository, https://github.com/k3-mt/zu
+License-Expression: Apache-2.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: zu-cli==0.2.6
+Requires-Dist: zu-core==0.2.11
+Provides-Extra: live
+Requires-Dist: playwright>=1.40; extra == 'live'
+Requires-Dist: zu-tools==0.2.7; extra == 'live'
+Description-Content-Type: text/markdown
+
+# zu-shadow — author an agent by demonstration (§2.8)
+
+A Shadow recording **is** the event bus run over a *human* session: the human is
+the policy for that one run, so recording costs almost nothing architecturally.
+You drive the task once, by hand; Shadow folds your clicks, types, navigations and
+the page/network metadata into `data.shadow.*` events on the same append-only log
+everything else in Zu uses, then synthesizes a production agent + a rail from it.
+
+```
+record (human session) ─▶ redact-at-capture ─▶ data.shadow.* on the log
+                                                     │
+                              synthesize (a Zu agent, ScriptedProvider offline)
+                                                     │
+                       agent spec + induced Fsm + Invariants + self-writing egress
+                                                     │
+                      verification-replay GATE (reuses zu-cli offline.py/build.py)
+                                                     │
+                          promote only if the recorded outcome reproduces
+```
+
+## Four load-bearing disciplines
+
+- **Redaction is DEFAULT-ON and runs BEFORE append** (`redaction.py`). Passwords,
+  `Authorization`/`Cookie`/`Set-Cookie` headers, token/API-key shapes, and
+  consumer-configured PII are stripped — *including the "why" intent text* — before
+  any event reaches `EventSink.append`. The secret is gone before the event is
+  hashed into the audit chain. This is conformance requirement **ZU-AUDIT-4**.
+- **Capture is SEMANTIC** (`capture.py`). Every action is named by its target's
+  `{role, name, label}` (the core `zu_core.surface` currency, shared with the §4
+  locator and §5 `SurfaceView`) — never a CSS selector or pixel coordinate, so the
+  synthesized agent re-resolves on a changed page instead of breaking.
+- **The synthesizer is a Zu agent** (`synthesizer.py`). It is *driven by a*
+  `ModelProvider` (offline-tested with `ScriptedProvider`). The model writes only
+  the policy prompt + goal; the egress allowlist, the induced `Fsm`, and the
+  `Invariant`s are **derived deterministically** from the log — the egress allowlist
+  *writes itself* from the recorded `network.response` hosts. No new FSM/invariant
+  types: it emits `zu_core.reachability.Fsm` and `zu_core.invariants.Invariant`.
+- **Promotion is GATED by reproduced outcome** (`replay_gate.py`). A synthesized
+  agent does not run on real data until it reproduces the recorded outcome, reusing
+  zu-cli's `offline.py`/`build.py`. The "why" resolutions are surfaced for **review**,
+  never auto-promoted.
+
+## The honest scope
+
+Robustness comes from the runtime machinery — semantic re-resolution, detectors,
+replay, the rail — not from a single recording. On a structurally different site
+the honest behaviour is to **escalate**, not silently err. The live human recorder
+(real Chromium + a real human over CDP) is demo/manual, behind the `live` extra and
+a manual entrypoint (`live.py`); the offline core is fully tested against a
+synthetic input/CDP stream at $0.
+
+## CLI
+
+```
+zu shadow record   <stream.json> --site <url> -o recording.json   # synthetic/live stream → recording
+zu shadow synthesize <recording.json> --instruction "…"           # recording → agent + rail proposal
+zu shadow scale    <agent> --rows rows.csv --var <name>           # one governed run per CSV row
+```

zu_shadow-0.1.13/README.md

@@ -0,0 +1,58 @@
+# zu-shadow — author an agent by demonstration (§2.8)
+
+A Shadow recording **is** the event bus run over a *human* session: the human is
+the policy for that one run, so recording costs almost nothing architecturally.
+You drive the task once, by hand; Shadow folds your clicks, types, navigations and
+the page/network metadata into `data.shadow.*` events on the same append-only log
+everything else in Zu uses, then synthesizes a production agent + a rail from it.
+
+```
+record (human session) ─▶ redact-at-capture ─▶ data.shadow.* on the log
+                                                     │
+                              synthesize (a Zu agent, ScriptedProvider offline)
+                                                     │
+                       agent spec + induced Fsm + Invariants + self-writing egress
+                                                     │
+                      verification-replay GATE (reuses zu-cli offline.py/build.py)
+                                                     │
+                          promote only if the recorded outcome reproduces
+```
+
+## Four load-bearing disciplines
+
+- **Redaction is DEFAULT-ON and runs BEFORE append** (`redaction.py`). Passwords,
+  `Authorization`/`Cookie`/`Set-Cookie` headers, token/API-key shapes, and
+  consumer-configured PII are stripped — *including the "why" intent text* — before
+  any event reaches `EventSink.append`. The secret is gone before the event is
+  hashed into the audit chain. This is conformance requirement **ZU-AUDIT-4**.
+- **Capture is SEMANTIC** (`capture.py`). Every action is named by its target's
+  `{role, name, label}` (the core `zu_core.surface` currency, shared with the §4
+  locator and §5 `SurfaceView`) — never a CSS selector or pixel coordinate, so the
+  synthesized agent re-resolves on a changed page instead of breaking.
+- **The synthesizer is a Zu agent** (`synthesizer.py`). It is *driven by a*
+  `ModelProvider` (offline-tested with `ScriptedProvider`). The model writes only
+  the policy prompt + goal; the egress allowlist, the induced `Fsm`, and the
+  `Invariant`s are **derived deterministically** from the log — the egress allowlist
+  *writes itself* from the recorded `network.response` hosts. No new FSM/invariant
+  types: it emits `zu_core.reachability.Fsm` and `zu_core.invariants.Invariant`.
+- **Promotion is GATED by reproduced outcome** (`replay_gate.py`). A synthesized
+  agent does not run on real data until it reproduces the recorded outcome, reusing
+  zu-cli's `offline.py`/`build.py`. The "why" resolutions are surfaced for **review**,
+  never auto-promoted.
+
+## The honest scope
+
+Robustness comes from the runtime machinery — semantic re-resolution, detectors,
+replay, the rail — not from a single recording. On a structurally different site
+the honest behaviour is to **escalate**, not silently err. The live human recorder
+(real Chromium + a real human over CDP) is demo/manual, behind the `live` extra and
+a manual entrypoint (`live.py`); the offline core is fully tested against a
+synthetic input/CDP stream at $0.
+
+## CLI
+
+```
+zu shadow record   <stream.json> --site <url> -o recording.json   # synthetic/live stream → recording
+zu shadow synthesize <recording.json> --instruction "…"           # recording → agent + rail proposal
+zu shadow scale    <agent> --rows rows.csv --var <name>           # one governed run per CSV row
+```

zu_shadow-0.1.13/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "zu-shadow"
+version = "0.1.13"
+description = "Zu Shadow: author a production agent by demonstration — record a human session, redact at capture, synthesize an agent + rail (§2.8)"
+readme = "README.md"
+requires-python = ">=3.11"
+license = "Apache-2.0"
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Libraries :: Application Frameworks",
+    "Typing :: Typed",
+]
+# A Shadow recording IS the event bus run over a HUMAN session, so the recorder
+# only needs zu-core (events + surface types + the sink seam). The synthesizer is
+# itself a Zu agent (a ModelProvider), and the verification-replay promotion gate
+# REUSES zu-cli's offline.py/build.py — so zu-cli is a dependency too.
+dependencies = ["zu-core==0.2.11", "zu-cli==0.2.6"]
+
+[project.optional-dependencies]
+# The LIVE recorder/capture binds a real Chrome over CDP (Playwright connects to it;
+# no extra browser download) — opt-in, never needed for the offline core.
+live = ["zu-tools==0.2.7", "playwright>=1.40"]
+
+[project.urls]
+Homepage = "https://github.com/k3-mt/zu"
+Repository = "https://github.com/k3-mt/zu"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/zu_shadow"]

zu_shadow-0.1.13/src/zu_shadow/__init__.py

@@ -0,0 +1,46 @@
+"""zu-shadow — author a production agent by DEMONSTRATION (§2.8).
+
+A Shadow recording *is* the event bus run over a HUMAN session: the human is the
+policy for that one run, so recording costs almost nothing architecturally — the
+recorder folds an abstract input/CDP stream into ``data.shadow.*`` events on the
+same append-only log everything else uses. Four disciplines are load-bearing:
+
+* **Redaction is DEFAULT-ON and runs BEFORE append** (``redaction``): secrets —
+  passwords, ``Authorization``/``Cookie`` headers, tokens/API keys, configured PII
+  — never reach :meth:`EventSink.append`. The "why" intent text is redacted too.
+* **Capture is SEMANTIC** (``capture``): a user action is named by its target's
+  ``{role, name, label}`` (the core ``surface`` currency, shared with §4 handles /
+  §5 SurfaceView) — never a CSS selector or pixel coordinate.
+* **The synthesizer is itself a Zu agent** (``synthesizer``): driven by a
+  ``ModelProvider`` (offline-tested with ``ScriptedProvider``), it PROPOSES an
+  agent spec + an induced ``Fsm`` + ``Invariant``s; the egress allowlist writes
+  itself from the recorded ``network.response`` hosts.
+* **Promotion is GATED by reproduced outcome** (``replay_gate``): a synthesized
+  agent does not run on real data until it reproduces the recorded outcome, reusing
+  zu-cli's ``offline.py``/``build.py``. The "why" resolutions are reviewed, never
+  auto-promoted.
+"""
+
+from __future__ import annotations
+
+from .capture import SemanticTarget, capture_click, capture_navigate, capture_type
+from .recorder import RecordedSession, Recorder
+from .redaction import RedactionPolicy, redact_event, redact_text
+from .replay_gate import PromotionVerdict, verify_and_gate
+from .synthesizer import SynthesisResult, Synthesizer
+
+__all__ = [
+    "PromotionVerdict",
+    "RecordedSession",
+    "Recorder",
+    "RedactionPolicy",
+    "SemanticTarget",
+    "SynthesisResult",
+    "Synthesizer",
+    "capture_click",
+    "capture_navigate",
+    "capture_type",
+    "redact_event",
+    "redact_text",
+    "verify_and_gate",
+]

zu_shadow-0.1.13/src/zu_shadow/capture.py

@@ -0,0 +1,119 @@
+"""SEMANTIC-TARGET capture — name an action by WHAT it acts on, not WHERE.
+
+Every captured user action identifies its target by ``{role, name, label}`` — the
+same accessibility-grounded currency the core ``surface`` types speak (§4 handles /
+§5 ``SurfaceView``). NEVER a CSS selector, an XPath, or a pixel coordinate: those
+are brittle (a redesign breaks them) and untransferable (they cannot feed the §4
+locator / §5 recognizer). A semantic target re-resolves on a changed page, which is
+the whole reason a synthesized agent can be *resilient* rather than pixel-frozen.
+
+``SemanticTarget`` is a thin, frozen value object that reuses ``role``/``label``
+exactly as :class:`zu_core.surface.SurfaceAffordance` does, plus the accessible
+``name`` (the click target's accessible name). The capture helpers turn a raw
+abstract-stream event into a redaction-ready ``data.shadow.*`` payload.
+"""
+
+from __future__ import annotations
+
+from pydantic import BaseModel
+
+from zu_core import events as ev
+from zu_core.surface import SurfaceAffordance
+
+# Target role/name/label tokens that mark an input as a CREDENTIAL field, so the
+# recorder records its typed value under a credential-named key the redaction stage
+# blanks wholesale — a password is never recorded verbatim, even pre-redaction-sweep.
+_CREDENTIAL_TARGET_HINTS: tuple[str, ...] = ("password", "passwd", "secret", "token",
+                                             "api key", "api_key", "apikey", "otp",
+                                             "cvv", "cvc", "pin", "security code",
+                                             # payment-card secrets — the agent must NEVER hold
+                                             # these; a real payment goes through the §8 broker.
+                                             "card number", "cardnumber", "card no",
+                                             "credit card", "debit card", "expiration", "expiry",
+                                             "iban", "sort code", "account number")
+
+
+class SemanticTarget(BaseModel):
+    """A user-action target, identified the way the core surface currency does:
+    ``role`` (a free string, e.g. ``button``/``link``/``textbox``), the accessible
+    ``name``, and a human ``label``. NO selector, NO coordinates — re-resolvable on
+    a changed page. Frozen so it is a stable value on the log."""
+
+    model_config = {"frozen": True}
+
+    role: str
+    name: str = ""
+    label: str = ""
+
+    @classmethod
+    def from_affordance(cls, a: SurfaceAffordance, *, name: str = "") -> SemanticTarget:
+        """Build a target from a core ``SurfaceAffordance`` — the bridge from a §5
+        SurfaceView the live recorder reduced to a recorded action target. The
+        affordance's ``label`` carries through; ``name`` is the accessible name the
+        CDP locate step resolved (the affordance has no separate name field)."""
+        return cls(role=a.role, name=name or a.label, label=a.label)
+
+    def to_payload(self) -> dict:
+        return {"role": self.role, "name": self.name, "label": self.label}
+
+
+def capture_click(target: SemanticTarget, *, intent: str | None = None) -> tuple[str, dict]:
+    """A ``data.shadow.user.click`` (type, payload). ``intent`` is the OPTIONAL,
+    reviewed "why" narration — carried but NEVER auto-promoted into the agent."""
+    payload: dict = {"target": target.to_payload()}
+    if intent is not None:
+        payload["intent"] = intent
+    return ev.SHADOW_USER_CLICK, payload
+
+
+def _is_credential_target(target: SemanticTarget) -> bool:
+    """A type target whose role/name/label marks it as a credential input — so its
+    value is recorded under a credential-named key the redaction stage blanks."""
+    blob = f"{target.role} {target.name} {target.label}".lower()
+    return any(h in blob for h in _CREDENTIAL_TARGET_HINTS)
+
+
+def capture_type(target: SemanticTarget, value: str, *,
+                 intent: str | None = None) -> tuple[str, dict]:
+    """A ``data.shadow.user.type`` (type, payload). The recorder MARKS a credential
+    target: a password/secret field's value goes under a ``password`` key that the
+    redaction stage (run before append) blanks wholesale, so a credential is never
+    recorded verbatim. A non-credential value rides under ``value`` and is still
+    swept for token shapes by redaction. Capture marks; redaction enforces the floor."""
+    payload: dict = {"target": target.to_payload()}
+    if _is_credential_target(target):
+        payload["password"] = value  # credential-named ⇒ redaction blanks it wholesale
+    else:
+        payload["value"] = value
+    if intent is not None:
+        payload["intent"] = intent
+    return ev.SHADOW_USER_TYPE, payload
+
+
+def capture_navigate(url: str, *, intent: str | None = None) -> tuple[str, dict]:
+    """A ``data.shadow.user.navigate`` (type, payload). The URL is redaction-swept
+    (credentials/tokens in the query stripped) before it reaches the log."""
+    payload: dict = {"url": url}
+    if intent is not None:
+        payload["intent"] = intent
+    return ev.SHADOW_USER_NAVIGATE, payload
+
+
+def capture_page_loaded(url: str, title: str) -> tuple[str, dict]:
+    """A ``data.shadow.page.loaded`` (type, payload) — a settled page; the locus a
+    subsequent action's semantic target re-resolves against."""
+    return ev.SHADOW_PAGE_LOADED, {"url": url, "title": title}
+
+
+def capture_network_response(url: str, status: int, host: str) -> tuple[str, dict]:
+    """A ``data.shadow.network.response`` (type, payload) — METADATA only (no body,
+    no headers beyond the host). The synthesized agent's egress allowlist is induced
+    from the ``host`` values across these events."""
+    return ev.SHADOW_NETWORK_RESPONSE, {"url": url, "status": status, "host": host}
+
+
+def capture_scroll(direction: str, y: int = 0) -> tuple[str, dict]:
+    """A ``data.shadow.user.scroll`` (type, payload) — a settled scroll up/down. Context,
+    not an action step: it records that the human had to scroll to reach the next thing."""
+    d = direction if direction in ("up", "down") else "down"
+    return ev.SHADOW_USER_SCROLL, {"direction": d, "y": int(y)}