matrx-connect 0.1.73__tar.gz

matrx_connect-0.1.73/.gitignore

@@ -0,0 +1,299 @@
+*.pyc
+secrets/
+ignore/
+temp/
+logs/
+# The broad `logs/` rule above is for RUNTIME log output, but it also matched
+# the dashboard's SOURCE directory and silently swallowed an entire feature's
+# files (only the pre-existing index.tsx stayed tracked), breaking the prod
+# Docker build with "Could not resolve ./structured-tab". Re-include the source.
+!apps/dashboard/src/features/logs/
+!apps/dashboard/src/features/logs/**
+todo
+text_notes/
+aidream/secrets/2.env
+automation_matrix/matrix_processing/temp/*
+cd
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+.venv/
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+# The blanket lib/ rule above is from the standard Python .gitignore template
+# and was silently swallowing TS source under the SPA `src/lib/` folders.
+# Re-allow them explicitly so frontend builds don't ship without their lib layer.
+!apps/dashboard/src/lib/
+!apps/dashboard/src/lib/**
+!apps/dashboard/src/features/crawler/lib/
+!apps/dashboard/src/features/crawler/lib/**
+!apps/workflow-studio/src/lib/
+!apps/workflow-studio/src/lib/**
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+ai/tests/clean_response.json
+ai/tests/cx_storage_response.json
+ai/tests/execution_test.py
+ai/tests/final_response.json
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.env_remote
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+.env.armanonly
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# random armani files
+/armani_dev/secrets/
+/armani/
+/_armani/
+
+
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+.idea/
+.vscode/
+/node_modules/
+
+# Frontend pnpm workspace (apps/) — node_modules at the workspace root and any
+# member, plus Vite caches and build output. The unified lockfile (apps/pnpm-lock.yaml)
+# IS committed; everything below is regenerated.
+node_modules/
+apps/**/.vite/
+apps/**/dist/
+.vite/
+
+dump.rdb
+
+frontend/
+
+# AME Temp Files and directory structure
+# Ignore all files in the temp directory and its subdirectories
+/temp/**/*
+/tmp/**/*
+
+# Allow .gitkeep files to retain directory structure
+!/temp/**/.gitkeep
+!/tmp/**/.gitkeep
+
+# Armani
+.history*
+.history/
+/local_data/
+local_reports_data/
+webscraper/quick_scrapes/temp/
+automation_matrix/ai_apis/fireworks/_dev/*
+automation_matrix/ai_apis/fireworks/_dev/fireworks_sample.py
+*.pdf
+*.flac
+*.mp3
+*.wav
+miniconda.sh
+/database/python_sql/temp_data/
+ .history*
+ .history/
+.history/
+
+_dev/
+/_dev/
+requirements_filtered.txt
+
+# matrx-dev-tools backups
+.env-backups/
+# Matrx Ship config (contains API key)
+.matrx-ship.json
+
+# Matrx config (contains API keys)
+.matrx.json
+.matrx-tools.conf
+
+# Claude Code local worktrees and per-user settings
+.claude/worktrees/
+.worktrees/
+.claude/settings.local.json
+
+# Append-only snapshots from matrx_utils.update_history (unbounded; do not commit)
+common/utils/data_in_code/data_history.json
+packages/matrx-utils/matrx_utils/data_in_code/data_history.json
+
+# Tool-dispatch debug logs — one file per server start, never committed
+.matrx-debug/
+
+# macOS Finder metadata
+.DS_Store
+**/.DS_Store
+
+# Environment files
+.env
+.env.*
+*.env
+*.env.*
+
+# Keep safe templates trackable
+!.env.example
+!.env.sample
+!.env.template
+
+# Never commit local OAuth client/token artifacts
+credentials.json
+token*.pickle
+token*.json
+tests_trials/rag_tests/kg_export_output/
+
+# Pooler-contamination watch output (machine-local evidence, not source)
+db/evidence/
+
+# Pooler contamination evidence — machine-local forensic capture, not source
+db/evidence/
+db/mirror/.env.mirror
+supabase/.temp/
+**/supabase/.temp/
+# Packed npm artifacts. Kept locally for a one-time bootstrap publish; never
+# committed (they are large binaries regenerable with `pnpm pack`).
+apps/shared/*/*.tgz
+
+# Guard forcing-function scratch packages. These tests plant real files inside
+# the real tree the guard scans (a temp dir would prove nothing) and delete
+# them in teardown — ignoring them keeps the auto-commit tooling from
+# capturing one mid-run.
+_kind_boundary_scratch*/
+_kind_marker_law_scratch*/

matrx_connect-0.1.73/AGENTS.md

@@ -0,0 +1 @@
+CLAUDE.md

matrx_connect-0.1.73/CLAUDE.md

@@ -0,0 +1,285 @@
+# CLAUDE.md — matrx-connect
+
+> **Operating Principle: Build the platform, not the artifact.** Every task is a probe that exposes a missing capability — build it, then consume it. Code that only serves one artifact is forbidden. Full doctrine: [/PRINCIPLES.md](../../PRINCIPLES.md).
+
+**Package:** `matrx-connect` (PyPI) — Python 3.12+ — currently v0.1.70
+**Role in the graph:** Tier 1. Depends only on `matrx-utils`. Used by `matrx-graph`, `matrx-scraper` (optional), and `matrx-ai`.
+
+---
+
+## Read this first
+
+`matrx-connect` is the FastAPI-integration layer for the Matrx ecosystem: **auth middleware, streaming response infrastructure, request-scoped `AppContext`, and the `Emitter` protocol.** It is *not* a database connectivity package (despite the name — "connect" refers to connecting FastAPI apps to the Matrx streaming / auth contract).
+
+Any aidream-specific concept that belongs in a package belongs here, **not in the packages above**. If a sibling package needs auth or streaming primitives, it should `configure_ext(...)` or import from matrx-connect; aidream should never be touched.
+
+---
+
+## What this package provides
+
+- **`AppContext`** (`matrx_connect.context.app_context`) — request-scoped identity + metadata held in a `ContextVar`. Fields: `user_id`, `email`, `auth_type`, `is_authenticated`, `is_admin`, `request_id`, `fingerprint_id`, `api_keys`, `metadata`, `emitter`. **Frozen** (Phase 1) — derive a modified copy with `with_overrides(**kwargs)` and install it via `set_app_context(...)` (the old mutating `extend()` was removed and now raises). Persistence intent fields: `store` (False = write NOTHING, ephemeral) and `system_run` (True = internal machine call; consumers persist only the cost spine and skip transcript machinery — set via matrx-ai `run_agent(system_run=True)`, semantics documented on the field). Durable work uses the paired, validated `execution_kind` + UUID `execution_id` fields via `with_execution_attribution(...)`; persistence stamps these directly instead of scraping mutable metadata or inferring ownership from conversation IDs. Forked contexts preserve the attribution.
+- **`system_app_context(feature, *, user_id, ...)`** (`matrx_connect.context.app_context`) — **the ONE way to get an ambient `AppContext` for work that runs OUTSIDE a request** (a `detached_task`, queue worker, NOTIFY listener, scheduled run). Those start with EMPTY contextvars, so anything downstream that calls `get_app_context()` — **every agent run does**, via `run_agent` → `child_agent_context` — dies with *"No AppContext is set. Ensure AuthMiddleware is registered…"*, an error that blames middleware for a request that never existed. **Idempotent**: yields the existing context untouched when one is ambient, so it is safe to wrap an entry point that has both an HTTP and a background caller. Non-persisting by default (`store=False`, `is_internal_agent=True`); pass any AppContext field as a kwarg to override (`system_run=True` to keep the cost spine). **Install it at the FUNNEL every background dispatcher passes through, not at each agent call site** — a guard placed one layer too deep is exactly how aidream's PDF page cleaner silently embedded raw text for every PDF (2026-07-11) while the NER stage below it worked fine. Never hand-roll `AppContext(...)` + `set_app_context` + `finally: clear_app_context` again — that copy-paste block is what this replaced.
+- **Auth middleware** (`matrx_connect.middleware.auth`) — pluggable Supabase JWT / fingerprint resolution. Constructor takes JWT verification settings plus injected `is_admin_resolver`, `resolve_guest`, and optional organization resolver callbacks; static admin tokens are forbidden.
+- **`Emitter` protocol + `StreamEmitter` / `ConsoleEmitter`** — async event emitters with the full vocabulary (`send_chunk`, `send_reasoning`, `send_data`, `send_phase`, tool events, `fatal_error`, `send_end`).
+- **`buffer_error_events()`** — task-local emitter overlay for retry ownership boundaries. Non-error events pass through immediately; ERROR events replay only when the scoped operation returns normally and are discarded when it raises. Stream/console emitters independently enforce at-most-one terminal ERROR per lifecycle.
+- **`create_streaming_response(...)`** (`matrx_connect.streaming.response`) — the canonical streaming response wrapper. Spawns the task, manages lifecycle, emits heartbeats, clears the ContextVar on exit. Opens every stream with `phase="connected"` followed by the route's `initial_message` as an INFO event (`code="initial_message"`, mirrored into `system_message`/`user_message`) — the client's opening progress line. (Before 2026-08-10 the parameter was accepted and silently dropped.)
+  Detached `StreamEmitter`s keep publishing after the first HTTP reader leaves, retain every sequenced NDJSON frame for the task's live lifetime, and expose `generate_replay()` for owner-authorized rejoin routes. `register_active_stream(request_id, emitter)` / `get_active_stream` / identity-guarded `unregister_active_stream` are the ONE process-local delivery registry; durable cross-process status/result recovery stays in the host runtime spine.
+  Classified exceptions may attach `error_info.code` and `error_info.details`; the wrapper
+  forwards both and adds the current request ID to details so clients retain the full
+  diagnostic contract instead of receiving only a generic label.
+- **Failure capture** (`matrx_connect.streaming.error_capture`) — the injected host sink for
+  durable `system_error` records. Pytest processes are denied by default; local fake sinks must
+  opt in explicitly. Contract: [`matrx_connect/streaming/FEATURE.md`](matrx_connect/streaming/FEATURE.md).
+- **`context_dep`** — FastAPI `Depends()` helper for pulling `AppContext` into a route handler.
+- **`require_organization_context`** — route dependency that declares required `X-Organization-Id`, validates its UUID, and rejects unresolved or mismatched middleware context before handler code. Admin callers use the same target-org contract; the returned `AppContext` preserves their admin tier.
+- **Event data types** (`matrx_connect.context.events`, `.data_types`, `.operations`, `.tool_event_data`, `.render_blocks`) — Pydantic event payload schemas shared with the frontend.
+
+Public exports from `matrx_connect/__init__.py`: `AppContext`, `Emitter`, `EventType`, `StreamEvent`, `StreamEmitter`, `ConsoleEmitter`, `BufferedErrorEmitter`, `BufferedErrorEvent`, `buffer_error_events`, `context_dep`, `require_organization_context`, `get_app_context`, `set_app_context`, `try_get_app_context`, `clear_app_context`, `system_app_context`.
+
+---
+
+## The streaming endpoint pattern (canonical)
+
+Every streaming route in aidream and in any external consumer is expected to follow this shape. Do not deviate inside this package:
+
+```python
+from matrx_connect import AppContext, context_dep
+from matrx_connect.streaming import create_streaming_response
+from fastapi import Depends
+
+@router.post("/thing")
+async def do_thing(payload: ThingRequest, ctx: AppContext = Depends(context_dep)):
+    return create_streaming_response(
+        ctx, _run, payload,
+        initial_message="Starting…", debug_label="ThingFlow",
+    )
+
+async def _run(emitter, payload: ThingRequest):
+    # AppContext is already in the ContextVar; use get_app_context() if you need it.
+    result = await do_work(payload)
+    await emitter.send_data(ResultPayload(...).model_dump())
+    await emitter.send_end()
+```
+
+Rules (enforce these on contributions to matrx-connect and consumers):
+
+- `create_streaming_response` owns the context **teardown** — it `clear_app_context`s on exit; never `clear_app_context` yourself. A route that needs a route-scoped field (e.g. `conversation_id`) for the streaming task installs it with `ctx = ctx.with_overrides(field=value); set_app_context(ctx)` BEFORE calling `create_streaming_response`. AppContext is **frozen** (Phase 1), so `set_app_context` is REQUIRED to install the derived ctx into the ContextVar — omit it and the override is silently dropped.
+- The task function takes `emitter` as its first arg, then business args. It does not handle `CancelledError` or generic exceptions; infrastructure does.
+- Heartbeats and client-disconnect cleanup are the middleware's job, not the task's.
+- **Detach preserves delivery, not only work.** Closing the primary generator must never set `StreamEmitter.cancelled` when `detach_on_disconnect=True`; doing so makes every later `send_*` a no-op and leaves rejoining clients with lifecycle-only status.
+- **`debug_label` is a LOG label, never an error identity.** It carries per-run ids
+  (`VisionInterview:72175649`), so the crash path derives the emitted `error_type`
+  through `stable_error_type()`, which strips them. Emitting
+  `f"{debug_label.lower()}_error"` mints a NEW error type for every session —
+  ungroupable, uncountable, unalertable, and the Error Inspector shows
+  `operation: unknown`.
+
+### The crash path never blames the user, and never ships a terminal banner
+
+[`failure_classification.py`](matrx_connect/streaming/failure_classification.py) sits on
+the ONE path every stream crash takes. Three rules, each earned on 2026-08-17 when
+`agent.slot_definition` was renamed to `agent.mandate` under a running build:
+
+1. **A recognized infrastructure failure gets a stable type and an honest message.**
+   `database_schema_mismatch` / `database_unavailable` / `database_permission_denied`,
+   each saying *"nothing you did caused it and your work is saved"*. The old generic
+   text — *"Please try again or adjust your settings"* — told a non-technical user to
+   go fix settings when the server could not reach a renamed table.
+2. **Classification is by exception CLASS NAME.** matrx-connect must not depend on
+   matrx-orm or asyncpg to recognize a database error; the whole `__mro__` and
+   `__cause__`/`__context__` chain is walked so a service that wraps the DB error in
+   its own type still classifies.
+3. **Everything client-bound is ANSI-stripped.** Operator banners are colour-escaped;
+   a browser renders `` literally. The raw text is preserved in the durable
+   `system_error` record — strip at the wire, never at the sink.
+
+An ordinary application bug stays unclassified on purpose: only infrastructure earns
+"not your fault", or a real bug gets laundered into "try again in a moment".
+
+---
+
+## The auth contract (read this if you're touching `middleware/auth.py`)
+
+`AuthMiddleware` is the only place `ctx.user_id` gets set. The contract is intentionally narrow:
+
+- **Accepted token types:** Supabase-signed JWTs, plus — when the host injects `resolve_api_key` — `mx_`-prefixed AI Matrx API keys (see the API-key lane below). Every other bearer falls through. For JWTs the middleware verifies the signature with `jwt_secret`/JWKS, checks `aud="authenticated"`, and pulls the user_id from `sub`.
+- **The API-key lane (ratified C16, 2026-08-29).** ONE `Authorization: Bearer` header for every credential kind, disambiguated by SHAPE: three-dot JWTs go to the one verifier, `mx_`-prefixed bearers go to the host-injected `resolve_api_key` callback (`ApiKeyResolver → ApiKeyIdentity | None`). The HOST owns the lookup, hashing, constant-time compare, and caching — the middleware never sees a secret store. Success sets `auth_type="api_key"`, `user_id` = the key's REAL service-identity `auth.users` row, and admits the key's ONE bound organization as the request org (`X-Organization-Id` present must EQUAL it → 409 `organization_context_mismatch`; absent → the key's org is admitted). Rejection (None or a raising resolver) leaves the request anonymous with compact INFO logging — never a 500, never token material in logs. Nothing downstream branches on "is this an API key". Design: `common-docs/projects/npm-package-extraction/API-KEY-LANE-DESIGN.md`.
+- **Preserve verified JWT lifecycle metadata.** `VerifiedToken.expires_at` and `.issuer` survive crypto validation, single-flight sharing, and cache hits so OAuth resource-server adapters can populate their native access-token contract without decoding the bearer a second time.
+- **A removed signing key stays removed.** An unknown `kid` forces at most one bounded JWKS refresh; if the refreshed document still omits it, reject the token. Supabase operators must keep a rotated key in `previously used` until every token it signed has expired—never weaken verification to compensate for premature revocation.
+- **An unknown JWKS `kid` is a denied credential, not an application failure.** Log the rejection at WARNING so it remains correlatable without entering the ERROR-only repair queue; do not create `system_error` for expected stale, foreign, or forged tokens.
+- **An HS256 token at an asymmetric-only verifier is also a denied credential.** Reject it at WARNING when `jwt_secret` is absent; it must not enter the ERROR-only repair queue or create `system_error`.
+- **An authenticated request with no admitted organization is expected caller denial.** Return `organization_required` and log `[AUTH][REJECT]` at WARNING; deliberate HTTP 400 and WebSocket 4400 control flow never enters the ERROR-only repair queue or creates `system_error`.
+- **The file-session cookie lane has a DRIFT DETECTOR (2026-08-30 outage class).** `mx_files_session` cookies are HMAC-signed with `guest_fingerprint_secret`; with per-process random fallback secrets across containers, cookies mint on one instance and fail on another — every private `<img>` 401s silently. The middleware now classifies cookie failures (`session_cookie.classify_session_cookie_value`): a PRESENT, structurally valid, UNEXPIRED cookie failing HMAC logs CRITICAL with the machine-greppable token `MX_COOKIE_SIGNATURE_MISMATCH` (rate-limited per burst window, `COOKIE_DRIFT_LOG_WINDOW_SECONDS`); expired/malformed stay silent-anonymous. `AuthMiddleware.secret_is_ephemeral` + `session_cookie.get_session_cookie_secret_source()` expose whether the process fell back to a random secret — matrx-files' `POST /files/session` mint response carries it as `"secret_source": "stable"|"ephemeral"` so clients can scream. Hosts MUST inject one stable secret on every instance.
+- **No static admin token, no env-mapped user_id.** The legacy `admin_token` + `admin_user_id` escape hatch was removed (intentionally — see [`SCRAPER_SECURITY_BUGS.md`](../../docs/archive/2026/scraper__SCRAPER_SECURITY_BUGS.md) for why). Do not re-add it.
+- **`is_admin` resolution:** an injected `is_admin_resolver(user_id) → bool` consults the host's `public.admins` table on every authenticated request. The middleware itself never decides admin status.
+- **Local dev / agent authentication:** the host's dev-login endpoint mints a real Supabase JWT for a real user_id (gated by an env flag never set in prod). Agents use that JWT like any other client. There is no agent-specific code path in this package, and there must never be one.
+
+If you find yourself wanting to add a fast path that maps a token-or-secret to a user_id without going through JWT validation, stop. The two sanctioned non-JWT lanes are exactly the injected ones above — guest fingerprints and the C16 API-key lane, both resolving to REAL `auth.users` identities via host callbacks. Anything else belongs in the host as an explicitly-named dev-only endpoint that *produces* a real JWT.
+
+### Server-to-server: the approved-server handshake (`service_auth.py`) — NOT the middleware
+
+The rule above is about the **middleware**. There is a separate, sanctioned way for one of our **servers** (not a person) to call a protected route: `matrx_connect.service_auth`. This is **opt-in per route** — a host applies it explicitly to specific routers; it is never wired into `AuthMiddleware`, and the middleware stays JWT-only.
+
+- `require_authenticated_or_service(secret, *, require_user=False)` — a router dependency that admits EITHER a real user login (`is_authenticated` already set) OR an approved-server call: the bearer equals the shared `secret` (constant-time), and the acting user is named by the `X-Matrx-User-Id` header. It stamps that user into the request context.
+- `verify_approved_server(request, *, secret, secret_header="authorization", require_user=False)` — the pure check (used by callers with a custom header or their own 503-when-unconfigured signal, e.g. the sandbox bridge / media-healer).
+
+**Why this is safe and NOT the removed escape hatch:** the old hatch hard-bound every call to ONE fixed `admin_user_id` env var, so writes were mis-attributed to a fake admin and users' data mixed (the incident in [`SCRAPER_SECURITY_BUGS.md`](../../docs/archive/2026/scraper__SCRAPER_SECURITY_BUGS.md) §D). This primitive **never defaults to a user** — the acting user is exactly what the caller names, or empty for ephemeral no-write calls; a write endpoint enforces its own owner and raises on a missing user. It's the reusable form of the pattern the sandbox bridge (`cloud_files_bridge`) already used; the scraper, sandbox bridge, and media-healer now share this one primitive. Adding a per-route service path here is allowed; adding a token→user path to the middleware is not.
+
+---
+
+## The configuration / injection pattern
+
+matrx-connect is itself a "thing other packages and apps inject into". Its own injection surface:
+
+- `AuthMiddleware(...)` constructor — pass `jwt_secret`, `is_admin_resolver`, and `resolve_guest` callback from the host. (`admin_token` / `admin_user_id` are no longer accepted — see "The auth contract" above.)
+- `AppContext.with_overrides(**kwargs)` + `set_app_context(...)` — host apps install request-scoped overrides (e.g. `conversation_id`) before the streaming task runs. AppContext is frozen; `extend()` was removed in Phase 1 and now raises.
+- `Emitter` is a `typing.Protocol` — any object that satisfies the shape works; `StreamEmitter` is just the default HTTP/JSONL implementation.
+
+Inside this package:
+
+- **Never** read environment variables at import time. The middleware takes its config as constructor args.
+- **Never** assume a specific database or ORM. The only persistence matrx-connect does is NDJSON over HTTP.
+
+---
+
+## Dependency rules specific to this package
+
+- ✅ May import `matrx_utils` (currently only for `vcprint` dev-logging).
+- ❌ No `from matrx_orm import …`
+- ❌ No `from matrx_graph import …`
+- ❌ No `from matrx_scraper import …`
+- ❌ No `from matrx_ai import …`
+- ❌ No `from aidream import …`, no root-module imports.
+
+If a feature seems to need an ORM (e.g. auth wants to load users from a DB), inject the loader function — don't pull in matrx-orm.
+
+---
+
+## The event vocabulary — when to use which
+
+`matrx_connect.context.events.EventType` is THE registry for everything Python emits to the client. Every event has a Pydantic payload class registered in `PAYLOAD_REGISTRY`; `build_event(event_type, payload)` is the only sanctioned constructor (rejects payload-class mismatches at emit time). Adding a new event type is a coordinated change: enum entry + payload class + registry entry + Emitter Protocol method + StreamEmitter/ConsoleEmitter implementations + (optional) re-exports in `context/__init__.py`.
+
+The most-confused choice points — read these before you reach for a new emitter call:
+
+| You want to tell the client… | Use | Don't use |
+|---|---|---|
+| Token-by-token text or reasoning TEXT | `send_chunk` / `send_reasoning_chunk` | DATA |
+| **The model started / stopped THINKING (content-less)** | **`send_reasoning_state`** | `send_phase`, `send_chunk` |
+| Lifecycle stage (e.g. "connected", "tool_loop_started") | `send_phase` | INFO |
+| Structured payload the UI renders inline (search results, conversation IDs, image URLs, …) | `send_data` | DATA + secret intent encoding |
+| Operational note that decorates a successful response | `send_warning` (with severity) | ERROR |
+| The response itself failed before/during stream | `send_error` (then `send_end`) | WARNING |
+| Tool started / progress / completed | `send_tool_event` | INFO |
+| A matrx-orm record was reserved / updated | `send_record_reserved` / `send_record_update` | RESOURCE_CHANGED |
+| **A non-orm resource changed; client should refetch if displaying it** | **`send_resource_changed`** | DATA, INFO |
+| **An agent finished and its declared `output_schema` was applied to the response** | **`send_structured_output`** | DATA, RECORD_UPDATE |
+| **A provider attached a citation to the streaming answer** | **`send_citation`** | DATA, INFO |
+
+### `REASONING` — content-less thinking lifecycle (added 2026-07-05)
+
+`send_reasoning_state("started" | "stopped")` — the model opened or closed a thinking/reasoning block, **independent of whether reasoning text streams**. Every provider signals a reasoning block start even when the reasoning text is suppressed (Anthropic `display="omitted"`, OpenAI `reasoning_summary="never"`, Gemini `include_thoughts=False`) — this event carries that signal so the UI shows a "thinking" state instead of a silent heartbeat gap.
+
+- **This is NOT `reasoning_chunk`.** `reasoning_chunk` carries reasoning TEXT (only when requested); `reasoning` is the on/off lifecycle and fires regardless.
+- **This is NOT a `phase`.** A model can think while the overall phase is `generating` or `using_tools`; reasoning is a sub-state that pairs `started`/`stopped`, not a one-way progress stage.
+- **Emitted by the provider stream parsers**, not the orchestrator — each provider fires it at its own reasoning-block boundary (`matrx_ai/providers/*/*_api.py`). Balanced per turn; a turn with no thinking emits nothing.
+
+### `CITATION` — live citation stream (added 2026-07-17)
+
+`send_citation(CitationPayload(block_index, citation))` — one **normalized** citation (the canonical cross-provider `NormalizedCitation` shape owned by `matrx_ai.config.citations`; matrx-connect carries it as a dict) attached to the streaming answer. Emitted live by the provider stream parsers (Anthropic `citations_delta`, OpenAI `response.output_text.annotation.added`) or at stream settle for terminal-only providers (Gemini grounding). The same citations also persist on the text part's top-level `citations` (`TextPart.citations`) — the stream event is the live channel, storage is the settle channel; both carry the identical shape.
+
+### `RESOURCE_CHANGED` — generic refresh hint (added 2026-04-27)
+
+Use `send_resource_changed(*, kind, action, resource_id, sandbox_id?, user_id?, metadata?)` whenever your code mutates something on the server side that a client may be displaying or caching, and **no other event type already covers it**. This is the generic "refetch this" primitive — distinct from RECORD_UPDATE (matrx-orm-row-specific) and DATA (which carries actual content, not change hints).
+
+`kind` is namespaced and intentionally open-ended; consumers register their own. Established conventions:
+
+- `fs.file`, `fs.directory` — sandbox / agent workspace filesystem (emitted by matrx-ai's `fs_write`, `fs_patch`, `fs_mkdir` — see `matrx_ai/tools/_change_events.py`)
+- `cld_files` — AI Dream cloud-files row changed (size, version, mime_type)
+- `sandbox.cwd` — agent's working directory shifted
+- `cache.<key>` — explicit cache invalidation hint
+- `active_tools` — the agent's active tool set changed mid-loop. Emitted by matrx-ai's `tools/dynamic_drain.py` whenever a registered tool calls `ctx.queue_tool_changes(...)` and the orchestrator drains it between iterations. **Exactly one event per non-empty drain**; an empty drain (no pending mutations) emits nothing — the channel is silent when the active set is unchanged. `metadata` carries:
+  - `added_tools: list[str]` — names of tools just added to the active set (in queue order; possibly empty when this drain only removed).
+  - `removed_tools: list[str]` — names of tools just removed (deduped, order preserved).
+  - `active_count: int` — total tools active *after* the drain (`config.tools` + `config.custom_tools`); use this to keep client-side counters in sync without reconciling lists.
+  - `added: int`, `removed: int`, `sources: list[str]` — back-compat scalars from the original payload; safe to ignore in new consumers.
+
+  The FE re-renders any "active tools" UI on this signal — see [cx_chat__TOOL_INJECTION_REFACTOR.md](../../docs/archive/2026/cx_chat__TOOL_INJECTION_REFACTOR.md) (archived) for the full pattern. The matrx-extend Chrome extension consumes this event into its `useActiveToolsStore` to update the Tools-tab badge and `loaded_categories` hint without polling.
+
+`action` is a closed `Literal`: `created` | `modified` | `deleted` | `moved` | `renamed` | `invalidated`. Use `invalidated` as the batch/coalesced "I changed many things under this scope" hint (a directory wipe, a `git checkout` that touches dozens of files) instead of fanning out to one event per file.
+
+For move/rename, `resource_id` is the **new** identifier and `metadata.previous_id` carries the old one.
+
+The wire shape is auto-emitted into `aidream/api/generated/stream-events.ts` via the `/schema/all` endpoint, so the FE gets typed `ResourceChangedPayload` after running `pnpm sync-types`.
+
+When in doubt: there is **no shame in adding a more specific event type later** if `kind` strings start to feel like a sub-type system. The point of RESOURCE_CHANGED is to give every package a typed home for invalidation hints without forcing matrx-connect to know about every domain.
+
+### `STRUCTURED_OUTPUT` — agent's parsed response (added 2026-05-12)
+
+Emitted automatically by `matrx_ai.agents.Agent.execute` whenever the agent has an `output_schema` declared and the execution loop has produced its final assistant message. Carries the full JSON Schema AND the parsed payload, so the FE can drive generic renderers (tables, forms, type-checked accessors) without re-running extraction or re-fetching the agent record.
+
+- Fires AFTER all `CHUNK` events — prose-first streaming behavior is preserved; the structured event is a capstone, not a replacement.
+- Fires whether extraction succeeded or failed. `success: false` (with `data: null` and a human-readable `reason`) tells the FE the agent declared a contract and the parser couldn't honour it — surface the gap, don't silently drop.
+- Fires once per agent completion. Sub-agents emit their own events, tagged with the matching `operation_id` so the FE can attribute them to the right sub_agent init/completion pair.
+- The schema is in the payload (verbatim from `agent.output_schema["schema"]`) — clients should not have to look up the agent to know the contract.
+- Kind-bound executions also carry `kind`, `kind_version`, `kind_checked`, and
+  `kind_errors`. Treat `kind_checked=false` as skipped/unverified, never as pass.
+
+Wire shape:
+
+```json
+{
+  "event": "structured_output",
+  "data": {
+    "schema_name": "combined_sdt_index",
+    "json_schema": { "type": "object", "required": ["items"], "properties": { … } },
+    "data": { "items": [ … ] },
+    "success": true,
+    "reason": "",
+    "match_count": 1,
+    "agent_name": "WC Medical & Legal Report Extractor",
+    "operation_id": null,
+    "kind": "agent_io_7f3a_output_ab12cd34",
+    "kind_version": 1,
+    "kind_checked": true,
+    "kind_errors": []
+  }
+}
+```
+
+The FE picks up the typed `StructuredOutputPayload` automatically after running `pnpm sync-types`.
+
+**Do not** hand-roll a `send_data` event to ship structured agent output. The funnel is `Agent.output_schema` + automatic emission. If a caller needs the parsed value server-side without an agent (e.g. a raw `execute_ai_request` call), call `parse_agent_output(text, schema)` directly — but the standard agent path emits the event for you.
+
+---
+
+## Python standards (same as root)
+
+- Full type hints on every signature. `Emitter` / `AppContext` shapes are part of the public contract; changing them is a breaking change.
+- No docstrings unless the symbol is user-facing (the ones in `context/`, `streaming/`, `middleware/` qualify — keep them one line).
+- Explicit exception handling. `fatal_error` is the one-way door for unrecoverable stream errors.
+
+---
+
+## Testing this package in isolation
+
+```bash
+uv run pytest packages/matrx-connect/tests
+```
+
+Tests must run without any sibling package beyond `matrx-utils`. Use FastAPI's `TestClient` for middleware; mock `resolve_guest` rather than reaching into aidream.
+
+### This package ships the ambient-AppContext leak guard for the WHOLE monorepo
+
+`matrx_connect/pytest_plugin.py` holds the autouse `_no_leaked_app_context` fixture — Layer 2 of the ambient-AppContext contract, which FAILS by name any test that leaves `_app_context` set (Layer 1 is each test cleaning up after itself). It lives here because this package owns that process-global ContextVar.
+
+It is registered as a **`pytest11` entry point**, not a conftest, and that is load-bearing: matrx-ai, matrx-assignment, matrx-files, matrx-orm, matrx-runtime and matrx-seo each declare their own `[tool.pytest.ini_options]`, so `pytest packages/<pkg>/tests` makes that package the rootdir and the repo-root `conftest.py` is never loaded — the guard used to not run at all in a standalone package run, which is how a real leak in matrx-ai survived until it happened to be collected alongside matrx-connect. An entry-point plugin is auto-loaded from installed distribution metadata whatever the rootdir, so it reaches even the packages that (correctly) do not depend on matrx-connect, with no conftest to copy-paste and no new sibling dependency anywhere.
+
+- **ONE definition — edit it here.** Never copy the fixture into a conftest; copies drift and that drift is the bug this replaced.
+- Standalone independence is untouched: where matrx-connect isn't installed, neither is the ContextVar, so there is nothing to guard.
+- Adding the entry point does not change any consumer's version floor (nothing imports the module). Opt out of a run with `-p no:matrx_app_context_guard`.
+
+---
+
+## Known issues
+
+Tracked centrally in the root `PACKAGES_MIGRATION_PLAN.md`. This package is the cleanest of the bunch — no known import violations or hardcoded paths.