graphjin 3.20.70 → 3.20.72

package/README.md

@@ -20,7 +20,8 @@ Works with PostgreSQL, MySQL, MongoDB, SQLite, Oracle, MSSQL, Snowflake, Redshif
 - **Smart discovery before action** - Agents start with `query_catalog(search: "<user instruction>")`, `graphql_help`, relationship evidence, examples, config recipes, and safety notes before writing or running queries.
 - **Guarded action, not raw access** - Source-mode access, query allow-lists, read-only boundaries, policy-aware MCP tools, local encrypted secrets, and `gj_config` preview/apply keep changes auditable.
 - **Operational awareness** - `gj_security`, `gj_runtime`, and the built-in console expose policy and bounded runtime status so agents can check what is safe before they act.
-- **Durable memory and standing questions** - Saved queries, fragments, and workflows live in the owner-scoped `gj_artifacts` store; cursor-backed watches (`gj_watch`) run standing questions under the owner's permissions, resume from persisted subscription cursors, and deliver fired events to a durable inbox (`gj_watch_event`), webhooks, or workflows. Normal watches are durable by default; explicit ephemeral watches use TTL leases.
+- **Standing questions, not just answers** - Hand a cursor-paginated subscription to `gj_watch` and GraphJin keeps answering it under the owner's permissions, resumes from persisted cursors across restarts, treats absence as a first-class event, and files what it finds in a durable inbox (`gj_watch_event`), a webhook, or a workflow. Evaluation runs on database polls, not model calls - a model writes the watch once, and nothing calls one again unless you turn on optional per-watch triage. See [Watches](#watches-and-standing-questions).
+- **Durable memory** - Saved queries, fragments, and workflows live in the owner-scoped `gj_artifacts` store. Normal watches are durable by default; explicit ephemeral watches use TTL leases.
 
 ## Installation
 
@@ -608,7 +609,151 @@ Instead of your client chaining `query_catalog` → `validate_where_clause` →
 - **Machine-actionable refusals:** blocked responses carry a structured `refusal` (code, reasons, unblock steps, `policy_final`/`retryable`) so a calling agent can course-correct in one step instead of guessing.
 - **Server-owned model:** `ask_graphjin_agent` always uses the provider, model, and credential environment variable configured under `agent`. Without server credentials it fails closed with `model_credentials_required`.
 
-It is an RLM loop — the model writes JavaScript that calls the discovery tools, and the typed result is parsed from `key: value` output. It needs strong **code generation**, not provider tool-calling. Ax requests structured JSON for typed stages, choosing the mechanism from the named deployment profile (`agent.provider`) and its rules for the selected model; `agent.structured_output_mode: auto` is the default, with `native`, `function`, and `json_object` available as overrides. See [AGENTIC.md](AGENTIC.md#server-side-agent) and [CONFIG.md](CONFIG.md#agent-configuration).
+It is an RLM loop — the model writes JavaScript that calls the discovery tools, and the typed result is parsed from `key: value` output. It needs strong **code generation**, not provider tool-calling. Ax requests structured JSON for typed stages, choosing the mechanism from the named deployment profile (`agent.provider`) and its rules for the selected model; `agent.structured_output_mode: auto` is the default, with `native`, `function`, and `json_object` available as overrides. `agent.service_tier` similarly defaults to provider-delegated `auto`, with portable `standard`, `flex`, and `priority` requests available when the profile/model supports them. See [AGENTIC.md](AGENTIC.md#server-side-agent) and [CONFIG.md](CONFIG.md#agent-configuration).
+
+## Watches And Standing Questions
+
+Asking is a pull: it only produces value when someone thinks to ask. A watch is the
+push direction. You hand `gj_watch` a cursor-paginated subscription once, and GraphJin
+keeps answering it against live data, under the owner's stored identity and role.
+
+```graphql
+mutation {
+  gj_watch(insert: {
+    name: "failed_invoices"
+    description: "Alert when a failed invoice changes."
+    query: "subscription failed_invoices { invoices(where: {status: {eq: \"failed\"}}, first: 25, after: $cursor) { id account_id status attempts } invoices_cursor }"
+  }) { id name status enabled }
+}
+```
+
+- **It costs database polls, not model calls.** The runner rides GraphJin's existing
+  cursor-backed subscriptions, compares a hash of each result, and writes a cursor
+  checkpoint even when nothing changed. No model is consulted to decide whether
+  something happened. A model writes the watch once; after that the only optional model
+  cost is per-watch triage in `enrich_json`, which is off by default and capped per day.
+- **Absence is a first-class event.** A watch with `absence_json` fires when the
+  expected thing *does not* arrive, so silence stops being indistinguishable from
+  health.
+- **It survives restarts.** Cursors are persisted, so a watch resumes where it left off
+  rather than replaying or skipping.
+- **It cannot outrun its owner.** A watch only ever sees what its owner could already
+  query, and both `gj_watch` and `gj_watch_event` are owner-scoped.
+- **Waking you and acting are different permissions.** Inbox delivery is immediate;
+  autonomous webhook or workflow delivery stays paused until the exact current
+  `action_hash` is approved. Alerts fail open, actions fail closed.
+
+The built-in demo registers four standing questions on a first run (declared in
+`examples/saas-ops/seed/watches.yml`), three of which have already fired by the time the
+console opens. Management, review, cleanup, and MCP resource subscriptions are covered
+under [MCP Tools](#mcp-tools); the full model is in
+[AGENTIC.md](AGENTIC.md#watches-standing-questions-with-a-durable-inbox).
+
+## Train And Measure Agents On Your Own Graph
+
+The same machinery that grades GraphJin's public benchmark can grade agents on
+*your* data, and serve as a reinforcement-learning environment for tuning small
+models on it.
+
+```bash
+# Generate a verified task suite from your catalog, with a train/eval split
+graphjin eval create --demo --writable --scale 500 --composition coverage \
+  --verify-concurrency 8 --split 0.8
+
+# Serve it: pooled isolated worlds, one graded episode per request
+graphjin env serve --path ./graphjin-demo --suite eval/suite.yml --pool 4 \
+  --split eval/suite.split.json --side train --freeze-time 2026-08-01T12:00:00Z
+```
+
+- **Tasks come from your schema.** Each carries a hidden oracle — a read-only
+  query that computes the answer in the database — so being plausible earns
+  nothing. Writes are graded by the state the database ended in *and* by every
+  other row staying put.
+- **Worlds are isolated and resettable.** An episode leases one, so a task that
+  writes changes only the world it was given.
+- **Your policy plugs in by configuration**, not code: point `agent.base_url` at
+  any OpenAI-compatible endpoint.
+- **Your real schema, without your real data.** `graphjin env clone --url
+  <server>` learns a running GraphJin server's schema from its catalog and
+  writes a local SQLite copy filled with synthetic rows. No data is read: the
+  only real values that cross over are the closed sets the catalog already
+  publishes. The clone is writable and resettable, so write tasks and training
+  work against it while production is never touched.
+- **Nothing memorable to overfit.** `graphjin env new-world` writes a fresh
+  organization — schema, data and all — deterministically from a seed, so you
+  can train on some companies and measure on others. Worlds can be asked for the
+  awkwardness real schemas have: one word meaning two things, a stale column
+  that still looks authoritative, fields that are usually null.
+- **Any industry, not three.** `graphjin env new-world --describe "genome
+  sequencing lab"` asks a capable model to name the records that business would
+  actually keep, checks every name, and saves the description as
+  `world-pack.json` inside the world. From then on the world is rebuilt from
+  that file with `--pack`: deterministic, and with no model involved.
+- **A big model writes the questions a schema cannot derive.** Counting and
+  filtering follow from column statistics; knowing that *failed invoices are
+  worth alerting on* does not. `graphjin eval author` asks a capable model —
+  configured separately from the small one being trained, via `GJ_GENERATOR_*` —
+  to choose what is worth watching and phrase it as a colleague would. Every
+  table, column and value it names must exist, and every task it produces is
+  verified against the live database before it counts.
+- **Questions no single source answers.** Real answers are often half in the
+  database and half in something somebody wrote down. Clones carry over the
+  document sources the original served — the names only, never a file — and
+  authoring plants the standard it grades against in a document of its own, so
+  the ground truth is true by construction rather than assumed.
+- **Runs export as training data.** `graphjin eval export` writes trajectories
+  as JSONL, marking the programs GraphJin's runtime wrote itself so they are not
+  mistaken for the policy's.
+- **Collecting is not measuring.** `graphjin eval sample --repeats 8
+  --temperature 0.8` draws many attempts at each task instead of judging one.
+  It reaches no verdict and promotes nothing, because a temperature raised on
+  purpose loses against a greedy baseline every time — and it records which side
+  of the split it drew from, so `eval export` can refuse to build a training
+  corpus out of held-out work.
+
+```bash
+# Learn a real server's schema; write a local synthetic copy
+graphjin env clone --url https://graphjin.internal --out ./clone-acme
+
+# Have a capable model author the richer families for it
+export GJ_GENERATOR_MODEL=<a-strong-model>
+graphjin eval author --demo --path ./clone-acme --kinds watch,confirmation,file --yes
+```
+
+### Driving Episodes Your Own Way
+
+Three ways in, all grading through the same contract, so a number from one is a
+number from any:
+
+- **Let GraphJin call your endpoint** — the default. Point `agent.base_url` at
+  anything OpenAI-compatible.
+- **Supply each completion yourself** — `env serve --step`. The episode runs
+  normally, but when the model is needed the call is parked and handed to you as
+  an observation; you post the completion back and it resumes. Useful when the
+  weights being updated live inside your training process and standing up an
+  inference server just to be called back is machinery you do not want.
+- **Bring your own agent entirely** — `env serve --external`. You get the task,
+  an MCP endpoint and a deadline, do the work with your own scaffold, and post
+  an answer. The server records every tool call, so the method and behavior
+  rules apply exactly as they do to a hosted run — an answer with no work behind
+  it scores zero.
+
+An agent run is several model calls with different jobs, and they need not all
+be the policy's. `--support-model` (or `GJ_SUPPORT_MODEL`) puts a fixed capable
+model in front of the distiller and responder stages while the policy answers
+the executor, so a small model is measured on the work being trained rather than
+through bottlenecks it did not create. The stages that write the final answer
+stay with the policy: letting a stronger model write those would score its care
+as the policy's grounding.
+
+```bash
+# Train only the executor; a fixed model condenses and phrases
+graphjin env serve --path ./clone-acme --suite eval/suite.yml --pool 4 \
+  --step --support-model <a-fast-model>
+```
+
+See [training/README.md](training/README.md) for the client, an example policy
+server, and what to record alongside a result.
 
 ## JS Workflows (GraphQL + REST)
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "graphjin",
-  "version": "3.20.70",
+  "version": "3.20.72",
   "description": "GraphJin — one governed graph for AI agents: GraphQL + MCP over your databases, files, APIs, and code",
   "bin": {
     "graphjin": "bin/graphjin.js"