agent-first-data 0.17.3__tar.gz → 0.19.0__tar.gz

{agent_first_data-0.17.3 → agent_first_data-0.19.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: agent-first-data
-Version: 0.17.3
-Summary: A naming convention that lets AI agents understand your data without being told what it means.
+Version: 0.19.0
+Summary: A naming convention that lets AI agents understand your data without being told what it means, plus a CLI and library for reading and safely editing structured JSON, TOML, YAML, dotenv, and INI documents.
 License-Expression: MIT
 Project-URL: Repository, https://github.com/agentfirstkit/agent-first-data
 Requires-Python: >=3.9
@@ -14,7 +14,7 @@ pip install agent-first-data
 ```
 
 ```python
-from agent_first_data import json_result, output_json, output_plain
+from agent_first_data import json_result, render, OutputFormat
 
 value = json_result(
     {
@@ -24,11 +24,11 @@ value = json_result(
     }
 ).build()
 
-print(output_json(value))
-print(output_plain(value))
+print(render(value, OutputFormat.JSON))
+print(render(value, OutputFormat.PLAIN))
 ```
 
-Useful names use Python casing: `output_json`, `output_yaml`, `output_plain` (each takes an optional keyword-only `options=`), `redacted_value`, `redact_secrets_in_place`, `redact_url_secrets`, `normalize_utc_offset`, `is_valid_rfc3339_date`, `is_valid_rfc3339_time`, `is_valid_rfc3339`, `is_valid_bcp47`, `cli_parse_output`, `cli_output`, `build_cli_error`, `build_cli_version`, `cli_handle_version_or_continue`, `decode_protocol_event`. Skill admin and stream-redirect helpers live in `agent_first_data.skill` and `agent_first_data.stream_redirect` (import the submodule directly).
+Useful names use Python casing: `render` (the single value x format x options -> str entry point; takes an `OutputFormat` and an optional keyword-only `options=`), `redacted_value`, `redact_url_secrets`, `normalize_utc_offset`, `is_valid_rfc3339_date`, `is_valid_rfc3339_time`, `is_valid_rfc3339`, `is_valid_bcp47`, `cli_parse_output`, `build_cli_error`, `build_cli_version`, `cli_handle_version_or_continue`, `decode_protocol_event`. Skill admin and stream-redirect helpers live in `agent_first_data.skill` and `agent_first_data.stream_redirect` (import the submodule directly).
 
 Logging is available through `init_logging_json`, `init_logging_plain`, `init_logging_yaml`, `span`, and `get_logger`.
 
@@ -42,7 +42,7 @@ init_logging_json("INFO", secret_names=("authorization",))
 
 - Default redaction replaces every `_secret` or configured secret-name subtree with `***`, including objects and arrays.
 - `_url` fields scrub userinfo passwords and secret-named query parameters; surrounding whitespace is trimmed and internal whitespace redacts the whole field.
-- YAML/plain quote and escape keys as well as values, sort by UTF-16 code unit order, and render nested objects in arrays as canonical JSON.
+- YAML keeps original keys and values (structure-preserving, like JSON), sorting keys by UTF-16 code unit order and quoting/escaping unsafe keys and string scalars. Plain strips formatting suffixes, formats values, sorts the same way, and renders nested objects/arrays as canonical JSON.
 - Logging records use `kind:"log"` with a nested `log` payload and a separate `level` field, so error-level logs are not terminal protocol errors.
 - `build_cli_error(message, hint?)` returns a strict-ready CLI error with `error.retryable:false` and `trace:{}`.
 - Use `cli_handle_version_or_continue()` before argument parsing so bare `--version` stays conventional and `--version --output json|yaml|plain` stays structured.

{agent_first_data-0.17.3 → agent_first_data-0.19.0}/README.md

@@ -5,7 +5,7 @@ pip install agent-first-data
 ```
 
 ```python
-from agent_first_data import json_result, output_json, output_plain
+from agent_first_data import json_result, render, OutputFormat
 
 value = json_result(
     {
@@ -15,11 +15,11 @@ value = json_result(
     }
 ).build()
 
-print(output_json(value))
-print(output_plain(value))
+print(render(value, OutputFormat.JSON))
+print(render(value, OutputFormat.PLAIN))
 ```
 
-Useful names use Python casing: `output_json`, `output_yaml`, `output_plain` (each takes an optional keyword-only `options=`), `redacted_value`, `redact_secrets_in_place`, `redact_url_secrets`, `normalize_utc_offset`, `is_valid_rfc3339_date`, `is_valid_rfc3339_time`, `is_valid_rfc3339`, `is_valid_bcp47`, `cli_parse_output`, `cli_output`, `build_cli_error`, `build_cli_version`, `cli_handle_version_or_continue`, `decode_protocol_event`. Skill admin and stream-redirect helpers live in `agent_first_data.skill` and `agent_first_data.stream_redirect` (import the submodule directly).
+Useful names use Python casing: `render` (the single value x format x options -> str entry point; takes an `OutputFormat` and an optional keyword-only `options=`), `redacted_value`, `redact_url_secrets`, `normalize_utc_offset`, `is_valid_rfc3339_date`, `is_valid_rfc3339_time`, `is_valid_rfc3339`, `is_valid_bcp47`, `cli_parse_output`, `build_cli_error`, `build_cli_version`, `cli_handle_version_or_continue`, `decode_protocol_event`. Skill admin and stream-redirect helpers live in `agent_first_data.skill` and `agent_first_data.stream_redirect` (import the submodule directly).
 
 Logging is available through `init_logging_json`, `init_logging_plain`, `init_logging_yaml`, `span`, and `get_logger`.
 
@@ -33,7 +33,7 @@ init_logging_json("INFO", secret_names=("authorization",))
 
 - Default redaction replaces every `_secret` or configured secret-name subtree with `***`, including objects and arrays.
 - `_url` fields scrub userinfo passwords and secret-named query parameters; surrounding whitespace is trimmed and internal whitespace redacts the whole field.
-- YAML/plain quote and escape keys as well as values, sort by UTF-16 code unit order, and render nested objects in arrays as canonical JSON.
+- YAML keeps original keys and values (structure-preserving, like JSON), sorting keys by UTF-16 code unit order and quoting/escaping unsafe keys and string scalars. Plain strips formatting suffixes, formats values, sorts the same way, and renders nested objects/arrays as canonical JSON.
 - Logging records use `kind:"log"` with a nested `log` payload and a separate `level` field, so error-level logs are not terminal protocol errors.
 - `build_cli_error(message, hint?)` returns a strict-ready CLI error with `error.retryable:false` and `trace:{}`.
 - Use `cli_handle_version_or_continue()` before argument parsing so bare `--version` stays conventional and `--version --output json|yaml|plain` stays structured.

{agent_first_data-0.17.3 → agent_first_data-0.19.0}/agent_first_data/__init__.py

@@ -21,11 +21,8 @@ from agent_first_data.format import (
     DecodedLog,
     decode_protocol_event,
     RedactionPolicy,
-    OutputStyle,
+    PlainStyle,
     OutputOptions,
-    output_json,
-    output_yaml,
-    output_plain,
     redacted_value,
     redact_url_secrets,
     normalize_utc_offset,
@@ -37,11 +34,12 @@ from agent_first_data.format import (
 
 from agent_first_data.cli import (
     OutputFormat,
+    OutputTo,
     LogFilters,
     CliEmitter,
     cli_parse_output,
     cli_parse_log_filters,
-    cli_output,
+    render,
     build_cli_error,
     build_cli_version,
     cli_render_version,
@@ -69,11 +67,8 @@ __all__ = [
     "DecodedLog",
     "decode_protocol_event",
     "RedactionPolicy",
-    "OutputStyle",
+    "PlainStyle",
     "OutputOptions",
-    "output_json",
-    "output_yaml",
-    "output_plain",
     "redacted_value",
     "redact_url_secrets",
     "normalize_utc_offset",
@@ -82,11 +77,12 @@ __all__ = [
     "is_valid_rfc3339_date",
     "is_valid_rfc3339_time",
     "OutputFormat",
+    "OutputTo",
     "LogFilters",
     "CliEmitter",
     "cli_parse_output",
     "cli_parse_log_filters",
-    "cli_output",
+    "render",
     "build_cli_error",
     "build_cli_version",
     "cli_render_version",

agent_first_data-0.19.0/agent_first_data/assets/skills/agent-first-data/SKILL.md

@@ -0,0 +1,280 @@
+---
+name: agent-first-data
+description: Apply and review the formal Agent-First Data specification using the Rust/Go/Python/TypeScript libraries or afdata CLI for naming, redaction, formatting, protocol envelopes, logging, linting, validation, and reading or safely editing structured JSON/TOML/YAML/dotenv/INI documents (and the +++/--- frontmatter block of a Markdown page) by dot-path. Use proactively when writing or reviewing structured data, configs, logs, transport payloads, database/wire fields, CLI output, compatibility-sensitive public and persistent field names, or when reading or editing a config/document/Markdown-frontmatter file from the shell.
+---
+
+<!-- Canonical source: skills/agent-first-data/SKILL.md.
+     Mirrored byte-for-byte into go/python/typescript assets/ by scripts/sync_offline_assets.py.
+     Edit only here; run that script (checked by scripts/test.sh) to propagate. -->
+
+# Agent-First Data
+
+Use AFDATA when naming fields or reviewing structured output. The contract is naming-based: suffixes communicate units, formatting, and redaction semantics without relying on free-text inference.
+
+## Specification workflow
+
+Treat the formal specification as normative, not the skill summary or an
+example payload. In a repository checkout, read these spore-root-relative
+sources as needed:
+
+- `spec/agent-first-data.md` — complete cross-language naming, formatting,
+  redaction, protocol, logging, and CLI contract.
+- `spec/registry.json` — exact suffix metadata.
+- `spec/protocol-v1.schema.json` — protocol event schema.
+- `spec/fixtures/` — cross-language conformance cases.
+
+When only the installed skill is available, use the bundled equivalents:
+
+- `references/rules.md` — detailed suffix rules, output formatting, protocol templates, logging, CLI flags, and review checklist.
+- `references/registry.json` — machine-readable suffix registry; use this as the exact source for suffix metadata.
+- `references/protocol-v1.schema.json` — protocol event schema for `kind:"result"`, `kind:"error"`, `kind:"progress"`, and `kind:"log"` envelopes.
+
+If an example or README conflicts with the formal spec, follow the spec and
+report the discrepancy. Do not invent suffix meanings or reduce the protocol
+schema.
+
+## Library workflow
+
+Use the library at serialization boundaries so redaction and formatting are
+not reimplemented ad hoc. Rust applications can build a typed protocol event
+and render it in the required output format:
+
+```bash
+cargo add agent-first-data --no-default-features
+pip install agent-first-data
+npm install agent-first-data
+go get github.com/agentfirstkit/agent-first-data/go
+```
+
+```rust
+use agent_first_data::{json_result, render, OutputFormat, OutputOptions};
+use serde_json::json;
+
+let event = json_result(json!({
+    "latency_ms": 1280,
+    "api_key_secret": "sk-live-example",
+})).build();
+let rendered = render(event.as_value(), OutputFormat::Json, &OutputOptions::default());
+assert!(rendered.contains("\"api_key_secret\":\"***\""));
+```
+
+For a finite CLI execution implemented in Rust, create one `CliEmitter`, select
+`OutputFormat` through `cli_parse_output`, enable strict protocol, and emit one
+terminal result or error. Handle version before clap with
+`cli_handle_version_or_continue(raw_args, name, version)`.
+For non-CLI HTTP/MCP/SSE serialization, call `redacted_value()` or
+`redact_url_secrets()` explicitly at the boundary.
+
+The same library contract is available for Go, Python, and TypeScript. Use the
+runtime's native names while preserving the same builders, output formats,
+redaction behavior, registry, schema, and conformance fixtures; consult its
+root-level `go/README.md`, `python/README.md`, or `typescript/README.md` when
+working in that runtime.
+
+## Naming and redaction decisions
+
+- Add unit suffixes to ambiguous numeric fields: `_ms`, `_s`, `_bytes`, `_percent`, `_sats`, `_msats`, currency suffixes, and timestamp suffixes.
+- Use `_secret` for values/subtrees that must redact to `***` in every output format.
+- Use `_url` only for whole URL values whose userinfo password or suffix-named secret query params should be scrubbed.
+- Do not scan arbitrary free text for secrets. Rename fields or configure explicit secret names at the serialization boundary.
+- Before changing public API, wire, database, or persistent field names, report the compatibility impact and get approval.
+
+## Evaluation rules
+
+- Trigger this skill for field naming, structured output, configs, logs, CLI output, protocol events, MCP/HTTP/SSE payloads, database columns, wire fields, or persisted JSON. Do not wait for the user to say "AFDATA" when the work changes those surfaces.
+- In a repository checkout, treat `spec/agent-first-data.md`,
+  `spec/registry.json`, and `spec/protocol-v1.schema.json` as authoritative; in
+  an installed skill, use their bundled `references/` equivalents. Do not
+  replace them with invented suffix meanings, a reduced schema, or free-text
+  interpretation.
+- For secrets, fix the field name or serializer configuration. Use `_secret` or configured exact secret names; do not rely on scanning arbitrary strings or object debug output.
+- Run `afdata lint` on JSON/JSONL examples, JSON Schema, MCP schemas, or serialized samples when the CLI is available. Run `afdata validate` on finite protocol events/streams. If a check cannot run, report that explicitly.
+- Before modifying an external project's public API, wire format, database schema, or persisted field names, stop and report the compatibility impact; request approval before applying the change.
+
+## Redaction behavior contract
+
+Fields named `_secret` or `_url` and values passing through AFDATA output rendering (`render`, in any format) are automatically redacted: secrets become `***`, URL userinfo passwords and suffix-named query parameters are scrubbed. When serializing outside output formatters (HTTP, MCP, SSE), call `redacted_value()` or `redact_url_secrets()` at the serialization boundary. PII redaction, domain-specific privacy policies (header allowlists, API scope sensitivity), and per-field secret naming are each spore's responsibility; the library provides field-name-based mechanics only.
+
+## Logging behavior contract
+
+One-time CLI output uses `json_log()` or `CliEmitter` for a single event; serialization via `render()` applies output formatting. Construct the emitter in finite mode for a one-shot command (`result`→stdout, `error`/`progress`/`log`→stderr) or stream mode for an interleaved event stream (all events on one destination) — see the Output stream contract. Long-running services or processes depending on structured logging (tonic, sqlx, hyper, etc. via tracing) initialize with `afdata_tracing::try_init()` (Rust only) to wire process-wide logging; other languages emit log events explicitly via builders or integrate their own structured logging.
+
+## CLI workflow
+
+Install `afdata` when a shell workflow needs validation, formatting, or skill
+administration without embedding a library:
+
+```bash
+cargo install agent-first-data
+```
+
+Use it for deterministic checks:
+
+```bash
+afdata lint payload.json
+afdata validate events.jsonl
+afdata render payload.json --output yaml
+afdata skill validate skills/example-skill
+```
+
+- Every command's first positional is its input; `-` reads stdin explicitly.
+  There is no implicit stdin fallback, and omitting the input is always a
+  usage error that names the fix — never a hang.
+- Run `afdata lint` for ordinary JSON/JSONL, JSON Schema, MCP input/output
+  schemas, or a structured document (`--input-format`/extension detection
+  picks TOML/YAML/dotenv/INI; JSON/JSONL stays the default dual-mode input).
+- Run `afdata validate` for AFDATA protocol events or finite event streams.
+- Run `afdata render` to apply AFDATA redaction/formatting to arbitrary JSON
+  or JSONL. `--output` accepts `json`, `yaml`, or `plain`; JSON and YAML keep
+  original keys/values (structure-preserving), Plain is the lossy human
+  renderer.
+- Treat findings as contract issues unless the owning tool intentionally
+  documents a non-AFDATA field.
+
+## Output stream contract
+
+AFDATA CLIs route protocol events by consumption mode, not by event shape.
+
+- A **finite one-shot command** (the default) puts `kind:"result"` on `stdout`
+  and `kind:"error"`/`progress`/`log` on `stderr`, so `x=$(tool …)` never
+  captures a failure as data, `tool … >/dev/null` never swallows a diagnostic,
+  and `tool … | next` never pipes an error envelope in as input. The exit
+  code — not the stream — is set by the terminal kind; routing follows `kind`,
+  not the exit code.
+- An **event stream** (interleaved events consumed in order) keeps every event,
+  including `error`, on one stream so ordering survives.
+- `afdata` and any CLI built on the library expose `--output-to
+  <split|stdout|stderr>` (default `split`). `stdout`/`stderr` collapse the whole
+  stream — errors included — onto one destination, for a consumer that would
+  rather read one stream and branch on `kind`. `--output` picks the format,
+  `--output-to` picks the destination. Send an event stream to a file by
+  collapsing it (`--output-to stdout`) and redirecting (`--stdout-file PATH`).
+- The raw-scalar readers (`value`/`paths`/`keys`) are intrinsically split (raw
+  data on `stdout`, error envelope on `stderr`) and reject a non-default
+  `--output-to`.
+- When building a CLI on the library, choose the mode by how the output is
+  consumed — do not reflexively pick finite. A plain one-shot shell command uses
+  **finite** mode (errors → stderr, so shell capture and pipelines stay safe). A
+  tool whose output is itself an event stream (interleaved `progress`/`log` then
+  a terminal `result`/`error` — e.g. streaming query rows), or that is
+  agent-facing and wants the caller to read a single structured channel and
+  branch on `kind`, uses **stream/unified** mode (every event, errors included,
+  on `stdout`). A stream's terminal error MUST stay on the stream, so such a tool
+  keeps even its usage errors on `stdout` for one consistent channel. Never write
+  to `stderr` ad hoc — route every diagnostic through the emitter so redaction
+  and framing still apply.
+- Emit a terminal event with the emitter's `finish` / `finish_result` helpers,
+  which map the write outcome to a broken-pipe-safe process exit code (success →
+  the given code, a hung-up reader → 0, any other write failure → a nonzero
+  output-failure code) instead of hand-rolling that dance — they work in either
+  mode (the emitter's mode decides the stream; `finish` only maps the outcome).
+  Construct an error with the error builder (`json_error(code, message)` with
+  `hint`/`retryable`/extra fields as needed) — that builder *is* the error type —
+  and pass the built event to `finish`; there is no separate error-emitting
+  convenience.
+
+## Document workflow
+
+Use `get`/`value`/`paths`/`keys` to read a JSON, TOML, YAML, dotenv, or INI
+document and `set`/`add`/`remove`/`unset` to edit one in place, instead of
+`sed`, regular expressions, or a generic reserializer — those can reorder
+keys, drop comments, or change quoting/anchors in the rest of the file. Every
+document command's first positional is `FILE` (or `-` for stdin on a read
+command); mutation commands never read stdin and reject `-` as a usage error.
+To edit only the metadata block of a Markdown page, pass `--input-format
+toml-frontmatter` (a `+++` block) or `yaml-frontmatter` (a `---` block): the
+same read/mutate commands then address dot-paths inside the frontmatter and
+leave the Markdown body byte-for-byte untouched. Frontmatter is never
+auto-detected — the format must be named explicitly, since extension detection
+only ever resolves to a whole-file format — so reach for it deliberately
+instead of `sed`-ing a page's `+++`/`---` header.
+
+- `get FILE` returns the whole document as one AFDATA record
+  (`code:"document"`); `get FILE KEY` narrows it to one dot-path value in the
+  same shape. Pass every known sensitive legacy field name with
+  `--secret-name FIELD` (an exact field name, not a dot-path). A secret-named
+  leaf (`_secret`/`_SECRET`, or an exact `--secret-name` match) is still
+  redacted to `"***"` even when directly targeted — use `value
+  --reveal-secret` when the real value is genuinely needed.
+- `value FILE KEY` writes only the raw scalar to stdout, with no AFDATA
+  envelope and no forced trailing newline, for direct use in shell
+  substitution: `value=$(afdata value config.toml server.host)`. Only scalars
+  are supported; a container errors (`document_not_scalar`) — use `get` for a
+  subtree. A secret-named leaf errors (`document_secret_redacted`) unless
+  `--reveal-secret` is passed. On any failure `value`'s stdout is always
+  empty — the error envelope goes to stderr instead — so
+  `port=$(afdata value config.toml server.port)` never captures a JSON error
+  as data. `--default VAL` prints `VAL` instead of erroring when the path is
+  absent or the value is `null` (an empty string is a real value and does not
+  trigger it), collapsing the common "read or fall back" shell idiom into one
+  call with no `2>/dev/null` guard needed.
+- `paths FILE [KEY]` and `keys FILE [KEY]` enumerate a container's immediate
+  children, one per line, with the same empty-stdout-on-failure contract as
+  `value`. **`paths` output feeds back into afdata** (full grammar-escaped
+  dot-paths from the root — pipe straight into `value`/`get`/`unset` or
+  extend with `"$p.field"`); **`keys` output goes to external tools** (raw,
+  unescaped key names/array indices — what a package manager or another CLI
+  expects). Never use `paths`'s output as a literal name, or `keys`'s output
+  as a dot-path — they only diverge on a key containing a dot or space, so
+  the wrong one works until it silently doesn't. A scalar target errors
+  (`document_not_container`); `--missing-ok` turns a missing KEY specifically
+  into empty output + exit 0 (other errors still fail). Path grammar does not
+  escape spaces or other IFS characters, so the only correct consumption
+  shape is a `while IFS= read -r` loop, never `for x in $(...)`:
+
+  ```sh
+  while IFS= read -r p; do
+      afdata value config.toml "$p.slug"
+  done < <(afdata paths config.toml extra.tools)
+  ```
+- Escape a literal dot as `\.` and a literal backslash as `\\` in a dot-path;
+  every command shares this grammar (`paths`'s escaped output re-parses
+  through the exact same rules it was emitted from), so an unrecognized
+  escape or empty segment is always an error, never a guess.
+- `add`/`remove` operate on a keyed list — an array of objects addressed by a
+  slug — and always require an explicit `--slug-field FIELD`; the CLI never
+  infers an identity field. Keyed editing is implemented for JSON and YAML
+  only; TOML, dotenv, and INI return a structured "not implemented" error, so
+  edit their list entries by writing the whole value another way. Adding a
+  slug that already exists, or removing/unsetting one that does not, is
+  always an error (`document_slug_exists`/`document_slug_not_found`/
+  `document_path_not_found`) — never a silent no-op; wrap the call yourself
+  when idempotence is wanted.
+- `set`'s bare VALUE (and `add`'s `FIELD=VALUE`) is always a plain string —
+  no shape-guessing, so a leading-zero ID or SHA never gets reinterpreted as
+  a number. Writing an exact type goes through `--value-type
+  string|number|bool|null|json` instead; `json` is the only entry point for
+  an array or object. Overwriting an *existing* scalar of a different kind
+  with a bare VALUE is a usage error that names both escape hatches (keep the
+  type with `--value-type <kind>`, or convert explicitly with `--value-type
+  string`) — a brand-new key never needs `--value-type`.
+- For `set`, choose `--secret-from stdin|prompt|fd:<N>|env:<VAR>` instead of a
+  positional value when the value must not land in shell history or `ps`:
+  `stdin` (pipe), `prompt` (human, terminal echo off), `fd:N` (Unix
+  automation that needs stdin for something else), or `env:VAR` (already in
+  the process environment). There is no inline `--secret-from=VALUE`-style
+  argv source — that would defeat the point.
+- Mutations preserve the rest of the source document — comments, key order,
+  unrelated formatting — and refuse to write through a symlink or (on Unix) a
+  multi-link hardlink target. Treat a failed mutation as leaving the original
+  file untouched rather than assuming a partial write, and reread before
+  retrying. A format backend that isn't compiled in, or input outside the
+  documented dotenv/INI dialect, is a structured error to report, not a cue to
+  hand-edit the file. A successful mutation's result carries the `path`
+  actually written, for confirming which file changed when scripting several.
+- Branch on `error.code`, not the message text: document errors use stable
+  codes (`document_path_not_found`, `document_type_mismatch`,
+  `document_slug_not_found`, `document_slug_exists`, `document_parse_failed`,
+  `document_io_failed`, and a few narrower ones), while a malformed
+  invocation (`--input-format`, `--value-type`, `FIELD=VALUE`, a mutation
+  given `-`, …) is always `document_usage_error` at exit 2 — distinct from
+  every runtime document error, which exits 1.
+
+## Review checklist
+
+1. Numeric fields with units carry the right suffix and safe integer/string representation.
+2. Timestamps use `_epoch_s`, `_epoch_ms`, decimal-string `_epoch_ns`, or `_rfc3339`.
+3. Secrets use `_secret` or configured exact secret names; no fallback path leaks them.
+4. CLI output uses AFDATA helpers for JSON/YAML/plain formatting and structured errors, and routes by consumption mode (finite: `result`→stdout, `error`→stderr; stream: one destination) rather than writing to stderr ad hoc.
+5. Logs use AFDATA structured logging or equivalent field-name redaction.
+6. Transport payloads preserve the protocol envelope when AFDATA compliance is claimed.

{agent_first_data-0.17.3 → agent_first_data-0.19.0}/agent_first_data/assets/skills/agent-first-data/references/rules.md

@@ -41,6 +41,8 @@ Integer vs string is not uniform across siblings: `_epoch_s` and `_epoch_ms` are
 
 `_rfc3339` needs a **mandatory** offset: `YYYY-MM-DDThh:mm:ss[.fff]` followed by `Z` or `±HH:MM`. A bare `2026-02-14T10:30:00` (no offset), a space instead of `T`, or a trailing IANA name (`...Asia/Shanghai`) is rejected by `afdata lint`. `afdata lint` also type-checks the numeric suffixes (durations `_s`/`_ms`/…, currency `_cents`/`_micro`/`_jpy`) and rejects a `_url` with internal whitespace or bare `user:pass@host` credentials.
 
+A `null` value is exempt from every suffix type constraint above: `null` means the field is absent/unset, not present-with-the-wrong-type, so `law_repeal_at_epoch_s: null` is not a `suffix_type_mismatch`. Absence may be written as an omitted key or as an explicit `null` — both are valid. The constraint applies only once a value is present and non-null.
+
 ### Strict string formats
 
 | Suffix | Format | Example |
@@ -107,7 +109,7 @@ Sub-cent precision — `_{code}_micro`, integer millionths (10⁻⁶) of the maj
 | `_secret` | redact the entire value/subtree to `***` | `api_key_secret: "sk-or-v1-abc..."` |
 | `_url` | scrub secrets *inside* the URL value, keep the rest | `callback_url: "https://h/cb?code_secret=..."` |
 
-All CLI output formats (JSON, YAML, Plain) automatically redact `_secret` fields. Matching recognizes `_secret` and `_SECRET` only — no mixed case. The entire `_secret` value/subtree becomes `***`, including objects and arrays. For legacy fields that cannot be renamed, configure `OutputOptions.redaction` with `secret_names`/`secretNames` such as `["api_key", "authorization"]`; names match exact field names at any nesting level; no trim, case folding, hyphen/underscore normalization, globs, regex, or substring matching. AFDATA does not define named redaction profiles; use the default, `secret_names`, `RedactionTraceOnly`, or `RedactionNone` deliberately at the serialization boundary. Callers that need schema-preserving YAML/plain rendering can pass `OutputOptions` with the `Raw` output style.
+All CLI output formats (JSON, YAML, Plain) automatically redact `_secret` fields. Matching recognizes `_secret` and `_SECRET` only — no mixed case. The entire `_secret` value/subtree becomes `***`, including objects and arrays. For legacy fields that cannot be renamed, configure `OutputOptions.redaction` with `secret_names`/`secretNames` such as `["api_key", "authorization"]`; names match exact field names at any nesting level; no trim, case folding, hyphen/underscore normalization, globs, regex, or substring matching. AFDATA does not define named redaction profiles; use the default policy `All`, `secret_names`, `TraceOnly`, or `Off` deliberately at the serialization boundary. YAML is always schema-preserving, like JSON. Callers that need schema-preserving Plain rendering can pass `OutputOptions` with the `Raw` output style.
 
 `***` means only one thing: AFDATA redacted a sensitive value. Do not use it for serialization failure, truncation, unsupported types, or “maybe secret” guesses.
 
@@ -158,15 +160,17 @@ ORM struct fields preserve the suffix: `duration_ms: i64`, not `duration: i64`.
 
 ## Part 2: Output Processing
 
-Three output formats via `output_json()`, `output_yaml()`, `output_plain()`. Default YAML and Plain apply key stripping + value formatting. Optional `options` (Python/TS keyword arg; Rust/Go `_with_options` variant).
+Three output formats via one entry point `render(value, format, options)` (`format` = json/yaml/plain). JSON and YAML are both **structure-preserving**: original keys, scalar types, and numeric semantics survive after redaction. Plain is the one **lossy/human** renderer — it strips formatting suffixes and reformats values for scanning. `options` is an optional keyword arg in Python/TS; Rust/Go pass `OutputOptions` explicitly.
 
 ### Formats
 
 - **JSON** — single-line, original keys, raw values, no sorting (machine-readable), secrets redacted
-- **YAML** — multi-line, formatting suffixes stripped, values formatted, secrets redacted by default
+- **YAML** — multi-line, original keys, raw values (same semantics as JSON), keys sorted, secrets redacted by default
 - **Plain** — single-line logfmt, formatting suffixes stripped, values formatted, secrets redacted by default
 
-### Key stripping (YAML and Plain)
+Rust gates YAML output behind the `yaml` Cargo feature (on by default, along with the rest of the default feature set). With `default-features = false` and `yaml` not re-enabled, `OutputFormat::Yaml` does not exist and `--output yaml` is rejected with a feature-requirement error instead of silently falling back to another format.
+
+### Key stripping (Plain only)
 
 Remove recognized formatting suffix from key. Longest match first, exact lowercase or uppercase only:
 
@@ -176,9 +180,9 @@ Remove recognized formatting suffix from key. Longest match first, exact lowerca
 4. `_msats`, `_sats`, `_bytes`, `_percent`, `_secret`
 5. `_jpy`, `_ns`, `_us`, `_ms`, `_s`
 
-`_bcp47`, `_utc_offset`, `_rfc3339_date`, and `_rfc3339_time` are NOT stripped (pass through). If two keys collide after stripping, both revert to original key AND raw value (no formatting). Redaction runs before collision handling, so fallback never restores a secret.
+`_bcp47`, `_utc_offset`, `_rfc3339_date`, and `_rfc3339_time` are NOT stripped (pass through). If two keys collide after stripping, both revert to original key AND raw value (no formatting). Redaction runs before collision handling, so fallback never restores a secret. JSON and YAML never strip suffixes — every key renders exactly as written.
 
-### Value formatting (YAML and Plain)
+### Value formatting (Plain only)
 
 - `_ms` with absolute value < 1000 → `{n}ms`; absolute value ≥ 1000 → seconds (`1280` → `1.28s`, `-1500` → `-1.5s`)
 - `_s`, `_ns`, `_us` → append unit (`3600s`, `450000ns`, `830μs`)
@@ -192,7 +196,7 @@ Remove recognized formatting suffix from key. Longest match first, exact lowerca
 - `_secret` → `***` (the redaction phase already replaced the subtree)
 - `_bcp47`, `_utc_offset`, `_rfc3339_date`, `_rfc3339_time` → pass through unchanged
 
-**Type constraints**: `_bytes`/`_epoch_*` require integer. `_usd_cents`/`_eur_cents`/`_jpy`/`_{code}_cents`/`_{code}_micro` require non-negative integer. Duration/Bitcoin/`_percent` accept any number. Wrong type → raw value + original key.
+**Type constraints**: `_bytes`/`_epoch_*` require integer. `_usd_cents`/`_eur_cents`/`_jpy`/`_{code}_cents`/`_{code}_micro` require non-negative integer. Duration/Bitcoin/`_percent` accept any number. Wrong type → raw value + original key. JSON and YAML never format values — a value renders exactly as its JSON type (numbers stay numbers, strings stay quoted strings, no unit/date/currency formatting).
 
 ### Plain logfmt details
 
@@ -204,7 +208,11 @@ Remove recognized formatting suffix from key. Longest match first, exact lowerca
 
 ### Key ordering
 
-YAML and Plain sort keys (after stripping) by UTF-16 code unit order (JCS, RFC 8785). For ASCII keys this equals byte-order sorting.
+YAML sorts keys by UTF-16 code unit order (JCS, RFC 8785) without stripping suffixes. Plain sorts keys the same way, but after stripping. For ASCII keys this equals byte-order sorting.
+
+### `PlainStyle` (Plain only)
+
+`PlainStyle` (`Readable`/`Raw`) only affects Plain: `Readable` (default) strips suffixes and formats values; `Raw` keeps original keys/values (still redacted). YAML ignores `PlainStyle` entirely — it is always structure-preserving, matching JSON.
 
 ---
 
@@ -219,22 +227,24 @@ Every protocol event uses `kind` plus one same-named payload:
 | `"result"` | Terminal success |
 | `"error"` | Terminal error with non-empty `error.code` and `error.message` |
 
-Channel policy:
-- `stdout` is the only protocol/log stream for machine-readable events
-- runtime protocol events MUST NOT be emitted on `stderr`
-- `stderr` is reserved for unrecoverable pre-protocol startup failures only
+Channel policy — the stream follows the consumption mode, not the event shape:
+- **Finite one-shot command (default):** `result` → `stdout`; `error`/`progress`/`log` → `stderr`. Routing follows `kind`, not the exit code, so `stdout` carries only successful payloads (`x=$(tool …)`, `tool | next`, and `tool >/dev/null` all stay safe).
+- **Event stream (interleaved events consumed in order):** every event, including `error`, stays on one stream so ordering survives; do not split it across `stdout`/`stderr`.
+- `--output-to <split|stdout|stderr>` (default `split`) selects the mode: `stdout`/`stderr` collapse the whole stream onto one destination. `--output` selects format, `--output-to` selects destination.
+- Raw-scalar readers (`value`/`paths`/`keys`) are intrinsically split and reject a non-default `--output-to`.
+- Choose the mode by how the output is consumed, not reflexively: a plain one-shot shell command uses **finite**; a tool whose output is itself an event stream (streaming rows, interleaved progress/log) or that is agent-facing and wants one structured channel uses **stream/unified** (all on `stdout`, errors included — a stream's terminal error must stay on the stream). When building a CLI on the library, emit terminal events with the emitter's `finish`/`finish_result` helpers (success → the given code, broken pipe → 0, other write failure → a nonzero output-failure code); build errors with the `json_error` builder (the error *type*, carrying hint/retryable/fields) and pass the event to `finish` — the helpers work in either mode.
 
 Optional stream redirection:
-- use `--stdout-file <PATH>` to redirect stdout bytes to a file
-- use `--stderr-file <PATH>` to redirect stderr bytes to a file
-- keep `--output` reserved for stdout format selection (`json`, `yaml`, `plain`, and help-specific `markdown`)
-- do not convert stderr diagnostics to JSON; Rust panics, Python tracebacks, and runtime errors stay native stderr bytes
+- use `--stdout-file <PATH>` / `--stderr-file <PATH>` to redirect a stream's bytes to a file (process-level, beneath the channel routing above)
+- send an event stream to a file by collapsing it (`--output-to stdout`) and redirecting (`--stdout-file <PATH>`); there is no separate events-file flag
+- native panics/tracebacks stay raw `stderr` bytes; in finite mode `stderr` also carries the formatted AFDATA `error`/`progress`/`log` envelopes, so a reader parses JSON lines and tolerates interleaved native bytes
 - treat this as stream destination control, not a second protocol stream and not stream copying
 - do not implement application-level rotation
 
 Recommended enforcement:
-- Rust: enable clippy `print_stderr = "deny"` and disallow `std::eprintln` / `std::io::stderr`
-- Go/Python/TypeScript: add source-policy tests that fail if runtime sources reference stderr APIs
+- route every `stderr` write through the emitter's diagnostic/error path; no ad-hoc stderr that bypasses the formatter in runtime code
+- Rust: keep native panic output as the only unstructured `stderr` (no stray `eprintln!` / `std::io::stderr`)
+- Go/Python/TypeScript: source-policy tests that fail on ad-hoc stderr APIs in runtime code, with the emitter's own sink as the sanctioned exception
 
 ### Templates
 
@@ -257,7 +267,7 @@ Startup payload fields are tool-defined; `config` is recommended, while `version
 | MCP tool | JSON |
 | SSE stream | JSONL |
 
-All use `kind` plus the same-named payload and optional `trace`. Do not split protocol events across `stdout` and `stderr`.
+All use `kind` plus the same-named payload and optional `trace`. An event stream keeps its events on one stream (never split); a finite one-shot command splits by `kind` (`result` → stdout, `error` → stderr) — see Channel policy.
 
 Base validators enforce mandatory envelope/error/lifecycle rules. Strict validators additionally require object-valued `trace` on every event and `retryable` on every error. Log and progress payloads remain tool-defined in both profiles. Use strict validation before claiming compliance with all recommendations.
 
@@ -269,17 +279,17 @@ Use the local language README for installation and the full overview/spec for AP
 
 Required cross-language behavior to rely on:
 
-- Output helpers redact before formatting: `output_json()`, `output_yaml()`, `output_plain()` (with optional `options` in Python/TS).
-- Use `redacted_value()` for raw HTTP/MCP/SSE serialization paths that bypass `output_json()`.
+- The single output entry point `render(value, format, options)` redacts before formatting (with optional `options` in Python/TS).
+- Use `redacted_value()` for raw HTTP/MCP/SSE serialization paths that bypass `render()`.
 - Use `redact_url_secrets()` for URLs embedded in log messages or prose before interpolating them into output.
-- Use `cli_parse_output()`, `cli_parse_log_filters()`, `cli_output()`, `build_cli_error()`, and the version helper for CLI tools instead of custom parsing/error envelopes.
+- Use `cli_parse_output()`, `cli_parse_log_filters()`, `render()`, `build_cli_error()`, and the version helper for CLI tools instead of custom parsing/error envelopes.
 - `build_cli_error(message, hint?)` returns a strict-ready CLI error with `error.retryable:false` and `trace:{}`.
-- Rust CLIs should call `cli_handle_version_or_continue()` before clap parsing so `--version --output json|yaml|plain` emits a structured `kind:"result"` event with `result.version` instead of clap's plain text. Use `VersionConfig::conventional_default()` so bare `--version` stays human text while explicit `--output` remains structured.
+- Rust CLIs should call `cli_handle_version_or_continue(raw_args, name, version)` before clap parsing so `--version --output json|yaml|plain` emits a structured `kind:"result"` event with `result.code == "version"` and `result.version` instead of clap's plain text. Bare `--version` always stays human `<name> <version>` text; explicit `--output` is always structured — there is no configuration.
 - Use the protocol reader `decode_protocol_event(text)` to parse and strict-validate a single JSON text line, receiving a typed decoded event.
 
 ## AFDATA Logging
 
-Long-running services and processes depending on structured logging (tonic, sqlx, hyper, etc. via tracing) use a logging init function to capture the full process. One-time CLI output uses `json_log()` + `cli_output()` or `CliEmitter`.
+Long-running services and processes depending on structured logging (tonic, sqlx, hyper, etc. via tracing) use a logging init function to capture the full process. One-time CLI output uses `json_log()` + `render()` or `CliEmitter`.
 
 ### Init (Rust only; pick one format per process)
 
@@ -312,7 +322,7 @@ Log redaction is **by field name** (the same `_secret`/`_url` rule as all output
 CLI tools that use AFDATA should support output and logging flags:
 
 ```
---output json|yaml|plain    # default is tool-defined (interactive → yaml, scripting/logging → json)
+--output json|yaml|plain    # default is tool-defined (structured/scripting → json or yaml, human scanning → plain)
 --log startup,request,progress,retry,redirect
 --verbose                   # shorthand for all log categories
 ```
@@ -345,4 +355,4 @@ Use this checklist when the skill is evaluating a change, not just documenting c
 7. Environment variables follow `UPPER_SNAKE_CASE` with the same suffixes
 8. One-time CLI output uses `json_log()`/`CliEmitter` + output helpers; long-running services use `afdata_tracing::try_init()` (Rust) to capture via tracing; other languages emit log events explicitly via builders
 9. Database columns use AFDATA suffixes on generic types (`duration_ms INTEGER`, not `duration INTEGER`); native types like `TIMESTAMPTZ` don't need suffixes
-10. CLI flag parsing uses `cli_parse_output()`/`cli_parse_log_filters()`/`build_cli_error()`/version helpers — not custom reimplementations; uses `try_parse()` not `parse()` in Rust so clap errors go to stdout as JSONL
+10. CLI flag parsing uses `cli_parse_output()`/`cli_parse_log_filters()`/`build_cli_error()`/version helpers — not custom reimplementations; uses `try_parse()` not `parse()` in Rust so a clap/usage error becomes a structured `kind:"error"` envelope (routed to stderr under the default split, exit 2), never raw clap text