@xberg-io/opencode-xberg 1.3.7 → 1.3.11

package/.pi/skills/xberg-picking-a-format/SKILL.md

@@ -0,0 +1,104 @@
+---
+description: Use when choosing an output format for extracted documents — plain text, markdown, djot, HTML, JSON, or DocTags. Maps consumer (LLM, parser, archive) to the right `--format` / `--content-format` pair.
+name: xberg-picking-a-format
+---
+
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:a2f204c34c2eb303812f0b7b0a9acf29c879edd1985b8cc3312555b347378e1e
+Source-Hash: blake3:662aa704cf97ad20f02258c9dcd1c73822213a426e27ec7ebc292d3f599fb7c2
+Schema-Version: v1
+-->
+
+# Picking a format
+
+Xberg has two orthogonal format knobs. Get them right up front and the
+downstream code stays simple.
+
+| Knob                | What it controls                                  | Values                                 | Default          |
+| ------------------- | ------------------------------------------------- | -------------------------------------- | ---------------- |
+| `--format`          | How the CLI prints the result                     | `text`, `json`, `toon`                 | `text` (`extract`), `json` (`batch`) |
+| `--content-format`  | How extracted content is rendered inside `result` | `plain`, `markdown`, `djot`, `html`, `json`, `doctags` | `plain`     |
+| `--token-reduction` | Strip whitespace / boilerplate for LLM contexts   | `off`, `light`, `moderate`, `aggressive`, `maximum` | `off`  |
+
+`--format json` returns an envelope wrapping the `ExtractedDocument` — the
+document lives under `.result` for `extract` and under `.results[]` for
+`batch`, with `content`, `metadata`, `tables`, and `images` as fields of
+that nested document. `--format text` prints just `content`.
+`--content-format` is what shows up inside that `content` field.
+
+## Decision tree
+
+```text
+Who consumes the output?
+├── LLM (Claude, GPT, Gemini, local) — embed/prompt context
+│       --format text --content-format markdown
+├── Vector store / RAG indexer
+│       --format json --content-format markdown
+│       (markdown preserves structure for chunking)
+├── Downstream parser that expects machine-readable JSON
+│       --format json --content-format plain
+│       (cleanest text + structured metadata)
+├── Human review / archival
+│       --format text --content-format markdown
+├── HTML re-rendering / web display
+│       --format json --content-format html
+├── Lossless intermediate for pandoc / academic tooling
+│       --format json --content-format djot
+└── Token-budget-constrained pipeline
+        --format text --content-format plain
+        (drops markup; add --token-reduction moderate for further savings)
+```
+
+## Examples
+
+Feed a PDF directly into an LLM:
+
+```bash
+xberg extract paper.pdf --content-format markdown
+```
+
+Index a corpus into a RAG store with tables and headings preserved:
+
+```bash
+xberg batch docs/*.pdf --format json --content-format markdown \
+  | jq -c '.result | {content: .content, tables: .tables}'
+```
+
+Strip a file to bare text for a token-tight summarizer:
+
+```bash
+xberg extract long.pdf \
+  --content-format plain \
+  --token-reduction moderate
+```
+
+Pull metadata only, ignore content:
+
+```bash
+xberg extract file.pdf --format json | jq '.result.metadata'
+```
+
+## When in doubt
+
+- **Default to `markdown`** as the content format. It is the best
+  compromise across LLMs, RAG, and human review, and Xberg has the
+  most faithful renderer for it.
+- Reach for `plain` only when downstream cannot tolerate any markup.
+- Reach for `djot` only if you're already in a djot/pandoc pipeline.
+- Reach for `html` only when re-rendering for the web.
+- Reach for `json` for a heading-driven content tree, or `doctags` for Docling-compatible output.
+
+## Token-reduction (orthogonal)
+
+`--token-reduction` collapses whitespace, strips repeated headers/footers,
+and trims boilerplate. It composes with any `--content-format`:
+
+- `off` (default), `light`, `moderate`, `aggressive`, `maximum`.
+
+Use `moderate` as a safe starting point for LLM context windows. `maximum`
+is lossy — verify before relying on it.
+
+See `references/cli-reference.md` for the full flag set and
+`references/configuration.md` for the equivalent `output_format` and
+`token_reduction` keys in `xberg.toml`.

package/.pi/skills/xberg-picking-a-format/index.md

@@ -0,0 +1,10 @@
+<!--
+AI-RULEZ :: GENERATED FILE — DO NOT EDIT
+Content-Hash: blake3:d81bf6504823199be6d747c4b5e058732f6d6e5ee6f97aae83374eb2f0982d9a
+Source-Hash: blake3:662aa704cf97ad20f02258c9dcd1c73822213a426e27ec7ebc292d3f599fb7c2
+Schema-Version: v1
+-->
+
+# Concepts
+
+* [Picking A Format](SKILL.md) - Use when choosing an output format for extracted documents — plain text, markdown, djot, HTML, JSON, or DocTags. Maps consumer (LLM, parser, archive) to the right `--format` / `--content-format` pair.