@dboio/cli 0.20.3 → 0.20.4

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dboio/cli",
-  "version": "0.20.3",
+  "version": "0.20.4",
   "description": "CLI for the DBO.io framework",
   "type": "module",
   "bin": {

package/plugins/claude/dbo/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "dbo",
-  "version": "0.8.3",
+  "version": "0.8.4",
   "description": "DBO.io CLI integration for Claude Code",
   "author": {
     "name": "DBO.io"

package/plugins/claude/dbo/docs/dbo-cli-readme.md

@@ -64,6 +64,204 @@ Once installed, use `/dbo` in Claude Code:
 
 The plugin source lives in `plugins/claude/dbo/` at the repository root. Installed copies in `.claude/plugins/` are gitignored and managed by `dbo install`. Each plugin's installation scope (project or global) is stored in `~/.dbo/settings.json`.
 
+### "Add / Create `<file>` **" — decision tree
+
+When a user says **"add `<file>` "**, **"create `<file>`"**, **"push `<file>`"**, or **"upload `<file>`"**, use this decision tree to determine the correct command sequence. The words **add** and **create** are treated identically here — both refer to the CREATE step of CRUD.
+
+#### Step 1 — Find the companion metadata file
+
+Companion `.metadata.json` files are usually co-located with the content file, but there is one notable exception:
+
+| Content file | Where to look for metadata |
+|---|---|
+| `lib/bins/**/*` | Same directory as the file |
+| `lib/extension/**/*` | Same directory as the file |
+| `docs/*.md` | `lib/extension/documentation/<filename>.metadata.json` (when `ExtensionDocumentationMDPlacement` = `"root"`) |
+| Root-level files listed in `.app/config.json` `rootContentFiles` | Default bin directory (e.g., `lib/bins/app/<filename>.metadata.json`); the metadata's `@reference` uses `@/<filename>` to point back to the root |
+| Any other file | Same directory first; if not found, search for `"@<filename>"` or `"@/<filename>"` across all `.metadata.json` files — a `@reference` entry pointing to the file identifies its companion |
+
+> **Edge cases where metadata is not co-located with the file:**
+> - `docs/Chat.md` → metadata in `lib/extension/documentation/Chat.metadata.json`
+> - `CLAUDE.md` (root) → metadata in `lib/bins/app/CLAUDE.metadata.json` (or `CLAUDE.md.metadata.json` for newly-adopted files)
+>
+> When you can't find a `.metadata.json` beside a file, grep for `"@<filename>"` or `"@/<filename>"` across the project before concluding the file is untracked.
+
+#### Step 2 — Check server presence
+
+```
+Does <file>.metadata.json exist?
+│
+├─ YES → Does it contain "_CreatedOn" or "_LastUpdated"?
+│         │
+│         ├─ YES → Record exists on server
+│         │         → dbo push <file> [--ticket <id>]
+│         │
+│         └─ NO  → Metadata was adopted locally but never pushed (no UID yet)
+│                   → dbo push <file> [--ticket <id>]
+│                     (push detects missing UID and routes to insert)
+│
+└─ NO  → Does the local file exist?
+          │
+          ├─ YES → File is untracked — create metadata first
+          │         → dbo adopt <file> -e <entity>
+          │         → dbo push <file> [--ticket <id>]
+          │
+          └─ NO  → File doesn't exist locally — create it on the server first
+                    → POST /api/input/submit   (create record via REST API)
+                    → dbo pull                 (sync new record to local)
+```
+
+**User says "add" or "create" with no file specified:**
+→ Use REST API `/api/input/submit` to create the record on the server, then `dbo pull` to sync locally.
+
+#### Ticket ID
+
+Include `--ticket <id>` when the project has `RepositoryIntegrationID` configured (ticketing integration is active) and a ticket ID is known for the change. If omitted, the CLI will prompt interactively when the server requires one. A stored ticket in `.app/ticketing.local.json` is applied automatically — check there first if you're unsure whether one exists.
+
+### "Read / view `<record>`" — decision tree
+
+When a user says **"read"**, **"show"**, **"view"**, **"what's on the server for `<file>`"**, or **"get `<record>`"**, they are in the READ step of CRUD. There are two distinct layers:
+
+#### Layer 1 — Query what's on the server (REST)
+
+Use the raw REST output to inspect live server data without touching local files:
+
+```bash
+# Output by entity
+GET /api/o/e/<entityUid>
+# Output direct
+GET /api/o/<outputUid>
+```
+
+Alternatively the CLI `output` command returns clean stylized JSON:
+
+```bash
+# Query records for an entity
+dbo output -e <entity>
+
+# Query a specific record by UID
+dbo output -e <entity>/<uid>
+
+# Run a named output query
+dbo output <uid>
+```
+
+Use this when you want to inspect current server state, compare values, or look up a UID before pulling.
+
+#### Layer 2 — Sync server state to local files
+
+The local `.metadata.json` file is a representation of the server record — it reflects what was on the server at the time of the last `dbo clone` or `dbo pull`. Use these commands to bring local files in sync with the server:
+
+> **After any REST API add/insert/create** (`/api/input/submit`): always follow with `dbo pull` (or `dbo pull <uid>`) so the new record's `.metadata.json` and companion content files are written locally. The REST API only creates the record server-side — local files do not exist until a pull is performed.
+
+```
+Is this the first time cloning this app, or do you need a full reset?
+│
+├─ YES → dbo clone
+│         Pulls all records for the app, writes .metadata.json + companion
+│         content files, and moves any local records deleted server-side to trash/.
+│
+└─ NO  → Already have local files, just want to refresh a specific record or entity?
+          │
+          ├─ Specific file/UID → dbo pull <file>   or   dbo pull <uid>
+          │                       (also use after REST API insert to create local files)
+          │
+          └─ All records for an entity → dbo pull -e <entity>
+                                          or just: dbo pull   (entire app)
+```
+
+> **What `.metadata.json` represents**: each `.metadata.json` is a snapshot of the server record at last sync — column values, UIDs, `_CreatedOn`, `_LastUpdated`, and `@reference` pointers to companion content files. It is the local source of truth for push/diff operations, not an independent copy. If local metadata diverges from the server (another user edited the record), `dbo push` will detect the conflict via `_LastUpdated` comparison.
+
+### "Update / Edit `<file>`" — decision tree
+
+When a user says **"edit `<file>`"**, **"update `<file>`"**, **"change `<column>` on `<record>`"**, or **"modify `<file>`"**, they are in the UPDATE step of CRUD.
+
+#### File given (local edit + push)
+
+This is the standard path. Make changes locally, then push:
+
+1. **Edit the companion content file** — modify the file itself (e.g. `docs/readme.md`, `lib/bins/app/app.js`)
+2. **Edit the `.metadata.json`** — update any column values that should change (e.g. `Name`, `Status`, `Type`). Valid columns for each entity are listed in `.app/<appname>.metadata_schema.json`. Do not modify `_CreatedOn`, `_LastUpdated`, `UID`, or `ID` — these are server-managed.
+3. **Push**:
+   ```bash
+   dbo push <file> [--ticket <id>]
+   ```
+
+> Only modify the fields that need to change. Leave all other columns in `.metadata.json` as-is — push sends a diff, not a full replace.
+
+> If you only need to update content (not column values), you can skip editing `.metadata.json` and push the content file directly. If you only need to update column values (not content), edit `.metadata.json` and use `--meta-only`:
+> ```bash
+> dbo push <file> --meta-only [--ticket <id>]
+> ```
+
+#### No file given (server-side edit via REST)
+
+When editing column values only and no local file needs to change, use the REST API directly:
+
+```bash
+# Update a specific record
+POST /api/input/submit
+  RowID:<rowId>;column:<entity>.<column>=<value>
+  RowID:<rowId>;column:<entity>.<column>=<value>
+```
+
+Always follow with a pull to keep local files in sync:
+
+```bash
+dbo pull <uid>      # sync a single record
+# or
+dbo pull            # sync all (if multiple records changed)
+```
+
+> **Never** skip the `dbo pull` after a REST update — the local `.metadata.json` will be stale and the next `dbo push` may overwrite the server change with old values.
+
+### "Remove / Delete `<file>`" — decision tree
+
+When a user says **"delete `<file>`"**, **"remove `<record>`"**, **"rm `<file>`"**, or **"drop `<record>`"**, they are in the DELETE step of CRUD.
+
+#### File given (local + server delete)
+
+This is the standard path. The CLI stages the deletion and renames local files before the server is touched:
+
+```bash
+dbo rm <file> [--ticket <id>]
+dbo push      [--ticket <id>]
+```
+
+What `dbo rm` does:
+1. Reads the companion `.metadata.json` to identify the entity and row ID
+2. Stages the delete expression in `.dbo/synchronize.json`
+3. Removes the record from `app.json`
+4. Renames local files to `__WILL_DELETE__<filename>` (default) so they are excluded from future operations but not immediately gone
+   - `--hard` — immediately deletes local files instead of renaming
+   - `--keep-local` — only stages the server deletion; leaves local files untouched
+
+`dbo push` then sends the staged delete to the server and removes the `__WILL_DELETE__` files.
+
+#### No file given (server-side delete via REST)
+
+When deleting a record that has no local representation, use the REST API:
+
+```bash
+POST /api/input/submit
+  RowID:del<rowId>;entity:<entity>=true
+```
+
+Follow with a pull to confirm local state is consistent — though if the record had local files, `dbo pull` will **not** remove them (see note below):
+
+```bash
+dbo pull -e <entity>    # re-sync the entity (does not delete local orphans)
+```
+
+For local orphaned files after a server-side delete, run `dbo rm <file>` without the subsequent `dbo push` (the server record is already gone) — or use `--keep-local` to skip local file changes and only update `synchronize.json`.
+
+> **Server-side deletions and local sync**: `dbo clone` and `dbo pull` handle server-side deletions in two ways:
+>
+> 1. **Explicit `deleted` list** — when the server response includes a `deleted` map (keyed by entity, each containing `UID`, `Name`, and `AssetLastUpdated`), matching local files are automatically moved to `trash/`. This is the authoritative path and works correctly in delta/pull mode.
+> 2. **UID diff fallback** — after processing the deleted list, clone compares all remaining local UIDs against all server UIDs. Any local record whose UID is absent from the server (and not already staged in `synchronize.json`) is also moved to `trash/`.
+>
+> In both cases the files are moved to `trash/` — not permanently deleted. For records that have no local files at all, nothing happens. If you need to force-clean a local orphan that neither path caught, run `dbo rm <file>` manually.
+
 ---
 
 ## Upgrading
@@ -156,9 +354,7 @@ my-project/
 ├── test/               # Project-level tests
 ├── trash/              # Staged soft-deleted files from dbo rm
 ├── docs/               # Project documentation and docs entities
-├── manifest.json       # PWA web app manifest (root content file — server-tracked)
-├── CLAUDE.md           # Claude Code instructions (root content file — server-tracked)
-├── README.md           # Project readme (root content file — server-tracked)
+├── manifest.json       # PWA web app manifest (auto-generated from app metadata)
 ├── .gitignore          # Tells Git to ignore files in the repo sync
 ├── .dboignore          # Tells dbo cli to ignore files in commands
 ├── package.json        # Metadata, scripts, and dependency list
@@ -247,7 +443,6 @@ Use `dbo status` to see how many pending migrations exist.
 | `TicketSuggestionOutput` | string | Output UID for ticket suggestions (auto-set during init, default `ojaie9t3o0kfvliahnuuda`) |
 | `cloneSource` | `"default"` \| file path \| URL | Where the last `dbo clone` fetched app JSON from. `"default"` = server fetch via `AppShortName`; any other value = the explicit local file path or URL used. Set automatically after each successful clone. |
 | `ContentPlacement` | `bin` \| `path` | Where to place content files during clone (default: `bin`) |
-| `UserMedia` | `true` \| `false` | Whether to download user media (`/media/<app>/user/...`) during clone. Set on first clone via prompt (or `false` in non-interactive mode). |
 | `<Entity>FilenameCol` | column name | Filename column for entity-dir records (e.g., `ExtensionFilenameCol`) |
 | `dependencies` | string[] | Array of app short-names to auto-clone as read-only local checkouts. Default: `["_system"]` |
 | `dependencyLastUpdated` | object | Map of dependency short-name to last-cloned `_LastUpdated` ISO string |
@@ -290,49 +485,20 @@ Clone of the original app JSON from the server with entity entries replaced by `
 
 The `_domain` field in the app metadata file stores the project's reference domain (set during `dbo clone`). This is committed to git and used for domain-change detection when running `dbo init --force` or `dbo clone --domain`. It provides a stable cross-user baseline — all collaborators share the same reference domain.
 
-#### Root content files (`manifest.json`, `CLAUDE.md`, `README.md`)
-
-Certain files live at the **project root** (not inside `lib/`) but are still tracked and synced to the server as `content` entities. Their metadata companions live in `lib/bins/app/` with a root-relative `@/` reference (`Content: "@/manifest.json"`).
-
-| File | Server entity | Public | Description |
-|------|--------------|--------|-------------|
-| `manifest.json` | content | Yes | [PWA web app manifest](https://developer.mozilla.org/en-US/docs/Web/Manifest) |
-| `CLAUDE.md` | content | No | Claude Code project instructions |
-| `README.md` | content | Yes | Project readme |
-| `package.json` | content | No | Node.js package metadata and scripts |
-| `.dboignore` | content | No | DBO CLI ignore patterns |
-| `.gitignore` | content | No | Git ignore patterns |
-
-**`dbo clone`** writes all six to the project root. If the server has no record for a file, a stub is generated automatically (`manifest.json` from app metadata, `CLAUDE.md` and `README.md` from app name/description, `package.json` as a minimal stub). Matching works regardless of the server record's `BinID`: records are matched by name, `@`-reference, or `Path`, then relocated to the root — whether they live in `lib/bins/app/`, another bin, or have `BinID=null`. Records with `BinID=null` that aren't in the configured list are also promoted to the project root automatically.
-
-**`dbo push`** auto-creates the companion metadata in `lib/bins/app/` for any listed root file that exists locally but has no tracked metadata yet — no manual `dbo adopt` needed before the first push.
-
-**`dbo adopt <filename>`** called from the project root also works for any root content file: it bypasses the `.dboignore` check and entity inference, and writes the metadata directly to `lib/bins/app/`.
+#### `manifest.json`
 
-**`manifest.json`** generated fields (when produced as a stub):
+A [PWA web app manifest](https://developer.mozilla.org/en-US/docs/Web/Manifest) auto-generated during scaffolding (`dbo init` or `dbo clone`). Values are derived from `app.json` when available:
 
 | Field | Source |
 |-------|--------|
-| `name` | `AppName` |
+| `name` | `<AppName> \| <domain>` |
 | `short_name` | `AppShortName` |
 | `description` | `App.Description` |
 | `start_url` / `scope` | `/app/<ShortName>/ui/` |
-| `background_color` | Extracted from the `widget` extension matching `ShortName` (field `String4`), defaults to `#ffffff` |
+| `background_color` | Extracted from the `widget` extension matching the app's `ShortName` (field `String4`), defaults to `#ffffff` |
 | `theme_color` | `#000000` |
 
-#### `rootContentFiles` config
-
-The list of root content files is stored in `.app/config.json` under `rootContentFiles`:
-
-```json
-{
-  "rootContentFiles": ["CLAUDE.md", "README.md", "manifest.json", "package.json", ".dboignore", ".gitignore"]
-}
-```
-
-- **Absent**: defaults are written on the first `dbo push` — `["CLAUDE.md", "README.md", "manifest.json", "package.json", ".dboignore", ".gitignore"]`
-- **`[]`, `false`, or `null`**: feature disabled — no root content files are auto-managed
-- **Custom list**: any filename can be added; metadata is derived from the file extension
+The file is only created if absent — it is never overwritten by subsequent clones, so local edits (icons, screenshots, display overrides) are preserved. A companion `manifest.metadata.json` is auto-created by `dbo push` if missing, allowing the manifest to be pushed to the server as a content record.
 
 ### `.dboignore`
 
@@ -493,11 +659,11 @@ The project's reference domain is stored in `.app/<shortName>.metadata.json._dom
 6. **Creates directories** — processes `children.bin` to build the directory hierarchy based on `ParentBinID` relationships
 7. **Saves `.app/directories.json`** — maps BinIDs to directory paths for file placement
 8. **Writes content files** — decodes base64 content, creates `*.metadata.json` + content files in the correct bin directory. Filename columns and companion file extraction preferences are auto-applied with sensible defaults on first clone (use `--configure` to re-prompt)
-9. **Downloads media files** — classifies records into three scopes before downloading: `app` (`/media/<app>/app/...` or no FullPath — always downloaded), `user` (`/media/<app>/user/...` — opt-in, preference saved as `UserMedia` in `config.json`), and `foreign` (`/media/<other_app>/...` — silently skipped with stale metadata written). Fetches using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`. 404 errors create stale metadata to prevent re-prompting. Errors are logged to `.app/errors.log`
+9. **Downloads media files** — fetches binary files (images, CSS, fonts) from the server using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`, and saves with metadata. 404 errors create stale metadata to prevent re-prompting. Errors are logged to `.app/errors.log`
 10. **Processes entity-dir records** — entities matching project directories (`extension`, `app_version`, `data_source`, `entity`, `site`, `group`, `integration`, `automation`) are saved as `.metadata.json` files in their corresponding directory. Note: `entity` (table definitions) shares `lib/data_source/` with `data_source` records rather than using a separate directory
 11. **Processes other entities** — remaining entities with a `BinID` are placed in the corresponding bin directory
 12. **Saves `.app/<shortName>.metadata.json`** — clone of the original JSON with processed entries replaced by `@path/to/*.metadata.json` references
-13. **Orphan cleanup** — any local `.metadata.json` files whose UID is absent from the server response are automatically moved to `trash/` along with their companion content and media files. This prevents stale records (deleted server-side) from causing false positives in `dbo push`. Skipped during `--entity-filter` clones
+13. **Orphan cleanup** — server-deleted records are moved to `trash/` in two passes: first via the explicit `deleted` map in the server response (authoritative; works in delta/pull mode), then via a full UID diff of all local `.metadata.json` files against the server response (fallback for anything not in the deleted list). Companion content and media files are moved alongside their metadata. Skipped during `--entity-filter` clones
 
 #### Clone source tracking
 
@@ -559,24 +725,6 @@ This helps keep your app clean by removing database records for media files that
 
 In non-interactive mode (`-y`), stale cleanup is skipped (conservative default).
 
-#### Media scope filtering
-
-During `dbo clone`, media records are classified by their `FullPath` relative to the app short name:
-
-| Scope | Path pattern | Behavior |
-|-------|-------------|----------|
-| `app` | `/media/<app>/app/...` or no `FullPath` | Always downloaded |
-| `user` | `/media/<app>/user/...` | Downloaded only if `UserMedia=true` in `.app/config.json` |
-| `foreign` | `/media/<other_app>/...` | Never downloaded — stale metadata written to prevent re-prompts |
-
-On the first clone with user media present, you'll be prompted once:
-
-```
-App has 14 user media file(s) (e.g. "avatar.jpg"). Download user media? (saves preference)
-```
-
-The answer is saved as `UserMedia` in `.app/config.json` and reused on subsequent clones. In non-interactive mode (`-y`), user media defaults to skipped (`UserMedia=false`).
-
 #### Path resolution
 
 When a content record has both `Path` and `BinID`, the CLI prompts:
@@ -1594,11 +1742,18 @@ Files matching `.dboignore` patterns are skipped — both in directory mode (`db
 
 #### Single file
 
-```bash
-# Single file — entity inferred from directory
-dbo adopt lib/bins/app/assets/css/colors.css
+> **Always pass `-e <entity>`.**
+> Auto-inference via `resolveDirective()` only works when the file's directory path exactly matches a registered directive template. For files in `lib/bins/` (HTML, CSS, JS, SQL), inference commonly fails because `lib/bins/` maps to multiple entity types — the CLI cannot determine whether the file should be `content`, `media`, or another entity without an explicit flag. Omitting `-e` for these files will cause `adopt` to fail with an "entity cannot be inferred" error.
+>
+> | File type | Correct `-e` flag |
+> |-----------|-------------------|
+> | HTML, CSS, JS, SQL in `lib/bins/` | `-e content` |
+> | Images, fonts, binaries in `lib/bins/` | `-e media` |
+> | Files in `lib/extension/` | `-e extension` (or `-e extension.{Column}`) |
+> | Files in `docs/` | `-e content` |
 
-# Single file — explicit entity
+```bash
+# Single file — explicit entity (recommended, always works)
 dbo adopt lib/bins/app/assets/css/colors.css -e content
 
 # Extension with explicit column
@@ -1606,9 +1761,10 @@ dbo adopt lib/extension/widget/button.html -e extension.String5
 
 # Skip all confirmation prompts
 dbo adopt lib/bins/app/assets/css/colors.css -e content -y
-```
 
-If the file lives in a recognized directory (e.g. `lib/bins/`, `lib/extension/`, `docs/`), the entity is auto-inferred via `resolveDirective()`. Otherwise, pass `-e <entity>` or follow the interactive wizard.
+# Single file — entity inferred from directory (only works for exact directive matches)
+dbo adopt lib/bins/app/assets/css/colors.css
+```
 
 #### Attach to existing record (`--into`)
 
@@ -1791,6 +1947,20 @@ Stores ticket IDs for automatic application during submissions:
 - `records` — Per-record tickets (auto-cleared after successful submission)
 - `--ticket` flag always takes precedence over stored tickets
 
+#### Commands that accept `--ticket`
+
+The `--ticket <id>` flag is available on all submission commands:
+
+| Command | Flag effect |
+|---------|-------------|
+| `dbo push [path]` | Attaches ticket ID to all records pushed in this invocation |
+| `dbo add` | Attaches ticket ID to the new record being created |
+| `dbo content deploy <uid> <file>` | Attaches ticket ID to the content deploy submission |
+| `dbo deploy <target>` | Attaches ticket ID to the deploy submission |
+| `dbo input -d <expr>` | Passes `_OverrideTicketID` directly to `/api/input/submit` |
+
+When `--ticket` is provided it bypasses both the stored ticket prompt and any interactive ticket prompt on server error.
+
 #### ModifyKey protection (locked/production apps)
 
 When an app has a `ModifyKey` set (production/locked mode), the CLI guards all submission commands (`push`, `input`, `add`, `content deploy`, `deploy`) with an interactive prompt:

package/plugins/claude/track/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "track",
-  "version": "1.0.1",
+  "version": "1.0.2",
   "description": "Changelog and Track API logging for Claude Code — automatically logs changes to changelog.md and the remote Track task_log on every session",
   "author": {
     "name": "DBO.io"