@dboio/cli 0.19.4 → 0.20.0

package/plugins/claude/dbo/skills/cookbook/SKILL.md

@@ -0,0 +1,162 @@
+---
+name: cookbook
+description: >-
+  DBO project cookbook — operational DOs and DON'Ts for working correctly in any
+  DBO project. Load this skill when editing files, adding new features, debugging,
+  or when the white-paper provides architecture context but you need behavioral
+  rules. Covers file operations, build recipes, metadata handling, and common
+  gotchas.
+user-invocable: true
+---
+
+# DBO Project Cookbook
+
+Operational rules for working correctly in a DBO project. This is a living document — rules are added as new patterns and pitfalls are discovered.
+
+For architecture context, load the **white-paper** skill. For CLI command reference, use `/dbo`. This cookbook focuses on *how to behave* when making changes.
+
+---
+
+## DOs
+
+- **DO use UIDs, not numeric IDs, in templates.** Numeric IDs change across environments (dev → staging → prod). UIDs are stable and portable.
+- **DO always pass `-e <entity>` when running `dbo adopt`.** Every file in `lib/` needs a metadata companion. `dbo adopt path/to/file -e {entity}` creates one. Without it, `dbo push` won't know what server record to target. **Never omit `-e`** — auto-inference fails for most `lib/bins/` files because the directory maps to multiple entity types. Use `-e content` for HTML/CSS/JS/SQL, `-e media` for images and binaries.
+- **DO build from `src/` before pushing compiled assets.** Check `.app/scripts.json` for the build command. Run `npm run build` or the specific build script, then push the compiled output from `lib/`.
+- **DO check `.app/scripts.json` for build/postpush chains before manual deploys.** A file may have postpush hooks that auto-deploy dependent assets. Pushing manually with `dbo deploy` bypasses these chains.
+- **DO read the metadata file before modifying a server-tracked file.** The `.metadata.json` companion tells you what entity, column, and record the file maps to (check the `UID` field inside the JSON). This context prevents wrong-target pushes.
+- **DO use `dbo diff` to verify local vs server state before bulk pushes.** Catches unexpected server-side changes that would be overwritten.
+- **DO check `app_dependencies/` for parent app patterns when building extensions.** Descriptor definitions, CSS design tokens, and JS APIs from the parent app define what's available.
+- **DO know that `dbo clone` skips dependencies that are already up to date.** Before re-cloning a dependency, the CLI hits `/api/app/object/<name>?UpdatedAfter=<stored-date>` and compares `_LastUpdated`. If the server hasn't changed, the dep is skipped silently. Use `--force` or `--schema` to bypass this check and force a re-clone.
+- **DO use the REST API with `.app/cookies.txt` for data operations, CLI for file operations.** The CLI handles metadata sync; the API handles data reads/writes. Don't cross these boundaries.
+- **DO check `.app/deploy_config.json` for existing shorthands before creating deploy commands.** Shorthands are auto-generated by `dbo clone` and `dbo adopt`.
+- **DO use `dbo push --confirm false` to validate before committing a push.** Dry-run catches issues without touching the server.
+
+## DON'Ts
+
+- **DON'T push `src/`, `test/`, `node_modules/`, `app_dependencies/`, or `trash/` to the server.** These are strictly local. Only `lib/` and `docs/` contents map to server records.
+- **DON'T use numeric IDs in templates.** They break across environments. Always use UIDs.
+- **DON'T create files in `lib/` without a metadata companion.** Use `dbo adopt` to assign one. Files without metadata are invisible to the CLI and won't sync.
+- **DON'T bypass `dbo push` by deploying raw files.** Push handles build scripts, metadata sync, and postpush hooks. Deploying directly skips all of that.
+- **DON'T assume a deploy shorthand exists — check `.app/deploy_config.json` first.** Not all files have shorthands. Use `dbo push` for files without one.
+- **DON'T modify vendor files in `node_modules/` or `app_dependencies/`.** These are read-only references. Vendor changes belong upstream; dependency changes belong in the source project.
+- **DON'T edit files in `app_dependencies/` directly.** They are cloned read-only mirrors. Edit in the source project and re-clone.
+- **DON'T push minified files without also pushing their unminified source.** The server tracks both. Pushing only `.min.css` leaves the content record out of sync.
+- **DON'T delete metadata files manually.** Use `dbo rm` to remove a file and its metadata together. Orphaned metadata causes ghost records.
+
+---
+
+## File Operation Recipes
+
+### Add a new file to the server
+```bash
+# 1. Create the file in the correct lib/ subdirectory
+# 2. Create metadata companion (entity is usually "content" for text, "media" for binary)
+dbo adopt lib/bins/app/assets/css/new-file.css -e content
+
+# 3. Push — inserts a new server record
+dbo push lib/bins/app/assets/css/new-file.css
+```
+
+### Modify an existing server-tracked file
+```bash
+# 1. Edit the file locally
+# 2. Push the change (metadata already exists from clone)
+dbo push lib/bins/app/assets/css/colors.css
+```
+
+### Delete a file from the server
+```bash
+# Removes the local file, its metadata, and the server record
+dbo rm lib/bins/app/assets/css/old-file.css
+```
+
+### Rename or move a file
+There is no rename command. Instead:
+```bash
+# 1. Create the new file with the desired name/location
+# 2. Adopt it as a new record
+dbo adopt lib/bins/app/assets/css/renamed.css -e content
+# 3. Push the new file
+dbo push lib/bins/app/assets/css/renamed.css
+# 4. Remove the old file
+dbo rm lib/bins/app/assets/css/old-name.css
+```
+
+### Move a file between bins
+Same as rename — the bin is determined by the directory path. Moving to a new directory means a new BinID, which requires a new record.
+
+---
+
+## Build Recipes
+
+### CSS: SASS source → compiled → minified
+```bash
+# Build compiles src/sass/ → lib/bins/.../css/ and minifies
+npm run build
+# Or run the specific target from .app/scripts.json:
+sass src/sass/colors:lib/bins/app/assets/css
+csso lib/bins/app/assets/css/colors.css --output lib/bins/app/assets/css/colors.min.css
+```
+Then push both the unminified and minified versions.
+
+### JS: Rollup bundle
+```bash
+# Bundles ES6 modules from src/ into lib/bins/.../js/app.min.js
+rollup --config src/rollup.config.mjs
+# Then push
+dbo push lib/bins/app/assets/js/app.min.js
+```
+
+### When does push auto-compile?
+It doesn't. `dbo push` triggers **postpush** scripts (defined in `.app/scripts.json` under `targets.{file}.postpush`), but these are for deploying *dependent* assets, not for compiling. Always build first, then push.
+
+### Check what builds exist
+```bash
+# Read the scripts config
+cat .app/scripts.json
+```
+Look at `targets.{file}.build` for compile commands and `targets.{file}.postpush` for post-push deploy chains.
+
+---
+
+## Gotchas
+
+### Metadata mismatch after manual file creation
+**Symptom:** `dbo push` says "no changes detected" for a file you just created.
+**Cause:** No metadata companion exists. The CLI doesn't track unregistered files.
+**Fix:** Run `dbo adopt path/to/file -e {entity}` first.
+
+### Postpush chain didn't fire
+**Symptom:** You pushed a JS file but the documentation deploy didn't happen.
+**Cause:** You used `dbo deploy` instead of `dbo push`. Deploy bypasses postpush hooks.
+**Fix:** Use `dbo push` for files with postpush chains, or manually run the postpush commands.
+
+### Content and media records out of sync
+**Symptom:** CSS looks wrong on the server even though you pushed the file.
+**Cause:** You pushed only the content record but not the media companion (or vice versa).
+**Fix:** Push both — check `deploy_config.json` for paired `{name}` and `{name}_media` entries.
+
+### `dbo adopt` fails — entity cannot be inferred
+**Symptom:** `dbo adopt path/to/file` errors with "entity cannot be inferred" or similar.
+**Cause:** `-e <entity>` was omitted. Auto-inference only works for exact directive template matches. Files in `lib/bins/` (HTML, CSS, JS, SQL) almost never infer correctly because `lib/bins/` maps to multiple entity types.
+**Fix:** Always pass `-e content` for text/code files, `-e media` for images and binaries:
+```bash
+dbo adopt lib/bins/app/user_first_names.html -e content
+dbo adopt lib/bins/app/assets/img/logo.png -e media
+```
+
+### Wrong entity type on adopt
+**Symptom:** Pushed file lands in unexpected server location.
+**Cause:** Used wrong `-e` flag on `dbo adopt` (e.g., `-e media` for a text file).
+**Fix:** Remove with `dbo rm`, re-adopt with correct entity type, re-push.
+
+### Extension files not picked up
+**Symptom:** New extension file doesn't appear in the app.
+**Cause:** Extension files follow strict naming: `{Name}.{ContentColumnName}.{ext}` with metadata companion.
+**Fix:** Verify the file name matches the pattern and the descriptor type exists in `app_dependencies/`.
+
+---
+
+## Project-Specific Overlays
+
+Individual DBO projects may provide an additional conventions file at `docs/Operator_Claude.md` (or equivalent). When present, load it alongside this cookbook for project-specific DOs, DON'Ts, and patterns. When the project is a dependency of another app, the overlay appears at `app_dependencies/{app}/docs/{App}_Claude.md`.

package/plugins/claude/dbo/skills/white-paper/SKILL.md

@@ -49,10 +49,9 @@ project-root/
 ├── lib/                     # Server-mapped files (mirrors DBO table data)
 │   ├── bins/                # Bin entities and their children (see Bin Mapping below)
 │   ├── extension/           # Extension records, sub-foldered by Descriptor
-│   ├── entity/              # Entity metadata files
+│   ├── data_source/         # Data source configs + entity (table) definitions
 │   ├── site/                # Site definitions
 │   ├── security/            # Security rule records
-│   ├── data_source/         # Data source configurations
 │   └── app_version/         # App version metadata
 │
 ├── docs/                    # Project documentation (scripts, embeds, pages, components)
@@ -83,11 +82,11 @@ Files belonging to a `bin` entity are placed in directories matching their BinID
 }
 ```
 
-So `lib/bins/app/assets/css/colors.css` lives inside the bin whose `fullPath` is `app/assets/css`. Every file has a companion `.metadata.json` (or `.metadata~{uid}.json`) containing the server record fields.
+So `lib/bins/app/assets/css/colors.css` lives inside the bin whose `fullPath` is `app/assets/css`. Every file has a companion `.metadata.json` (e.g. `colors.metadata.json`) containing the server record fields, including the `UID` field.
 
 ### Non-bin entities (no BinID column)
 
-Entities without a BinID column get their own directory under `lib/`, named after the entity: `lib/extension/`, `lib/entity/`, `lib/site/`, `lib/security/`, `lib/data_source/`, `lib/app_version/`.
+Entities without a BinID column get their own directory under `lib/`, named after the entity: `lib/extension/`, `lib/site/`, `lib/security/`, `lib/data_source/`, `lib/app_version/`. Exception: `entity` (table definitions) shares `lib/data_source/` rather than getting its own directory — both `_entity: "data_source"` and `_entity: "entity"` records live there.
 
 ### Extension special case
 
@@ -96,7 +95,7 @@ The `extension` entity has sub-folders based on the **Descriptor** column value:
 - `lib/extension/Widget/` — Widget extensions
 - `lib/extension/Descriptor Definition/` — Descriptor definitions themselves
 
-Each extension file follows: `{Name}.{ContentColumnName}.{ext}` with a companion `{Name}.metadata~{uid}.json`.
+Each extension file follows: `{Name}.{ContentColumnName}.{ext}` with a companion `{Name}.metadata.json` (UID stored inside the JSON).
 
 ## .app/ Configuration
 
@@ -152,10 +151,10 @@ When `app_dependencies/{app_name}/` exists (e.g., `app_dependencies/operator/`),
 ## CLI vs REST API
 
 ### Use the DBO CLI for:
-- **File operations**: `pull`, `push`, `add`, `clone`, `rm`, `diff`
+- **File operations**: `pull`, `push`, `adopt`, `clone`, `rm`, `diff` — for `adopt`, always pass `-e <entity>` (e.g., `-e content` for HTML/CSS/JS, `-e media` for images/binaries); entity cannot be inferred for standalone files in `lib/bins/`
 - **Deployment**: `deploy` (shorthand or UID), content deploy
 - **Project setup**: `init`, `login`, `status`
-- **Batch operations**: pushing directories, scanning for un-added files
+- **Batch operations**: pushing directories, scanning for unadopted files
 
 The CLI handles metadata management, change detection, and file-to-record synchronization automatically.
 
@@ -182,6 +181,48 @@ RowID:del{id};entity:entityName=true            # Delete
 ```
 Always include `_confirm=true`. Special values: `_{now}` (datetime), `_{null}` (null).
 
+## Creating New Assets
+
+Two approaches — choose based on whether the file exists locally first or on the server first.
+
+### File-first (CLI)
+
+Use when you're creating a new file locally and want the server to create the record.
+
+```bash
+# 1. Write the file in the correct lib/ subdirectory
+# 2. Create the metadata companion (required before push)
+dbo adopt -e content lib/bins/app/my-file.html   # HTML/CSS/JS/SQL → content
+dbo adopt -e media lib/bins/app/assets/img/x.png # images/binaries → media
+
+# 3. Push — inserts a new record on the server, assigns a UID, updates local metadata
+dbo push lib/bins/app/my-file.html
+```
+
+Always pass `-e <entity>` to `adopt`. Entity cannot be inferred for standalone files in `lib/bins/` because the directory maps to multiple entity types.
+
+### Server-first (REST API)
+
+Use when you want to create the server record first, then pull the file locally. Required fields for the insert come from three local config files:
+
+- **`.app/<app_short_name>.metadata_schema.json`** — entity schema: which columns are required, their types and allowed values
+- **`.app/directories.json`** — BinID for the target directory (needed for bin-based entities like `content` and `media`)
+- **`.app/config.json`** — `AppID`, `AppUID`, and other app-level fields required on most records
+
+```bash
+# 1. Insert the record via REST API (RowID:add1 — must use RowID:, not RowUID:)
+curl -b .app/cookies.txt \
+  "https://{{domain}}/api/input/submit?_confirm=true" \
+  --data-urlencode 'RowID:add1;column:content.BinID=11042' \
+  --data-urlencode 'RowID:add1;column:content.AppID={{appID}}' \
+  --data-urlencode 'RowID:add1;column:content.Name=My File'
+
+# 2. Pull to create the local file and metadata companion
+dbo pull -e content --filter 'Name=My File'
+```
+
+After pull, the file appears in `lib/bins/` at the path matching its BinID in `directories.json`, with a `.metadata.json` companion.
+
 ## docs/ Directory
 
 Contains Markdown documentation for this project's components — scripts, embeds, pages, UI features. These are project-specific references generated or maintained alongside development. Read these to understand documented behaviors of the current app.
@@ -213,7 +254,7 @@ The `test/` directory is for developer and Claude Code test scripts. Tests targe
 | Write/update data | REST API | `POST /api/input/submit` |
 | Pull files from server | CLI | `dbo pull -e content` |
 | Push local changes | CLI | `dbo push path/to/file` |
-| Add new file to server | CLI | `dbo add path/to/file` |
+| Add new file to server | CLI | `dbo adopt path/to/file -e <entity>` |
 | Deploy by shorthand | CLI | `dbo deploy css:colors` |
 | Deploy by UID | CLI | `dbo deploy {uid}` |
 | Compare with server | CLI | `dbo diff` |

package/plugins/claude/dbo/skills/white-paper/references/api-reference.md

@@ -87,7 +87,7 @@ RowID:{ref};column:{entity}.{Column}={value}
 | `{id}` or `{uid}` | Update existing record |
 | `del{id}` | Delete record |
 
-`RowID:` and `RowUID:` are interchangeable. Use `RowUID:` for cross-environment stability.
+`RowID:` and `RowUID:` are interchangeable for updates and deletes. Use `RowUID:` for cross-environment stability. **Inserts (`add1`, `add2`, …) must use `RowID:` — `RowUID:` does not work for inserts.**
 
 ### Cross-referencing within a batch
 ```

package/plugins/claude/track/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "track",
-  "version": "1.0.0",
+  "version": "1.0.1",
   "description": "Changelog and Track API logging for Claude Code — automatically logs changes to changelog.md and the remote Track task_log on every session",
   "author": {
     "name": "DBO.io"

package/src/commands/adopt.js

@@ -3,7 +3,7 @@ import { readFile, writeFile, stat, mkdir } from 'fs/promises';
 import { join, dirname, basename, extname, relative } from 'path';
 import { log } from '../lib/logger.js';
 import { loadIgnore } from '../lib/ignore.js';
-import { loadAppConfig, loadAppJsonBaseline, loadExtensionDocumentationMDPlacement, loadDescriptorFilenamePreference } from '../lib/config.js';
+import { loadAppConfig, loadAppJsonBaseline, loadExtensionDocumentationMDPlacement, loadDescriptorFilenamePreference, loadRootContentFiles } from '../lib/config.js';
 import {
   resolveDirective, resolveTemplateCols, assembleMetadata,
   promptReferenceColumn, getTemplateCols, setTemplateCols,
@@ -11,7 +11,7 @@ import {
 } from '../lib/metadata-schema.js';
 import { loadStructureFile, findBinByPath, BINS_DIR } from '../lib/structure.js';
 import { isMetadataFile } from '../lib/filenames.js';
-import { findUnaddedFiles, MIME_TYPES, seedMetadataTemplate, detectDocumentationFile, detectManifestFile } from '../lib/insert.js';
+import { findUnaddedFiles, MIME_TYPES, seedMetadataTemplate, detectDocumentationFile } from '../lib/insert.js';
 import { runPendingMigrations } from '../lib/migrations.js';
 
 export const adoptCommand = new Command('adopt')
@@ -79,6 +79,15 @@ function parseEntitySpec(spec, metadataSchema) {
   return { entity, descriptor: null, column: sub };
 }
 
+/**
+ * Metadata templates for known root content files (mirrors push.js ROOT_FILE_TEMPLATES).
+ */
+const ROOT_FILE_TEMPLATES = {
+  'manifest.json': { Extension: 'JSON', Public: 1, Active: 1, Title: 'PWA Manifest' },
+  'CLAUDE.md':     { Extension: 'MD',   Public: 0, Active: 1, Title: 'Claude Code Instructions' },
+  'README.md':     { Extension: 'MD',   Public: 1, Active: 1, Title: 'README' },
+};
+
 /**
  * Build entity-specific metadata for a file in or outside lib/bins/.
  */
@@ -144,6 +153,62 @@ async function buildBinMetadata(filePath, entity, appConfig, structure) {
 async function adoptSingleFile(filePath, entityArg, options) {
   const ig = await loadIgnore();
   const relPath = relative(process.cwd(), filePath).replace(/\\/g, '/');
+  const fileName = basename(filePath);
+
+  // --- Root content file check (before ignore): CLAUDE.md, README.md, manifest.json, etc. ---
+  // These files live at the project root but their metadata goes in lib/bins/app/.
+  // They bypass the dboignore check and entity inference.
+  const rootFiles = await loadRootContentFiles();
+  if (rootFiles.some(f => f.toLowerCase() === fileName.toLowerCase()) && relPath === fileName) {
+    const appConfig = await loadAppConfig();
+    const structure = await loadStructureFile();
+    const appBin = findBinByPath('app', structure);
+    const binsAppDir = join(process.cwd(), BINS_DIR, 'app');
+    const ext = extname(fileName).replace('.', '').toUpperCase() || 'TXT';
+    const stem = basename(fileName, extname(fileName));
+
+    const metaFilename = `${stem}.metadata.json`;
+    const metaPath = join(binsAppDir, metaFilename);
+
+    // Check for existing metadata
+    let existingMeta = null;
+    try { existingMeta = JSON.parse(await readFile(metaPath, 'utf8')); } catch {}
+    if (existingMeta) {
+      if (existingMeta.UID || existingMeta._CreatedOn) {
+        log.warn(`"${fileName}" is already on the server (has UID/_CreatedOn) — skipping.`);
+        return;
+      }
+      if (!options.yes) {
+        const inquirer = (await import('inquirer')).default;
+        const { overwrite } = await inquirer.prompt([{
+          type: 'confirm', name: 'overwrite',
+          message: `Metadata already exists for "${fileName}" (no UID). Overwrite?`,
+          default: false,
+        }]);
+        if (!overwrite) return;
+      }
+    }
+
+    const tmplKey = Object.keys(ROOT_FILE_TEMPLATES).find(k => k.toLowerCase() === fileName.toLowerCase());
+    const tmpl = (tmplKey ? ROOT_FILE_TEMPLATES[tmplKey] : null) || { Extension: ext, Public: 0, Active: 1, Title: fileName };
+    const meta = {
+      _entity: 'content',
+      _companionReferenceColumns: ['Content'],
+      ...tmpl,
+      Content: `@/${fileName}`,
+      Path: fileName,
+      Name: fileName,
+    };
+    if (appBin) meta.BinID = appBin.binId;
+    if (appConfig.AppID) meta.AppID = appConfig.AppID;
+
+    await mkdir(binsAppDir, { recursive: true });
+    await writeFile(metaPath, JSON.stringify(meta, null, 2) + '\n');
+    log.success(`Created ${metaFilename} for ${fileName}`);
+    log.dim(`  Run "dbo push" to insert this record on the server.`);
+    return;
+  }
+
   if (ig.ignores(relPath)) {
     log.dim(`Skipped (dboignored): ${relPath}`);
     return;
@@ -152,7 +217,6 @@ async function adoptSingleFile(filePath, entityArg, options) {
   const dir = dirname(filePath);
   const ext = extname(filePath);
   const base = basename(filePath, ext);
-  const fileName = basename(filePath);
 
   // Check for existing metadata
   const metaPath = join(dir, `${base}.metadata.json`);
@@ -204,16 +268,6 @@ async function adoptSingleFile(filePath, entityArg, options) {
 
   const appConfig = await loadAppConfig();
 
-  // --- Special case: manifest.json ---
-  const rel = relative(process.cwd(), filePath).replace(/\\/g, '/');
-  if (rel.toLowerCase() === 'manifest.json') {
-    const manifestResult = await detectManifestFile(filePath);
-    if (manifestResult) {
-      log.success(`Created manifest metadata at ${manifestResult.metaPath}`);
-      return;
-    }
-  }
-
   // --- content or media entities: build entity-specific metadata ---
   if ((entity === 'content' || entity === 'media') && !column) {
     const structure = await loadStructureFile();
@@ -238,7 +292,8 @@ async function adoptSingleFile(filePath, entityArg, options) {
       Descriptor: 'documentation',
       Name: docBase,
     };
-    const contentCol = 'String10';
+    if (appConfig.AppID) docMeta.AppID = appConfig.AppID;
+    const contentCol = 'Text';
     docMeta[contentCol] = `@/${relPath}`;
     docMeta._companionReferenceColumns = [contentCol];