@dboio/cli 0.15.2 → 0.16.2

package/README.md

@@ -134,8 +134,7 @@ my-project/
 │   ├── entity_column_value/ # Column value definitions
 │   ├── extension/      # Extension entity records (organized by descriptor type)
 │   │   ├── widget/
-│   │   ├── documentation/
-│   │   └── _unsupported/
+│   │   └── documentation/
 │   ├── integration/    # Integration entity records
 │   ├── security/       # Security policy records
 │   ├── security_column/ # Column-level security records
@@ -148,6 +147,7 @@ my-project/
 ├── trash/              # Staged soft-deleted files from dbo rm
 ├── docs/               # Project documentation and docs entities
 ├── app.json            # Clone of original app JSON with @references
+├── schema.json         # Instance-level entity/column schema fetched from the server during `dbo init`. Refreshed explicitly with `dbo clone --schema`.
 ├── manifest.json       # PWA web app manifest (auto-generated from app metadata)
 ├── .gitignore          # Tells Git to ignore files in the repo sync
 ├── .dboignore          # Tells dbo cli to ignore files in commands
@@ -178,7 +178,7 @@ All configuration is **directory-scoped**. Each project folder maintains its own
 | `credentials.json` | Username, user ID, UID, name, email (no password) | Gitignored (per-user) |
 | `cookies.txt` | Session cookie (Netscape format) | Gitignored (per-user) |
 | `structure.json` | Bin directory mapping (created by `dbo clone`) | Committable (shared) |
-| `metadata_templates.json` | Column templates per entity/descriptor (auto-generated by `dbo clone`) | Committable (shared) |
+| `metadata_schema.json` | Column templates per entity/descriptor (auto-generated by `dbo clone`; renamed from `metadata_templates.json`) | Committable (shared) |
 | `synchronize.json` | Pending deletions and sync state (updated by `dbo rm` and `dbo push`) | Committable (shared) |
 | `.app_baseline.json` | Server-state snapshot for delta detection — stores column values at last clone/push so `dbo push` can detect which columns changed locally. Read-only (chmod 444); auto-migrated from legacy root `.app.json`. | Committable (shared) |
 | `scripts.json` | Build/push lifecycle hooks (see [Script Hooks](#script-hooks)) | Committable (shared) |
@@ -201,6 +201,8 @@ All configuration is **directory-scoped**. Each project folder maintains its own
 > | Output | `Sales.metadata~ghi789.json` |
 > | Extension | `MyExtension.metadata~jkl012.json` |
 
+> **Upgrading to 0.16.0+**: `.dbo/metadata_templates.json` is renamed to `.dbo/metadata_schema.json`. The `_contentColumns` field in metadata files is renamed to `_companionReferenceColumns`. Extension companion file prompts during `dbo clone` are replaced by automatic derivation from `descriptor_definition` `form-control-code` data. Migration 011 runs automatically on first use.
+
 #### Automatic migrations
 
 When the CLI version is upgraded, one-time migration scripts in `tools/cli/src/migrations/` run automatically on the first command invocation in a project directory. Completed migration IDs are recorded in `.dbo/config.local.json` under `_completedMigrations` (per-user, gitignored) so each developer runs them independently and they never re-fire.
@@ -241,9 +243,37 @@ Use `dbo status` to see how many pending migrations exist.
 | `cloneSource` | `"default"` \| file path \| URL | Where the last `dbo clone` fetched app JSON from. `"default"` = server fetch via `AppShortName`; any other value = the explicit local file path or URL used. Set automatically after each successful clone. |
 | `ContentPlacement` | `bin` \| `path` | Where to place content files during clone (default: `bin`) |
 | `<Entity>FilenameCol` | column name | Filename column for entity-dir records (e.g., `ExtensionFilenameCol`) |
+| `dependencies` | string[] | Array of app short-names to auto-clone as read-only local checkouts. Default: `["_system"]` |
+| `dependencyLastUpdated` | object | Map of dependency short-name to last-cloned `_LastUpdated` ISO string |
 
 **File placement:** All content, media, and output files are placed by BinID into the `lib/bins/` directory structure. Records without a BinID go into `lib/bins/` root. Media always uses BinID — no FullPath option. Bins with `Name=null` (legacy) map directly to `lib/bins/` root.
 
+### App Dependencies
+
+The CLI automatically maintains local read-only checkouts of related apps under `.dbo/dependencies/<shortname>/`. By default, the `_system` app (which contains all entity, column, and extension descriptors for the connected instance) is always included.
+
+#### How it works
+
+- After `dbo init` or `dbo clone`, dependency apps are cloned into `.dbo/dependencies/`.
+- If an `app.json` contains a `Dependencies` column, those app short-names are merged into `.dbo/config.json` and also cloned.
+- You can add dependencies explicitly with `dbo clone --dependencies operator,launchpad`.
+- Dependencies are only re-cloned when the server's `_LastUpdated` is newer than the locally stored timestamp — so subsequent runs are fast.
+- Dependency clone output is suppressed — a single summary line shows synced/failed/up-to-date status.
+- `.dbo/dependencies/` is excluded from `.gitignore` and `.dboignore` automatically.
+
+#### Schema merging from dependencies
+
+After dependency cloning, extension descriptor definitions from dependency schemas (`.dbo/dependencies/<name>/.dbo/metadata_schema.json`) are merged into the local `.dbo/metadata_schema.json`. This is useful when an app has no `descriptor_definition` extensions of its own but depends on an app (like `operator`) that does — the operator's descriptor definitions (widget, include, control, etc.) become available for local extension processing.
+
+#### Flags
+
+| Flag | Commands | Effect |
+|------|----------|--------|
+| `--no-deps` | `dbo init`, `dbo clone` | Skip dependency syncing |
+| `--dependencies <list>` | `dbo clone` | Add and sync specific dependencies (comma-separated short-names) |
+| `--force` | `dbo clone` | Re-clone all dependencies regardless of staleness |
+| `--schema` | `dbo clone` | Re-clone all dependencies regardless of staleness |
+
 ### Root-level project files
 
 #### `app.json`
@@ -367,6 +397,7 @@ dbo init --scaffold --yes                             # scaffold dirs non-intera
 | `--dboignore` | Create `.dboignore` with default patterns (use with `--force` to overwrite existing) |
 | `-y, --yes` | Skip all interactive prompts (legacy migration, Claude Code setup) |
 | `--non-interactive` | Alias for `--yes` |
+| `--no-deps` | Skip dependency cloning after init |
 
 ---
 
@@ -402,9 +433,14 @@ dbo clone -e extension --descriptor-types false   # Clone extensions flat (no de
 | `--documentation-only` | When used with `-e extension`, clone only documentation extensions |
 | `--descriptor-types <bool>` | Sort extensions into descriptor sub-directories (default: `true`). Set to `false` to use flat `extension/` layout |
 | `--domain <host>` | Override domain. Triggers a domain-change confirmation prompt when it differs from the project reference domain |
+| `--schema` | Re-fetch `schema.json` from server before cloning |
 | `--force` | Skip source mismatch confirmation and change detection; re-processes all files |
 | `-y, --yes` | Auto-accept all prompts (also skips source mismatch confirmation) |
 | `-v, --verbose` | Show HTTP request details |
+| `--no-deps` | Skip dependency cloning after clone |
+| `--dependencies <list>` | Comma-separated app short-names to add and sync as dependencies (e.g. `--dependencies operator,launchpad`) |
+| `--configure` | Re-prompt for filename columns, companion file extraction, and documentation placement preferences |
+| `--media-placement <type>` | Set media placement: `fullpath` or `binpath` (default: `bin`) |
 
 #### Domain change detection
 
@@ -414,17 +450,19 @@ The project's reference domain is stored in `app.json._domain` (committed to git
 
 #### What clone does
 
-1. **Loads app JSON** — from a local file, server API, or interactive prompt. A spinner shows progress while fetching from the server (responses can be slow as the JSON is assembled on demand)
-2. **Updates `.dbo/config.json`** — saves `AppID`, `AppUID`, `AppName`, `AppShortName`, `AppModifyKey` (if the app is locked), and `cloneSource` (the source used for this clone)
-3. **Updates `package.json`** — populates `name`, `productName`, `description`, `homepage`, and `deploy` script
-4. **Creates directories** — processes `children.bin` to build the directory hierarchy based on `ParentBinID` relationships
-5. **Saves `.dbo/structure.json`** — maps BinIDs to directory paths for file placement
-6. **Writes content files** — decodes base64 content, creates `*.metadata.json` + content files in the correct bin directory. When a record's `Extension` field is empty, prompts you to choose from `css`, `js`, `html`, `xml`, `txt`, `md`, `cs`, `json`, `sql` (or skip to keep no extension). The chosen extension is saved back into the `metadata.json` `Extension` field
-7. **Downloads media files** — fetches binary files (images, CSS, fonts) from the server using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`, and saves with metadata. Errors are logged to `.dbo/errors.log`
-8. **Processes entity-dir records** — entities matching project directories (`extension`, `app_version`, `data_source`, `site`, `group`, `integration`, `automation`) are saved as `.metadata.json` files in their corresponding directory (e.g., `extension/`, `data_source/`)
-9. **Processes other entities** — remaining entities with a `BinID` are placed in the corresponding bin directory
-10. **Saves `app.json`** — clone of the original JSON with processed entries replaced by `@path/to/*.metadata.json` references
-11. **Orphan cleanup** — any local `.metadata.json` files whose UID is absent from the server response are automatically moved to `trash/` along with their companion content and media files. This prevents stale records (deleted server-side) from causing false positives in `dbo push`. Skipped during `--entity-filter` clones
+1. **Fetches schema** — downloads `schema.json` from the server if missing or explicitly requested (`--schema`). Regenerates `.dbo/metadata_schema.json` from the schema
+2. **Syncs dependencies** — clones dependency apps into `.dbo/dependencies/<shortname>/` (unless `--no-deps`). Merges extension descriptor definitions from dependency schemas into the local `.dbo/metadata_schema.json` (e.g., operator provides descriptor definitions for apps that lack their own)
+3. **Loads app JSON** — from a local file, server API, or interactive prompt. A spinner shows progress while fetching from the server (responses can be slow as the JSON is assembled on demand)
+4. **Updates `.dbo/config.json`** — saves `AppID`, `AppUID`, `AppName`, `AppShortName`, `AppModifyKey` (if the app is locked), and `cloneSource` (the source used for this clone)
+5. **Updates `package.json`** — populates `name`, `productName`, `description`, `homepage`, and `deploy` script
+6. **Creates directories** — processes `children.bin` to build the directory hierarchy based on `ParentBinID` relationships
+7. **Saves `.dbo/structure.json`** — maps BinIDs to directory paths for file placement
+8. **Writes content files** — decodes base64 content, creates `*.metadata.json` + content files in the correct bin directory. Filename columns and companion file extraction preferences are auto-applied with sensible defaults on first clone (use `--configure` to re-prompt)
+9. **Downloads media files** — fetches binary files (images, CSS, fonts) from the server using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`, and saves with metadata. 404 errors create stale metadata to prevent re-prompting. Errors are logged to `.dbo/errors.log`
+10. **Processes entity-dir records** — entities matching project directories (`extension`, `app_version`, `data_source`, `site`, `group`, `integration`, `automation`) are saved as `.metadata.json` files in their corresponding directory (e.g., `extension/`, `data_source/`)
+11. **Processes other entities** — remaining entities with a `BinID` are placed in the corresponding bin directory
+12. **Saves `app.json`** — clone of the original JSON with processed entries replaced by `@path/to/*.metadata.json` references
+13. **Orphan cleanup** — any local `.metadata.json` files whose UID is absent from the server response are automatically moved to `trash/` along with their companion content and media files. This prevents stale records (deleted server-side) from causing false positives in `dbo push`. Skipped during `--entity-filter` clones
 
 #### Clone source tracking
 
@@ -454,6 +492,8 @@ If the initial clone attempt fails (network error, file not found, empty respons
 
 When cloning an app that was already cloned locally, the CLI detects existing files and compares modification times against the server's `_LastUpdated`. You'll be prompted to overwrite, compare differences, or skip — same as `dbo pull`. Use `-y` to auto-accept all changes.
 
+Change detection only checks companion content and media files for user edits — metadata file mtimes are not considered, since they can be bumped by CLI-internal operations (migrations, legacy companion renames).
+
 #### Collision detection
 
 When multiple records would create files at the same path (e.g., a `content` record and a `media` record both named `colors.css`), the CLI detects the collision before writing any files and prompts you to choose which record to keep:
@@ -471,7 +511,9 @@ In non-interactive mode (`-y`), the first record is kept and others are auto-sta
 
 #### Stale media cleanup
 
-During media downloads, files returning 404 (no longer exist on the server) are collected as "stale records". After all downloads complete, you'll be prompted to stage them for deletion:
+During media downloads, files returning 404 (no longer exist on the server) are collected as "stale records". Metadata is still written for 404 files (marked with `_stale404: true`) so they are not re-prompted on subsequent clones.
+
+After all downloads complete, you'll be prompted to stage stale records for deletion:
 
 ```
 Found 3 stale media record(s) (404 - files no longer exist on server)
@@ -507,9 +549,9 @@ Entity types that correspond to project directories (`extension`, `app_version`,
 
 For each entity type, the CLI prompts to choose which column becomes the filename (defaults to `Name`, fallback `UID`). The choice is saved per entity type in `config.json` (e.g., `ExtensionFilenameCol`).
 
-If any columns contain base64-encoded content, the CLI prompts to extract them as companion files. Extracted columns produce files named `<name>.<Column>.<ext>` alongside the metadata, with `@reference` entries in the metadata and a `_contentColumns` array.
+If any columns contain base64-encoded content, the CLI prompts to extract them as companion files. Extracted columns produce files named `<name>.<Column>.<ext>` alongside the metadata, with `@reference` entries in the metadata and a `_companionReferenceColumns` array (legacy name: `_contentColumns`, supported for backward compatibility).
 
-**Companion file naming convention:** Only `.metadata.json` files carry the `~UID` suffix (e.g. `Add-Asst-Execute-Security~wxl6ivcwfkix3zgantszjg.metadata.json`). Companion content files use the natural base name without `~UID` (e.g. `Add-Asst-Execute-Security.String16.js`). If two records share the same name within a directory, the second gets a `-1` suffix (e.g. `Add-Asst-Execute-Security-1.String16.js` with metadata `Add-Asst-Execute-Security-1~otheruid.metadata.json`). Migration 007 automatically renames legacy `~UID` companion files to the natural convention.
+**Companion file naming convention:** Only `.metadata.json` files carry the `~UID` suffix (e.g. `Add-Asst-Execute-Security.metadata~wxl6ivcwfkix3zgantszjg.json`). Companion content files use the natural base name without `~UID` (e.g. `Add-Asst-Execute-Security.String5.html`). When a `{title}` is defined in the `@reference` expression (see [Metadata Schema](#metadata-schema)), the title replaces the column name in the filename (e.g. `Add-Asst-Execute-Security.Add-Form-Parameters.js` instead of `Add-Asst-Execute-Security.String5.js`). If two records share the same name within a directory, the second gets a `-1` suffix. Migration 007 automatically renames legacy `~UID` companion files to the natural convention.
 
 Use `-y` to skip prompts (uses `Name` column, no content extraction).
 
@@ -521,7 +563,7 @@ During clone, the CLI:
 
 1. **Pre-scans** all extension records for `descriptor_definition` entries and builds a `String1 → Name` mapping
 2. **Creates sub-directories** under `extension/` for each mapped descriptor (e.g., `extension/Documentation/`, `extension/Includes/`)
-3. **Always creates** `extension/_unsupported/` — extensions with an unmapped or null `Descriptor` are placed here
+3. **Places unmapped extensions** directly in `extension/` — extensions with an unmapped or null `Descriptor` are placed in the root `extension/` directory (the legacy `_unsupported/` sub-directory is migrated automatically by Migration 011)
 4. **Prompts per descriptor** — filename column and content extraction prompts fire once per unique `Descriptor` value (not once for all extensions)
 5. **Persists the mapping** in `.dbo/structure.json` under `descriptorMapping`
 
@@ -575,7 +617,6 @@ project/
       MyDoc.uid1.String10.md     # ← extracted content column
     Includes/
       Header.uid2.metadata.json
-    _unsupported/            # ← always created (unmapped/null descriptors)
   docs/                      # ← root-placed documentation MD files (optional)
     MyDoc.md
   data_source/
@@ -683,9 +724,9 @@ Key points:
 - `CustomSQL` content is extracted to companion `.sql` files per extraction rules
 - Re-cloning moves orphaned old-format child `.json` files to `/trash`
 
-#### Metadata Templates
+#### Metadata Schema
 
-During `dbo clone`, the CLI auto-generates `.dbo/metadata_templates.json` — a file that records which columns each entity/descriptor uses. This file is seeded from the first cloned record of each type and can be manually edited afterwards.
+During `dbo clone`, the CLI auto-generates `.dbo/metadata_schema.json` (formerly `metadata_templates.json`) — a file that records which columns each entity/descriptor uses. This file is seeded from the first cloned record of each type and can be manually edited afterwards.
 
 **File format:**
 
@@ -706,9 +747,46 @@ During `dbo clone`, the CLI auto-generates `.dbo/metadata_templates.json` — a
 | `Name` | Plain column — populated from filename or left empty |
 | `Descriptor=documentation` | Literal value — always set to `documentation` |
 | `String10=@reference` | Content reference — links to the file being added |
+| `String5=@reference:@Name[js]` | Content reference with filename source and extension |
 
 `dbo add` uses these templates to auto-detect the entity type from a file's directory placement and generate metadata without prompting. For example, `dbo add extension/include/nav.html` resolves to entity `extension` / descriptor `include` and applies the matching template.
 
+#### `@reference` expression syntax
+
+The `=@reference` expression controls how companion files are named during `dbo clone` and `dbo add`. The full grammar:
+
+```
+Column=@reference[:filenameSource][extPart][{title}]
+```
+
+**Components:**
+
+| Part | Syntax | Description |
+|------|--------|-------------|
+| Filename source | `:@ColumnName` | Use another column's value as the base filename (e.g., `:@Name`) |
+| Filename fallback | `:@Name\|\|fallback` | Use `fallback` if the column value is empty |
+| Extension | `[ext]` | Literal extension (e.g., `[js]`, `[css]`, `[html]`) |
+| Extension from column | `[@ColumnName]` | Read extension from another column (e.g., `[@Extension]`) |
+| Extension fallback | `[@Extension\|\|html]` | Use `html` if the column value is empty |
+| Title | `{Title Text}` | Human-readable name for the companion file segment (replaces column name) |
+
+**Examples:**
+
+Given a record with `Name="My-Widget"`, `Extension="html"`, `Ext=""` (empty):
+
+| Expression | Companion filename | Description |
+|---|---|---|
+| `Content=@reference` | `My-Widget.Content.txt` | Basic — column name as segment, default ext |
+| `Content=@reference:@Name[@Extension]` | `My-Widget.html` | Name and extension both from record columns |
+| `Content=@reference:@Name\|\|untitled[@Extension]` | `My-Widget.html` | Uses Name; would use `untitled` if Name were empty |
+| `String5=@reference:@Name[js]` | `My-Widget.String5.js` | Literal `.js` extension |
+| `String5=@reference:@Name[js]{Add Form Parameters}` | `My-Widget.Add-Form-Parameters.js` | Title replaces `String5` in filename |
+| `CustomSQL=@reference[sql]` | `My-Widget.CustomSQL.sql` | No filename source, literal extension |
+| `Text=@reference:@Name[@Ext\|\|js]` | `My-Widget.Text.js` | `Ext` column is empty, falls back to `.js` |
+| `Text=@reference:@Name[@Extension\|\|js]` | `My-Widget.Text.html` | `Extension` column has value, fallback not used |
+
+**Auto-generation from `descriptor_definition`**: During `dbo clone`, the CLI parses `form-control-code` from `descriptor_definition` extension records to auto-populate `@reference` entries per descriptor. If `form-control-title` is also present, the column's title is included as the `{title}` suffix — producing human-readable companion filenames instead of raw column names like `String5`.
+
 ---
 
 ### `dbo login`
@@ -778,7 +856,7 @@ Plugin scopes (project or global) are displayed when plugins have been installed
 
 ### `dbo input`
 
-Submit CRUD operations (add, edit, delete records) to DBO.io. This is the core data manipulation command.
+Submit raw CRUD operations (add, edit, delete records) to DBO.io. This is a low-level command that sends input expressions directly to the server without any AppID injection or metadata awareness — use `dbo push` for metadata-driven deployments.
 
 ```bash
 # Add a record
@@ -1285,7 +1363,7 @@ After a pull, metadata files contain `@filename` references for content columns:
 ```json
 {
   "_entity": "content",
-  "_contentColumns": ["Content"],
+  "_companionReferenceColumns": ["Content"],
   "UID": "abc123",
   "Name": "colors",
   "Content": "@colors.css",
@@ -1481,7 +1559,7 @@ Minimal (no optional fields provided):
   "Path": "assets/css/colors.css",
   "Content": "@colors.css",
   "_entity": "content",
-  "_contentColumns": ["Content"]
+  "_companionReferenceColumns": ["Content"]
 }
 ```
 
@@ -1495,7 +1573,7 @@ With optional fields (only included if the user provides values during the wizar
   "Path": "assets/css/colors.css",
   "Content": "@colors.css",
   "_entity": "content",
-  "_contentColumns": ["Content"]
+  "_companionReferenceColumns": ["Content"]
 }
 ```
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dboio/cli",
-  "version": "0.15.2",
+  "version": "0.16.2",
   "description": "CLI for the DBO.io framework",
   "type": "module",
   "bin": {

package/plugins/claude/dbo/docs/dbo-cli-readme.md

@@ -134,8 +134,7 @@ my-project/
 │   ├── entity_column_value/ # Column value definitions
 │   ├── extension/      # Extension entity records (organized by descriptor type)
 │   │   ├── widget/
-│   │   ├── documentation/
-│   │   └── _unsupported/
+│   │   └── documentation/
 │   ├── integration/    # Integration entity records
 │   ├── security/       # Security policy records
 │   ├── security_column/ # Column-level security records
@@ -148,6 +147,7 @@ my-project/
 ├── trash/              # Staged soft-deleted files from dbo rm
 ├── docs/               # Project documentation and docs entities
 ├── app.json            # Clone of original app JSON with @references
+├── schema.json         # Instance-level entity/column schema fetched from the server during `dbo init`. Refreshed explicitly with `dbo clone --schema`.
 ├── manifest.json       # PWA web app manifest (auto-generated from app metadata)
 ├── .gitignore          # Tells Git to ignore files in the repo sync
 ├── .dboignore          # Tells dbo cli to ignore files in commands
@@ -178,7 +178,7 @@ All configuration is **directory-scoped**. Each project folder maintains its own
 | `credentials.json` | Username, user ID, UID, name, email (no password) | Gitignored (per-user) |
 | `cookies.txt` | Session cookie (Netscape format) | Gitignored (per-user) |
 | `structure.json` | Bin directory mapping (created by `dbo clone`) | Committable (shared) |
-| `metadata_templates.json` | Column templates per entity/descriptor (auto-generated by `dbo clone`) | Committable (shared) |
+| `metadata_schema.json` | Column templates per entity/descriptor (auto-generated by `dbo clone`; renamed from `metadata_templates.json`) | Committable (shared) |
 | `synchronize.json` | Pending deletions and sync state (updated by `dbo rm` and `dbo push`) | Committable (shared) |
 | `.app_baseline.json` | Server-state snapshot for delta detection — stores column values at last clone/push so `dbo push` can detect which columns changed locally. Read-only (chmod 444); auto-migrated from legacy root `.app.json`. | Committable (shared) |
 | `scripts.json` | Build/push lifecycle hooks (see [Script Hooks](#script-hooks)) | Committable (shared) |
@@ -201,6 +201,8 @@ All configuration is **directory-scoped**. Each project folder maintains its own
 > | Output | `Sales.metadata~ghi789.json` |
 > | Extension | `MyExtension.metadata~jkl012.json` |
 
+> **Upgrading to 0.16.0+**: `.dbo/metadata_templates.json` is renamed to `.dbo/metadata_schema.json`. The `_contentColumns` field in metadata files is renamed to `_companionReferenceColumns`. Extension companion file prompts during `dbo clone` are replaced by automatic derivation from `descriptor_definition` `form-control-code` data. Migration 011 runs automatically on first use.
+
 #### Automatic migrations
 
 When the CLI version is upgraded, one-time migration scripts in `tools/cli/src/migrations/` run automatically on the first command invocation in a project directory. Completed migration IDs are recorded in `.dbo/config.local.json` under `_completedMigrations` (per-user, gitignored) so each developer runs them independently and they never re-fire.
@@ -241,9 +243,37 @@ Use `dbo status` to see how many pending migrations exist.
 | `cloneSource` | `"default"` \| file path \| URL | Where the last `dbo clone` fetched app JSON from. `"default"` = server fetch via `AppShortName`; any other value = the explicit local file path or URL used. Set automatically after each successful clone. |
 | `ContentPlacement` | `bin` \| `path` | Where to place content files during clone (default: `bin`) |
 | `<Entity>FilenameCol` | column name | Filename column for entity-dir records (e.g., `ExtensionFilenameCol`) |
+| `dependencies` | string[] | Array of app short-names to auto-clone as read-only local checkouts. Default: `["_system"]` |
+| `dependencyLastUpdated` | object | Map of dependency short-name to last-cloned `_LastUpdated` ISO string |
 
 **File placement:** All content, media, and output files are placed by BinID into the `lib/bins/` directory structure. Records without a BinID go into `lib/bins/` root. Media always uses BinID — no FullPath option. Bins with `Name=null` (legacy) map directly to `lib/bins/` root.
 
+### App Dependencies
+
+The CLI automatically maintains local read-only checkouts of related apps under `.dbo/dependencies/<shortname>/`. By default, the `_system` app (which contains all entity, column, and extension descriptors for the connected instance) is always included.
+
+#### How it works
+
+- After `dbo init` or `dbo clone`, dependency apps are cloned into `.dbo/dependencies/`.
+- If an `app.json` contains a `Dependencies` column, those app short-names are merged into `.dbo/config.json` and also cloned.
+- You can add dependencies explicitly with `dbo clone --dependencies operator,launchpad`.
+- Dependencies are only re-cloned when the server's `_LastUpdated` is newer than the locally stored timestamp — so subsequent runs are fast.
+- Dependency clone output is suppressed — a single summary line shows synced/failed/up-to-date status.
+- `.dbo/dependencies/` is excluded from `.gitignore` and `.dboignore` automatically.
+
+#### Schema merging from dependencies
+
+After dependency cloning, extension descriptor definitions from dependency schemas (`.dbo/dependencies/<name>/.dbo/metadata_schema.json`) are merged into the local `.dbo/metadata_schema.json`. This is useful when an app has no `descriptor_definition` extensions of its own but depends on an app (like `operator`) that does — the operator's descriptor definitions (widget, include, control, etc.) become available for local extension processing.
+
+#### Flags
+
+| Flag | Commands | Effect |
+|------|----------|--------|
+| `--no-deps` | `dbo init`, `dbo clone` | Skip dependency syncing |
+| `--dependencies <list>` | `dbo clone` | Add and sync specific dependencies (comma-separated short-names) |
+| `--force` | `dbo clone` | Re-clone all dependencies regardless of staleness |
+| `--schema` | `dbo clone` | Re-clone all dependencies regardless of staleness |
+
 ### Root-level project files
 
 #### `app.json`
@@ -367,6 +397,7 @@ dbo init --scaffold --yes                             # scaffold dirs non-intera
 | `--dboignore` | Create `.dboignore` with default patterns (use with `--force` to overwrite existing) |
 | `-y, --yes` | Skip all interactive prompts (legacy migration, Claude Code setup) |
 | `--non-interactive` | Alias for `--yes` |
+| `--no-deps` | Skip dependency cloning after init |
 
 ---
 
@@ -402,9 +433,14 @@ dbo clone -e extension --descriptor-types false   # Clone extensions flat (no de
 | `--documentation-only` | When used with `-e extension`, clone only documentation extensions |
 | `--descriptor-types <bool>` | Sort extensions into descriptor sub-directories (default: `true`). Set to `false` to use flat `extension/` layout |
 | `--domain <host>` | Override domain. Triggers a domain-change confirmation prompt when it differs from the project reference domain |
+| `--schema` | Re-fetch `schema.json` from server before cloning |
 | `--force` | Skip source mismatch confirmation and change detection; re-processes all files |
 | `-y, --yes` | Auto-accept all prompts (also skips source mismatch confirmation) |
 | `-v, --verbose` | Show HTTP request details |
+| `--no-deps` | Skip dependency cloning after clone |
+| `--dependencies <list>` | Comma-separated app short-names to add and sync as dependencies (e.g. `--dependencies operator,launchpad`) |
+| `--configure` | Re-prompt for filename columns, companion file extraction, and documentation placement preferences |
+| `--media-placement <type>` | Set media placement: `fullpath` or `binpath` (default: `bin`) |
 
 #### Domain change detection
 
@@ -414,17 +450,19 @@ The project's reference domain is stored in `app.json._domain` (committed to git
 
 #### What clone does
 
-1. **Loads app JSON** — from a local file, server API, or interactive prompt. A spinner shows progress while fetching from the server (responses can be slow as the JSON is assembled on demand)
-2. **Updates `.dbo/config.json`** — saves `AppID`, `AppUID`, `AppName`, `AppShortName`, `AppModifyKey` (if the app is locked), and `cloneSource` (the source used for this clone)
-3. **Updates `package.json`** — populates `name`, `productName`, `description`, `homepage`, and `deploy` script
-4. **Creates directories** — processes `children.bin` to build the directory hierarchy based on `ParentBinID` relationships
-5. **Saves `.dbo/structure.json`** — maps BinIDs to directory paths for file placement
-6. **Writes content files** — decodes base64 content, creates `*.metadata.json` + content files in the correct bin directory. When a record's `Extension` field is empty, prompts you to choose from `css`, `js`, `html`, `xml`, `txt`, `md`, `cs`, `json`, `sql` (or skip to keep no extension). The chosen extension is saved back into the `metadata.json` `Extension` field
-7. **Downloads media files** — fetches binary files (images, CSS, fonts) from the server using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`, and saves with metadata. Errors are logged to `.dbo/errors.log`
-8. **Processes entity-dir records** — entities matching project directories (`extension`, `app_version`, `data_source`, `site`, `group`, `integration`, `automation`) are saved as `.metadata.json` files in their corresponding directory (e.g., `extension/`, `data_source/`)
-9. **Processes other entities** — remaining entities with a `BinID` are placed in the corresponding bin directory
-10. **Saves `app.json`** — clone of the original JSON with processed entries replaced by `@path/to/*.metadata.json` references
-11. **Orphan cleanup** — any local `.metadata.json` files whose UID is absent from the server response are automatically moved to `trash/` along with their companion content and media files. This prevents stale records (deleted server-side) from causing false positives in `dbo push`. Skipped during `--entity-filter` clones
+1. **Fetches schema** — downloads `schema.json` from the server if missing or explicitly requested (`--schema`). Regenerates `.dbo/metadata_schema.json` from the schema
+2. **Syncs dependencies** — clones dependency apps into `.dbo/dependencies/<shortname>/` (unless `--no-deps`). Merges extension descriptor definitions from dependency schemas into the local `.dbo/metadata_schema.json` (e.g., operator provides descriptor definitions for apps that lack their own)
+3. **Loads app JSON** — from a local file, server API, or interactive prompt. A spinner shows progress while fetching from the server (responses can be slow as the JSON is assembled on demand)
+4. **Updates `.dbo/config.json`** — saves `AppID`, `AppUID`, `AppName`, `AppShortName`, `AppModifyKey` (if the app is locked), and `cloneSource` (the source used for this clone)
+5. **Updates `package.json`** — populates `name`, `productName`, `description`, `homepage`, and `deploy` script
+6. **Creates directories** — processes `children.bin` to build the directory hierarchy based on `ParentBinID` relationships
+7. **Saves `.dbo/structure.json`** — maps BinIDs to directory paths for file placement
+8. **Writes content files** — decodes base64 content, creates `*.metadata.json` + content files in the correct bin directory. Filename columns and companion file extraction preferences are auto-applied with sensible defaults on first clone (use `--configure` to re-prompt)
+9. **Downloads media files** — fetches binary files (images, CSS, fonts) from the server using a fallback chain: `FullPath` directly (`/media/{app}/{path}`) → `/dir/` route → `/api/media/{uid}`, and saves with metadata. 404 errors create stale metadata to prevent re-prompting. Errors are logged to `.dbo/errors.log`
+10. **Processes entity-dir records** — entities matching project directories (`extension`, `app_version`, `data_source`, `site`, `group`, `integration`, `automation`) are saved as `.metadata.json` files in their corresponding directory (e.g., `extension/`, `data_source/`)
+11. **Processes other entities** — remaining entities with a `BinID` are placed in the corresponding bin directory
+12. **Saves `app.json`** — clone of the original JSON with processed entries replaced by `@path/to/*.metadata.json` references
+13. **Orphan cleanup** — any local `.metadata.json` files whose UID is absent from the server response are automatically moved to `trash/` along with their companion content and media files. This prevents stale records (deleted server-side) from causing false positives in `dbo push`. Skipped during `--entity-filter` clones
 
 #### Clone source tracking
 
@@ -454,6 +492,8 @@ If the initial clone attempt fails (network error, file not found, empty respons
 
 When cloning an app that was already cloned locally, the CLI detects existing files and compares modification times against the server's `_LastUpdated`. You'll be prompted to overwrite, compare differences, or skip — same as `dbo pull`. Use `-y` to auto-accept all changes.
 
+Change detection only checks companion content and media files for user edits — metadata file mtimes are not considered, since they can be bumped by CLI-internal operations (migrations, legacy companion renames).
+
 #### Collision detection
 
 When multiple records would create files at the same path (e.g., a `content` record and a `media` record both named `colors.css`), the CLI detects the collision before writing any files and prompts you to choose which record to keep:
@@ -471,7 +511,9 @@ In non-interactive mode (`-y`), the first record is kept and others are auto-sta
 
 #### Stale media cleanup
 
-During media downloads, files returning 404 (no longer exist on the server) are collected as "stale records". After all downloads complete, you'll be prompted to stage them for deletion:
+During media downloads, files returning 404 (no longer exist on the server) are collected as "stale records". Metadata is still written for 404 files (marked with `_stale404: true`) so they are not re-prompted on subsequent clones.
+
+After all downloads complete, you'll be prompted to stage stale records for deletion:
 
 ```
 Found 3 stale media record(s) (404 - files no longer exist on server)
@@ -507,9 +549,9 @@ Entity types that correspond to project directories (`extension`, `app_version`,
 
 For each entity type, the CLI prompts to choose which column becomes the filename (defaults to `Name`, fallback `UID`). The choice is saved per entity type in `config.json` (e.g., `ExtensionFilenameCol`).
 
-If any columns contain base64-encoded content, the CLI prompts to extract them as companion files. Extracted columns produce files named `<name>.<Column>.<ext>` alongside the metadata, with `@reference` entries in the metadata and a `_contentColumns` array.
+If any columns contain base64-encoded content, the CLI prompts to extract them as companion files. Extracted columns produce files named `<name>.<Column>.<ext>` alongside the metadata, with `@reference` entries in the metadata and a `_companionReferenceColumns` array (legacy name: `_contentColumns`, supported for backward compatibility).
 
-**Companion file naming convention:** Only `.metadata.json` files carry the `~UID` suffix (e.g. `Add-Asst-Execute-Security~wxl6ivcwfkix3zgantszjg.metadata.json`). Companion content files use the natural base name without `~UID` (e.g. `Add-Asst-Execute-Security.String16.js`). If two records share the same name within a directory, the second gets a `-1` suffix (e.g. `Add-Asst-Execute-Security-1.String16.js` with metadata `Add-Asst-Execute-Security-1~otheruid.metadata.json`). Migration 007 automatically renames legacy `~UID` companion files to the natural convention.
+**Companion file naming convention:** Only `.metadata.json` files carry the `~UID` suffix (e.g. `Add-Asst-Execute-Security.metadata~wxl6ivcwfkix3zgantszjg.json`). Companion content files use the natural base name without `~UID` (e.g. `Add-Asst-Execute-Security.String5.html`). When a `{title}` is defined in the `@reference` expression (see [Metadata Schema](#metadata-schema)), the title replaces the column name in the filename (e.g. `Add-Asst-Execute-Security.Add-Form-Parameters.js` instead of `Add-Asst-Execute-Security.String5.js`). If two records share the same name within a directory, the second gets a `-1` suffix. Migration 007 automatically renames legacy `~UID` companion files to the natural convention.
 
 Use `-y` to skip prompts (uses `Name` column, no content extraction).
 
@@ -521,7 +563,7 @@ During clone, the CLI:
 
 1. **Pre-scans** all extension records for `descriptor_definition` entries and builds a `String1 → Name` mapping
 2. **Creates sub-directories** under `extension/` for each mapped descriptor (e.g., `extension/Documentation/`, `extension/Includes/`)
-3. **Always creates** `extension/_unsupported/` — extensions with an unmapped or null `Descriptor` are placed here
+3. **Places unmapped extensions** directly in `extension/` — extensions with an unmapped or null `Descriptor` are placed in the root `extension/` directory (the legacy `_unsupported/` sub-directory is migrated automatically by Migration 011)
 4. **Prompts per descriptor** — filename column and content extraction prompts fire once per unique `Descriptor` value (not once for all extensions)
 5. **Persists the mapping** in `.dbo/structure.json` under `descriptorMapping`
 
@@ -575,7 +617,6 @@ project/
       MyDoc.uid1.String10.md     # ← extracted content column
     Includes/
       Header.uid2.metadata.json
-    _unsupported/            # ← always created (unmapped/null descriptors)
   docs/                      # ← root-placed documentation MD files (optional)
     MyDoc.md
   data_source/
@@ -683,9 +724,9 @@ Key points:
 - `CustomSQL` content is extracted to companion `.sql` files per extraction rules
 - Re-cloning moves orphaned old-format child `.json` files to `/trash`
 
-#### Metadata Templates
+#### Metadata Schema
 
-During `dbo clone`, the CLI auto-generates `.dbo/metadata_templates.json` — a file that records which columns each entity/descriptor uses. This file is seeded from the first cloned record of each type and can be manually edited afterwards.
+During `dbo clone`, the CLI auto-generates `.dbo/metadata_schema.json` (formerly `metadata_templates.json`) — a file that records which columns each entity/descriptor uses. This file is seeded from the first cloned record of each type and can be manually edited afterwards.
 
 **File format:**
 
@@ -706,9 +747,46 @@ During `dbo clone`, the CLI auto-generates `.dbo/metadata_templates.json` — a
 | `Name` | Plain column — populated from filename or left empty |
 | `Descriptor=documentation` | Literal value — always set to `documentation` |
 | `String10=@reference` | Content reference — links to the file being added |
+| `String5=@reference:@Name[js]` | Content reference with filename source and extension |
 
 `dbo add` uses these templates to auto-detect the entity type from a file's directory placement and generate metadata without prompting. For example, `dbo add extension/include/nav.html` resolves to entity `extension` / descriptor `include` and applies the matching template.
 
+#### `@reference` expression syntax
+
+The `=@reference` expression controls how companion files are named during `dbo clone` and `dbo add`. The full grammar:
+
+```
+Column=@reference[:filenameSource][extPart][{title}]
+```
+
+**Components:**
+
+| Part | Syntax | Description |
+|------|--------|-------------|
+| Filename source | `:@ColumnName` | Use another column's value as the base filename (e.g., `:@Name`) |
+| Filename fallback | `:@Name\|\|fallback` | Use `fallback` if the column value is empty |
+| Extension | `[ext]` | Literal extension (e.g., `[js]`, `[css]`, `[html]`) |
+| Extension from column | `[@ColumnName]` | Read extension from another column (e.g., `[@Extension]`) |
+| Extension fallback | `[@Extension\|\|html]` | Use `html` if the column value is empty |
+| Title | `{Title Text}` | Human-readable name for the companion file segment (replaces column name) |
+
+**Examples:**
+
+Given a record with `Name="My-Widget"`, `Extension="html"`, `Ext=""` (empty):
+
+| Expression | Companion filename | Description |
+|---|---|---|
+| `Content=@reference` | `My-Widget.Content.txt` | Basic — column name as segment, default ext |
+| `Content=@reference:@Name[@Extension]` | `My-Widget.html` | Name and extension both from record columns |
+| `Content=@reference:@Name\|\|untitled[@Extension]` | `My-Widget.html` | Uses Name; would use `untitled` if Name were empty |
+| `String5=@reference:@Name[js]` | `My-Widget.String5.js` | Literal `.js` extension |
+| `String5=@reference:@Name[js]{Add Form Parameters}` | `My-Widget.Add-Form-Parameters.js` | Title replaces `String5` in filename |
+| `CustomSQL=@reference[sql]` | `My-Widget.CustomSQL.sql` | No filename source, literal extension |
+| `Text=@reference:@Name[@Ext\|\|js]` | `My-Widget.Text.js` | `Ext` column is empty, falls back to `.js` |
+| `Text=@reference:@Name[@Extension\|\|js]` | `My-Widget.Text.html` | `Extension` column has value, fallback not used |
+
+**Auto-generation from `descriptor_definition`**: During `dbo clone`, the CLI parses `form-control-code` from `descriptor_definition` extension records to auto-populate `@reference` entries per descriptor. If `form-control-title` is also present, the column's title is included as the `{title}` suffix — producing human-readable companion filenames instead of raw column names like `String5`.
+
 ---
 
 ### `dbo login`
@@ -778,7 +856,7 @@ Plugin scopes (project or global) are displayed when plugins have been installed
 
 ### `dbo input`
 
-Submit CRUD operations (add, edit, delete records) to DBO.io. This is the core data manipulation command.
+Submit raw CRUD operations (add, edit, delete records) to DBO.io. This is a low-level command that sends input expressions directly to the server without any AppID injection or metadata awareness — use `dbo push` for metadata-driven deployments.
 
 ```bash
 # Add a record
@@ -1285,7 +1363,7 @@ After a pull, metadata files contain `@filename` references for content columns:
 ```json
 {
   "_entity": "content",
-  "_contentColumns": ["Content"],
+  "_companionReferenceColumns": ["Content"],
   "UID": "abc123",
   "Name": "colors",
   "Content": "@colors.css",
@@ -1481,7 +1559,7 @@ Minimal (no optional fields provided):
   "Path": "assets/css/colors.css",
   "Content": "@colors.css",
   "_entity": "content",
-  "_contentColumns": ["Content"]
+  "_companionReferenceColumns": ["Content"]
 }
 ```
 
@@ -1495,7 +1573,7 @@ With optional fields (only included if the user provides values during the wizar
   "Path": "assets/css/colors.css",
   "Content": "@colors.css",
   "_entity": "content",
-  "_contentColumns": ["Content"]
+  "_companionReferenceColumns": ["Content"]
 }
 ```