everything-dev 2.0.0-rc.7 → 2.0.0-rc.9

package/skills/init-upgrade/SKILL.md

@@ -1,13 +1,13 @@
 ---
 name: init-upgrade
-description: bos init, bos sync, and bos upgrade workflows — template download, snapshot-based conflict detection, package version bumps, and how init/sync select and own files. Use when scaffolding new projects, syncing upstream changes, or upgrading framework packages.
+description: bos init, bos sync, bos import, and bos upgrade workflows — template download, snapshot-based conflict detection, workspace import from upstream repos with divergence reporting, package version bumps, and how init/sync select and own files. Use when scaffolding new projects, syncing upstream changes, importing a workspace subtree from another repo, or upgrading framework packages.
 metadata:
-  sources: "packages/everything-dev/src/cli/init.ts,packages/everything-dev/src/cli/sync.ts,packages/everything-dev/src/cli/upgrade.ts,packages/everything-dev/src/cli/snapshot.ts,packages/everything-dev/src/cli.ts,packages/everything-dev/src/merge.ts"
+  sources: "packages/everything-dev/src/cli/init.ts,packages/everything-dev/src/cli/sync.ts,packages/everything-dev/src/cli/import.ts,packages/everything-dev/src/commands/import.ts,packages/everything-dev/src/cli/upgrade.ts,packages/everything-dev/src/cli/snapshot.ts,packages/everything-dev/src/cli.ts,packages/everything-dev/src/merge.ts"
 ---
 
 > **Config form:** the authored config is `bos.app.ts` (canonical, preferred when both exist). A legacy `bos.config.json` is still supported for older children. Where this doc says `bos.config.json` for the *local authored file*, read "the authored config". The published artifact on FastKV keeps the key name `bos.config.json`.
 
-# bos init, sync, upgrade
+# bos init, sync, import, upgrade
 
 ## bos init
 
@@ -182,6 +182,76 @@ bos upgrade --dry-run    # Preview without making changes
 
 Workspace packages use `catalog:` refs so a single version bump in root catalog propagates everywhere. Skips `workspace:*` and `file:` refs.
 
+## bos import
+
+Imports a workspace subtree (e.g. `plugins/auth`) from an upstream GitHub repo or a local directory — no clone of the full history, just a tarball (or direct local read) of one ref. The upstream can also be resolved through the authored config's `extends` chain (a bos:// URL centric source — see below):
+
+```bash
+bos import plugins/auth                          # resolved through the extends chain
+bos import plugins/auth --from bos://base.near/base
+bos import plugins/auth --from NEARBuilders/everything-dev
+bos import plugins/auth --from NEARBuilders/everything-dev@v2.1.0
+bos import plugins/auth --from NEARBuilders/everything-dev@everything-dev@2.0.0-rc.8
+bos import plugins/auth --from https://github.com/owner/repo@develop
+bos import plugins/auth --from ./local-checkout --dry-run
+bos import plugins/auth --from owner/repo --yes
+```
+
+**Flags:** `--from <bos://<account>/<gateway> | owner/repo[@ref] | url | dir>` (optional — resolve through the extends chain when omitted), `--yes`, `--dry-run`, `--no-install`, `--json`. The GitHub ref splits on the **first** `@`, so changesets-style tags keep their own `@`.
+
+### Source resolution
+
+`--from` takes three shapes:
+
+- **`bos://<account>/<gateway>`** — the config published to FastKV by that runtime. The extends chain of that runtime is walked outward to the **nearest ancestor whose own config declares the workspace locally** (`local:<workspace-path>`); the tarball comes from that ancestor's `repository` field (continuing rootward when the declaring runtime itself lacks one).
+- **omitted** — same chain walk, starting from the authored `bos.app.ts` (v2 config only). Errors when the workspace is declared locally in the child (you own it — nothing to import), when no runtime in the chain declares the workspace, or when no ancestor along the chain carries a `repository`.
+- **`<owner/repo>[@ref]`, GitHub URL, or local dir** — explicit source, unchanged.
+
+**Ref pinning:** the ancestor's published config pins each workspace to an immutable version manifest. When the ancestor publishes source archives (`publish.sourcePublish` on its authored config, ADR 0030), the import **byte-sources** from the pinned archive: the manifest itself is SRI-verified against the config slot pin, the archive URL + server SRI come from the manifest's `source` field, the bytes are SRI-verified and extracted (repo-root-relative — a drop-in for the GitHub tarball root), and no GitHub dependency exists (the parent's `repository` may be absent or private). If archive fetch fails or its SRI mismatches, the import falls back to the chain-resolved GitHub coordinate with a `Note:`. When the manifest records no `source`, GitHub remains the source: a recorded `git sha` pins the exact deployed commit; otherwise the import falls back to the **default branch (HEAD)** with a `Note:` line. The report's Source section shows both layers: the `bos:// …` extends ref, the source archive URL when used, and the dereferenced `repository @ ref`.
+
+### Classification
+
+Each file shared between the local workspace and the upstream subtree is classified against a baseline: `.bos/import-snapshot.json` (written by previous imports), falling back to `.bos/sync-snapshot.json`.
+
+| Bucket | Meaning | Action |
+|---|---|---|
+| added | upstream-only | written |
+| unchanged | identical content | nothing (baseline recorded) |
+| updated | local matches baseline, upstream moved | overwritten with upstream (backed up first) |
+| preserved-local | local-only, upstream lacks it | never deleted |
+| preserved-local-edit | upstream matches baseline, local edited | local wins, untouched |
+| merged | diverged `package.json` | structured merge — template wins shared keys, local-only keys and deps preserved |
+| conflict | both changed since baseline, or no baseline exists | gated |
+
+Local-only files are never deleted, even when upstream no longer carries them.
+
+### Gating
+
+Conflicts — and a migration chain that cannot be applied verbatim (next section) — gate the import:
+
+- **Non-TTY without `--yes`:** nothing is written, the report is emitted, exit 1. `--yes` applies upstream over conflicting files (local copies backed up to `.bos/import-backup/<timestamp>/`) and proceeds.
+- **TTY:** an interactive confirm (default No; declining exits 1).
+
+A diverged `package.json` is never a conflict when both sides parse — it merges instead.
+
+### Migration chain reconciliation
+
+When both sides have `src/db/migrations/` journals, the chain is **reconciled** instead of overwritten:
+
+- **Applied entries win.** Every local journal entry is presumed applied to the database and is kept verbatim — never renumbered or rehashed.
+- **Colliding upstream migrations are renumbered and appended.** An upstream entry that shares an idx or lands below the local tail but is not present locally is renumbered after the local tail in upstream order: upstream `0011_team-member-role` with a local tail at idx 12 becomes `0013_team-member-role`. The `.sql` file is copied to the new name, `meta/<idx>_snapshot.json` is renamed to match, and the migration journal under `meta` (`_journal.json`) is rebuilt (local entries verbatim + appended entries carrying upstream's `when`/`version`).
+- **Content matching keeps repeat imports convergent.** An upstream entry whose SQL content hash matches a local entry (e.g. one imported under a renumbered tag earlier) is treated as already imported — nothing is appended twice.
+- **Report.** The import report and console output show the kept set (applied), the appended set (`fromTag → toTag`, new idx), and what the next boot will apply versus what is already applied. Renamed snapshots record upstream intermediate schema states — review `drizzle-kit generate` output against the merged chain before generating new migrations.
+- **Held back.** Only unresolvable drift holds the chain now: a shared tag whose SQL content differs locally (already-applied hashes would drift), or an unreadable journal. Held chain files are not written; the rest of the workspace imports normally. A held chain gates the run even when no other file conflicts.
+
+### Route-set delta
+
+Both sides' contract surface is scanned for literal `.route({ method, path })` pairs — the monolithic `<workspace>/src/contract.ts` plus any split `<workspace>/src/contract/**` files, unioned per side (so a monolithic local contract compares correctly against a re-export + split-file upstream). The report lists ours-only and theirs-only `METHOD path` routes so API drift is visible before you port local routes onto the new upstream contract.
+
+### Report and baselines
+
+Every run — dry-run, refused, or applied — writes `.bos/import-report.md`: summary table, conflict checklist with reasons and held markers, the migration-chain reconciliation plan (kept/appended/held), route delta, bucket lists, and next steps. Written and unchanged files are recorded to `.bos/import-snapshot.json` (and merged into `.bos/sync-snapshot.json` when one exists) so the next import detects drift instead of starting from zero.
+
 ## bos publish
 
 See `everything-dev#publish-sync` for the full publish workflow. A quick reference:
@@ -206,6 +276,8 @@ Unknown keys go after known keys. See `everything-dev#extends-config` for full o
 
 **"Sync reports conflicted files."** Your local edits diverged from the snapshot; upgrade applies the template version and leaves your copy in `.bos/sync-backup/` — diff and merge back, or `bos sync --force` deliberately.
 
+**"Import a workspace from the base platform."** `bos import <workspace> --dry-run` (defaults to the extends chain; or `--from <owner/repo>` for an explicit upstream) first, read `.bos/import-report.md` for the conflict/migration/route picture, then real-run: plain invoke prompts in a TTY, `--yes` applies upstream over conflicts (backed up), and a flagged migration chain stays held until reconciliation.
+
 **"Upgrade framework packages."** `bos upgrade --dry-run` first, then `bos upgrade` — it bumps the root catalog, rewrites workspace refs to `catalog:`, syncs the template, and rewrites legacy imports.
 
 ## What comes back when it refuses
@@ -214,5 +286,6 @@ Unknown keys go after known keys. See `everything-dev#extends-config` for full o
 |---|---|
 | file listed as `conflicted` in sync results | local file modified since the snapshot AND the template changed — take the template version (your copy is backed up at `.bos/sync-backup/`) or re-apply your change on top |
 | sync keeps skipping my file | app-owned files that diverged from the snapshot are skipped without `--force`; framework-owned files always update when the template changes |
+| import exits 1 listing files | unresolvable conflicts (or a held migration chain) — read `.bos/import-report.md`, then re-run with `--yes` to apply upstream (local copies land in `.bos/import-backup/`), or port local changes manually first |
 | `Circular extends detected while resolving upgrade source` | the upgrade source's extends chain loops — resolve from a non-cyclic parent |
 | init copies fewer files than expected | `buildInitPatterns` filters to the selected plugins + their routes; pass `--overrides ui,api,host` to widen the scaffold |

package/skills/talk-to-the-app/SKILL.md

@@ -36,19 +36,46 @@ There is no network parameter anywhere — the origin is the choice.
 
 ## Authentication
 
-All API operations require authentication. Use an **API key**:
+Every API surface — `/api/*` (REST), `/api/rpc/*` (oRPC RPC), and `/api/mcp`
+(MCP) — requires a session or an API key. Anonymous requests get `401` with a
+`WWW-Authenticate` hint (this includes MCP `tools/list`: the tool catalog is
+not disclosed to unauthenticated callers). Use an **API key**:
+
+**Path A — create one in the app UI.**
 
 1. Sign in with your NEAR wallet (SIWN) at the app.
 2. Navigate to **Settings → API Keys** at `/settings/api-keys`.
 3. Create a new key — the full secret (`api_...`) is shown once, copy it immediately.
-4. Pass it on every request:
+
+**Path B — no UI needed: `bos login`** (for CLI sessions and agents with a
+browser available):
+
+```bash
+bos login
+```
+
+Opens a browser, you approve once (NEAR SIWN), and the CLI persists a
+device-named API key in `.bos/session.json` (gitignored, 0600). Same key, same
+header — pass it on every request:
 
 ```
 x-api-key: api_your_key_here
 ```
 
-The header works for `/api/*` (REST), `/api/rpc/*` (oRPC RPC), and `/api/mcp`
-(MCP). Keys carry optional permissions; a call outside them refuses with 403.
+Keys carry optional permissions; a call outside them refuses with 403. Keys
+expire (90 days by default) and are revocable at `/settings/api-keys` —
+re-run `bos login` (or create a fresh key in the UI) when a key dies.
+
+**Other credentials:**
+
+- `bos login --key` mints a publish-scoped function-call access key
+  (`~/.near-credentials`) for the publish path — not an API-key replacement.
+- Device-link session tokens (`POST /device/token` +
+  `POST /device-link/claim`, one-time hashed claim) yield a real Better Auth
+  session that also passes the gate — cookie transport, no revocation UI.
+  For MCP and agent work, prefer the API key.
+- For CI or a browser-less agent: create a key once (UI or `bos login` from a
+  machine with a browser) and pass it as `x-api-key` in the pipeline.
 
 ## MCP
 
@@ -128,7 +155,7 @@ with write permission.
 
 | status | do |
 |---|---|
-| `401` | no or bad key — re-read `/settings/api-keys`; check the `x-api-key` header reached the request |
+| `401` | no key sent, a bad/revoked key, or an anonymous caller — if you have a key, check the `x-api-key` header reached the request; if not, get one (Paths A/B above); MCP `tools/list` is also gated |
 | `403` | the key lacks permission — the owner edits the key's permissions; do not retry unchanged |
 | `404` | wrong surface or path — re-check `/api/spec.json`; plugin RPC mounts are `/api/rpc/<pluginKey>` |
 | `429` | rate limited — wait as the response says, then once more |