inferative 0.2.0-next.20261023154401 → 0.2.0

package/README.md

@@ -2,7 +2,8 @@
 
 The Inferative command line. It is on public npm, ships built JavaScript and runs under plain `node` (version 24 or
 later). It has the command `init`, which creates a project, the group `site`: the commands a website runs against
-its channel, the group `backup`: one archive of a project's database and files, its restore and its check, and the group `plugin`: the MCP server and the skill of a coding agent. To start a project, run `npm create inferative@latest my-site`: that command needs no registry setting.
+its channel (its import of entries and media included), the group `backup`: one archive of a project's database and files, its restore and its check, the group `plugin`: the MCP server and the skill of a coding agent, and the commands `setup-token` and
+`chat-webhook-secret`, which print a secret that a host derives from its root secret. To start a project, run `npm create inferative@latest my-site`: that command needs no registry setting.
 
 ```sh
 npm create inferative@latest my-site
@@ -37,7 +38,7 @@ npm create inferative@latest my-site
 | Argument or flag | What it does |
 | :-- | :-- |
 | `directory` | The folder to create. When you leave it out, the current folder is the target. |
-| `--starter <name>` | The starter to use: `nuxt-cloudflare` (a client site, the default) or `dashboard-cloudflare` (the dashboard). |
+| `--starter <name>` | The starter to use: `nuxt-cloudflare` (a client site, the default), `dashboard-cloudflare` (the dashboard) or `nuxt-sdk` (a website with its own server that reads a deployment with the SDK). |
 | `--source <giget source>` | Download this giget source alone, instead of the default sources. |
 | `--local <folder>` | Copy `<folder>/<starter>` instead of a download. For tests and for work on the starters. |
 | `--no-install` | Do not run `pnpm install`. |
@@ -81,6 +82,7 @@ file is the site's own TypeScript. `--site <path>` names another declarations mo
 | `inferative site sync` | Publishes the manifest to the channel, then waits until the channel serves that version. `--first-sync` publishes the first-sync catalog. `--manifest <file>` publishes a prebuilt envelope as it is. |
 | `inferative site doctor [origin]` | Reports registry access, then reads the diagnostic of a running site at `<origin>/_site/health`. `--local` diagnoses from the site's configuration instead. `--path <path>` names a diagnostic mounted elsewhere. |
 | `inferative site config` | Prints what the command line read from the declarations and the runtime configuration. |
+| `inferative site import <folder>` | Loads entries and media files from a folder into the channel, through the editor API. `--dry-run` writes nothing. |
 
 `sync` presents the key in `INFERATIVE_MANIFEST_TOKEN`. The channel and the API come from the `site` runtime config
 section, or from the `INFERATIVE_SITE_*` variables. Every `site` command first reads the `.env` file of the site root,
@@ -88,6 +90,51 @@ the file the connect guide of the dashboard fills. A variable that is already se
 file, and a missing file is not an error. The agent skill (`docs/skills/inferative`, the page
 `site-deploy-and-topologies`) lists every variable and every exit code.
 
+### `inferative site import`
+
+`inferative site import <folder>` loads a site's entries and media from files. It presents the `ak_` key in
+`INFERATIVE_IMPORT_TOKEN` (`content:read`, `content:write`, `files:read` and `files:write`; `content:publish` for
+`--publish` and `files:delete` for `--release`). The channel and the API come from `--channel` and `--api`, or from the
+`site.sync` group as for `sync`. The channel must serve a schema: run `sync` first.
+
+- `entries/<type>/<language>/<slug>.md`, `.html`, `.yaml` or `.json` is one entry in one language. The type is a
+  served key, with or without its namespace. A Markdown or HTML file has YAML or JSON front matter for the fields, and
+  its body goes to the type's one rich-text field (`_body` names another). The write converts the body with
+  `RichTextImport`, sent as `{ format, text }`. The same slug in two language folders is one entry with two languages.
+- `media/<path>` is one file. A side file `<path>.yaml` (or `.yml`, `.json`) holds its `title`, `alt`, `caption`,
+  `credit`, `focalPoint`, `visibility` and `contentType`. A media field names the file by `<path>`; a reference field
+  names an entry by `<type>/<slug>`.
+
+The run is idempotent. The entry id is a UUIDv7 that the channel, the type and the slug derive. An entry with no row
+at that id is looked up by each unique field that the type lets a list filter on (`wire.filter`), with the editor
+read `GET /content/editor/types/:typeKey/unique-holder`. A field inside an object is looked up by its dotted path, and
+a localized field, or a field inside a localized object, is looked up in each language that the files give it. The read answers the entry that the uniqueness check stores as the holder: the whole value folded
+(`CaseFold`), never cut to `indexedTextLength`, an empty value included, and for a field unique per channel only an
+entry that the channel owns (a shared entry holds its value in another scope). Two holders of one field (in two
+languages) stop the entry with an error. A create refused because another entry holds a unique value (`content/unique-conflict`,
+`content/single-entry-taken`) updates the entry that the answer names in `holdingEntryId`. A second run patches only
+the served fields and languages whose value differs; it converts a rich-text body locally with `RichTextImport` and
+compares the whole document, so an unchanged entry gets no new draft and a removed mark is a change. Objects, lists,
+sections and reference fields (`{ to, meta }`) are converted by their own fields, and a localized child field joins
+its languages; a section is matched to its stored twin by `id`. A part that the files do not name keeps its stored
+value, a reference's own fields and their languages included (also when the files write the reference as a bare
+key), with only the members that the channel's schema
+declares at every depth. A stored section of a type that the channel does not serve, or that its field does not take,
+is left out of the write, and the server keeps it in its place. The write does not convert a rich-text field of a reference's own fields, so the command
+sends its document. An entry that names another entry of the folder is written after it; in a cycle the reference is written
+once both exist. A media file is found by its stored name (`FileNameRule.storedFormOf`), its size and its visibility,
+and its bytes are compared by SHA-256 with the digest that the files read answers (`sha256`). A file with no stored
+digest is read through the delivery route when it is ready. The key goes to the API origin only: a redirect to another
+origin is followed with no credential. A file that waits for a release is reused when its stored digest matches, so a
+repeat run uploads nothing; `--release` releases it when it waits only for a release (`scan-pending`). A file held for
+another reason stays held, with a note. A held file with no stored digest has bytes that no key reads, so
+the command uploads a new file beside it. A missing file is uploaded in one request (`PUT /files/uploads`), then its
+credit is set. `--publish` sends the draft revision that the run wrote or read as `expectedDraftRevision`, so a later
+draft refuses the publish. A refused file stops the run before any write. The report has one line for each entry and
+each media file, and a `note` line for each part of a rich-text body that was kept as text, changed or removed.
+`--dry-run` makes the reads and writes nothing. The exit code is `0` when every file was read and every write landed,
+and `1` otherwise.
+
 ## The `backup` commands
 
 Run them in the project folder. A project made from a starter runs them as `pnpm backup`. The full procedure is the
@@ -95,8 +142,8 @@ runbook [Backup and restore](https://inferative.dev/docs/deploy/backup-and-resto
 
 | Command | What it does |
 | :-- | :-- |
-| `inferative backup create --out <file>` | Writes one archive (`.tar.gz.age`) of the Postgres database and the file store, with a manifest of the row count and digest of each table and the size and SHA-256 of each object, encrypted with age to the public keys of `--recipient` or `INFERATIVE_BACKUP_AGE_RECIPIENT`. `--unencrypted` writes a plain `.tar.gz`. The archive must not exist. |
-| `inferative backup restore --archive <file>` | Restores an archive into an empty database and an empty file store, then runs the check of `verify`. It applies the migrations of the backup from `server/persistence/pg/migrations` (`--migrations` names another folder). |
+| `inferative backup create --out <file>` | Writes one archive (`.tar.gz.age`) of the Postgres or MySQL database and the file store, with a manifest of the row count and digest of each table and the size and SHA-256 of each object, encrypted with age to the public keys of `--recipient` or `INFERATIVE_BACKUP_AGE_RECIPIENT`. `--unencrypted` writes a plain `.tar.gz`. The archive must not exist. |
+| `inferative backup restore --archive <file>` | Restores an archive into an empty database of its dialect and an empty file store, then runs the check of `verify`. It applies the migrations of the backup from `server/persistence/pg/migrations`, or from `server/persistence/mysql/migrations` for a `mysql://` address (`--migrations` names another folder). It holds the work of the agents of each organization with an agent (a restore flag in `agent_pause_flags`) until a person releases the hold. A backup and `verify` leave the restore flags out. |
 | `inferative backup verify --archive <file>` | Compares the database and the file store with the archive. `--archive-only` checks the archive alone. |
 | `inferative backup upload --archive <file>` | Copies an encrypted archive into the R2 bucket of archives of the four `INFERATIVE_BACKUP_R2_*` values (`--bucket`, `--r2-endpoint`, `--key`), in one write with `If-None-Match: *`, so it never replaces an object. It takes at most 5 GiB less 5 MiB, the largest single write of R2. |
 | `inferative backup keygen --out <file>` | Writes a new age private key to a file of mode 600 and prints its public key. |
@@ -127,7 +174,16 @@ or a cleanup debt names is not required. The lease is 120
 minutes; `--fence-lease <minutes>` names another, up to 1440. A crashed run leaves its row until the lease ends. A run
 that outlives its lease fails and writes no archive. The archive holds no rows of the fence table.
 Exit status 0 means that the run ended and the result is the same as the backup, 1 is a failure or a difference, and
-2 is a wrong option. Postgres only.
+2 is a wrong option.
+
+The address names the dialect: `postgres://` or `postgresql://` for Postgres, `mysql://` for MySQL (PlanetScale
+included). On MySQL the snapshot is a read-only transaction at `REPEATABLE READ` whose first statement reads one row
+of each table, so a migration that changes a table waits for the backup; PlanetScale refuses `LOCK TABLES`. Every table must use InnoDB: the command refuses a MyISAM or MEMORY table. Each value
+reads as the text that MySQL prints for it; a byte string goes in hexadecimal and a `BIT` value as a number. A MySQL
+restore keeps every foreign key on: it writes the tables in the order of their keys, writes the columns of a
+self-reference or a cycle after the rows, then sets each `AUTO_INCREMENT` value. Each of its statements commits on its
+own, and it refuses a table with a trigger. PlanetScale ends a transaction after 20 seconds, so a MySQL database there
+must read in 20 seconds; a larger one keeps the backups of PlanetScale.
 
 ## The `plugin` commands
 
@@ -166,6 +222,27 @@ own sign-in: you sign in to the dev host as the owner in the browser and allow t
 The access token lives one hour by default (`identity.oauthServer.accessTokenTtlMs`), and the agent keeps it in its own
 credential store, outside the project. The host must import `McpModule` and set `identity.oauthServer.enabled`.
 
+## The derived secrets
+
+A host derives some secrets from its root secret (`INFERATIVE_ROOT_SECRET`) when their own variable is not set. Two of
+them are for a person: the token of the first-run step, and the secret that Telegram sends with each update of the
+member chat. These commands print the value that the host derives, on one line, and nothing else:
+
+```sh
+pnpm exec inferative setup-token
+pnpm exec inferative chat-webhook-secret --env-file .env.worker
+```
+
+- The command reads the variables of the environment and of `.env` in the folder; a variable that is set wins. With
+  `--env-file <path>`, the values of that file win over the environment.
+- When the secret's own variable is set (`INFERATIVE_IDENTITY_SETUP_TOKEN`,
+  `INFERATIVE_NOTIFICATIONS_CHAT_TELEGRAM_WEBHOOK_SECRET`), the host uses it, so the command prints it.
+- With neither value, the command prints which variable to set and exits with 1.
+
+The setup token of a host with a root never changes, and the first-run step closes after its first use. Register the
+webhook secret with `setWebhook` again after each rotation of the root: until then, the host still accepts the
+secret of the previous root (`INFERATIVE_ROOT_SECRET_PREVIOUS`).
+
 ## How it is built
 
 `vp pack` bundles `src/bin.ts` into `dist/bin.mjs`, and `src/index.ts` into `dist/index.mjs`. The index is the library