@securecoms/agent-runtime 0.1.0 → 0.3.0

package/PROVENANCE.md

@@ -26,3 +26,21 @@ own transport, so no row is seeded for it here.
 
 This table is complete as of 2026-09-08 and should be re-reviewed if the
 source repo's files move or are re-cut.
+
+## SKILL.md format (2026-09-09)
+
+The `SKILL.md` + YAML-frontmatter skill layout follows the convention used by
+Hermes Agent, OpenClaw and Claude Code, so a skill written for one of those is
+recognisable here. **No code was cloned from any of them.** The frontmatter
+parser in `src/skill-md.ts` is original and deliberately accepts only the
+documented subset; the built-ins are written against this package's own
+`BuiltinContext`.
+
+## Managed configuration directory (2026-09-09)
+
+`src/config-dirs.ts` and the search-path form of the persona, memory and
+skills.json readers are original to this package. The convention
+`~/.securecoms/runtime-config/<identity-id>/` is SecureComs' own: the runtime
+computes the path from the home directory and its own identity rather than
+being told it, so no path crosses a process boundary and nothing can redirect
+a configuration read. No code was cloned for this round.

package/README.md

@@ -20,7 +20,8 @@ connection this process serves.
 ```
 securecoms-agent install --identity <identity_id> [--llm-endpoint <url>] \
   [--llm-model <model>] [--llm-key-file <path>] [--skills-dir <dir>] \
-  [--data-dir <dir>] [--exec-path <node>]
+  [--data-dir <dir>] [--brave-key-file <path>] [--tavily-key-file <path>] \
+  [--skills-config <path>] [--exec-path <node>]
 securecoms-agent install --identity <identity_id> --dry-run
 securecoms-agent uninstall --identity <identity_id>
 ```
@@ -28,13 +29,15 @@ securecoms-agent uninstall --identity <identity_id>
 Every install flag also reads its `SECURECOMS_*` variable from the environment
 when the flag is absent (`SECURECOMS_IDENTITY`, `SECURECOMS_LLM_ENDPOINT`,
 `SECURECOMS_LLM_MODEL`, `SECURECOMS_LLM_API_KEY_FILE`, `SECURECOMS_SKILLS_DIR`,
-`SECURECOMS_DATA_DIR`), so an operator who already exports the run-mode
+`SECURECOMS_DATA_DIR`, `SECURECOMS_BRAVE_KEY_FILE`, `SECURECOMS_TAVILY_KEY_FILE`,
+`SECURECOMS_SKILLS_CONFIG`), so an operator who already exports the run-mode
 environment can just run `securecoms-agent install`. The flag always wins.
 `--exec-path` is the one exception: it is never taken from the environment.
 `--dry-run` prints the resolved Node path, the service file path, the log
-path, the key file, and the rendered plist/unit **without writing anything or
-calling `launchctl`/`systemctl`** — use it to inspect an install before making
-it.
+path, the key file, the brave/tavily key files and the skills config path —
+each as a PATH only, never a key value — and the rendered plist/unit
+**without writing anything or calling `launchctl`/`systemctl`** — use it to
+inspect an install before making it.
 
 `install` writes a per-user service that starts the agent at login and
 restarts it on crash — a launchd LaunchAgent at
@@ -88,10 +91,13 @@ All configuration is via `SECURECOMS_*` environment variables (see
 | `SECURECOMS_LLM_MODEL` | `llama3` | Model name to request |
 | `SECURECOMS_LLM_API_KEY` | unset | API key for the LLM endpoint, if it requires one |
 | `SECURECOMS_LLM_API_KEY_FILE` | unset (or `--llm-key-file <path>`) | File holding the API key (trimmed on read). Preferred: it wins over `SECURECOMS_LLM_API_KEY`, and it is the only key form the service installer accepts. An unreadable or empty file exits 2 |
+| `SECURECOMS_BRAVE_KEY_FILE` | unset (or `--brave-key-file <path>`) | File holding a Brave Search API key. A path, never a value — required for `web_search` |
+| `SECURECOMS_TAVILY_KEY_FILE` | unset (or `--tavily-key-file <path>`) | File holding a Tavily API key. Used only when the Brave key file is absent |
 | `SECURECOMS_SKILLS_DIR` | this package's own `skills/` | Directory of skill manifests to load |
 | `SECURECOMS_DATA_DIR` | `~/.securecoms/agent-runtime` | Root directory for per-channel state |
+| `SECURECOMS_SKILLS_CONFIG` | `<dataDir>/skills.json` (or `--skills-config <path>`) | Path to `skills.json` — named endpoints, databases and flags |
 | `SECURECOMS_LOG_LEVEL` | `info` | One of `debug`, `info`, `warn`, `error` |
-| `SECURECOMS_TOOL_ALLOW` | `echo` | Comma-separated tool allow-list |
+| `SECURECOMS_TOOL_ALLOW` | `echo`, `web_search`, `fetch_url`, `http_request`, `sql_query`, `current_time`, `remember` | Comma-separated tool allow-list |
 | `SECURECOMS_TOOL_DENY` | unset | Comma-separated tool deny-list (wins over allow) |
 | `SECURECOMS_TOOL_DEFAULT` | `deny` | Fallback policy for a tool matched by neither list |
 
@@ -103,21 +109,226 @@ history and captured memories never leak into another channel's turn.
 
 ## Adding a skill
 
-Drop a manifest under `SECURECOMS_SKILLS_DIR`, either as a flat
-`<skillsDir>/<name>.json` file or as `<skillsDir>/<dir>/manifest.json`. A
-manifest needs `name`, `description`, a JSON-Schema `parameters` object, and
-a `handler`:
+A skill is either a JSON manifest or a `SKILL.md` file, dropped under
+`SECURECOMS_SKILLS_DIR` in one of three layouts:
 
-- `builtin:<name>` — one of the runtime's built-in handlers (currently
-  `echo`, `timestamp`).
+- flat `<skillsDir>/<name>.json`
+- `<skillsDir>/<dir>/manifest.json`
+- `<skillsDir>/<dir>/SKILL.md`
+
+All three keep working side by side. The loader reads them in that order —
+flat `.json`, then `manifest.json`, then `SKILL.md` — sorted within each
+bucket for a deterministic result; if two files declare the same skill name,
+the one loaded later (later bucket, or later alphabetically) wins, with a
+warning naming both paths.
+
+A `SKILL.md` is YAML frontmatter (between two `---` lines) followed by
+free-text guidance appended to the model-facing description (capped at 1 kB).
+The frontmatter accepts a deliberately small subset of YAML — anything else
+throws with the file and line, rather than silently misreading it:
+
+- scalars (`key: value`, optionally quoted)
+- flow lists (`key: [a, b, c]`)
+- flow maps (`key: { a: 1, b: 2 }`)
+- nested mappings (an indented block under a bare `key:`), two-space indents
+  only, no tabs
+- comments (`#`) and blank lines
+- block scalars (`|`, `>`), anchors, aliases and other YAML constructs are
+  **not** supported — use a JSON manifest for anything that needs them
+
+Example:
+
+```
+---
+name: web_search
+description: Search the web for recent information
+parameters:
+  type: object
+  properties:
+    query: { type: string }
+    count: { type: number }
+  required: [query]
+handler: builtin:web_search
+requires: [search_provider]
+---
+Use this when the user asks about something time-sensitive or outside your
+training data. Prefer a specific query over a broad one.
+```
+
+`name`, `description`, `parameters` (a nested mapping) and `handler` are
+required. `handler` is one of:
+
+- `builtin:<name>` — one of the runtime's built-in handlers (see "Curated
+  skills" below).
 - `script:<path>` — an executable file, path relative to `skillsDir` and
   contained within it (symlinks included — a link that resolves outside
   `skillsDir` is refused). It runs with its arguments as JSON on stdin, cwd
   set to `skillsDir`, and a minimal environment (no inherited secrets).
 
-A manifest with any other handler shape is skipped with a warning, and a
-skill invocation that throws is turned into a `tool_failed` tool message
-rather than dropping the turn.
+A manifest or `SKILL.md` with any other handler shape, or a missing required
+field, is skipped with a warning, and a skill invocation that throws is
+turned into a `tool_failed` tool message rather than dropping the turn.
+
+### Gating a skill on configuration
+
+A manifest or `SKILL.md` may carry `requires: [...]`, and the skill is offered
+to the model only when every entry is configured: `endpoint:<name>` and
+`database:<name>` name entries in `skills.json`, `search_provider` means a
+readable, non-empty search key file, `any_endpoint` / `any_database` mean at
+least one endpoint or database is configured, and anything else is matched
+against `skills.json`'s `flags`.
+
+The shipped `http_request` and `sql_query` skills declare `requires:
+[any_endpoint]` and `requires: [any_database]`: the endpoint and database names
+belong to the customer, so the shipped copies cannot name one, but neither is
+offered to the model until `skills.json` gives it something to reach. With no
+endpoints configured, startup prints `unavailable: http_request (missing
+any_endpoint)`. To narrow one to a particular system, copy the folder and change
+the line — for example `requires: [endpoint:crm]` in
+`skills/http_request/SKILL.md`.
+
+## Curated skills
+
+Seven built-in handlers ship with the runtime. All network built-ins refuse
+`localhost` and any private, loopback or link-local resolved address, and
+every tool result is capped at 20 kB (truncated, not an error). A redirect to a
+different origin is followed without the caller's headers and without the
+request body, so a configured header value or a search key can never be handed
+to a host the operator did not name.
+
+| Skill | Arguments | Notes |
+| --- | --- | --- |
+| `echo` | `text` | Returns the text unchanged. No `requires`. |
+| `current_time` | `timezone` (an IANA name such as `America/New_York`; defaults to UTC) | The server clock, ISO 8601. No `requires`. |
+| `fetch_url` | `url` | Fetches a page and returns extracted text (HTML is stripped to text). Up to 3 redirects, 10 s timeout, 2 MB response cap. |
+| `web_search` | `query`, `count` (1–10, default 5) | `requires: [search_provider]` — needs a readable, non-empty `SECURECOMS_BRAVE_KEY_FILE` or `SECURECOMS_TAVILY_KEY_FILE`; Brave is used first when both are set. |
+| `http_request` | `name` (a configured `endpoint:<name>`), `path` (a query string is kept, a `#fragment` is dropped), `method` (GET/POST/PUT/DELETE), `body` | `requires: [any_endpoint]`. The model names an endpoint and a path, never a host; a path that would leave the endpoint's `baseUrl` origin is refused. |
+| `sql_query` | `name` (a configured `database:<name>`), `sql` (read-only `SELECT` only; a `;` or a write keyword anywhere in the text — inside a string literal or a comment included — is refused) | `requires: [any_database]`. SQLite or Postgres (`pg` required separately for Postgres). Results capped at 200 rows. |
+| `remember` | `fact` (≤300 chars) | Appends a dated line to the agent-wide `MEMORY.md`. |
+
+## skills.json
+
+`skills.json` names the endpoints and databases the curated skills are
+allowed to reach, plus arbitrary boolean `flags` for gating. It carries paths,
+never secrets — a header value or a Postgres connection string is read from
+the file it names, at call time, and never logged. Relative paths (a
+`sqlite`, `postgresUrlFile` or `headersFile` value) resolve against the data
+dir, not the current working directory.
+
+```json
+{
+  "endpoints": {
+    "crm": { "baseUrl": "https://crm.example.com/api", "headersFile": "crm-headers.json" }
+  },
+  "databases": {
+    "reports": { "sqlite": "reports.db" },
+    "warehouse": { "postgresUrlFile": "warehouse-url.txt" }
+  },
+  "flags": ["beta_tools"]
+}
+```
+
+A `headersFile` is a flat JSON object of header name to string value, read
+fresh on every `http_request` call — never cached, never logged. A Postgres
+database needs the optional `pg` package installed separately
+(`npm i -g pg`); SQLite needs nothing extra (`node:sqlite`).
+
+`skills.json` may also carry a `tools` block — `{"tools": {"allow": [...],
+"deny": [...]}}` — the owner's on/off choices for the curated skills. A skill
+that is switched off is not offered to the model at all. The environment wins:
+see "Managed configuration" below.
+
+By default `skills.json` lives at `<dataDir>/skills.json`; point elsewhere
+with `SECURECOMS_SKILLS_CONFIG` or `--skills-config <path>`. A managed copy
+(see below) wins over both. A missing file means no endpoints, no databases
+and no flags — not an error.
+
+## Persona and memory
+
+Three optional files make up an agent's persona, looked up along the config
+search path (see "Managed configuration" below) and read fresh
+on **every turn** — no restart needed after an edit:
+
+| File | Heading |
+| --- | --- |
+| `IDENTITY.md` | `## Identity` |
+| `SOUL.md` | `## Soul` |
+| `AGENTS.md` | `## Operating rules` |
+
+Whichever of the three exist are joined, in that fixed order, under their
+fixed headings; each is capped at 8 kB (truncated, with a warning, if
+larger). If none exist, a built-in default prompt is used instead. Either
+way, a fixed, non-removable safety preamble is appended last, so it is the
+part of the prompt read most recently.
+
+`securecoms-agent persona init [--identity <id>] [--data-dir <dir>]` copies
+the shipped default persona files into the data dir — it never overwrites a
+file that already exists, so a customized `IDENTITY.md` is safe to keep
+across a re-init. `securecoms-agent persona show` prints the assembled system
+prompt exactly as a turn would see it, useful for checking an edit landed.
+
+Separately, `MEMORY.md` in the data dir is **agent-wide** (not per-persona,
+not per-channel): the `remember` skill appends a dated, one-line fact to it,
+and its contents are injected as a system message on every turn across every
+channel. It is capped at 8 kB; once it overflows, the **oldest** lines are
+dropped first, on the theory that the newest remembered facts are the ones
+still likely to be true.
+
+## Managed configuration (from the web app)
+
+When an agent is managed from the SecureComs web app, the owner's saved
+configuration is written to disk by the customer's daemon at:
+
+```
+~/.securecoms/runtime-config/<identity-id>/
+```
+
+The runtime reads its configuration along a **search path**: that managed
+directory first, then the data dir. The lookup is per FILE, so an owner who
+saved only `IDENTITY.md` in the app keeps the machine's own `SOUL.md`; a file
+with no managed copy still comes from the data dir, and an agent with no
+managed directory at all behaves exactly as it did before. `skills.json` is
+looked up the same way, and re-read on every turn — a new endpoint or a skill
+switched off in the app takes effect on the next message, with no restart.
+
+Two deliberate exceptions:
+
+- **The environment still wins for the tool policy.** `SECURECOMS_TOOL_ALLOW`
+  overrides the `allow` list in the `tools` block of `skills.json`, so an
+  install that pins its allow list in its service unit is unaffected by a save
+  in the app. `SECURECOMS_TOOL_DENY` is UNIONED with the file's `deny` list
+  rather than replacing it: deny is a safety mechanism, so one entry in the
+  service unit must not silently un-deny every tool the owner switched off in
+  the app. Everything the environment denies stays denied either way.
+- **Writes go to the data dir**, not the managed directory — except `MEMORY.md`,
+  which is written wherever the agent already keeps it. An existing
+  `<dataDir>/MEMORY.md` is copied into the managed directory once, at startup,
+  so the owner can read and prune it in the app.
+
+The runtime also writes `local-config.json` into the managed directory when one
+exists: what this agent's OWN persona files say, and a single boolean for
+whether it has a readable search key. It is written at startup and rewritten
+whenever the search-key answer changes, so a key file dropped in (or taken
+away) mid-run is reported without a restart. It is a report the daemon passes
+upward — nothing in the runtime reads it back, and it holds no key value.
+
+### A known limitation: the generated tool paragraph
+
+The `AGENTS.md` the app delivers ends in a generated paragraph naming the tools
+this agent has. The backend derives that list from the saved configuration plus
+the search-key boolean above; it does not see the running process. Three things
+can therefore make it diverge from what the runtime actually offers:
+
+- a `SECURECOMS_TOOL_ALLOW` or `SECURECOMS_TOOL_DENY` set in the service unit,
+  which the backend cannot see;
+- a customer skill folder whose `requires:` is narrower than the shipped one, so
+  the runtime withholds a tool the backend counted;
+- a custom skill the backend knows nothing about, so it is offered but unnamed.
+
+In any of those cases the paragraph is advisory rather than exact. The gate is
+still authoritative — a tool named in the paragraph but denied by policy is
+still refused at call time — so the risk is a wrong claim in conversation, not
+a wrong permission.
 
 ## Provenance