sidegraph 0.4.0__tar.gz → 0.5.0__tar.gz

{sidegraph-0.4.0 → sidegraph-0.5.0}/.github/workflows/ci.yml

@@ -105,7 +105,6 @@ jobs:
             exit 0
           fi
           uv sync --locked
-          git fetch origin "$GITHUB_BASE_REF"
           base="$(git merge-base "origin/$GITHUB_BASE_REF" HEAD)"
           uv run sidegraph-verify --against "$base" --json
 

{sidegraph-0.4.0 → sidegraph-0.5.0}/CHANGELOG.md

@@ -7,6 +7,67 @@ interfaces, exactly, and what each one promises: [`docs/reference/stability.md`]
 
 ## [Unreleased]
 
+## [0.5.0] — 2026-09-23
+
+### Changed
+
+- **The public `main`, which the plugin manifests install from, moves only at releases.**
+  The plugin runs whatever `main` holds, so a snapshot pushed between releases used to reach
+  every plugin user at once. The release script now pushes `main` only for a release: it
+  refuses unless the version it carries is untagged in the public repository and
+  `CHANGELOG.md` dates it.
+
+### Fixed
+
+- **`sidegraph-init` now says to restart the Claude Code session after it writes
+  `SIDEGRAPH_RATIFY_POLICY`.** The Sidegraph MCP server reads the policy from the
+  environment it started with, so a server started before `sidegraph-init` kept proposing
+  under the old policy until the session restarted, and nothing said so.
+- **`sidegraph-verify --against` no longer flags a ratification of an already-committed
+  proposal, or a supersede/drop that touches a record written before a later schema addition
+  (`Provenance.commit`, `Domain.seed_anchors`/`path_prefixes`) existed.** Ratifying a
+  decision, fact, or domain stamps `ratified_at`/`ratified_by`, and neither field was on the
+  transition layer's mutable allow-list, so the stamp alone was reported as an illegal field
+  change. Separately, rewriting an old record that predates a field added to its schema
+  serializes that field back in as its default, which a plain top-level or whole-list
+  comparison also read as a change. The ratifier stamp may now be set once, from a proposed
+  record, landing on any state reached through accepted within the diffed range; an absent
+  field now counts as the same value as an explicit `null` or an empty list/object, at every
+  nesting depth and inside list items. A field whose default is a non-empty value is not
+  covered by this.
+- **`sidegraph-import --docs` no longer aborts on a file that is not valid UTF-8.** It
+  decoded every document as strict UTF-8 with no handler, so one document saved in
+  `cp1251` (or carrying a stray non-UTF-8 byte) raised and stopped the whole run, with
+  nothing after it imported and no report printed. The file is now a named skip
+  (`skipped_undecodable`) instead: the run continues, and both the real run and
+  `--dry-run` print the skipped path(s), last.
+- **`sidegraph-import --docs` now reads a UTF-8 document that starts with a byte-order mark
+  (BOM).** The BOM used to survive into the parsed text, so the H1 and any frontmatter never
+  matched, and a real decision document was silently counted `skipped_not_decision`. It is
+  now stripped before parsing. `sidegraph-bootstrap` still reads such a file without
+  stripping it.
+
+  Re-importing a BOM document that an earlier version did import can change its record,
+  once:
+  - On the `openspec` profile, a BOM `proposal.md` had imported under a title built from its
+    path. The re-import supersedes it with the real H1 title.
+  - Frontmatter the BOM hid is now honoured. A document marked `status: superseded` now
+    counts `skipped_superseded_frontmatter`, and its earlier live record stays as it is.
+  - A document marked as a draft, proposed, pending or under review keeps its earlier
+    `accepted` record: a re-import compares content, not status, and the content is
+    unchanged.
+
+  Retiring or re-proposing such a record is manual.
+
+### Security
+
+- **The lockfile moves `anyio` from 4.14.1 to 4.15.1**, past three advisories fixed in
+  4.14.2: TLS certificate spoofing through IDNA 2003 host-name encoding (critical),
+  `run_process` keeping the parent's supplementary groups (high), and process-pool workers
+  blocking on undrained stderr (moderate). The published package does not pin `anyio`, so
+  this changes development and CI environments and installs made from `uv.lock`; a fresh
+  install already resolves a fixed version.
+
 ## [0.4.0] — 2026-09-22
 
 ### Fixed

{sidegraph-0.4.0 → sidegraph-0.5.0}/CLAUDE.md

@@ -13,7 +13,7 @@ task-aware retrieval + decision memory riding on the engine's entity graph.
 continuing participant in the project. A core feature must preserve, organize, or deliver
 accumulated project understanding. Document search by itself is not the product.
 
-Sidegraph is pre-1.0 (v0.4.0) — the full loop (capture, ratification, mistakes-first
+Sidegraph is pre-1.0 (v0.5.0) — the full loop (capture, ratification, mistakes-first
 retrieval, refactor-surviving re-anchoring, semantic docs layer, mind-model domains, and the
 facts evidence layer) ships and is exercised end-to-end, but interfaces may still move before
 a stable release. See

{sidegraph-0.4.0 → sidegraph-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sidegraph
-Version: 0.4.0
+Version: 0.5.0
 Summary: Own the memory, rent the graph — a durable decision/lessons layer as a sidecar over a code-graph engine.
 Project-URL: Homepage, https://github.com/SantyagoSeaman/sidegraph
 Project-URL: Repository, https://github.com/SantyagoSeaman/sidegraph
@@ -84,9 +84,10 @@ only ever reads `graph.json`, so the plain install above is all it needs.
 uv tool install sidegraph        # from PyPI — puts sidegraph-init / sidegraph-mcp / … on PATH
 sidegraph-init
 
-# Prefer the latest unreleased build straight from git instead of PyPI? Swap step 3 for:
+# Prefer running straight from git instead of PyPI? Swap step 3 for:
 #   uvx --from git+https://github.com/SantyagoSeaman/sidegraph.git@main sidegraph-init
-#   `@main` is a mutable ref — it moves under you. Pin a tag or a SHA for CI.
+#   `@main` moves only when a release is cut. It is still a mutable ref: a later cache refresh can
+#   install a newer one. Pin a tag or a SHA for CI.
 ```
 
 4. **Name your domains** — turns the graph's communities into a described table of

{sidegraph-0.4.0 → sidegraph-0.5.0}/README.md

@@ -56,9 +56,10 @@ only ever reads `graph.json`, so the plain install above is all it needs.
 uv tool install sidegraph        # from PyPI — puts sidegraph-init / sidegraph-mcp / … on PATH
 sidegraph-init
 
-# Prefer the latest unreleased build straight from git instead of PyPI? Swap step 3 for:
+# Prefer running straight from git instead of PyPI? Swap step 3 for:
 #   uvx --from git+https://github.com/SantyagoSeaman/sidegraph.git@main sidegraph-init
-#   `@main` is a mutable ref — it moves under you. Pin a tag or a SHA for CI.
+#   `@main` moves only when a release is cut. It is still a mutable ref: a later cache refresh can
+#   install a newer one. Pin a tag or a SHA for CI.
 ```
 
 4. **Name your domains** — turns the graph's communities into a described table of

{sidegraph-0.4.0 → sidegraph-0.5.0}/docs/getting-started/installation.md

@@ -17,7 +17,8 @@ The `git+…@main` examples below track a mutable branch: a later `uvx` cache re
 install different code. That is useful while trying Sidegraph. For CI, shared setup, or a
 measured pilot, replace `@main` with a release tag or commit SHA, for example
 `git+https://github.com/SantyagoSeaman/sidegraph.git@<sha>`. The plugin marketplace is the
-exception: its public manifests intentionally track `main`. See
+exception: its public manifests intentionally track `main`, and the release process moves
+that `main` only when it cuts a release. See
 [`reference/stability.md`](../reference/stability.md) for the promises made by each surface.
 
 ## Install Sidegraph
@@ -54,11 +55,13 @@ PyPI package. See
 the integration details for [Claude Code](../integrations/claude-code.md#plugin-install-path)
 or [Codex](../integrations/codex.md#plugin-install-path).
 
-### Option C — uvx directly from git (latest / unreleased)
+### Option C — uvx directly from git
 
-Want the latest unreleased build, prefer manual wiring, or need an entry point outside Claude
-Code (e.g. `sidegraph-init` from a plain terminal)? Run it straight from the repository with
-[`uvx`](https://docs.astral.sh/uv/guides/tools/) — no persistent install, no local checkout:
+Prefer manual wiring, or need an entry point outside Claude Code (e.g. `sidegraph-init` from
+a plain terminal)? Run it straight from the repository with
+[`uvx`](https://docs.astral.sh/uv/guides/tools/), with no persistent install and no local
+checkout. `@main` moves only when a release is cut. For reproducibility, pin a tag or commit SHA instead
+(see "Mutable development references" above):
 
 ```bash
 uvx --from git+https://github.com/SantyagoSeaman/sidegraph.git@main sidegraph-init

{sidegraph-0.4.0 → sidegraph-0.5.0}/docs/reference/cli.md

@@ -132,6 +132,11 @@ the Sidegraph checkout):
      `--ratify-policy VALUE` sets the value with no prompt, still never overwriting an
      existing one, for a fully scripted setup that wants an explicit answer.
 
+   When the value is written, the line after it says to restart the Claude Code session:
+   the Sidegraph MCP server reads the policy from the environment it started with, so a
+   server started before `sidegraph-init` keeps proposing under the old policy until the
+   session restarts.
+
    Whatever happens above, a host without `.claude/settings.json` (Codex CLI, CI) gets the
    equivalent as a plain `export SIDEGRAPH_RATIFY_POLICY=<value>` line. A settings-file
    problem never fails the store creation in step 1.
@@ -632,24 +637,39 @@ import on their own. Both the real run and `--dry-run` print an extra line whene
 `skipped_degenerate_parent > 0`: `N split parent(s) skipped as degenerate (echoed context
 or empty choice) — children imported on their own`.
 
+**A file that is not valid UTF-8 is skipped, named, and the run continues.** Every document
+is decoded as `utf-8-sig` — plain UTF-8, with a leading byte-order mark stripped when one is
+present. No encoding is guessed. A file whose bytes don't decode (for example, one saved in
+`cp1251`) is counted `skipped_undecodable` instead of raising and aborting the whole run.
+Both the real run and `--dry-run` print an extra block, last, whenever
+`skipped_undecodable > 0`:
+
+```
+N file(s) skipped: not valid UTF-8, re-save as UTF-8 to import:
+  path/to/bad-file.md
+```
+
 **Real run:** `imported N decision(s), superseded M (skipped: A existing, B unanchorable,
 C not-decision-shaped, D unparseable, E superseded-frontmatter, F outside-profile)`, followed
 by the status-derived-proposed line above when applicable, followed by the template-skip line
-and the degenerate-parent line above when applicable. A non-zero `F` is the profile scope
-filter, not a parse failure — see the `--any-doc` note under the flag table above. Under a
-non-`manual` `SIDEGRAPH_RATIFY_POLICY`: `imported N decision(s), superseded M, auto-ratified K
-(skipped: …)` — only fresh `--propose` writes that land as a new `proposed` record are
-eligible; status-derived and `superseded` re-imports never auto-ratify; each failed
-auto-ratify attempt prints `auto-ratify failure: <id>: <reason>` on stderr; `--dry-run`
-never carries the segment.
+and the degenerate-parent line above when applicable, and the undecodable-file block above
+last when applicable. A non-zero `F` is the profile scope filter, not a parse failure — see
+the `--any-doc` note under the flag table above. Under a non-`manual`
+`SIDEGRAPH_RATIFY_POLICY`: `imported N decision(s), superseded M, auto-ratified K (skipped:
+…)` — only fresh `--propose` writes that land as a new `proposed` record are eligible;
+status-derived and `superseded` re-imports never auto-ratify; each failed auto-ratify attempt
+prints `auto-ratify failure: <id>: <reason>` on stderr; `--dry-run` never carries the
+segment.
 
 **`--dry-run`:** one line per candidate — `<path>: [imported|superseded] <title>`, with any
 skipped-anchor reasons indented underneath (`    anchor skipped: <name> (<reason>)`) —
 followed by a blank line, `would import N decision(s), supersede M (...)` (same skip breakdown
 as the real-run line above), the status-derived-proposed line when applicable (`N would land
 proposed (...)`), the template-skip line and the degenerate-parent line above when
-applicable, and a per-file breakdown. No decision or binding is written; opening the store
-may still migrate supported legacy data or rebuild the derived index.
+applicable, and a per-file breakdown. The undecodable-file block above prints last, after
+the per-file breakdown. Both indent their lines by two spaces, so breakdown lines printed
+after the block would read as more undecodable files. No decision or binding is written; opening the
+store may still migrate supported legacy data or rebuild the derived index.
 
 **Exit code:** `0` on success, including an empty run and every `--dry-run` invocation.
 Returns `1` before touching the graph or store if `--profile` names an unknown profile
@@ -903,21 +923,28 @@ plain-text line):
 | `illegal-deletion` | transition | a record file — or an already-published archive segment — was deleted with no sanctioned reason. The one exception: a decision/domain whose id is present in an `archive/*.jsonl` segment in the new tree (a legitimate `sidegraph-compact`) |
 
 **What's legally mutable** (transition layer only; derived from `store.py`'s write methods,
-not invented — see `verify.py`'s module docstring for the line-numbered derivation):
+not invented — see `verify.py`'s module comment above the mutable-field tables for exactly
+which methods):
 
 - **Decisions:** `status` (only along a real transition — `proposed→accepted`,
   `proposed→rejected`, `proposed→superseded`, `accepted→superseded`, `accepted→deprecated`),
-  `valid_to` (`null → value`, once). Everything else (`title`, `kind`, `context`, `choice`,
-  `rejected`, `consequences`, `layer`, `valid_from`, `supersedes`, `provenance`) is immutable.
-- **Facts:** the same shape, `status`/`valid_to`, but **without** `accepted→deprecated`
-  — no live write path ever sets a fact `deprecated` (`schema.py`: "DEPRECATED unused for
-  facts", unlike the decision-side forward-compat carve-out above), so that jump is *not*
-  legal here and is flagged `illegal-status-jump` if it appears. Legal: `proposed→accepted`,
-  `proposed→rejected`, `proposed→superseded`, `accepted→superseded`. Everything else
-  (`statement`, `source`, `supports`, `valid_from`, `supersedes`, `provenance`) is immutable.
+  `valid_to` (`null → value`, once). `ratified_at`/`ratified_by` may be set once, when a
+  proposed record is accepted — even if it moves on to a later state (`superseded`) within the
+  same diffed range — and never changes again after that. Everything else (`title`, `kind`,
+  `context`, `choice`, `rejected`, `consequences`, `layer`, `valid_from`, `supersedes`,
+  `provenance`) is immutable.
+- **Facts:** the same shape, `status`/`valid_to`/the ratifier stamp, but **without**
+  `accepted→deprecated` — no live write path ever sets a fact `deprecated` (`schema.py`:
+  "DEPRECATED unused for facts", unlike the decision-side forward-compat carve-out above), so
+  that jump is *not* legal here and is flagged `illegal-status-jump` if it appears. Legal:
+  `proposed→accepted`, `proposed→rejected`, `proposed→superseded`, `accepted→superseded`.
+  Everything else (`statement`, `source`, `supports`, `valid_from`, `supersedes`,
+  `provenance`) is immutable.
 - **Domains:** `status` (`proposed→accepted`, `proposed→dropped`, `accepted→dropped`,
   `proposed→superseded`, `accepted→superseded`, `dropped→superseded`). `communities` is
-  index-only and never appears in the canonical file.
+  index-only and never appears in the canonical file. `ratified_at`/`ratified_by` may be set
+  once, when a proposed record is accepted — even if it moves on to a later state (`dropped`,
+  `superseded`) within the same diffed range — and never changes again after that.
   Everything else (`domain_id`, `slug`, `title`, `summary`, `parent_id`, `path_prefixes`,
   `seed_anchors`, `provenance`) is immutable.
 - **Entities:** `descriptor` may change when sync confirms a durable leaf-file move.
@@ -933,6 +960,21 @@ not invented — see `verify.py`'s module docstring for the line-numbered deriva
   `A`) is legal; any modification or deletion of an already-published one is always
   illegal.
 
+A field an older file was written before its schema even had is simply absent from that
+file — whether its default is `null` (`Provenance.commit` today) or an empty list/object
+(`Domain.seed_anchors`/`path_prefixes` today). The next legal rewrite re-serializes the record
+and fills the field back in with that default. That alone is never a change: an absent field
+counts as the same value as an explicit `null` or an empty list/object, inside nested objects
+and inside the items of a list, everywhere this table calls a field immutable. A field whose
+default is a *non-empty* value (`Decision.scope`, `"repo"`) is **not** covered by this — an
+absent key there still reads as changed — but no file in this store lacks `scope` today.
+
+**Known limit:** `verify_against` compares only the net change across the whole diffed range
+(see "diffs the CURRENT WORKING TREE" below), not each intermediate commit. So a ratifier
+stamp appearing on a record that ends up `dropped`/`superseded` in that same range cannot be
+told apart from a record that was legally accepted and only later dropped or superseded — the
+store's own git history, not this lint, is the evidence for who ratified what and when.
+
 **`--against <ref>` diffs the CURRENT WORKING TREE against `<ref>`'s snapshot — not two fixed
 points in history.** Passing a moving branch name directly (`--against origin/main`) is only
 correct when your checkout's `HEAD` actually sits at that branch's tip; on a PR branch that's

{sidegraph-0.4.0 → sidegraph-0.5.0}/docs/reference/releasing.md

@@ -58,15 +58,41 @@ Prepare the release on internal `main`, then run the clean-tree preflight:
 7. Commit the release preparation on internal `main`. Both the internal and public checkouts
    must now be clean.
 8. Run `uv run python tools/preflight_release.py --online`. It verifies version agreement,
-   changelog shape, clean branches, target repository, and tag availability. Run it **after**
-   the release changes are committed: running it first can only report the work you have not
-   prepared yet.
+   changelog shape, clean branches, and target repository, and that `vX.Y.Z` is not already
+   tagged in the **internal** checkout or its origin. Run it **after** the release changes are
+   committed: running it first can only report the work you have not prepared yet.
 9. Cut the allowlisted public snapshot with
    `tools/release-public.sh "release: vX.Y.Z" --push`. This creates and pushes the public
-   `main` commit that will receive the release tag. Refresh `demo` separately when required.
+   `main` commit that will receive the release tag. `--push` itself refuses unless `vX.Y.Z`
+   is untagged in the **public** repository and `CHANGELOG.md` dates it (see "The public
+   main moves only at releases" below). Refresh `demo` separately when required.
 
 Only after all nine steps pass is the public snapshot ready to tag.
 
+### The public main moves only at releases
+
+Every plugin manifest installs from the public repository's `main`, and `uv` re-resolves that
+branch reference on every call, so pushing `main` reaches every plugin user at once. For that
+reason the public `main` moves only as part of a release: `tools/release-public.sh --push`
+refuses to push it unless the version being published is untagged in the public repository
+and `CHANGELOG.md` dates it, checked before anything is written and again right before the
+push. A docs-only or other non-release change waits for the next release, or ships in an
+urgent patch release.
+
+Two consequences follow. First, bump the version in `pyproject.toml` and date its
+`CHANGELOG.md` heading only as part of release preparation (steps 1 and 2 above). Once both
+are on `main`, every snapshot counts as a release until the tag exists, so doing them ahead of
+the release defeats the guard. Second, re-cutting a version whose tag-driven publish failed still needs
+its tag deleted in both the public checkout and its origin before the next `--push`;
+otherwise the push is refused.
+
+Between the release-preparation commit and the tag, every `--push` is a re-cut of that same
+release, and the tag goes on the last one. Tag promptly after the push, so the window stays
+short.
+
+Never push the public `main` by hand: the gate lives in `tools/release-public.sh`, and a
+manual `git push` from the public checkout bypasses it.
+
 ## Tag-driven publish flow
 
 PyPI publishing is driven entirely by a git tag. There is deliberately no manual dispatch: a

{sidegraph-0.4.0 → sidegraph-0.5.0}/plugin/sidegraph/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "sidegraph",
   "displayName": "Sidegraph",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Durable decision/lessons memory over your code graph — ADRs, gotchas, and constraints that survive refactors, retrieved mistakes-first.",
   "author": { "name": "SantyagoSeaman", "url": "https://github.com/SantyagoSeaman" },
   "homepage": "https://github.com/SantyagoSeaman/sidegraph",

{sidegraph-0.4.0 → sidegraph-0.5.0}/plugin/sidegraph/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "sidegraph",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Durable decision/lessons memory over your code graph — ADRs, gotchas, and constraints that survive refactors, retrieved mistakes-first.",
   "license": "Apache-2.0",
   "skills": "./skills/",

{sidegraph-0.4.0 → sidegraph-0.5.0}/plugin/sidegraph/skills/import-adrs/SKILL.md

@@ -49,8 +49,8 @@ Costs nothing, writes nothing, and shows exactly what a real run would do. Read
 report closely:
 
 - `would import N decision(s), supersede M (skipped: A existing, B unanchorable, C
-  not-decision-shaped, D unparseable, E superseded-frontmatter)` — plus per-file breakdown
-  and per-anchor `anchor skipped: <name> (<reason>)` lines.
+  not-decision-shaped, D unparseable, E superseded-frontmatter, F outside-profile)` — plus
+  per-file breakdown and per-anchor `anchor skipped: <name> (<reason>)` lines.
 - **Lots of `unanchorable` + you passed an absolute path?** That's the wrong-cwd trap from
   preflight — the command itself warns when ≥ half of anchor-attempted docs miss. Re-run
   from the repo root with a relative path.
@@ -63,6 +63,11 @@ report closely:
   (Context/Decision/Status/... or Trigger/Design/...); narrative or table-structured docs
   don't qualify (by design, for now) — record their content via
   `sidegraph:record-decision` if it matters.
+- **`N file(s) skipped: not valid UTF-8, re-save as UTF-8 to import:`** (printed last, with
+  the listed paths) — a file whose bytes aren't valid UTF-8 is skipped and named instead of
+  aborting the run. Re-save the listed file(s) as UTF-8 and re-run to import them. No
+  encoding is guessed on your behalf. A UTF-8 byte-order mark is fine: it is stripped and the
+  file imports, so it never appears in this list.
 
 ## Gate the real run on the human
 

{sidegraph-0.4.0 → sidegraph-0.5.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "sidegraph"
-version = "0.4.0"
+version = "0.5.0"
 description = "Own the memory, rent the graph — a durable decision/lessons layer as a sidecar over a code-graph engine."
 readme = "README.md"
 requires-python = ">=3.13"

{sidegraph-0.4.0 → sidegraph-0.5.0}/src/sidegraph/__init__.py

@@ -19,7 +19,7 @@ from .schema import (
 )
 from .store import Store
 
-__version__ = "0.4.0"
+__version__ = "0.5.0"
 
 __all__ = [
     "SCHEMA_VERSION",

{sidegraph-0.4.0 → sidegraph-0.5.0}/src/sidegraph/cli.py

@@ -145,6 +145,12 @@ def _report_ratify_policy_write(
     which already know `value` (whatever they decided to write) before calling this."""
     if result.outcome == "written":
         print(f"wrote {rel_settings}: env.{RATIFY_POLICY_ENV_VAR}={value}")
+        # The MCP server reads the policy once, from the environment it started with; a
+        # server already running keeps proposing under the old value.
+        print(
+            "  restart your Claude Code session so the Sidegraph MCP server picks it up: "
+            "a running server keeps the environment it started with"
+        )
     elif result.outcome == "already_set":
         print(
             f"{rel_settings} already sets "
@@ -923,6 +929,16 @@ def _import_docs_mode(args: argparse.Namespace) -> int:
             )
         for fp, n in sorted(report.by_file().items()):
             print(f"  {fp}: {n}")
+        # D4 (design/superpowers/specs/2026-09-23-doc-import-encoding-design.md): printed
+        # LAST. The by_file lines share the block's two-space indent, so any printed after
+        # the block would read as more undecodable files.
+        if report.skipped_undecodable:
+            print(
+                f"{report.skipped_undecodable} file(s) skipped: not valid UTF-8, re-save as "
+                "UTF-8 to import:"
+            )
+            for fp in report.undecodable_files:
+                print(f"  {fp}")
         return 0
 
     # Design D6: the count is added to the non-dry-run summary line only, only when the
@@ -954,6 +970,16 @@ def _import_docs_mode(args: argparse.Namespace) -> int:
             f"{report.skipped_degenerate_parent} split parent(s) skipped as degenerate "
             "(echoed context or empty choice) — children imported on their own"
         )
+    # D4 (design/superpowers/specs/2026-09-23-doc-import-encoding-design.md): printed last
+    # on stdout, after the degenerate-parent line — the auto-ratify-failures loop below
+    # goes to stderr, so this stays the last stdout line either way.
+    if report.skipped_undecodable:
+        print(
+            f"{report.skipped_undecodable} file(s) skipped: not valid UTF-8, re-save as "
+            "UTF-8 to import:"
+        )
+        for fp in report.undecodable_files:
+            print(f"  {fp}")
     for entry in report.auto_ratify_failures:
         print(f"auto-ratify failure: {entry}", file=sys.stderr)
     return 0

{sidegraph-0.4.0 → sidegraph-0.5.0}/src/sidegraph/doc_import.py

@@ -290,11 +290,8 @@ class _DocDisposition:
 class DocImportReport(BaseModel):
     """Counts (+ dry-run-only listing) for one ``import_docs`` run. Mirrors
     ``importer.ImportReport``'s shape, widened for doc-import's extra outcomes: a doc can
-    be *superseded* (edited since its last import) as well as freshly *imported*, and can
-    be skipped for four distinct reasons (existing/unanchorable/not-decision-shaped/
-    unparseable) plus a fifth doc-import-only reason (superseded/deprecated frontmatter —
-    a history doc, never (re-)imported as live) and a sixth (a document TEMPLATE — see
-    ``skipped_template``)."""
+    be *superseded* (edited since its last import) as well as freshly *imported*, and each
+    reason a document can be skipped for has its own ``skipped_*`` counter below."""
 
     imported: int = 0
     superseded: int = 0
@@ -332,6 +329,15 @@ class DocImportReport(BaseModel):
     # `any_doc=True` was NOT passed and at least one enumerated file falls outside the
     # profile's globs (the CLI's `--any-doc` flag restores today's no-filter behavior).
     skipped_outside_profile: int = 0
+    # design/superpowers/specs/2026-09-23-doc-import-encoding-design.md D1/D2/D3: a file
+    # that passed the profile-glob check above but whose bytes are not valid `utf-8-sig`
+    # (plain UTF-8, plus a stripped BOM — no encoding guessing). The decode is wrapped in a
+    # narrow try/except right where it happens, AFTER the profile check, so the file is a
+    # named skip and the run continues instead of raising `UnicodeDecodeError` and aborting
+    # the whole import. Distinct from `skipped_unparseable` (decision-shaped but no usable
+    # `choice`): this file was never even decoded far enough to be parsed.
+    skipped_undecodable: int = 0
+    undecodable_files: list[str] = Field(default_factory=list)  # on-disk rel_path, in order
     # E3 (design note §7, review Major 4/Blocker N1): a split-produced parent that is
     # never written because it is degenerate — either an echo of its own `context` (rule
     # 1) or empty even after every fallback (rule 2, which previously discarded the whole
@@ -1645,7 +1651,18 @@ def import_docs(
 
         source_bytes = Path(rel_path).read_bytes()
         source_hash = hashlib.sha256(source_bytes).hexdigest()
-        raw_text = source_bytes.decode().replace("\r\n", "\n").replace("\r", "\n")
+        # D1/D2 (design/superpowers/specs/2026-09-23-doc-import-encoding-design.md): decode
+        # as utf-8-sig (plain UTF-8 plus a stripped BOM, never guessed) inside a narrow
+        # try/except right where the decode happens — a file that fails is a named skip,
+        # not an aborted run. D3: this sits AFTER the profile-scope check above, so an
+        # out-of-profile file is still never even opened.
+        try:
+            decoded = source_bytes.decode("utf-8-sig")
+        except UnicodeDecodeError:
+            report.skipped_undecodable += 1
+            report.undecodable_files.append(rel_path)
+            continue
+        raw_text = decoded.replace("\r\n", "\n").replace("\r", "\n")
         # E2 (design note §6): the NORMALIZED path — not the on-disk `rel_path` — is what
         # gets stamped into `context`/children's `effective_ref`/`provenance.ref` (via
         # `DocWriteRequest.rel_path` below), so an unedited doc re-imported after an