mneme-cli 0.5.1__tar.gz → 0.5.2__tar.gz

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/AGENTS.md

@@ -280,6 +280,93 @@ plain-markdown `<details>` fallback so the page is useful outside
 Obsidian. Run after a large ingest, or whenever the wiki's shape
 changes meaningfully.
 
+### 3.9 TRACE — linking the full V-model chain
+
+The trace chain a notified body expects has two legs that both terminate
+at code and tests:
+
+```
+UN  ──implemented-by──┐
+                      ├──> REQ ──detailed-in──> DDS ──implemented-in──> codebase
+RMA ──mitigated-by────┘                             └──verified-by───> tests
+```
+
+The first three links (UN→REQ, RMA→REQ, REQ→DDS) are created
+automatically by the CSV mappings in `profiles/mappings/` (or by
+`mneme trace add` when ingesting structured sources). The last two
+links (DDS→codebase, DDS→tests) close the V-model and are the agent's
+responsibility when a user passes you one or more repositories.
+
+**When the user passes you a repo path, you must:**
+
+```bash
+# 1. Inventory: what code modules / test files exist?
+mneme scan-repo <repo-path> <client>
+# → reports which wiki pages reference the repo's modules, and which do not.
+
+# 2. For each DDS page that corresponds to a code module, add the link.
+#    The target is a git URL or an absolute repo path; mneme treats it
+#    as an opaque string (not a wiki slug) — the target may live outside
+#    the workspace.
+mneme trace add <client>/dds-cyb-001 \
+                "github.com/<org>/<repo>/blob/main/src/auth/password_policy.py" \
+                implemented-in
+
+# 3. For each DDS page that has a corresponding test, add the link.
+#    The test target can be a wiki page (for test-plan docs) or an
+#    external path (for a test file in a repo).
+mneme trace add <client>/dds-cyb-001 <client>/test-auth-001 verified-by
+mneme trace add <client>/dds-cyb-001 \
+                "github.com/<org>/<repo>/blob/main/tests/test_password_policy.py" \
+                verified-by
+```
+
+Do this for every DDS page that has implementing code or a verifying
+test. When there are tens or hundreds of links to create (typical for
+a real medical-device codebase):
+
+```bash
+# Batch approach — the agent parses the repo, maps DDS → files,
+# then writes a shell script of `mneme trace add` lines and runs it.
+# mneme has no bulk-trace-add subcommand yet; scripting is the way.
+for pair in dds-cyb-001:src/auth/password_policy.py \
+            dds-cyb-002:src/auth/mfa.py \
+            dds-cyb-003:src/auth/rate_limiter.py; do
+  dds=${pair%%:*}; file=${pair##*:}
+  mneme trace add <client>/$dds "<repo-url>/$file" implemented-in
+done
+```
+
+**Verify the chain is now complete:**
+
+```bash
+mneme trace gaps <client>
+# → should report 0 hazards without mitigation, 0 DDS without
+#   implementation link, 0 DDS without verification link
+
+mneme trace show <client>/un-001
+# → UN.001
+#     implemented-by -> REQ.SYS.001
+#         detailed-in -> DDS.CYB.001
+#             implemented-in -> github.com/.../password_policy.py
+#             verified-by    -> github.com/.../test_password_policy.py
+```
+
+**Relationship vocabulary — use exactly these strings:**
+
+| Relationship | From → To | Semantics |
+|---|---|---|
+| `implemented-by` | UN → REQ | The user need is met by this requirement |
+| `mitigated-by` | RMA → REQ | The hazard is mitigated by this requirement |
+| `derived-from` | REQ → UN / REQ → higher-level REQ | Parent requirement |
+| `detailed-in` | REQ → DDS | The requirement is elaborated by this design spec |
+| `implemented-in` | DDS → codebase | The design spec is realised by this source file / module |
+| `verified-by` | DDS → test / REQ → test | The spec/requirement is verified by this test |
+| `validated-by` | DDS → clinical/usability study | Validation (not verification) evidence |
+
+Stick to this vocabulary. Custom relationships confuse downstream
+matrix exports and break the default `trace gaps` heuristics.
+
 ---
 
 ## 4. Profiles and the writing-style contract
@@ -572,7 +659,53 @@ file.
 Stop conditions: inbox is empty, `mneme stats` shows a plausible page
 count, and `mneme lint` reports no critical issues.
 
-### 6.6 Pre-submission readiness check before sending to a notified body
+### 6.6 Close the V-model by linking DDS to codebase and tests
+
+The user has just handed you one or more repositories. Your job is to
+connect every DDS page to the implementing source file(s) and the
+verifying test file(s) so `mneme trace show` walks end-to-end from a
+user need / hazard all the way to the exact line of code and the exact
+test that exercises it.
+
+```
+1. mneme profile show                            # sanity check
+2. mneme trace matrix <client>                   # baseline — which DDS exist?
+3. For each repo the user passes:
+     a. mneme scan-repo <repo-path> <client>     # surface module gaps
+     b. Read the repo tree and README yourself.
+        Build a mapping: DDS ID -> [source files]
+                         DDS ID -> [test files]
+        Prefer explicit evidence (comments referencing the DDS ID,
+        module/function names that mirror the DDS title, docstrings
+        that cite the requirement). When evidence is weak, flag the
+        DDS as ambiguous and surface it — do not guess.
+4. For each confident (DDS, file) pair:
+     mneme trace add <client>/<dds-slug> "<repo-url-or-path>/<file>" implemented-in
+     mneme trace add <client>/<dds-slug> "<repo-url-or-path>/<test-file>" verified-by
+   Batch these in a shell loop — there is no bulk-trace-add subcommand.
+5. mneme trace gaps <client>                     # should trend to zero
+6. mneme trace show <client>/un-001              # spot-check: full chain
+                                                   from UN to test file?
+7. mneme trace matrix <client> --csv --out trace-matrix.csv
+                                                 # DHF-ready export
+```
+
+Stop conditions: (a) every DDS page either has both `implemented-in`
+and `verified-by` trace links OR is explicitly flagged ambiguous in a
+report to the user, AND (b) `trace gaps` reports zero open chains.
+
+Hard rules:
+- Do not fabricate file paths. If the repo has no file matching a DDS,
+  report the gap and stop — the user must either point you at another
+  repo or add the link manually.
+- Trace targets for external files are opaque strings. Use a stable
+  form the team can resolve later (a git URL with a pinned commit is
+  ideal; a bare relative path is fine when the repo lives alongside
+  the workspace).
+- Never rewrite a DDS page's body to embed the code link. The link
+  lives in `schema/traceability.json` only. Wiki pages stay prose.
+
+### 6.7 Pre-submission readiness check before sending to a notified body
 
 ```
 1. mneme profile show                            # confirm active profile

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/CHANGELOG.md

@@ -4,6 +4,51 @@ All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.2] - 2026-04-14
+
+### Changed
+
+- **`ingest-dir --preserve-structure` is now the default.** The wiki now
+  mirrors the source directory layout unless you pass `--flat`. This avoids
+  silent same-basename collisions (e.g. multiple `INSTRUCTIONS.md` files from
+  different source directories overwriting each other). Closes suggestion #15.
+- **`mneme ingest` (single-file) also mirrors by default.** When the source
+  lives under `sources/<client>/`, its relative position becomes a wiki
+  subpath automatically. Pass `--flat` to opt out.
+
+### Fixed
+
+- **`mneme profile list`** now discovers profiles correctly. Previously it
+  filtered files by `.json` (wrong extension — profiles are markdown) and
+  only checked the bundled directory, which meant the shipped `eu-mdr.md`
+  and `iso-13485.md` profiles appeared as "No profiles found". Now unions
+  workspace + bundled, marks origin, and flags shadowed bundled profiles.
+  Closes suggestion #25 discovery bug.
+
+### Added
+
+- **`ingest-dir --flat`** — explicit opt-out for the new preserve-structure
+  default.
+- **`ingest --flat`** — opt-out for the single-file command.
+- **xlsx support is now built-in.** `openpyxl` moved from
+  `[project.optional-dependencies].xlsx` to `dependencies`. The `[xlsx]`
+  extra is kept for backwards compatibility but is no longer required.
+
+### Documentation
+
+- **README**: expanded the agent end-to-end example. Step 3 now covers
+  bulk tagging (`tags bulk-suggest` + `bulk-apply`), Step 3b adds entity
+  typing (`entity suggest` + `bulk-apply`), and Step 3c walks the full
+  V-model trace chain (UN→REQ→DDS and RMA→REQ→DDS, terminating at code
+  and tests).
+- **AGENTS.md**: new section 3.9 "TRACE — linking the full V-model
+  chain" documents the `implemented-in` / `verified-by` relationships
+  and the DDS-to-codebase linking agents must perform when the user
+  passes repositories. New task template 6.6 "Close the V-model by
+  linking DDS to codebase and tests" gives the exact procedure, stop
+  conditions, and hard rules (no fabricated paths, trace targets are
+  opaque strings, never embed code links in page bodies).
+
 ## [0.5.0] - 2026-04-13
 
 ### Breaking Changes

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mneme-cli
-Version: 0.5.1
+Version: 0.5.2
 Summary: mneme - CLI tool that turns documents into a searchable second brain. Ingest once, query forever.
 Author-email: Tolis Moustaklis <apostolos.moustaklis@gmail.com>
 License-Expression: MIT
@@ -29,13 +29,13 @@ Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: portalocker>=2.0.0
+Requires-Dist: openpyxl>=3.1.0
 Provides-Extra: pdf
 Requires-Dist: pymupdf>=1.23.0; extra == "pdf"
 Provides-Extra: xlsx
 Requires-Dist: openpyxl>=3.1.0; extra == "xlsx"
 Provides-Extra: all
 Requires-Dist: pymupdf>=1.23.0; extra == "all"
-Requires-Dist: openpyxl>=3.1.0; extra == "all"
 Provides-Extra: release
 Requires-Dist: build>=1.0.0; extra == "release"
 Requires-Dist: twine>=5.0.0; extra == "release"
@@ -183,7 +183,7 @@ One installed CLI serves many projects — each workspace is just a directory.
 | `mneme stats` | Health overview |
 | `mneme repair` | Fix corrupted archives |
 
-**Formats:** `.md`, `.txt`, `.pdf`, `.xlsx` (with `pip install "mneme-cli[xlsx]"`)
+**Formats:** `.md`, `.txt`, `.pdf`, `.xlsx` (built-in), plus `.csv` via `mneme ingest-csv`
 
 ---
 
@@ -238,38 +238,115 @@ Creates the workspace tree, sets the EU MDR writing-style profile, and initializ
 cp -r ~/Downloads/parkinson-research/* inbox/
 mneme tornado --client parkiwatch
 
-# Or ingest individual files
+# Or ingest individual files (auto-mirrors sources/<client>/ layout into wiki/)
 mneme ingest research-paper.pdf parkiwatch
-mneme ingest-csv risk-register.csv parkiwatch --mapping risk-register
 mneme ingest spec-table.xlsx parkiwatch          # .xlsx renders sheets as markdown tables
-mneme ingest-dir docs/ parkiwatch --recursive    # walk subdirectories
+mneme ingest-dir docs/ parkiwatch --recursive    # walk subdirectories, preserve structure
+
+# Structured CSV ingestion — one row becomes one wiki page + trace links.
+# Mappings live in <workspace>/profiles/mappings/ or are auto-detected.
+mneme ingest-csv user-needs.csv    parkiwatch --mapping parkiwatch-user-needs
+mneme ingest-csv requirements.csv  parkiwatch --mapping parkiwatch-req
+mneme ingest-csv design-specs.csv  parkiwatch --mapping parkiwatch-dds
+mneme ingest-csv risk-register.csv parkiwatch --mapping parkiwatch-rma
 ```
 
-What happens per ingest: source file → wiki page in `wiki/parkiwatch/` → frontmatter with auto-extracted entities → entry in `index.md` → row in the FTS5 search DB → log entry.
+What happens per ingest: source file → wiki page in `wiki/parkiwatch/<mirrored-subpath>/` → frontmatter with auto-extracted proper-noun entities → entry in `index.md` → row in the FTS5 search DB → log entry. CSV ingests additionally create trace links (e.g. UN→REQ `implemented-by`, REQ→DDS `detailed-in`) in `schema/traceability.json`.
 
-### Step 3 — Tag the new pages (LLM agent)
+### Step 3 — Tag many pages at once (LLM agent, bulk)
 
-The new pages have only the auto-applied `parkiwatch` client tag. The agent now adds meaningful tags:
+New pages have only the auto-applied `parkiwatch` client tag. The agent tags them in batches:
 
 ```bash
-# For each new page, the agent runs:
-mneme tags suggest parkiwatch/research-paper > /tmp/packet.md
+# 1. Pack up to 30 untagged pages into a single review packet.
+#    --filter scopes by wiki_path substring; omit for everything.
+mneme tags bulk-suggest --filter indicators --limit 30 \
+                        --json --out /tmp/tag-packet.json
 ```
 
-The packet contains the page body, the current tag taxonomy (every tag in the workspace + usage counts), and a ready-to-paste prompt. **The LLM reads the packet** — it understands the content and decides on tags, preferring existing taxonomy entries when they fit. The LLM's response is JSON:
+The packet contains, for each page: wiki_path, title, current tags, body excerpt, and the existing taxonomy with usage counts. **The LLM reads the packet** and returns a response JSON:
 
 ```json
-{"tags": ["clinical-trial", "iso-13485"], "new_tags": ["bradykinesia-detection"]}
+{
+  "pages": [
+    {"wiki_path": "parkiwatch/indicators/bda_algorithm_description.md",
+     "add": ["bradykinesia", "algorithm", "imu", "medical-device"]},
+    {"wiki_path": "parkiwatch/indicators/tremor_indicator_dataflow.md",
+     "add": ["tremor", "dataflow", "imu", "algorithm"]}
+  ]
+}
 ```
 
-The agent then runs:
+```bash
+# 2. Apply all decisions in one atomic call
+mneme tags bulk-apply /tmp/tag-response.json
+# → Pages updated: 9   Tags added: 42   Tags removed: 0
+```
+
+Each application rewrites the wiki page frontmatter, updates `schema/tags.json`, re-indexes the page in FTS5, and appends a log entry. Subsequent packets reuse the growing taxonomy, so the vocabulary converges.
+
+For single pages use `mneme tags suggest <slug>` + `mneme tags apply <slug> --add a,b,c`.
+
+### Step 3b — Classify entities by type (LLM agent)
+
+Ingest auto-extracts capitalized proper nouns (e.g. "Parkiwatch", "IEC 62304") into `schema/entities.json` with `type: unknown`. Typing is an LLM judgement call, handled the same packet way as tags:
 
 ```bash
-mneme tags apply parkiwatch/research-paper \
-  --add clinical-trial,iso-13485,bradykinesia-detection
+# 1. Build an entity-classification packet (up to 50 unclassified entities)
+mneme entity suggest --client parkiwatch --limit 50 \
+                     --json --out /tmp/entity-packet.json
+
+# 2. LLM reads the packet and returns classifications:
+#    [{"id": "iec-62304", "type": "standard"},
+#     {"id": "notified-body", "type": "organization"},
+#     {"id": "bradykinesia", "type": "concept"}, ...]
+
+# 3. Apply atomically
+mneme entity bulk-apply /tmp/entity-response.json
+# → Entities typed: 47   Errors: 0
+```
+
+Supported types include `standard`, `organization`, `person`, `concept`, `technology`, `regulation`, or any custom type the profile defines. Typed entities power filtered search and the knowledge graph.
+
+### Step 3c — Verify the trace chain (human, on demand)
+
+The CSV ingests in Step 2 created two parallel trace chains. Both converge at a requirement, drill into design specs, and finally terminate at **code** and **tests** — the complete QMS traceability an auditor expects:
+
+```
+Chain A:  UN ─┐
+              ├─> REQ ──> DDS ──┬─> codebase  (via `implemented-in`)
+Chain B:  RMA ┘                 └─> tests     (via `verified-by`)
 ```
 
-Atomic operation: rewrites the wiki page frontmatter, updates `schema/tags.json`, re-indexes the page in FTS5 (so search picks up the new tags immediately), appends a log entry. **Repeat for every page** — the taxonomy grows, and subsequent pages tend to reuse existing tags (consistency).
+Each arrow is a trace-link relationship type (`implemented-by`, `mitigated-by`, `detailed-in`, `implemented-in`, `verified-by`). The DDS→codebase link is stored as a frontmatter field on each DDS page (e.g. a git URL pointing at the implementing module). The DDS→tests link is a standard trace relationship added either by CSV ingest or by `mneme trace add`.
+
+Walk either chain from any root page:
+
+```bash
+# Chain A — from a user need forward to the specs that implement it
+mneme trace show parkiwatch/un-001
+# → UN.001 (secure sign-in)
+#     implemented-by -> REQ.SYS.001 (User Authentication)
+#         detailed-in -> DDS.CYB.001 (Strong Password Policy)
+#         detailed-in -> DDS.CYB.002 (Multi-Factor Authentication)
+#         ...
+
+# Chain B — from a hazard forward to the specs that mitigate it
+mneme trace show parkiwatch/rma-cyb-002
+# → RMA.CYB.002 (Unauthorized access -- weak passwords)
+#     mitigated-by -> REQ.SYS.001 (User Authentication)
+#         detailed-in -> DDS.CYB.001, DDS.CYB.002, ...
+#             implemented-in -> src/auth/password_policy.py   (codebase)
+#             verified-by    -> TEST.AUTH.001                 (tests)
+
+# Trace gaps for a notified body audit
+mneme trace gaps parkiwatch
+# → Hazards with no mitigation: ...
+#   User needs with no requirements: ...
+
+# Export the full traceability matrix for the DHF
+mneme trace matrix parkiwatch --csv --out trace-matrix.csv
+```
 
 ### Step 4 — Search the knowledge base (anyone)
 

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/README.md

@@ -140,7 +140,7 @@ One installed CLI serves many projects — each workspace is just a directory.
 | `mneme stats` | Health overview |
 | `mneme repair` | Fix corrupted archives |
 
-**Formats:** `.md`, `.txt`, `.pdf`, `.xlsx` (with `pip install "mneme-cli[xlsx]"`)
+**Formats:** `.md`, `.txt`, `.pdf`, `.xlsx` (built-in), plus `.csv` via `mneme ingest-csv`
 
 ---
 
@@ -195,38 +195,115 @@ Creates the workspace tree, sets the EU MDR writing-style profile, and initializ
 cp -r ~/Downloads/parkinson-research/* inbox/
 mneme tornado --client parkiwatch
 
-# Or ingest individual files
+# Or ingest individual files (auto-mirrors sources/<client>/ layout into wiki/)
 mneme ingest research-paper.pdf parkiwatch
-mneme ingest-csv risk-register.csv parkiwatch --mapping risk-register
 mneme ingest spec-table.xlsx parkiwatch          # .xlsx renders sheets as markdown tables
-mneme ingest-dir docs/ parkiwatch --recursive    # walk subdirectories
+mneme ingest-dir docs/ parkiwatch --recursive    # walk subdirectories, preserve structure
+
+# Structured CSV ingestion — one row becomes one wiki page + trace links.
+# Mappings live in <workspace>/profiles/mappings/ or are auto-detected.
+mneme ingest-csv user-needs.csv    parkiwatch --mapping parkiwatch-user-needs
+mneme ingest-csv requirements.csv  parkiwatch --mapping parkiwatch-req
+mneme ingest-csv design-specs.csv  parkiwatch --mapping parkiwatch-dds
+mneme ingest-csv risk-register.csv parkiwatch --mapping parkiwatch-rma
 ```
 
-What happens per ingest: source file → wiki page in `wiki/parkiwatch/` → frontmatter with auto-extracted entities → entry in `index.md` → row in the FTS5 search DB → log entry.
+What happens per ingest: source file → wiki page in `wiki/parkiwatch/<mirrored-subpath>/` → frontmatter with auto-extracted proper-noun entities → entry in `index.md` → row in the FTS5 search DB → log entry. CSV ingests additionally create trace links (e.g. UN→REQ `implemented-by`, REQ→DDS `detailed-in`) in `schema/traceability.json`.
 
-### Step 3 — Tag the new pages (LLM agent)
+### Step 3 — Tag many pages at once (LLM agent, bulk)
 
-The new pages have only the auto-applied `parkiwatch` client tag. The agent now adds meaningful tags:
+New pages have only the auto-applied `parkiwatch` client tag. The agent tags them in batches:
 
 ```bash
-# For each new page, the agent runs:
-mneme tags suggest parkiwatch/research-paper > /tmp/packet.md
+# 1. Pack up to 30 untagged pages into a single review packet.
+#    --filter scopes by wiki_path substring; omit for everything.
+mneme tags bulk-suggest --filter indicators --limit 30 \
+                        --json --out /tmp/tag-packet.json
 ```
 
-The packet contains the page body, the current tag taxonomy (every tag in the workspace + usage counts), and a ready-to-paste prompt. **The LLM reads the packet** — it understands the content and decides on tags, preferring existing taxonomy entries when they fit. The LLM's response is JSON:
+The packet contains, for each page: wiki_path, title, current tags, body excerpt, and the existing taxonomy with usage counts. **The LLM reads the packet** and returns a response JSON:
 
 ```json
-{"tags": ["clinical-trial", "iso-13485"], "new_tags": ["bradykinesia-detection"]}
+{
+  "pages": [
+    {"wiki_path": "parkiwatch/indicators/bda_algorithm_description.md",
+     "add": ["bradykinesia", "algorithm", "imu", "medical-device"]},
+    {"wiki_path": "parkiwatch/indicators/tremor_indicator_dataflow.md",
+     "add": ["tremor", "dataflow", "imu", "algorithm"]}
+  ]
+}
 ```
 
-The agent then runs:
+```bash
+# 2. Apply all decisions in one atomic call
+mneme tags bulk-apply /tmp/tag-response.json
+# → Pages updated: 9   Tags added: 42   Tags removed: 0
+```
+
+Each application rewrites the wiki page frontmatter, updates `schema/tags.json`, re-indexes the page in FTS5, and appends a log entry. Subsequent packets reuse the growing taxonomy, so the vocabulary converges.
+
+For single pages use `mneme tags suggest <slug>` + `mneme tags apply <slug> --add a,b,c`.
+
+### Step 3b — Classify entities by type (LLM agent)
+
+Ingest auto-extracts capitalized proper nouns (e.g. "Parkiwatch", "IEC 62304") into `schema/entities.json` with `type: unknown`. Typing is an LLM judgement call, handled the same packet way as tags:
 
 ```bash
-mneme tags apply parkiwatch/research-paper \
-  --add clinical-trial,iso-13485,bradykinesia-detection
+# 1. Build an entity-classification packet (up to 50 unclassified entities)
+mneme entity suggest --client parkiwatch --limit 50 \
+                     --json --out /tmp/entity-packet.json
+
+# 2. LLM reads the packet and returns classifications:
+#    [{"id": "iec-62304", "type": "standard"},
+#     {"id": "notified-body", "type": "organization"},
+#     {"id": "bradykinesia", "type": "concept"}, ...]
+
+# 3. Apply atomically
+mneme entity bulk-apply /tmp/entity-response.json
+# → Entities typed: 47   Errors: 0
+```
+
+Supported types include `standard`, `organization`, `person`, `concept`, `technology`, `regulation`, or any custom type the profile defines. Typed entities power filtered search and the knowledge graph.
+
+### Step 3c — Verify the trace chain (human, on demand)
+
+The CSV ingests in Step 2 created two parallel trace chains. Both converge at a requirement, drill into design specs, and finally terminate at **code** and **tests** — the complete QMS traceability an auditor expects:
+
+```
+Chain A:  UN ─┐
+              ├─> REQ ──> DDS ──┬─> codebase  (via `implemented-in`)
+Chain B:  RMA ┘                 └─> tests     (via `verified-by`)
 ```
 
-Atomic operation: rewrites the wiki page frontmatter, updates `schema/tags.json`, re-indexes the page in FTS5 (so search picks up the new tags immediately), appends a log entry. **Repeat for every page** — the taxonomy grows, and subsequent pages tend to reuse existing tags (consistency).
+Each arrow is a trace-link relationship type (`implemented-by`, `mitigated-by`, `detailed-in`, `implemented-in`, `verified-by`). The DDS→codebase link is stored as a frontmatter field on each DDS page (e.g. a git URL pointing at the implementing module). The DDS→tests link is a standard trace relationship added either by CSV ingest or by `mneme trace add`.
+
+Walk either chain from any root page:
+
+```bash
+# Chain A — from a user need forward to the specs that implement it
+mneme trace show parkiwatch/un-001
+# → UN.001 (secure sign-in)
+#     implemented-by -> REQ.SYS.001 (User Authentication)
+#         detailed-in -> DDS.CYB.001 (Strong Password Policy)
+#         detailed-in -> DDS.CYB.002 (Multi-Factor Authentication)
+#         ...
+
+# Chain B — from a hazard forward to the specs that mitigate it
+mneme trace show parkiwatch/rma-cyb-002
+# → RMA.CYB.002 (Unauthorized access -- weak passwords)
+#     mitigated-by -> REQ.SYS.001 (User Authentication)
+#         detailed-in -> DDS.CYB.001, DDS.CYB.002, ...
+#             implemented-in -> src/auth/password_policy.py   (codebase)
+#             verified-by    -> TEST.AUTH.001                 (tests)
+
+# Trace gaps for a notified body audit
+mneme trace gaps parkiwatch
+# → Hazards with no mitigation: ...
+#   User needs with no requirements: ...
+
+# Export the full traceability matrix for the DHF
+mneme trace matrix parkiwatch --csv --out trace-matrix.csv
+```
 
 ### Step 4 — Search the knowledge base (anyone)
 

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/mneme/__init__.py

@@ -5,4 +5,4 @@ Public API:
     from mneme.core import ingest_source_to_both, dual_search, ...
 """
 
-__version__ = "0.5.1"
+__version__ = "0.5.2"

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/mneme/core.py

@@ -565,7 +565,7 @@ def ingest_source_to_both(source_path: str, client_slug: str, force: bool = Fals
             raw_content = '\n\n'.join(sheets)
         except ImportError:
             raise ValueError(
-                'Excel extraction requires openpyxl. Install: pip install "mneme-cli[xlsx]"'
+                'Excel extraction requires openpyxl. Install: pip install openpyxl'
             )
     else:
         # Generic text fallback
@@ -2103,7 +2103,7 @@ def lint() -> dict:
 
 
 def ingest_dir(directory: str, client_slug: str, force: bool = False,
-               recursive: bool = False, preserve_structure: bool = False) -> dict:
+               recursive: bool = False, preserve_structure: bool = True) -> dict:
     """
     Batch ingest all supported files from a directory.
 
@@ -2112,9 +2112,10 @@ def ingest_dir(directory: str, client_slug: str, force: bool = False,
 
     When recursive=True, walks subdirectories as well.
 
-    When preserve_structure=True, each file's directory position relative to
-    ``directory`` becomes a wiki subdirectory under ``wiki/<client>/``. Also
-    naturally resolves same-basename collisions (suggestion #15).
+    When preserve_structure=True (the default), each file's directory position
+    relative to ``directory`` becomes a wiki subdirectory under
+    ``wiki/<client>/``. Also naturally resolves same-basename collisions
+    (suggestion #15). Pass preserve_structure=False for a flat wiki.
 
     Returns a summary of all ingestions.
     """
@@ -2154,10 +2155,18 @@ def ingest_dir(directory: str, client_slug: str, force: bool = False,
 
     for fpath in files:
         fname = os.path.basename(fpath)
-        # Compute subpath relative to the input directory when preserving structure
         if preserve_structure:
-            sub_rel = os.path.relpath(os.path.dirname(fpath), directory)
-            subpath = '' if sub_rel in ('', '.') else sub_rel
+            # Prefer the path relative to sources/<client>/ when the input lives
+            # there, so callers running `ingest-dir sources/<client>/SUBDIR` get
+            # the SUBDIR prefix in the wiki tree (rather than silently flattening
+            # because SUBDIR itself has no nested subdirectories). Falls back to
+            # relative-to-input-directory for sources outside the canonical tree.
+            auto = _auto_detect_subpath(fpath, client_slug)
+            if auto:
+                subpath = auto
+            else:
+                sub_rel = os.path.relpath(os.path.dirname(fpath), directory)
+                subpath = '' if sub_rel in ('', '.') else sub_rel
         else:
             subpath = ''
         try:
@@ -6241,6 +6250,8 @@ def main() -> None:
     ingest_parser.add_argument('file', help='Path to source file (.md, .txt, .pdf)')
     ingest_parser.add_argument('client_slug', help='Client slug (e.g. demo-retail, my-client)')
     ingest_parser.add_argument('--force', action='store_true', help='Re-ingest even if source was previously ingested')
+    ingest_parser.add_argument('--flat', action='store_true',
+                               help='Write the page directly to wiki/<client>/ without mirroring source subpath')
 
     # init
     init_parser = subparsers.add_parser('init', help='Initialize a new mneme workspace')
@@ -6256,8 +6267,13 @@ def main() -> None:
     ingest_dir_parser.add_argument('client_slug', help='Client slug (e.g. demo-retail, my-client)')
     ingest_dir_parser.add_argument('--force', action='store_true', help='Re-ingest even if sources were previously ingested')
     ingest_dir_parser.add_argument('--recursive', '-r', action='store_true', help='Recurse into subdirectories')
-    ingest_dir_parser.add_argument('--preserve-structure', dest='preserve_structure', action='store_true',
-                                   help='Mirror source directory structure into wiki/<client>/ subdirectories')
+    # Default-on since v0.5.2: mirror source directory structure into the wiki.
+    # --flat is the explicit opt-out for users who want a single-directory wiki.
+    ingest_dir_parser.add_argument('--preserve-structure', dest='preserve_structure',
+                                   action='store_true', default=True,
+                                   help='(default) Mirror source directory structure into wiki/<client>/ subdirectories')
+    ingest_dir_parser.add_argument('--flat', dest='preserve_structure', action='store_false',
+                                   help='Write all pages to wiki/<client>/ without subdirectories')
 
     # tornado
     tornado_parser = subparsers.add_parser('tornado', help='Process inbox: auto-detect, ingest, archive')
@@ -6546,7 +6562,12 @@ def main() -> None:
             print(f'Error: invalid client slug "{args.client_slug}". Use lowercase letters, numbers, hyphens only.', file=sys.stderr)
             sys.exit(1)
         try:
-            result = ingest_source_to_both(args.file, args.client_slug, force=args.force)
+            # Auto-mirror the source's position under sources/<client>/ into the
+            # wiki, unless --flat is passed. This makes single-file ingest match
+            # the default ingest-dir behavior (preserve structure) and avoids
+            # same-basename collisions across different source directories.
+            auto_sub = '' if args.flat else _auto_detect_subpath(args.file, args.client_slug)
+            result = ingest_source_to_both(args.file, args.client_slug, force=args.force, subpath=auto_sub)
             if not result:
                 # Skipped due to duplicate detection
                 sys.exit(0)
@@ -6821,21 +6842,42 @@ def main() -> None:
 
     elif args.command == 'profile':
         if args.profile_command == 'list':
-            if os.path.exists(PROFILES_DIR):
-                profiles = [f[:-5] for f in os.listdir(PROFILES_DIR) if f.endswith('.json')]
-                active = None
-                if os.path.exists(ACTIVE_PROFILE_FILE):
-                    with open(ACTIVE_PROFILE_FILE, 'r') as f:
-                        active = f.read().strip()
-                if profiles:
-                    print('Available profiles:\n')
-                    for p in sorted(profiles):
-                        marker = ' (active)' if p == active else ''
-                        print(f'  - {p}{marker}')
-                else:
-                    print('No profiles found in profiles/ directory.')
+            # Profiles are markdown files. Look in both the workspace profiles
+            # directory (per-project overrides) and the bundled profiles
+            # directory (shipped with mneme). Workspace entries shadow bundled
+            # ones with the same name.
+            workspace_profiles: dict[str, str] = {}
+            bundled_profiles: dict[str, str] = {}
+            if os.path.isdir(WORKSPACE_PROFILES_DIR):
+                for f in os.listdir(WORKSPACE_PROFILES_DIR):
+                    if f.endswith('.md'):
+                        workspace_profiles[f[:-3]] = 'workspace'
+            if os.path.isdir(PROFILES_DIR):
+                for f in os.listdir(PROFILES_DIR):
+                    if f.endswith('.md'):
+                        bundled_profiles[f[:-3]] = 'bundled'
+
+            merged = {**bundled_profiles, **workspace_profiles}  # workspace overrides
+            active = None
+            if os.path.exists(ACTIVE_PROFILE_FILE):
+                with open(ACTIVE_PROFILE_FILE, 'r') as f:
+                    active = f.read().strip()
+
+            if merged:
+                print('Available profiles:\n')
+                for p in sorted(merged):
+                    origin = merged[p]
+                    shadowed = origin == 'workspace' and p in bundled_profiles
+                    markers = []
+                    if p == active:
+                        markers.append('active')
+                    markers.append(origin)
+                    if shadowed:
+                        markers.append('shadows bundled')
+                    print(f'  - {p}  [{", ".join(markers)}]')
             else:
-                print('No profiles/ directory found.')
+                print('No profiles found.')
+                print(f'  Searched: {WORKSPACE_PROFILES_DIR} (workspace) and {PROFILES_DIR} (bundled).')
         elif args.profile_command == 'set':
             try:
                 set_active_profile(args.name)

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/pyproject.toml

@@ -32,12 +32,14 @@ classifiers = [
 ]
 dependencies = [
     "portalocker>=2.0.0",
+    "openpyxl>=3.1.0",
 ]
 
 [project.optional-dependencies]
 pdf = ["pymupdf>=1.23.0"]
+# Kept for backwards compatibility — xlsx support is now built-in.
 xlsx = ["openpyxl>=3.1.0"]
-all = ["pymupdf>=1.23.0", "openpyxl>=3.1.0"]
+all = ["pymupdf>=1.23.0"]
 release = [
     "build>=1.0.0",
     "twine>=5.0.0",

{mneme_cli-0.5.1 → mneme_cli-0.5.2}/tests/test_core.py

@@ -398,7 +398,13 @@ class TestIngestDirPreserveStructure:
             with open(full, 'w') as f:
                 f.write(content)
 
-    def test_flat_default_unchanged(self, sync_workspace):
+    def test_preserve_structure_is_default(self, sync_workspace):
+        """v0.5.2+: ingest-dir mirrors source structure by default.
+
+        Previously flat by default; flipped because flat wikis silently
+        overwrite pages that share a basename across different source dirs
+        (suggestion #15).
+        """
         from mneme.core import ingest_dir
         self._make_source_tree(sync_workspace, {
             'demo/REQUIREMENTS/req-001.md': '# req 1',
@@ -406,9 +412,25 @@ class TestIngestDirPreserveStructure:
         })
         ingest_dir(os.path.join(sync_workspace, 'sources', 'demo'),
                    'demo', recursive=True)
-        # Without --preserve-structure, both pages flatten
+        # Default now mirrors source layout
+        assert os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'requirements', 'req-001.md'))
+        assert os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'design', 'dds-001.md'))
+        # Flat-mode pages should NOT exist
+        assert not os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'req-001.md'))
+        assert not os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'dds-001.md'))
+
+    def test_flat_opt_out_still_works(self, sync_workspace):
+        """Callers can still request a flat wiki with preserve_structure=False."""
+        from mneme.core import ingest_dir
+        self._make_source_tree(sync_workspace, {
+            'demo/REQUIREMENTS/req-001.md': '# req 1',
+            'demo/DESIGN/dds-001.md': '# dds 1',
+        })
+        ingest_dir(os.path.join(sync_workspace, 'sources', 'demo'),
+                   'demo', recursive=True, preserve_structure=False)
         assert os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'req-001.md'))
         assert os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'dds-001.md'))
+        assert not os.path.exists(os.path.join(sync_workspace, 'wiki', 'demo', 'requirements', 'req-001.md'))
 
     def test_preserve_structure_creates_subdirs(self, sync_workspace):
         from mneme.core import ingest_dir
@@ -830,6 +852,127 @@ class TestCLI:
         rc, out, err = _run_mnemo('search', 'xyznonexistent12345qqqzzz')
         assert rc == 0
 
+    def test_profile_list_discovers_bundled_markdown_profiles(self):
+        """Regression: profile list used to filter .json (wrong ext) and only
+        look at the bundled dir. Bundled profiles ship as .md files."""
+        td = tempfile.mkdtemp(prefix='mneme-profile-list-')
+        try:
+            for sub in ('wiki', 'sources', 'schema'):
+                os.makedirs(os.path.join(td, sub), exist_ok=True)
+            with open(os.path.join(td, 'index.md'), 'w') as f:
+                f.write('# Index\n')
+            with open(os.path.join(td, 'log.md'), 'w') as f:
+                f.write('# Log\n')
+            for name, empty in (
+                ('entities.json', {'version': 1, 'updated': '2026-01-01', 'entities': []}),
+                ('tags.json', {'version': 1, 'updated': '2026-01-01', 'tags': {}}),
+                ('graph.json', {'version': 1, 'updated': '2026-01-01', 'nodes': [], 'edges': []}),
+            ):
+                with open(os.path.join(td, 'schema', name), 'w') as f:
+                    json.dump(empty, f)
+
+            rc, out, err = _run_mnemo('--workspace', td, 'profile', 'list')
+            assert rc == 0, f'profile list failed: {err}'
+            assert 'eu-mdr' in out, f'bundled eu-mdr not listed. got: {out}'
+            assert 'iso-13485' in out, f'bundled iso-13485 not listed. got: {out}'
+            assert 'bundled' in out, f'origin marker missing. got: {out}'
+        finally:
+            shutil.rmtree(td, ignore_errors=True)
+
+    def test_single_ingest_auto_detects_subpath_from_sources_tree(self):
+        """`mneme ingest` should mirror a file's position under sources/<client>/
+        by default (suggestion #15 — avoid silent basename collisions)."""
+        td = tempfile.mkdtemp(prefix='mneme-ingest-auto-')
+        try:
+            for sub in ('wiki', 'sources', 'schema'):
+                os.makedirs(os.path.join(td, sub), exist_ok=True)
+            with open(os.path.join(td, 'index.md'), 'w') as f:
+                f.write('# Index\n')
+            with open(os.path.join(td, 'log.md'), 'w') as f:
+                f.write('# Log\n')
+            for name, empty in (
+                ('entities.json', {'version': 1, 'updated': '2026-01-01', 'entities': []}),
+                ('tags.json', {'version': 1, 'updated': '2026-01-01', 'tags': {}}),
+                ('graph.json', {'version': 1, 'updated': '2026-01-01', 'nodes': [], 'edges': []}),
+            ):
+                with open(os.path.join(td, 'schema', name), 'w') as f:
+                    json.dump(empty, f)
+
+            src = os.path.join(td, 'sources', 'demo', 'TRACE', 'REQ', 'req-001.md')
+            os.makedirs(os.path.dirname(src), exist_ok=True)
+            with open(src, 'w') as f:
+                f.write('# Req 1\n\nFiller body content for a test.\n')
+
+            rc, out, err = _run_mnemo('--workspace', td, 'ingest', src, 'demo')
+            assert rc == 0, f'ingest failed: {err}'
+            nested = os.path.join(td, 'wiki', 'demo', 'trace', 'req', 'req-001.md')
+            flat = os.path.join(td, 'wiki', 'demo', 'req-001.md')
+            assert os.path.exists(nested), f'nested page missing at {nested}'
+            assert not os.path.exists(flat), 'flat page should not exist by default'
+        finally:
+            shutil.rmtree(td, ignore_errors=True)
+
+    def test_single_ingest_flat_flag_opts_out(self):
+        """`mneme ingest --flat` keeps the old flat-wiki behavior."""
+        td = tempfile.mkdtemp(prefix='mneme-ingest-flat-')
+        try:
+            for sub in ('wiki', 'sources', 'schema'):
+                os.makedirs(os.path.join(td, sub), exist_ok=True)
+            with open(os.path.join(td, 'index.md'), 'w') as f:
+                f.write('# Index\n')
+            with open(os.path.join(td, 'log.md'), 'w') as f:
+                f.write('# Log\n')
+            for name, empty in (
+                ('entities.json', {'version': 1, 'updated': '2026-01-01', 'entities': []}),
+                ('tags.json', {'version': 1, 'updated': '2026-01-01', 'tags': {}}),
+                ('graph.json', {'version': 1, 'updated': '2026-01-01', 'nodes': [], 'edges': []}),
+            ):
+                with open(os.path.join(td, 'schema', name), 'w') as f:
+                    json.dump(empty, f)
+
+            src = os.path.join(td, 'sources', 'demo', 'TRACE', 'req-001.md')
+            os.makedirs(os.path.dirname(src), exist_ok=True)
+            with open(src, 'w') as f:
+                f.write('# Req 1\n\nFiller body content for a test.\n')
+
+            rc, out, err = _run_mnemo('--workspace', td, 'ingest', src, 'demo', '--flat')
+            assert rc == 0, f'ingest failed: {err}'
+            flat = os.path.join(td, 'wiki', 'demo', 'req-001.md')
+            nested = os.path.join(td, 'wiki', 'demo', 'trace', 'req-001.md')
+            assert os.path.exists(flat)
+            assert not os.path.exists(nested)
+        finally:
+            shutil.rmtree(td, ignore_errors=True)
+
+    def test_profile_list_shows_workspace_override(self):
+        """Workspace profiles should shadow bundled ones of the same name."""
+        td = tempfile.mkdtemp(prefix='mneme-profile-override-')
+        try:
+            for sub in ('wiki', 'sources', 'schema', 'profiles'):
+                os.makedirs(os.path.join(td, sub), exist_ok=True)
+            with open(os.path.join(td, 'index.md'), 'w') as f:
+                f.write('# Index\n')
+            with open(os.path.join(td, 'log.md'), 'w') as f:
+                f.write('# Log\n')
+            for name, empty in (
+                ('entities.json', {'version': 1, 'updated': '2026-01-01', 'entities': []}),
+                ('tags.json', {'version': 1, 'updated': '2026-01-01', 'tags': {}}),
+                ('graph.json', {'version': 1, 'updated': '2026-01-01', 'nodes': [], 'edges': []}),
+            ):
+                with open(os.path.join(td, 'schema', name), 'w') as f:
+                    json.dump(empty, f)
+            with open(os.path.join(td, 'profiles', 'eu-mdr.md'), 'w') as f:
+                f.write('---\nname: Override\ndescription: local\n---\n# Principles\n- test\n')
+            with open(os.path.join(td, 'profiles', 'custom.md'), 'w') as f:
+                f.write('---\nname: Custom\ndescription: workspace-only\n---\n# Principles\n- test\n')
+
+            rc, out, err = _run_mnemo('--workspace', td, 'profile', 'list')
+            assert rc == 0
+            assert 'custom' in out
+            assert 'shadows bundled' in out, f'override marker missing. got: {out}'
+        finally:
+            shutil.rmtree(td, ignore_errors=True)
+
     def test_ingest_missing_file_exits_nonzero(self):
         rc, out, err = _run_mnemo('ingest', '/tmp/nonexistent_file_xyz_mnemo.md', 'test')
         assert rc == 1