vgxness 1.20.13 → 1.20.15

package/docs/sdd/sqlite-obsidian-knowledge-workflow/design.md

@@ -0,0 +1,118 @@
+# Design: SQLite + Obsidian MCP Knowledge Workflow
+
+## Technical Approach
+
+Introduce a knowledge workspace layer that projects existing VGXNESS SQLite/repository state into Obsidian through a small port. The port has an Obsidian MCP adapter as the preferred implementation and a filesystem adapter for tests/offline fallback. SQLite remains the source of truth for operational state.
+
+## Architecture Decisions
+
+### Decision: SQLite remains authoritative
+
+**Choice:** Keep memory, SDD artifacts, acceptance, runs, traces, checkpoints, and approvals in SQLite.
+
+**Alternatives considered:** Use Obsidian Markdown as the primary memory store; use bidirectional Obsidian/SQLite sync from the start.
+
+**Rationale:** Agent memory and SDD governance require structured queries, revisions, topic keys, readiness checks, and safe transactional state. Obsidian is excellent for human review, but not for authoritative runtime state.
+
+### Decision: Obsidian MCP is the preferred workspace transport
+
+**Choice:** Use an Obsidian MCP server for note read/write/search/patch operations.
+
+**Alternatives considered:** Direct filesystem writes only; implementing a VGXNESS-specific Obsidian plugin.
+
+**Rationale:** MCP gives agents a standard, inspectable transport and supports structured note operations such as patching, frontmatter management, tags, and open-in-UI without coupling VGXNESS directly to Obsidian internals.
+
+### Decision: Hide server-specific details behind a port
+
+**Choice:** Add a `KnowledgeWorkspacePort` seam before product services depend on Obsidian operations.
+
+**Alternatives considered:** Call a specific MCP package directly throughout exporters.
+
+**Rationale:** Community MCP servers can change. A port keeps VGXNESS code testable and allows a filesystem adapter for deterministic tests.
+
+### Decision: Do not mirror Obsidian tools in VGXNESS MCP
+
+**Choice:** Keep detailed note operations in the external Obsidian MCP server and add at most one grouped VGXNESS sync tool later.
+
+**Rationale:** The product direction is to minimize MCP/tool sprawl. VGXNESS should orchestrate product-level workflows; Obsidian MCP should own vault-level operations.
+
+## Data Flow
+
+```text
+SQLite/repo docs
+  -> projection query/read services
+  -> Markdown/Canvas renderers
+  -> KnowledgeWorkspacePort
+     -> ObsidianMcpWorkspacePort
+     -> FilesystemWorkspacePort (fallback/tests)
+  -> Obsidian vault / VGXNESS/
+```
+
+## Planned File Changes
+
+| File | Action | Description |
+|---|---|---|
+| `docs/knowledge-workspace.md` | Create | Canonical product boundary and workflow. |
+| `docs/obsidian-mcp.md` | Create | Setup and safety docs for Obsidian MCP. |
+| `src/domain/knowledge-workspace/schema.ts` | Create | Domain types for status, notes, writes, export kinds, and workspace mode. |
+| `src/application/knowledge-workspace/markdown-renderer.ts` | Create | Pure Markdown rendering for docs, memory summaries, and SDD boards. |
+| `src/application/knowledge-workspace/canvas-renderer.ts` | Create | Deterministic Obsidian Canvas JSON rendering. |
+| `src/application/knowledge-workspace/workspace-port.ts` | Create | Transport-agnostic workspace interface. |
+| `src/application/knowledge-workspace/obsidian-mcp-workspace-port.ts` | Create | Adapter mapping workspace operations to Obsidian MCP tool calls. |
+| `src/application/knowledge-workspace/filesystem-workspace-port.ts` | Create | Explicit fallback/test adapter with matching safety semantics. |
+| `src/application/knowledge-workspace/obsidian-export-service.ts` | Create | Orchestrates docs/memory/SDD exports through the workspace port. |
+| `src/interfaces/cli/commands/knowledge-workspace-dispatcher.ts` | Create | Grouped `vgxness knowledge status|init|export` commands. |
+
+## Interface Sketch
+
+```ts
+export interface KnowledgeWorkspacePort {
+  status(): Promise<KnowledgeWorkspaceStatus>;
+  listNotes(input: { pathPrefix: string; recursive?: boolean }): Promise<KnowledgeWorkspaceNoteRef[]>;
+  getNote(input: { path: string }): Promise<KnowledgeWorkspaceNote>;
+  writeGeneratedNote(input: { path: string; content: string; overwriteGeneratedOnly: boolean }): Promise<KnowledgeWorkspaceWriteResult>;
+  patchNote(input: { path: string; section: string; operation: 'append' | 'prepend' | 'replace'; content: string }): Promise<KnowledgeWorkspaceWriteResult>;
+  openInUi?(input: { path: string }): Promise<void>;
+}
+```
+
+CLI shape:
+
+```bash
+vgxness knowledge status --workspace obsidian-mcp --project vgxness
+vgxness knowledge init --workspace obsidian-mcp --project vgxness
+vgxness knowledge export --workspace obsidian-mcp --project vgxness --kind all
+vgxness knowledge export --workspace filesystem --vault <path> --project vgxness --kind all
+```
+
+## Testing Strategy
+
+| Layer | What to Test | Approach |
+|---|---|---|
+| Renderers | Stable Markdown/Canvas output | Pure unit tests with fixed inputs. |
+| Port adapters | Correct safety and call mapping | Mock Obsidian MCP client tests; temp-dir filesystem tests. |
+| Export service | Generated markers, no curated overwrite, correct targets | Unit tests with fake workspace port. |
+| SQLite projections | No raw traces/transcripts, no trace writes from read-only export | Temp SQLite tests. |
+| CLI | Status non-mutating, init/export dispatch | Node test runner with temp DB and fake workspace. |
+| Real integration | Obsidian MCP smoke | Opt-in test gated by `VGXNESS_OBSIDIAN_MCP_SMOKE=1`. |
+
+## Migration / Rollout
+
+No SQLite migration is required for the docs-first slice. Later code slices add new services and CLI commands without changing existing memory tables unless a future import/sync feature needs metadata.
+
+Roll out in slices:
+
+1. Docs and SDD artifacts.
+2. Pure renderers.
+3. Workspace port/adapters.
+4. Export service for docs.
+5. SQLite-backed memory and SDD projections.
+6. CLI commands.
+7. Optional grouped VGXNESS MCP sync tool.
+
+## Open Questions
+
+- Should VGXNESS connect to Obsidian MCP directly from its own runtime, or first rely on the agent host's MCP configuration?
+- Should the `VGXNESS/` folder name be configurable?
+- Should full memory export ever be allowed behind an explicit flag, or should summaries be the permanent limit?
+- Should archived SDD projections read from SQLite only, repo `docs/sdd/archive`, or both?

package/docs/sdd/sqlite-obsidian-knowledge-workflow/proposal.md

@@ -0,0 +1,80 @@
+# Proposal: SQLite + Obsidian MCP Knowledge Workflow
+
+## Intent
+
+Establish a durable VGXNESS knowledge workflow where SQLite remains the operational source of truth for agents and SDD governance, while Obsidian becomes a human-facing workspace reached through Obsidian MCP for documentation, research, boards, and canvas projections.
+
+## Scope
+
+### In Scope
+
+- Document the source-of-truth boundary between SQLite, Obsidian, Obsidian MCP, and repository docs.
+- Document Obsidian MCP setup, required Obsidian Local REST API configuration, and safety defaults.
+- Define a future `KnowledgeWorkspacePort` seam with Obsidian MCP as the preferred adapter and filesystem as a fallback/test adapter.
+- Plan CLI-level `knowledge` operations for status/init/export without adding many narrow VGXNESS MCP tools.
+- Keep generated Obsidian notes clearly marked as projections.
+
+### Out of Scope
+
+- Bidirectional sync from Obsidian back into SQLite.
+- Treating Obsidian notes as SDD artifact acceptance, verification, authorization, or governance state.
+- Mirroring every Obsidian MCP tool inside the VGXNESS MCP server.
+- Enabling Obsidian command-palette execution or destructive vault operations.
+- Cloud sync, shared vault operations, or team collaboration semantics.
+
+## Capabilities
+
+### New Capabilities
+
+- Knowledge workspace documentation explaining SQLite vs. Obsidian responsibilities.
+- Obsidian MCP setup guidance for human-facing workspace projections.
+- Planned workspace adapter seam for generated Markdown/Canvas exports.
+
+### Modified Capabilities
+
+- Storage documentation clarifies that SQLite remains authoritative even when Obsidian projections exist.
+- Product documentation links to the knowledge workspace model.
+
+## Approach
+
+Start with a docs and SDD slice, then implement the workflow in reviewable work units:
+
+1. Document the working model and Obsidian MCP setup.
+2. Add pure Markdown/Canvas renderers.
+3. Add `KnowledgeWorkspacePort` with Obsidian MCP and filesystem adapters.
+4. Export repo docs, memory summaries, and SDD board/artifact projections through that port.
+5. Add a grouped `vgxness knowledge` CLI surface.
+6. Add one VGXNESS MCP sync operation only if real agent workflows require it.
+
+## Affected Areas
+
+| Area | Impact | Description |
+|---|---|---|
+| `docs/knowledge-workspace.md` | New | Canonical SQLite/Obsidian boundary and workflow rules. |
+| `docs/obsidian-mcp.md` | New | Obsidian MCP setup and safety guidance. |
+| `docs/storage.md` | Modified | Clarifies Obsidian projections do not replace SQLite state. |
+| `README.md` | Modified | Links to knowledge workspace docs. |
+| `src/application/knowledge-workspace/*` | Future | Workspace port, renderers, and exporters. |
+| `src/interfaces/cli/commands/*` | Future | Grouped `knowledge` command surface. |
+
+## Risks
+
+| Risk | Likelihood | Mitigation |
+|---|---:|---|
+| Users confuse Obsidian boards with SDD governance | Medium | Generated files carry source-of-truth notices; docs state SQLite governs acceptance/readiness. |
+| Tool sprawl | Medium | Use external Obsidian MCP for vault operations and at most one grouped VGXNESS sync tool later. |
+| Sensitive memory export | Medium | Export summaries/previews by default; never export traces, raw transcripts, provider config, or secrets. |
+| Accidental overwrite of curated notes | Medium | Overwrite only generated notes carrying `generated_by: vgxness`. |
+| Obsidian MCP dependency instability | Medium | Hide server-specific details behind a workspace port and keep filesystem fallback for tests/offline export. |
+
+## Rollback Plan
+
+Revert the documentation, SDD artifacts, and any later `knowledge-workspace` code. Since SQLite remains authoritative and Obsidian exports are projections, rollback does not require data migration. Generated Obsidian notes can be removed from the vault manually if desired.
+
+## Success Criteria
+
+- [ ] Documentation clearly states SQLite is authoritative and Obsidian is a projection workspace.
+- [ ] Obsidian MCP setup is documented with safe defaults and path allowlists.
+- [ ] A future implementation path exists for a workspace port and grouped CLI surface.
+- [ ] No design requires Obsidian files to prove SDD acceptance, verification, or authorization.
+- [ ] No design requires mirroring all Obsidian MCP tools into VGXNESS MCP.

package/docs/sdd/sqlite-obsidian-knowledge-workflow/spec.md

@@ -0,0 +1,88 @@
+# Spec: SQLite + Obsidian MCP Knowledge Workflow
+
+## Requirements
+
+### Requirement 1: SQLite authority is preserved
+
+VGXNESS SHALL keep SQLite as the authoritative store for operational memory, SDD artifacts, artifact acceptance, run history, traces, checkpoints, approvals, and workflow governance.
+
+#### Scenario: Generated Obsidian board exists
+
+- **Given** an Obsidian SDD board has been generated for a change
+- **When** VGXNESS checks SDD readiness or acceptance
+- **Then** it SHALL read SQLite-backed VGXNESS state
+- **And** it SHALL NOT infer readiness or acceptance from the Obsidian board file
+
+### Requirement 2: Obsidian is a human-facing projection workspace
+
+VGXNESS SHALL treat Obsidian notes, boards, and canvas files as human-facing projections unless an explicit future import flow says otherwise.
+
+#### Scenario: Human edits a generated note
+
+- **Given** a generated note contains `generated_by: vgxness`
+- **When** VGXNESS exports the same projection again
+- **Then** VGXNESS MAY overwrite the generated note
+- **And** the generated note SHALL disclose that manual edits may be overwritten
+
+#### Scenario: Human creates a curated note
+
+- **Given** a note does not contain `generated_by: vgxness`
+- **When** VGXNESS exports generated projections
+- **Then** VGXNESS SHALL NOT overwrite that note by default
+
+### Requirement 3: Obsidian MCP is the preferred vault transport
+
+VGXNESS SHOULD use an Obsidian MCP adapter as the preferred write/search/read transport for the knowledge workspace.
+
+#### Scenario: Obsidian MCP is configured
+
+- **Given** Obsidian MCP is reachable and configured for the target vault
+- **When** `vgxness knowledge export --workspace obsidian-mcp --kind docs` runs
+- **Then** VGXNESS SHALL write generated documentation through the workspace port backed by Obsidian MCP
+- **And** it SHALL return note paths that were created or updated
+
+#### Scenario: Obsidian MCP is unavailable
+
+- **Given** Obsidian MCP is not configured or unreachable
+- **When** a knowledge command needs Obsidian MCP
+- **Then** VGXNESS SHALL report an actionable configuration error
+- **And** it SHALL NOT silently fall back to writing arbitrary filesystem paths unless `--workspace filesystem --vault <path>` is explicitly selected
+
+### Requirement 4: Path and command safety are enforced
+
+VGXNESS SHALL constrain generated Obsidian operations to the configured VGXNESS workspace folder.
+
+#### Scenario: Path allowlist is configured
+
+- **Given** the Obsidian MCP server supports path allowlists
+- **When** VGXNESS documents or installs recommended configuration
+- **Then** it SHALL scope reads and writes to `VGXNESS`
+- **And** it SHALL keep Obsidian command execution disabled by default
+
+#### Scenario: Export tries to escape workspace
+
+- **Given** an export target path resolves outside `VGXNESS/`
+- **When** VGXNESS prepares the write
+- **Then** it SHALL reject the operation before writing
+
+### Requirement 5: Memory exports are summarized and safe by default
+
+VGXNESS SHALL export memory summaries or previews by default, not raw sensitive operational records.
+
+#### Scenario: Memory summary export
+
+- **Given** project memory observations exist in SQLite
+- **When** VGXNESS exports `--kind memory`
+- **Then** the Obsidian projection SHALL include compact fields such as title, type, scope, topic key, updated time, and preview
+- **And** it SHALL NOT include raw traces, full transcripts, provider config, secrets, or hidden reasoning
+
+### Requirement 6: VGXNESS avoids MCP tool sprawl
+
+VGXNESS SHALL NOT mirror all Obsidian MCP note tools into its own MCP server.
+
+#### Scenario: Agent needs a high-level sync
+
+- **Given** agent workflows need to trigger a knowledge workspace sync from VGXNESS MCP
+- **When** VGXNESS adds an MCP tool
+- **Then** it SHALL add one grouped operation such as `knowledge_workspace_sync`
+- **And** it SHALL NOT add one VGXNESS MCP tool per Obsidian note operation

package/docs/sdd/sqlite-obsidian-knowledge-workflow/tasks.md

@@ -0,0 +1,70 @@
+# Tasks: SQLite + Obsidian MCP Knowledge Workflow
+
+## Review Workload Forecast
+
+| Item | Value |
+|---|---|
+| Expected total change | Large, multi-slice |
+| 400-line budget risk | High |
+| Chained PRs recommended | Yes |
+| Decision needed before apply | No — use reviewable work-unit slices |
+| Chain strategy | feature-branch-chain or stacked-to-main; first slice may target main independently |
+
+## Phase 1: Documentation and SDD foundation
+
+- [x] 1.1 Create SDD proposal/spec/design/tasks for the SQLite + Obsidian MCP knowledge workflow.
+- [x] 1.2 Create `docs/knowledge-workspace.md` documenting the SQLite/Obsidian source-of-truth boundary.
+- [x] 1.3 Create `docs/obsidian-mcp.md` documenting Obsidian MCP setup, safety defaults, and VGXNESS integration boundaries.
+- [x] 1.4 Update `docs/storage.md` to clarify Obsidian projections do not replace SQLite state.
+- [x] 1.5 Link the new docs from `README.md`.
+
+## Phase 2: Pure rendering layer
+
+- [x] 2.1 Add knowledge workspace domain types.
+- [x] 2.2 Add Markdown renderer tests for memory summaries and SDD board output.
+- [x] 2.3 Implement Markdown renderers.
+- [x] 2.4 Add Obsidian Canvas renderer tests.
+- [x] 2.5 Implement deterministic Canvas renderer.
+
+## Phase 3: Workspace port and adapters
+
+- [x] 3.1 Define `KnowledgeWorkspacePort` and shared result types.
+- [x] 3.2 Add mock/client contract tests for Obsidian MCP write/get/search/patch mapping.
+- [x] 3.3 Implement `ObsidianMcpWorkspacePort` behind the port.
+- [x] 3.4 Add filesystem fallback adapter tests with path escape and generated-marker cases.
+- [x] 3.5 Implement `FilesystemWorkspacePort` for tests/offline fallback.
+
+## Phase 4: Export service
+
+- [x] 4.1 Add export service tests for generated markers and curated-note overwrite protection.
+- [x] 4.2 Implement docs export through the workspace port.
+- [x] 4.3 Add memory summary export tests using safe previews.
+- [x] 4.4 Implement memory summary export without raw traces/transcripts.
+- [x] 4.5 Add SDD board/artifact projection tests.
+- [x] 4.6 Implement SDD board/artifact projection export.
+
+## Phase 5: CLI surface
+
+- [x] 5.1 Add CLI parser/dispatcher tests for `vgxness knowledge status|init|export`.
+- [x] 5.2 Implement grouped `knowledge` command dispatcher.
+- [x] 5.3 Document CLI usage in `docs/cli.md` and `docs/knowledge-workspace.md`.
+- [x] 5.4 Add optional real Obsidian MCP smoke test guidance/gate.
+
+## Phase 6: Optional VGXNESS MCP sync tool
+
+- [x] 6.1 Re-evaluate whether a VGXNESS MCP `knowledge_workspace_sync` tool is needed.
+- [x] 6.2 Not needed in this slice: the grouped `knowledge` CLI plus external Obsidian MCP keeps detailed note operations outside VGXNESS and avoids adding a preview-only MCP wrapper.
+- [x] 6.3 Not applicable until a real host-MCP bridge exists; current validation is covered by export service and CLI tests.
+
+## Verification
+
+Each implementation slice should run the narrowest relevant test first, then the standard VGXNESS gates before PR:
+
+```bash
+bun run verify:typecheck
+bun run verify:test
+bun run verify:bun-sqlite
+bun run package:bun:evidence
+```
+
+Docs-only slices may run `bun run verify:typecheck` and any docs tests if present, then broader gates as time allows.

package/docs/storage.md

@@ -2,6 +2,8 @@
 
 VGXNESS keeps all durable state in local SQLite. There is no required cloud account, no remote sync in the current local-first design, and no implicit sharing between scopes. Project data and personal/global data must not be collapsed into one scope.
 
+Human-facing knowledge workspace projections may be written to Obsidian through the [Obsidian MCP integration](./obsidian-mcp.md), but those notes, boards, and canvas files are not authoritative state. SQLite remains the source of truth for operational memory, SDD artifacts, acceptance, runs, traces, approvals, and workflow governance. See [Knowledge Workspace](./knowledge-workspace.md) for the source-of-truth boundary.
+
 ## Database location
 
 The default is a per-user global database. The path is resolved by `resolveMemoryDatabasePath(...)` in `src/memory/storage-paths.ts` with this precedence:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vgxness",
-  "version": "1.20.13",
+  "version": "1.20.15",
   "description": "CLI and MCP control plane for guided AI-agent workflows, SDD, memory, and OpenCode setup.",
   "license": "SEE LICENSE IN LICENSE",
   "repository": {