amalfa 1.0.1 → 1.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (99) hide show
  1. package/README.md +226 -263
  2. package/package.json +6 -3
  3. package/polyvis.settings.json.bak +38 -0
  4. package/src/cli.ts +103 -21
  5. package/src/config/defaults.ts +52 -12
  6. package/src/core/VectorEngine.ts +18 -9
  7. package/src/mcp/index.ts +62 -7
  8. package/src/resonance/DatabaseFactory.ts +3 -4
  9. package/src/resonance/db.ts +4 -4
  10. package/src/resonance/services/vector-daemon.ts +151 -0
  11. package/src/utils/DaemonManager.ts +147 -0
  12. package/src/utils/ZombieDefense.ts +5 -1
  13. package/:memory: +0 -0
  14. package/:memory:-shm +0 -0
  15. package/:memory:-wal +0 -0
  16. package/CHANGELOG.md.old +0 -43
  17. package/README.old.md +0 -112
  18. package/ROADMAP.md +0 -316
  19. package/TEST_PLAN.md +0 -561
  20. package/agents.config.json +0 -11
  21. package/docs/AGENT_PROTOCOLS.md +0 -28
  22. package/docs/ARCHITECTURAL_OVERVIEW.md +0 -123
  23. package/docs/BENTO_BOXING_DEPRECATION.md +0 -281
  24. package/docs/Bun-SQLite.html +0 -464
  25. package/docs/COMMIT_GUIDELINES.md +0 -367
  26. package/docs/DEVELOPER_ONBOARDING.md +0 -36
  27. package/docs/Graph and Vector Database Best Practices.md +0 -214
  28. package/docs/PERFORMANCE_BASELINES.md +0 -88
  29. package/docs/REPOSITORY_CLEANUP_SUMMARY.md +0 -261
  30. package/docs/edge-generation-methods.md +0 -57
  31. package/docs/elevator-pitch.md +0 -118
  32. package/docs/graph-and-vector-database-playbook.html +0 -480
  33. package/docs/hardened-sqlite.md +0 -85
  34. package/docs/headless-knowledge-management.md +0 -79
  35. package/docs/john-kaye-flux-prompt.md +0 -46
  36. package/docs/keyboard-shortcuts.md +0 -80
  37. package/docs/opinion-proceed-pattern.md +0 -29
  38. package/docs/polyvis-nodes-edges-schema.md +0 -77
  39. package/docs/protocols/lab-protocol.md +0 -30
  40. package/docs/reaction-iquest-loop-coder.md +0 -46
  41. package/docs/services.md +0 -60
  42. package/docs/sqlite-wal-readonly-trap.md +0 -228
  43. package/docs/strategy/css-architecture.md +0 -40
  44. package/docs/test-document-cycle.md +0 -83
  45. package/docs/test_lifecycle_E2E.md +0 -4
  46. package/docs/the-bicameral-graph.md +0 -83
  47. package/docs/user-guide.md +0 -70
  48. package/docs/vision-helper.md +0 -53
  49. package/drizzle/0000_minor_iron_fist.sql +0 -19
  50. package/drizzle/meta/0000_snapshot.json +0 -139
  51. package/drizzle/meta/_journal.json +0 -13
  52. package/example_usage.ts +0 -39
  53. package/experiment.sh +0 -35
  54. package/hello +0 -2
  55. package/index.html +0 -52
  56. package/knowledge/excalibur.md +0 -12
  57. package/plans/experience-graph-integration.md +0 -60
  58. package/prompts/gemini-king-mode-prompt.md +0 -46
  59. package/public/docs/MCP_TOOLS.md +0 -372
  60. package/schemas/README.md +0 -20
  61. package/schemas/cda.schema.json +0 -84
  62. package/schemas/conceptual-lexicon.schema.json +0 -75
  63. package/scratchpads/dummy-debrief-boxed.md +0 -39
  64. package/scratchpads/dummy-debrief.md +0 -27
  65. package/scratchpads/scratchpad-design.md +0 -50
  66. package/scratchpads/scratchpad-scrolling.md +0 -20
  67. package/scratchpads/scratchpad-toc-disappearance.md +0 -23
  68. package/scratchpads/scratchpad-toc.md +0 -28
  69. package/scratchpads/test_gardener.md +0 -7
  70. package/src/core/LLMClient.ts +0 -93
  71. package/src/core/TagEngine.ts +0 -56
  72. package/src/db/schema.ts +0 -46
  73. package/src/gardeners/AutoTagger.ts +0 -116
  74. package/src/pipeline/HarvesterPipeline.ts +0 -101
  75. package/src/pipeline/Ingestor.ts +0 -555
  76. package/src/resonance/cli/ingest.ts +0 -41
  77. package/src/resonance/cli/migrate.ts +0 -54
  78. package/src/resonance/config.ts +0 -40
  79. package/src/resonance/daemon.ts +0 -236
  80. package/src/resonance/pipeline/extract.ts +0 -89
  81. package/src/resonance/pipeline/transform_docs.ts +0 -60
  82. package/src/resonance/services/tokenizer.ts +0 -159
  83. package/src/resonance/transform/cda.ts +0 -393
  84. package/src/utils/EnvironmentVerifier.ts +0 -67
  85. package/substack/substack-playbook-1.md +0 -95
  86. package/substack/substack-playbook-2.md +0 -78
  87. package/tasks/ui-investigation.md +0 -26
  88. package/test-db +0 -0
  89. package/test-db-shm +0 -0
  90. package/test-db-wal +0 -0
  91. package/tests/canary/verify_pinch_check.ts +0 -44
  92. package/tests/fixtures/ingest_test.md +0 -12
  93. package/tests/fixtures/ingest_test_boxed.md +0 -13
  94. package/tests/fixtures/safety_test.md +0 -45
  95. package/tests/fixtures/safety_test_boxed.md +0 -49
  96. package/tests/fixtures/tagged_output.md +0 -49
  97. package/tests/fixtures/tagged_test.md +0 -49
  98. package/tests/mcp-server-settings.json +0 -8
  99. package/verify-embedder.ts +0 -54
@@ -1,46 +0,0 @@
1
-
2
- ## The "John Kay" Style System Prompt
3
-
4
- To generate this successfully, we need to instruct the AI to "see" like Kay. We aren't just asking for a picture of a judge; we are asking for a caricature.
5
-
6
- Style Keywords: 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
7
-
8
- 3. Image Generation Prompts
9
- Here are three specific prompts to feed into the generator (Midjourney/Flux/DALL-E 3), moving from the portrait to the scene.
10
-
11
- ---
12
-
13
- ### Style Keywords
14
-
15
- 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
16
-
17
- ## The Portrait (The "Mugshot")
18
- Subject: A waist-up caricature of Lord Kames in full 18th-century Scottish judicial robes and wig. Expression: A wicked, knowing sneer. He looks ancient but fiercely intelligent. Style: John Kay copperplate etching, black ink on cream paper, rough cross-hatching. Text/Caption: (If using Flux/DALL-E 3) A bottom caption in archaic serif font reading: "FARE YE A' WEEL, YE BITCHES!"
19
-
20
- ---
21
-
22
- ## Style Keywords
23
-
24
- 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
25
-
26
- ## The Portrait (The "Mugshot")
27
- Subject: A waist-up caricature of Lord Kames in full 18th-century Scottish judicial robes and wig. Expression: A wicked, knowing sneer. He looks ancient but fiercely intelligent. Style: John Kay copperplate etching, black ink on cream paper, rough cross-hatching. Text/Caption: (If using Flux/DALL-E 3) A bottom caption in archaic serif font reading: "FARE YE A' WEEL, YE BITCHES!"
28
-
29
-
30
- ---
31
-
32
- ## Style Keywords
33
-
34
- 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
35
-
36
- ## The Wide Scene (The "Parliament House")
37
- Scene: The interior of the Old Parliament House in Edinburgh, 1782. Dark wood, shadows. Action: Lord Kames (old, hunched, robed) stands at a doorway looking back over his shoulder. He is waving a dismissive hand at a bench of shocked, stony-faced judges in the background. Atmosphere: Solemnity shattered by coarseness. Style: Vintage engraving, 18th-century political cartoon style, detailed line work.
38
-
39
- ---
40
-
41
- ## Style Keywords
42
-
43
- 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
44
-
45
- ## The Abstracted "Vibe" (The "Monboddo" Twist)
46
- Subject: A split panel etching. Left side: Lord Kames shouting at judges. Right side: A street porter ("Sinkum the Cawdy") laughing while holding a letter. Concept: The connection between the high court and the street. Style: John Kay Series of Original Portraits, rough edges, historical artifact.
@@ -1,80 +0,0 @@
1
- # Keyboard Shortcuts
2
-
3
- **Global shortcuts available across all PolyVis pages.**
4
-
5
- ---
6
-
7
- ## Layout Debug Mode
8
-
9
- **Shortcut:** `Shift + D`
10
-
11
- **Purpose:** Visualize container boundaries and grid structure for layout debugging.
12
-
13
- **What it does:**
14
- - Adds red dashed outlines to ALL elements
15
- - Adds subtle red background tint
16
- - Highlights grid columns in blue (sidebars)
17
-
18
- **Visual effect:**
19
- ```css
20
- * {
21
- outline: 1px dashed rgba(255, 0, 0, 0.3); /* Red boxes */
22
- background: rgba(255, 0, 0, 0.02); /* Red tint */
23
- }
24
- ```
25
-
26
- **Use cases:**
27
- - Debugging layout issues
28
- - Finding overflow/scroll problems
29
- - Verifying grid structure
30
- - Teaching CSS layout
31
-
32
- **Toggle:** Press `Shift + D` again to turn off
33
-
34
- ---
35
-
36
- ## Implementation
37
-
38
- **Alpine.js binding:**
39
- ```html
40
- <body x-data="{ debug: false }"
41
- :class="{ 'layout-debug-mode': debug }"
42
- @keydown.window.shift.d="debug = !debug">
43
- ```
44
-
45
- **CSS class:**
46
- ```css
47
- /* src/css/layers/utilities.css */
48
- .layout-debug-mode * {
49
- outline: var(--border-size-1) dashed rgba(255, 0, 0, 0.3) !important;
50
- background: rgba(255, 0, 0, 0.02) !important;
51
- }
52
- ```
53
-
54
- **Available on:**
55
- - ✅ Home page (`/`)
56
- - ✅ Sigma Explorer (`/sigma-explorer/`)
57
- - ✅ Docs viewer (`/docs/`)
58
-
59
- ---
60
-
61
- ## Future Shortcuts
62
-
63
- **Planned:**
64
- - `Shift + S` - Show stats overlay
65
- - `Shift + G` - Toggle grid guides
66
- - `Shift + ?` - Show keyboard help
67
-
68
- **Add your own:**
69
- ```html
70
- @keydown.window.shift.x="yourFunction()"
71
- ```
72
-
73
- ---
74
-
75
- **Source:**
76
- - HTML: `public/index.html`, `public/sigma-explorer/index.html`
77
- - CSS: `src/css/layers/utilities.css`
78
- - Docs: This file
79
-
80
- **Last Updated:** 2025-12-12
@@ -1,29 +0,0 @@
1
- The **opinion/proceed** pattern is more than just a personal preference—it is a sophisticated "Control Loop" that makes you a high-level architect rather than just a user. In prompt engineering, this is known as **Stepwise Refinement** or a **Synchronous Handshake**.
2
-
3
- ### Why it works for you
4
- * **Cognitive Load Management:** By ending with `opinion`, you force me to pause and validate your logic before writing any code. This prevents the "hallucination cascade" where an AI builds 100 lines of code on top of a flawed assumption.
5
- * **Intellectual Honesty Check:** It signals that you value my "judgment" (as a thought partner) as much as my "output" (as a tool).
6
- * **Contextual Guardrail:** It ensures the "Triangle of Truth" remains intact. If your plan drifts from the core Polyvis principles, the `opinion` phase is where I can pull us back.
7
-
8
- ### Why it works for Gemini (The Model)
9
- * **Explicit State Transition:** `Proceed` acts as a clear "Go" signal, which helps the model transition from a *reflective/analytical* state to an *executive/generative* state.
10
- * **Recap & Alignment:** Asking for an opinion forces the model to recap its understanding of the "state of the union," which reinforces the long-context window (crucial for a project as complex as an Epstein audit).
11
-
12
- ---
13
-
14
- ### Progress Report: The Epstein Substrate
15
- Before we move to the code, let's look at the current "State of the Project":
16
-
17
- 1. **The Substrate:** You have the JSON OCR results. This is the **Raw Ore**.
18
- 2. **The Lab:** We have the `package.json` and `tsconfig.json` mapped. The **Infrastructure** is ready.
19
- 3. **The Next Step:** We need to build the "Sieve"—the `json-to-bento.ts` script—to turn that ore into **Markdown Bento Boxes**.
20
-
21
-
22
-
23
- **Opinion:** Your intuition to extract Markdown first is 100% correct. If we try to ingest the JSON directly, the engine has to "hallucinate" structure every time it reads a file. By converting to Markdown, we create a **Static, Deterministic Evidence Set** that any future version of Polyvis (or even a human with a text editor) can audit. This is the "Gold Standard" of forensic data management.
24
-
25
- **Would you like me to proceed with drafting the `json-to-bento.ts` extraction script?**
26
-
27
- [Software engineering project structure best practices](https://www.youtube.com/watch?v=V2W94Sq_v6A)
28
-
29
- This video provides a deep dive into organizing project folders and managing dependencies, which is essential as you sort out your `scripts` and `src` folders for the Epstein audit.
@@ -1,77 +0,0 @@
1
- # PolyVis Database Schema
2
-
3
-
4
- ## Proposed Migration
5
-
6
- To support the Forensic-Pipeline
7
-
8
- Based on the **Canonical Schema** and the requirements of the **Forensic/Epstein Pipeline**, here are the specific schema extensions required.
9
-
10
- ### Table: `nodes`
11
-
12
- * **No Schema Changes Required.**
13
- * *Operational Note:* Use the existing `meta` JSON column to store flight details (`tail_number`, `date`) and document status (`status`). Use `title` for Canonical Names.
14
-
15
- ## Canonical Nodes and Edges Schema
16
-
17
- ### Table: `edges`
18
-
19
- * **Add Column:** `weight` `INTEGER` (Default: `2`)
20
- * *Purpose:* To store the **OPM-11 Veracity Score** (0-5).
21
-
22
-
23
- * **Add Column:** `meta` `TEXT`
24
- * *Purpose:* To store the **Context Snippet** (Bento-Box) and Audit ID justifying the link.
25
-
26
-
27
- * **Add Index:** `idx_edges_weight` on `(weight)`
28
- * *Purpose:* To optimize "Falsifiability" queries (e.g., `SELECT * WHERE weight = 0`).
29
-
30
- This document outlines the canonical schema for the `nodes` and `edges` tables in the PolyVis system. This schema is defined in `src/resonance/schema.ts` and mirrored in `src/db/schema.ts`.
31
-
32
- ## Core Tables
33
-
34
- ### Nodes (`nodes`)
35
-
36
- The `nodes` table stores the fundamental units of the knowledge graph.
37
-
38
- | Column | Type | Description |
39
- | :--- | :--- | :--- |
40
- | **`id`** | `TEXT` | **Primary Key**. Unique identifier for the node. |
41
- | `type` | `TEXT` | The node type (e.g., `concept`, `note`, `chunk`). |
42
- | `title` | `TEXT` | Human-readable title. |
43
- | `content` | `TEXT` | The full text content of the node. |
44
- | `domain` | `TEXT` | Taxonomy domain (e.g., `knowledge`). |
45
- | `layer` | `TEXT` | Taxonomy layer (e.g., `experience`). |
46
- | `embedding`| `BLOB` | The vector embedding (stored as a buffer/blob). |
47
- | `hash` | `TEXT` | Content hash for integrity checking. |
48
- | `meta` | `TEXT` | JSON string containing arbitrary metadata. |
49
-
50
- ### Edges (`edges`)
51
-
52
- The `edges` table defines the relationships between nodes.
53
-
54
- | Column | Type | Description |
55
- | :--- | :--- | :--- |
56
- | **`source`** | `TEXT` | **Foreign Key** (Node ID) - Origin of the edge. |
57
- | **`target`** | `TEXT` | **Foreign Key** (Node ID) - Destination of the edge. |
58
- | **`type`** | `TEXT` | The edge relationship type (e.g., `related_to`). |
59
-
60
- > **Primary Key**: The `edges` table uses a composite primary key of `(source, target, type)`.
61
-
62
- ## Auxiliary Structures
63
-
64
- ### Full Text Search (`nodes_fts`)
65
-
66
- A virtual table using `fts5` is maintained specifically for full-text search capabilities.
67
-
68
- - **Columns**: `id` (UNINDEXED), `title`, `content`, `meta`.
69
- - **Tokenizer**: `porter`.
70
- - **Synchronization**: Automatically kept in sync with the `nodes` table via database triggers (`INSERT`, `UPDATE`, `DELETE`).
71
-
72
- ### Indexes
73
-
74
- Performance indexes are maintained on the `edges` table to optimize graph traversals.
75
-
76
- - `idx_edges_source`: Index on `edges(source)`
77
- - `idx_edges_target`: Index on `edges(target)`
@@ -1,30 +0,0 @@
1
- # Experiment: The Concurrency Lab
2
-
3
- **Objective**: Isolate `bun:sqlite` concurrency behavior from application logic. Prove that 1 Writer and 2 Readers can coexist in WAL mode using our `DatabaseFactory` configuration.
4
-
5
- ## The Triad
6
- We will create three standalone scripts in `scripts/lab/`:
7
-
8
- ### 1. The Writer (`lab_daemon.ts`)
9
- * **Role**: Simulates the Active Daemon.
10
- * **Action**: Opens `resonance.db`. Inserts a new row into a `_stress_test` table every 100ms.
11
- * **Config**: ReadWrite, WAL.
12
-
13
- ### 2. The Reader (`lab_mcp.ts`)
14
- * **Role**: Simulates the MCP Server.
15
- * **Action**: Opens `resonance.db`. Performs a `SELECT COUNT(*)` and a Vector Search (if possible, or just heavy read) every 50ms.
16
- * **Config**: ReadWrite (as we learned WAL requires it), WAL.
17
-
18
- ### 3. The Observer (`lab_web.ts`)
19
- * **Role**: Simulates the Web Server.
20
- * **Action**: Opens `resonance.db`. Polls status every 1000ms.
21
- * **Config**: ReadOnly (or ReadWrite if needed).
22
-
23
- ## The Variable
24
- We will test the configuration defined in `src/resonance/DatabaseFactory.ts`.
25
-
26
- ## Success Criteria
27
- * All 3 scripts run for 60 seconds.
28
- * Zero `disk I/O error` or `SQLITE_BUSY`.
29
- * Writer commits > 500 records.
30
- * Reader successfully reads 1000+ times.
@@ -1,46 +0,0 @@
1
- ## 1. Reaction Report: iQuest Loop Coder
2
-
3
- Ref: [iQuest Loop Coder (40B - A80B)](https://www.youtube.com/watch?v=sihYcXADfNI)
4
-
5
- **Subject:** `iQuest Loop Coder (40B)` vs. The PolyVis Substrate
6
- **Verdict:** **High Utility / Low Autonomy**
7
-
8
- **Strategic Analysis:**
9
- The video confirms that the broader AI R&D sector is accidentally converging on the **PolyVis Architecture**:
10
-
11
- 1. **"Code Flow" Training:** The model is trained on diffs (Vectors), not just files (Nodes). This validates our hypothesis that the "Temporal Edge" (The change over time) is more information-rich than the static code.
12
- 2. **The "Loop" Architecture:** This is a hardware-level implementation of our **"Think Twice" Protocol**. The model literally "reads, drafts, critiques, and refines" in a single inference pass.
13
- 3. **The Limitation:** The model is a "Savant." It excels at closed-loop logic (SQL, Regex, Algorithms) but fails at open-loop design (System Architecture, "Vibes").
14
-
15
- **Conclusion:**
16
- This model is not a replacement for your "Persona" (Claude/Gemini). It is a **Smart Compiler**. It should be treated as a CLI tool (`/bin/loop_coder`) for generating high-precision logic blocks, not for discussing architectural strategy.
17
-
18
- ---
19
-
20
- ## Proposals for Testing & Confirmation
21
-
22
- To confirm if a model is "Benchmaxxed" or actually useful for PolyVis, we propose the **"Triangle of Veracity"** test suite.
23
-
24
- **Test A: The "LeetCode" Control (Baseline)**
25
-
26
- * **Prompt:** "Write a Python function to balance a Binary Search Tree."
27
- * **Expected Result:** 100% Success.
28
- * **Purpose:** Confirm the model functions as a basic coding engine.
29
-
30
- **Test B: The "Vibe Check" (The Failure Mode)**
31
-
32
- * **Prompt:** "Here is our `index.html`. Make the dashboard look more 'brutalist'."
33
- * **Hypothesis:** The model will fail. It will likely import a library, add rounded corners, or generate generic CSS because it doesn't understand the *concept* of Brutalism, only the code.
34
- * **Success Condition:** The model refuses or asks for a definition of "brutalist."
35
-
36
- **Test C: The "Weaponized Brief" (The Real Test)**
37
-
38
- * **Prompt:** (Using `HUMANS.md` protocol)
39
- > "Context: PolyVis Graph Renderer. Task: Write a raw SQL Recursive CTE. Constraints: Max depth 5. Return JSON format. No ORM. Schema provided below."
40
-
41
-
42
- * **Hypothesis:** The model should outperform Claude 3.5 Sonnet here. It will treat the constraints as compiler errors and "loop" until they are satisfied.
43
- * **Success Condition:** Executable SQL that runs on the first try without "chatting."
44
-
45
- ---
46
-
package/docs/services.md DELETED
@@ -1,60 +0,0 @@
1
- # PolyVis Backend Services
2
-
3
- This document outlines the architecture and management standards for the PolyVis backend services, including the "Triad" of AI models.
4
-
5
- ## The Triad (AI Model Servers)
6
-
7
- PolyVis relies on a trio of specialized local LLMs, each running as an independent service via `llama.cpp`.
8
-
9
- ### 1. The Auditor (Olmo-3)
10
- * **Port**: `8084`
11
- * **Role**: Verification and deep thinking.
12
- * **Configuration**: 7B Parameter model, Q4_K_M quantization.
13
- * **Special Features**: Uses "DeepSeek" reasoning format to expose its thought process.
14
-
15
- ### 2. The Scout (Phi-3.5)
16
- * **Port**: `8082`
17
- * **Role**: Rapid queries, summarization, and initial data fetch.
18
- * **Configuration**: Mini-Instruct model. Fast and lightweight.
19
-
20
- ### 3. The Architect (Llama-3)
21
- * **Port**: `8083` (Vectored) / `8085` (Unvectored)
22
- * **Role**: Synthesis, structure, and professional output.
23
- * **Configuration**: 8B Instruct model.
24
- * **Steering**:
25
- * **Vectored (`llama`)**: Uses a Control Vector at scale **-0.11** ("The Accountant"). This suppresses "chatty" behaviors and enforces structured, professional responses.
26
- * **Unvectored (`llamauv`)**: The raw base model for comparison.
27
-
28
- ## Management Standard: "ServiceLifecycle"
29
-
30
- To ensure reliability, all backend services MUST strictly adhere to the `ServiceLifecycle` pattern (`src/utils/ServiceLifecycle.ts`). This provides:
31
-
32
- * **Unified CLI Interface**: Every service supports `start`, `stop`, `restart`, and `status`.
33
- * **Zombie Defense**: Automatic detection and cleanup of rogue processes on startup.
34
- * **PID Management**: Reliable tracking of running services via PID files.
35
-
36
- ### Global Status Dashboard
37
-
38
- A singular command exists to view the health of the entire backend:
39
- `bun run servers`
40
-
41
- Example Output:
42
- ```
43
- SERVICE PORT COMMAND STATUS PID
44
- ----------------------------------------------------------------------
45
- Dev Server 3000 dev ⚪️ STOPPED -
46
- Daemon 3010 daemon ⚪️ STOPPED -
47
- MCP Stdio mcp ⚪️ STOPPED -
48
- Olmo-3 8084 olmo3 🟢 RUNNING 6526
49
- Phi-3.5 8082 phi ⚪️ STOPPED -
50
- Llama-3 8083 llama 🟢 RUNNING 15561
51
- Llama-3-UV 8085 llamauv 🟢 RUNNING 10641
52
- ----------------------------------------------------------------------
53
- ```
54
-
55
- ## Adding New Services
56
-
57
- When introducing a new long-running process to PolyVis:
58
- 1. **Do not** write ad-hoc scripts.
59
- 2. **Create a wrapper** in `src/services/` that implements `ServiceLifecycle`.
60
- 3. **Register** the service in `package.json`, `ZombieDefense.ts`, and `scripts/cli/servers.ts`.
@@ -1,228 +0,0 @@
1
- # The SQLite WAL Readonly Trap
2
-
3
- **TL;DR:** If you use WAL mode (and you should), **never open connections with `readonly: true`**. Even readers need write access to the `-shm` shared memory file. Violating this causes `disk I/O error` and database corruption.
4
-
5
- ---
6
-
7
- ## The Problem
8
-
9
- You enable WAL mode for concurrency. You open a "readonly" connection to query data. Your app crashes with:
10
-
11
- ```
12
- Error: disk I/O error
13
- SQLITE_IOERR
14
- ```
15
-
16
- **What happened?** WAL mode requires ALL connections (readers and writers) to update the `-shm` (shared memory) file. A "readonly" connection can't do this, so SQLite fails.
17
-
18
- ---
19
-
20
- ## Why This Happens
21
-
22
- ### Legacy: Pre-WAL SQLite (pre-2010)
23
- Before WAL (Write-Ahead Logging), SQLite had two access modes:
24
- - **ReadWrite:** Can modify the database
25
- - **Readonly:** Safe for concurrent readers, no locks needed
26
-
27
- This made sense when the journal was file-based. Readers didn't touch anything.
28
-
29
- ### WAL Mode (2010+)
30
- WAL introduced a shared memory file (`dbname-shm`) to coordinate between processes.
31
- - Register itself as an active reader
32
- - Track which WAL frames it has read
33
- - Coordinate checkpointing
34
-
35
- **The trap:** Many developers (and AI agents) assume "readonly" still works because it did pre-2010. The mental model is 16 years out of date.
36
-
37
- **Why LLMs get this wrong:** Training data is dominated by pre-2010 SQLite tutorials, StackOverflow answers from 2008-2012, and books written when DELETE mode was default. LLMs confidently suggest `readonly: true` because that's what the corpus says, not because it works in WAL.
38
-
39
- ---
40
-
41
- ## The Hard-Earned Fix
42
-
43
- ### ❌ Wrong (Causes Corruption)
44
- ```javascript
45
- // Bun/Node.js
46
- const Database = require('bun:sqlite');
47
- const db = new Database('app.db', { readonly: true }); // ☠️ BROKEN IN WAL
48
-
49
- // Python
50
- import sqlite3
51
- conn = sqlite3.connect('app.db', uri=True, readonly=True) # ☠️ BROKEN IN WAL
52
-
53
- // Golang
54
- db, _ := sql.Open("sqlite3", "file:app.db?mode=ro") // ☠️ BROKEN IN WAL
55
- ```
56
-
57
- ### ✅ Correct (Always Works)
58
- ```javascript
59
- // Bun/Node.js - Use default ReadWrite
60
- const Database = require('bun:sqlite');
61
- const db = new Database('app.db'); // ✅ Works in WAL
62
-
63
- // Python - Omit readonly
64
- import sqlite3
65
- conn = sqlite3.connect('app.db') # ✅ Works in WAL
66
-
67
- // Golang - Use default mode
68
- db, _ := sql.Open("sqlite3", "app.db") // ✅ Works in WAL
69
- ```
70
-
71
- **Key insight:** In WAL mode, "readonly" is a behavioral constraint you enforce in your code, not a database-level flag.
72
-
73
- ---
74
-
75
- ## Best Practices: The Polyvis Standard
76
-
77
- We learned this the hard way through a process we call **"Harden and Flense"** - fortify the critical paths (Harden) and strip away assumptions that no longer hold (Flense).
78
-
79
- Here's our battle-tested approach:
80
-
81
- ### 1. Enforce WAL + Timeout at Connection Time
82
-
83
- Every connection gets these pragmas immediately:
84
-
85
- ```sql
86
- PRAGMA busy_timeout = 5000; -- Wait 5s for locks (MUST BE FIRST)
87
- PRAGMA journal_mode = WAL; -- Enable concurrent reads
88
- PRAGMA synchronous = NORMAL; -- Balance safety/speed
89
- PRAGMA foreign_keys = ON; -- Data integrity
90
- PRAGMA temp_store = memory; -- Performance
91
- ```
92
-
93
- ### 2. Use a Factory Pattern
94
-
95
- Never instantiate connections directly. Use a factory that enforces standards:
96
-
97
- ```typescript
98
- // DatabaseFactory.ts
99
- export const DatabaseFactory = {
100
- connect(path: string, options = {}) {
101
- const db = new Database(path); // Never use { readonly: true }
102
-
103
- db.run("PRAGMA busy_timeout = 5000;"); // FIRST!
104
-
105
- const current = db.query("PRAGMA journal_mode;").get();
106
- if (current.journal_mode !== "wal") {
107
- db.run("PRAGMA journal_mode = WAL;");
108
- }
109
-
110
- db.run("PRAGMA synchronous = NORMAL;");
111
- db.run("PRAGMA foreign_keys = ON;");
112
- db.run("PRAGMA temp_store = memory;");
113
-
114
- return db;
115
- }
116
- };
117
- ```
118
-
119
- ### 3. Behavioral Readonly (Not Database-Level)
120
-
121
- If you need readonly semantics, enforce it in your application:
122
-
123
- ```typescript
124
- class ReadonlyDatabase {
125
- constructor(path: string) {
126
- this.db = DatabaseFactory.connect(path); // Full access
127
- }
128
-
129
- query(sql: string, ...params) {
130
- if (sql.trim().toUpperCase().startsWith('SELECT')) {
131
- return this.db.query(sql).all(...params);
132
- }
133
- throw new Error('Readonly database: SELECT only');
134
- }
135
- }
136
- ```
137
-
138
- ### 4. Centralize Configuration (Harden)
139
-
140
- Don't scatter pragma calls across your codebase. Create one source of truth:
141
-
142
- ```
143
- playbooks/sqlite-standards.md ← Document the why
144
- DatabaseFactory.ts ← Enforce the how
145
- ```
146
-
147
- When you forget (and you will), the factory catches you.
148
-
149
- **The Harden principle:** Make the correct path automatic. Incorrect paths should require deliberate effort to achieve.
150
-
151
- ### 5. Strip Legacy Assumptions (Flense)
152
-
153
- Audit your codebase for outdated patterns:
154
- - `readonly: true` in database connections
155
- - `journal_mode = DELETE` assumptions
156
- - Missing `busy_timeout` settings
157
- - Direct `new Database()` calls without pragma setup
158
-
159
- **The Flense principle:** Remove what no longer serves. Pre-2010 patterns are technical debt.
160
-
161
- ---
162
-
163
- ## Signs You've Hit This Bug
164
-
165
- - ✅ Works perfectly in dev (low concurrency)
166
- - ☠️ Crashes in production (high concurrency)
167
- - ☠️ Error message: `disk I/O error`, `SQLITE_IOERR`, or `SQLITE_READONLY`
168
- - ☠️ Happens when multiple processes access the database
169
- - ☠️ `-shm` or `-wal` files exist but connection fails
170
-
171
- ---
172
-
173
- ## When This Doesn't Apply
174
-
175
- You're safe to skip this if:
176
- - You use `journal_mode = DELETE` (old default)
177
- - You have exactly one process, one connection, no concurrency
178
- - Your database is truly readonly (burned to CD-ROM)
179
-
180
- **But if you're here, you probably need WAL.**
181
-
182
- ---
183
-
184
- ## The Deeper Why: WAL Architecture
185
-
186
- WAL changed SQLite from a single-writer model to a concurrent model:
187
-
188
- ```
189
- Pre-2010 (DELETE mode):
190
- ┌─────────┐
191
- │ main DB │ ← Writers lock entire file
192
- └─────────┘ Readers wait
193
-
194
- 2010+ (WAL mode):
195
- ┌─────────┐ ┌─────────┐ ┌─────────┐
196
- │ main DB │ ←── │ WAL │ ──→ │ SHM │
197
- └─────────┘ └─────────┘ └─────────┘
198
- ↑ ALL connections must access
199
- ```
200
-
201
- The `-shm` file is a POSIX shared memory segment. It's not optional. It's not a cache. It's the coordination mechanism that makes WAL work.
202
-
203
- Opening a connection without write access to `-shm` is like trying to join a meeting without being able to raise your hand. The system doesn't know you exist.
204
-
205
- ---
206
-
207
- ## Recommended Reading
208
-
209
- - [SQLite WAL Mode Docs](https://www.sqlite.org/wal.html) - Official explanation
210
- - [Appropriate Uses For SQLite](https://www.sqlite.org/whentouse.html) - When to use WAL
211
- - [File Locking And Concurrency](https://www.sqlite.org/lockingv3.html) - Deep dive
212
-
213
- ---
214
-
215
- ## License & Attribution
216
-
217
- This document is born from production failures at Polyvis. Copy it. Share it. Save someone else the debugging pain.
218
-
219
- **Polyvis Standard:** See our full implementation in:
220
- - `src/resonance/DatabaseFactory.ts` - Factory pattern
221
- - `playbooks/sqlite-standards.md` - Team playbook
222
- - `debriefs/2025-12-16-database-stabilization.md` - The war stories
223
-
224
- ---
225
-
226
- ## Version History
227
-
228
- - **v1.0** (2026-01-05) - Initial release after repeated production incidents
@@ -1,40 +0,0 @@
1
- # CSS Architecture Strategy: "Zero Magic, High Predictability"
2
-
3
- ## The Problem
4
- We are currently "fighting the framework".
5
- 1. **Tailwind Preflight** aggressively strips all defaults (cursors, backgrounds) to ensure a blank canvas.
6
- 2. **Open Props** provides tokens but expects us to wire them up.
7
- 3. **Our Custom CSS** attempts to put styles back, but often clashes with the aggressive reset.
8
- 4. **The Build System** (Static Files) created a "Zombie Code" scenario where changes weren't reflected.
9
-
10
- ## The Solution
11
-
12
- ### 1. The Build: "Fresh Start" Protocol
13
- We must guarantee that `http://localhost:3000` never serves stale code.
14
- * **Action:** Update `scripts/cli/dev.ts` to **DELETE** `public/css/app.css` on startup.
15
- * **Result:** If the build fails, the site breaks (Good). If it works, it is fresh (Good). No more "Dark on Dark" ghosts.
16
-
17
- ### 2. The Reset: "PolyVis Base"
18
- Instead of importing a generic reset (like `open-props/normalize`) which previously broke our layout, we officially designate `src/css/layers/base.css` as our **Explicit Reset**.
19
- * **Rule:** If Tailwind strips it (like `cursor: pointer`), we explicitly put it back in `base.css`.
20
- * **Benefit:** We control the defaults. No hidden external styles.
21
-
22
- ### 3. The Future: "Utility-First"
23
- We should stop writing component classes like `.btn-structural` in CSS files.
24
- * **Direction:** Move styles into HTML using Tailwind classes (e.g., `class="cursor-pointer bg-surface-hover..."`).
25
- * **Benefit:** The HTML tells the truth. No CSS file hunting.
26
-
27
- ## Immediate Next Steps
28
- 1. [x] **Fix Cursors:** Done in `base.css`.
29
- 2. [ ] **Fix Build:** Modify `scripts/cli/dev.ts` to clean stale artifacts.
30
- ## Lessons Learned (2025-12-15)
31
- * **Layout Hierarchy:** Trying to control width via *Content* (Buttons) inside a Flex *Container* (Sidebar) creates friction and unpredictable stretching. Always control the **Container** first.
32
- * **Utility vs. Layout:** Utility classes are excellent for discrete styling (colors, spacing) but should be used carefully for varying layout contexts.
33
- * **Revert Culture:** If an experiment fails (like fixed-width buttons), revert to the known stable state (Content-based sizing) rather than patching the bad assumption.
34
-
35
- ## Phase 2: The CSS Remediation Plan
36
- To eliminate "CSS Friction":
37
- 1. **Strict Layering:** Enforce `reset` -> `theme` -> `layout` -> `utilities`.
38
- 2. **Container Queries:** Explore `@container` for components that need to adapt (like sidebar buttons) without relying on global viewport queries.
39
- 3. **Theme Variables for Layout:** Define layout constants (Sidebar Width, Header Height) in `theme.css` to centralize control, but allow components to read them, not drive them.
40
- 4. **Audit:** Systematically review `index.html` to remove mixed metaphors (e.g., inline styles fighting Tailwind classes).