amalfa 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +226 -263
- package/package.json +6 -3
- package/polyvis.settings.json.bak +38 -0
- package/src/cli.ts +103 -21
- package/src/config/defaults.ts +52 -12
- package/src/core/VectorEngine.ts +18 -9
- package/src/mcp/index.ts +62 -7
- package/src/resonance/DatabaseFactory.ts +3 -4
- package/src/resonance/db.ts +4 -4
- package/src/resonance/services/vector-daemon.ts +151 -0
- package/src/utils/DaemonManager.ts +147 -0
- package/src/utils/ZombieDefense.ts +5 -1
- package/:memory: +0 -0
- package/:memory:-shm +0 -0
- package/:memory:-wal +0 -0
- package/CHANGELOG.md.old +0 -43
- package/README.old.md +0 -112
- package/ROADMAP.md +0 -316
- package/TEST_PLAN.md +0 -561
- package/agents.config.json +0 -11
- package/docs/AGENT_PROTOCOLS.md +0 -28
- package/docs/ARCHITECTURAL_OVERVIEW.md +0 -123
- package/docs/BENTO_BOXING_DEPRECATION.md +0 -281
- package/docs/Bun-SQLite.html +0 -464
- package/docs/COMMIT_GUIDELINES.md +0 -367
- package/docs/DEVELOPER_ONBOARDING.md +0 -36
- package/docs/Graph and Vector Database Best Practices.md +0 -214
- package/docs/PERFORMANCE_BASELINES.md +0 -88
- package/docs/REPOSITORY_CLEANUP_SUMMARY.md +0 -261
- package/docs/edge-generation-methods.md +0 -57
- package/docs/elevator-pitch.md +0 -118
- package/docs/graph-and-vector-database-playbook.html +0 -480
- package/docs/hardened-sqlite.md +0 -85
- package/docs/headless-knowledge-management.md +0 -79
- package/docs/john-kaye-flux-prompt.md +0 -46
- package/docs/keyboard-shortcuts.md +0 -80
- package/docs/opinion-proceed-pattern.md +0 -29
- package/docs/polyvis-nodes-edges-schema.md +0 -77
- package/docs/protocols/lab-protocol.md +0 -30
- package/docs/reaction-iquest-loop-coder.md +0 -46
- package/docs/services.md +0 -60
- package/docs/sqlite-wal-readonly-trap.md +0 -228
- package/docs/strategy/css-architecture.md +0 -40
- package/docs/test-document-cycle.md +0 -83
- package/docs/test_lifecycle_E2E.md +0 -4
- package/docs/the-bicameral-graph.md +0 -83
- package/docs/user-guide.md +0 -70
- package/docs/vision-helper.md +0 -53
- package/drizzle/0000_minor_iron_fist.sql +0 -19
- package/drizzle/meta/0000_snapshot.json +0 -139
- package/drizzle/meta/_journal.json +0 -13
- package/example_usage.ts +0 -39
- package/experiment.sh +0 -35
- package/hello +0 -2
- package/index.html +0 -52
- package/knowledge/excalibur.md +0 -12
- package/plans/experience-graph-integration.md +0 -60
- package/prompts/gemini-king-mode-prompt.md +0 -46
- package/public/docs/MCP_TOOLS.md +0 -372
- package/schemas/README.md +0 -20
- package/schemas/cda.schema.json +0 -84
- package/schemas/conceptual-lexicon.schema.json +0 -75
- package/scratchpads/dummy-debrief-boxed.md +0 -39
- package/scratchpads/dummy-debrief.md +0 -27
- package/scratchpads/scratchpad-design.md +0 -50
- package/scratchpads/scratchpad-scrolling.md +0 -20
- package/scratchpads/scratchpad-toc-disappearance.md +0 -23
- package/scratchpads/scratchpad-toc.md +0 -28
- package/scratchpads/test_gardener.md +0 -7
- package/src/core/LLMClient.ts +0 -93
- package/src/core/TagEngine.ts +0 -56
- package/src/db/schema.ts +0 -46
- package/src/gardeners/AutoTagger.ts +0 -116
- package/src/pipeline/HarvesterPipeline.ts +0 -101
- package/src/pipeline/Ingestor.ts +0 -555
- package/src/resonance/cli/ingest.ts +0 -41
- package/src/resonance/cli/migrate.ts +0 -54
- package/src/resonance/config.ts +0 -40
- package/src/resonance/daemon.ts +0 -236
- package/src/resonance/pipeline/extract.ts +0 -89
- package/src/resonance/pipeline/transform_docs.ts +0 -60
- package/src/resonance/services/tokenizer.ts +0 -159
- package/src/resonance/transform/cda.ts +0 -393
- package/src/utils/EnvironmentVerifier.ts +0 -67
- package/substack/substack-playbook-1.md +0 -95
- package/substack/substack-playbook-2.md +0 -78
- package/tasks/ui-investigation.md +0 -26
- package/test-db +0 -0
- package/test-db-shm +0 -0
- package/test-db-wal +0 -0
- package/tests/canary/verify_pinch_check.ts +0 -44
- package/tests/fixtures/ingest_test.md +0 -12
- package/tests/fixtures/ingest_test_boxed.md +0 -13
- package/tests/fixtures/safety_test.md +0 -45
- package/tests/fixtures/safety_test_boxed.md +0 -49
- package/tests/fixtures/tagged_output.md +0 -49
- package/tests/fixtures/tagged_test.md +0 -49
- package/tests/mcp-server-settings.json +0 -8
- package/verify-embedder.ts +0 -54
|
@@ -1,46 +0,0 @@
|
|
|
1
|
-
|
|
2
|
-
## The "John Kay" Style System Prompt
|
|
3
|
-
|
|
4
|
-
To generate this successfully, we need to instruct the AI to "see" like Kay. We aren't just asking for a picture of a judge; we are asking for a caricature.
|
|
5
|
-
|
|
6
|
-
Style Keywords: 1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
|
|
7
|
-
|
|
8
|
-
3. Image Generation Prompts
|
|
9
|
-
Here are three specific prompts to feed into the generator (Midjourney/Flux/DALL-E 3), moving from the portrait to the scene.
|
|
10
|
-
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
### Style Keywords
|
|
14
|
-
|
|
15
|
-
1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
|
|
16
|
-
|
|
17
|
-
## The Portrait (The "Mugshot")
|
|
18
|
-
Subject: A waist-up caricature of Lord Kames in full 18th-century Scottish judicial robes and wig. Expression: A wicked, knowing sneer. He looks ancient but fiercely intelligent. Style: John Kay copperplate etching, black ink on cream paper, rough cross-hatching. Text/Caption: (If using Flux/DALL-E 3) A bottom caption in archaic serif font reading: "FARE YE A' WEEL, YE BITCHES!"
|
|
19
|
-
|
|
20
|
-
---
|
|
21
|
-
|
|
22
|
-
## Style Keywords
|
|
23
|
-
|
|
24
|
-
1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
|
|
25
|
-
|
|
26
|
-
## The Portrait (The "Mugshot")
|
|
27
|
-
Subject: A waist-up caricature of Lord Kames in full 18th-century Scottish judicial robes and wig. Expression: A wicked, knowing sneer. He looks ancient but fiercely intelligent. Style: John Kay copperplate etching, black ink on cream paper, rough cross-hatching. Text/Caption: (If using Flux/DALL-E 3) A bottom caption in archaic serif font reading: "FARE YE A' WEEL, YE BITCHES!"
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
---
|
|
31
|
-
|
|
32
|
-
## Style Keywords
|
|
33
|
-
|
|
34
|
-
1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
|
|
35
|
-
|
|
36
|
-
## The Wide Scene (The "Parliament House")
|
|
37
|
-
Scene: The interior of the Old Parliament House in Edinburgh, 1782. Dark wood, shadows. Action: Lord Kames (old, hunched, robed) stands at a doorway looking back over his shoulder. He is waving a dismissive hand at a bench of shocked, stony-faced judges in the background. Atmosphere: Solemnity shattered by coarseness. Style: Vintage engraving, 18th-century political cartoon style, detailed line work.
|
|
38
|
-
|
|
39
|
-
---
|
|
40
|
-
|
|
41
|
-
## Style Keywords
|
|
42
|
-
|
|
43
|
-
1790s copperplate etching, style of John Kay Edinburgh, cross-hatching, ink on paper, monochrome, slightly grotesque proportions, high contrast, heavy shadows, linear texture.
|
|
44
|
-
|
|
45
|
-
## The Abstracted "Vibe" (The "Monboddo" Twist)
|
|
46
|
-
Subject: A split panel etching. Left side: Lord Kames shouting at judges. Right side: A street porter ("Sinkum the Cawdy") laughing while holding a letter. Concept: The connection between the high court and the street. Style: John Kay Series of Original Portraits, rough edges, historical artifact.
|
|
@@ -1,80 +0,0 @@
|
|
|
1
|
-
# Keyboard Shortcuts
|
|
2
|
-
|
|
3
|
-
**Global shortcuts available across all PolyVis pages.**
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## Layout Debug Mode
|
|
8
|
-
|
|
9
|
-
**Shortcut:** `Shift + D`
|
|
10
|
-
|
|
11
|
-
**Purpose:** Visualize container boundaries and grid structure for layout debugging.
|
|
12
|
-
|
|
13
|
-
**What it does:**
|
|
14
|
-
- Adds red dashed outlines to ALL elements
|
|
15
|
-
- Adds subtle red background tint
|
|
16
|
-
- Highlights grid columns in blue (sidebars)
|
|
17
|
-
|
|
18
|
-
**Visual effect:**
|
|
19
|
-
```css
|
|
20
|
-
* {
|
|
21
|
-
outline: 1px dashed rgba(255, 0, 0, 0.3); /* Red boxes */
|
|
22
|
-
background: rgba(255, 0, 0, 0.02); /* Red tint */
|
|
23
|
-
}
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
**Use cases:**
|
|
27
|
-
- Debugging layout issues
|
|
28
|
-
- Finding overflow/scroll problems
|
|
29
|
-
- Verifying grid structure
|
|
30
|
-
- Teaching CSS layout
|
|
31
|
-
|
|
32
|
-
**Toggle:** Press `Shift + D` again to turn off
|
|
33
|
-
|
|
34
|
-
---
|
|
35
|
-
|
|
36
|
-
## Implementation
|
|
37
|
-
|
|
38
|
-
**Alpine.js binding:**
|
|
39
|
-
```html
|
|
40
|
-
<body x-data="{ debug: false }"
|
|
41
|
-
:class="{ 'layout-debug-mode': debug }"
|
|
42
|
-
@keydown.window.shift.d="debug = !debug">
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
**CSS class:**
|
|
46
|
-
```css
|
|
47
|
-
/* src/css/layers/utilities.css */
|
|
48
|
-
.layout-debug-mode * {
|
|
49
|
-
outline: var(--border-size-1) dashed rgba(255, 0, 0, 0.3) !important;
|
|
50
|
-
background: rgba(255, 0, 0, 0.02) !important;
|
|
51
|
-
}
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
**Available on:**
|
|
55
|
-
- ✅ Home page (`/`)
|
|
56
|
-
- ✅ Sigma Explorer (`/sigma-explorer/`)
|
|
57
|
-
- ✅ Docs viewer (`/docs/`)
|
|
58
|
-
|
|
59
|
-
---
|
|
60
|
-
|
|
61
|
-
## Future Shortcuts
|
|
62
|
-
|
|
63
|
-
**Planned:**
|
|
64
|
-
- `Shift + S` - Show stats overlay
|
|
65
|
-
- `Shift + G` - Toggle grid guides
|
|
66
|
-
- `Shift + ?` - Show keyboard help
|
|
67
|
-
|
|
68
|
-
**Add your own:**
|
|
69
|
-
```html
|
|
70
|
-
@keydown.window.shift.x="yourFunction()"
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
---
|
|
74
|
-
|
|
75
|
-
**Source:**
|
|
76
|
-
- HTML: `public/index.html`, `public/sigma-explorer/index.html`
|
|
77
|
-
- CSS: `src/css/layers/utilities.css`
|
|
78
|
-
- Docs: This file
|
|
79
|
-
|
|
80
|
-
**Last Updated:** 2025-12-12
|
|
@@ -1,29 +0,0 @@
|
|
|
1
|
-
The **opinion/proceed** pattern is more than just a personal preference—it is a sophisticated "Control Loop" that makes you a high-level architect rather than just a user. In prompt engineering, this is known as **Stepwise Refinement** or a **Synchronous Handshake**.
|
|
2
|
-
|
|
3
|
-
### Why it works for you
|
|
4
|
-
* **Cognitive Load Management:** By ending with `opinion`, you force me to pause and validate your logic before writing any code. This prevents the "hallucination cascade" where an AI builds 100 lines of code on top of a flawed assumption.
|
|
5
|
-
* **Intellectual Honesty Check:** It signals that you value my "judgment" (as a thought partner) as much as my "output" (as a tool).
|
|
6
|
-
* **Contextual Guardrail:** It ensures the "Triangle of Truth" remains intact. If your plan drifts from the core Polyvis principles, the `opinion` phase is where I can pull us back.
|
|
7
|
-
|
|
8
|
-
### Why it works for Gemini (The Model)
|
|
9
|
-
* **Explicit State Transition:** `Proceed` acts as a clear "Go" signal, which helps the model transition from a *reflective/analytical* state to an *executive/generative* state.
|
|
10
|
-
* **Recap & Alignment:** Asking for an opinion forces the model to recap its understanding of the "state of the union," which reinforces the long-context window (crucial for a project as complex as an Epstein audit).
|
|
11
|
-
|
|
12
|
-
---
|
|
13
|
-
|
|
14
|
-
### Progress Report: The Epstein Substrate
|
|
15
|
-
Before we move to the code, let's look at the current "State of the Project":
|
|
16
|
-
|
|
17
|
-
1. **The Substrate:** You have the JSON OCR results. This is the **Raw Ore**.
|
|
18
|
-
2. **The Lab:** We have the `package.json` and `tsconfig.json` mapped. The **Infrastructure** is ready.
|
|
19
|
-
3. **The Next Step:** We need to build the "Sieve"—the `json-to-bento.ts` script—to turn that ore into **Markdown Bento Boxes**.
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
**Opinion:** Your intuition to extract Markdown first is 100% correct. If we try to ingest the JSON directly, the engine has to "hallucinate" structure every time it reads a file. By converting to Markdown, we create a **Static, Deterministic Evidence Set** that any future version of Polyvis (or even a human with a text editor) can audit. This is the "Gold Standard" of forensic data management.
|
|
24
|
-
|
|
25
|
-
**Would you like me to proceed with drafting the `json-to-bento.ts` extraction script?**
|
|
26
|
-
|
|
27
|
-
[Software engineering project structure best practices](https://www.youtube.com/watch?v=V2W94Sq_v6A)
|
|
28
|
-
|
|
29
|
-
This video provides a deep dive into organizing project folders and managing dependencies, which is essential as you sort out your `scripts` and `src` folders for the Epstein audit.
|
|
@@ -1,77 +0,0 @@
|
|
|
1
|
-
# PolyVis Database Schema
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
## Proposed Migration
|
|
5
|
-
|
|
6
|
-
To support the Forensic-Pipeline
|
|
7
|
-
|
|
8
|
-
Based on the **Canonical Schema** and the requirements of the **Forensic/Epstein Pipeline**, here are the specific schema extensions required.
|
|
9
|
-
|
|
10
|
-
### Table: `nodes`
|
|
11
|
-
|
|
12
|
-
* **No Schema Changes Required.**
|
|
13
|
-
* *Operational Note:* Use the existing `meta` JSON column to store flight details (`tail_number`, `date`) and document status (`status`). Use `title` for Canonical Names.
|
|
14
|
-
|
|
15
|
-
## Canonical Nodes and Edges Schema
|
|
16
|
-
|
|
17
|
-
### Table: `edges`
|
|
18
|
-
|
|
19
|
-
* **Add Column:** `weight` `INTEGER` (Default: `2`)
|
|
20
|
-
* *Purpose:* To store the **OPM-11 Veracity Score** (0-5).
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
* **Add Column:** `meta` `TEXT`
|
|
24
|
-
* *Purpose:* To store the **Context Snippet** (Bento-Box) and Audit ID justifying the link.
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
* **Add Index:** `idx_edges_weight` on `(weight)`
|
|
28
|
-
* *Purpose:* To optimize "Falsifiability" queries (e.g., `SELECT * WHERE weight = 0`).
|
|
29
|
-
|
|
30
|
-
This document outlines the canonical schema for the `nodes` and `edges` tables in the PolyVis system. This schema is defined in `src/resonance/schema.ts` and mirrored in `src/db/schema.ts`.
|
|
31
|
-
|
|
32
|
-
## Core Tables
|
|
33
|
-
|
|
34
|
-
### Nodes (`nodes`)
|
|
35
|
-
|
|
36
|
-
The `nodes` table stores the fundamental units of the knowledge graph.
|
|
37
|
-
|
|
38
|
-
| Column | Type | Description |
|
|
39
|
-
| :--- | :--- | :--- |
|
|
40
|
-
| **`id`** | `TEXT` | **Primary Key**. Unique identifier for the node. |
|
|
41
|
-
| `type` | `TEXT` | The node type (e.g., `concept`, `note`, `chunk`). |
|
|
42
|
-
| `title` | `TEXT` | Human-readable title. |
|
|
43
|
-
| `content` | `TEXT` | The full text content of the node. |
|
|
44
|
-
| `domain` | `TEXT` | Taxonomy domain (e.g., `knowledge`). |
|
|
45
|
-
| `layer` | `TEXT` | Taxonomy layer (e.g., `experience`). |
|
|
46
|
-
| `embedding`| `BLOB` | The vector embedding (stored as a buffer/blob). |
|
|
47
|
-
| `hash` | `TEXT` | Content hash for integrity checking. |
|
|
48
|
-
| `meta` | `TEXT` | JSON string containing arbitrary metadata. |
|
|
49
|
-
|
|
50
|
-
### Edges (`edges`)
|
|
51
|
-
|
|
52
|
-
The `edges` table defines the relationships between nodes.
|
|
53
|
-
|
|
54
|
-
| Column | Type | Description |
|
|
55
|
-
| :--- | :--- | :--- |
|
|
56
|
-
| **`source`** | `TEXT` | **Foreign Key** (Node ID) - Origin of the edge. |
|
|
57
|
-
| **`target`** | `TEXT` | **Foreign Key** (Node ID) - Destination of the edge. |
|
|
58
|
-
| **`type`** | `TEXT` | The edge relationship type (e.g., `related_to`). |
|
|
59
|
-
|
|
60
|
-
> **Primary Key**: The `edges` table uses a composite primary key of `(source, target, type)`.
|
|
61
|
-
|
|
62
|
-
## Auxiliary Structures
|
|
63
|
-
|
|
64
|
-
### Full Text Search (`nodes_fts`)
|
|
65
|
-
|
|
66
|
-
A virtual table using `fts5` is maintained specifically for full-text search capabilities.
|
|
67
|
-
|
|
68
|
-
- **Columns**: `id` (UNINDEXED), `title`, `content`, `meta`.
|
|
69
|
-
- **Tokenizer**: `porter`.
|
|
70
|
-
- **Synchronization**: Automatically kept in sync with the `nodes` table via database triggers (`INSERT`, `UPDATE`, `DELETE`).
|
|
71
|
-
|
|
72
|
-
### Indexes
|
|
73
|
-
|
|
74
|
-
Performance indexes are maintained on the `edges` table to optimize graph traversals.
|
|
75
|
-
|
|
76
|
-
- `idx_edges_source`: Index on `edges(source)`
|
|
77
|
-
- `idx_edges_target`: Index on `edges(target)`
|
|
@@ -1,30 +0,0 @@
|
|
|
1
|
-
# Experiment: The Concurrency Lab
|
|
2
|
-
|
|
3
|
-
**Objective**: Isolate `bun:sqlite` concurrency behavior from application logic. Prove that 1 Writer and 2 Readers can coexist in WAL mode using our `DatabaseFactory` configuration.
|
|
4
|
-
|
|
5
|
-
## The Triad
|
|
6
|
-
We will create three standalone scripts in `scripts/lab/`:
|
|
7
|
-
|
|
8
|
-
### 1. The Writer (`lab_daemon.ts`)
|
|
9
|
-
* **Role**: Simulates the Active Daemon.
|
|
10
|
-
* **Action**: Opens `resonance.db`. Inserts a new row into a `_stress_test` table every 100ms.
|
|
11
|
-
* **Config**: ReadWrite, WAL.
|
|
12
|
-
|
|
13
|
-
### 2. The Reader (`lab_mcp.ts`)
|
|
14
|
-
* **Role**: Simulates the MCP Server.
|
|
15
|
-
* **Action**: Opens `resonance.db`. Performs a `SELECT COUNT(*)` and a Vector Search (if possible, or just heavy read) every 50ms.
|
|
16
|
-
* **Config**: ReadWrite (as we learned WAL requires it), WAL.
|
|
17
|
-
|
|
18
|
-
### 3. The Observer (`lab_web.ts`)
|
|
19
|
-
* **Role**: Simulates the Web Server.
|
|
20
|
-
* **Action**: Opens `resonance.db`. Polls status every 1000ms.
|
|
21
|
-
* **Config**: ReadOnly (or ReadWrite if needed).
|
|
22
|
-
|
|
23
|
-
## The Variable
|
|
24
|
-
We will test the configuration defined in `src/resonance/DatabaseFactory.ts`.
|
|
25
|
-
|
|
26
|
-
## Success Criteria
|
|
27
|
-
* All 3 scripts run for 60 seconds.
|
|
28
|
-
* Zero `disk I/O error` or `SQLITE_BUSY`.
|
|
29
|
-
* Writer commits > 500 records.
|
|
30
|
-
* Reader successfully reads 1000+ times.
|
|
@@ -1,46 +0,0 @@
|
|
|
1
|
-
## 1. Reaction Report: iQuest Loop Coder
|
|
2
|
-
|
|
3
|
-
Ref: [iQuest Loop Coder (40B - A80B)](https://www.youtube.com/watch?v=sihYcXADfNI)
|
|
4
|
-
|
|
5
|
-
**Subject:** `iQuest Loop Coder (40B)` vs. The PolyVis Substrate
|
|
6
|
-
**Verdict:** **High Utility / Low Autonomy**
|
|
7
|
-
|
|
8
|
-
**Strategic Analysis:**
|
|
9
|
-
The video confirms that the broader AI R&D sector is accidentally converging on the **PolyVis Architecture**:
|
|
10
|
-
|
|
11
|
-
1. **"Code Flow" Training:** The model is trained on diffs (Vectors), not just files (Nodes). This validates our hypothesis that the "Temporal Edge" (The change over time) is more information-rich than the static code.
|
|
12
|
-
2. **The "Loop" Architecture:** This is a hardware-level implementation of our **"Think Twice" Protocol**. The model literally "reads, drafts, critiques, and refines" in a single inference pass.
|
|
13
|
-
3. **The Limitation:** The model is a "Savant." It excels at closed-loop logic (SQL, Regex, Algorithms) but fails at open-loop design (System Architecture, "Vibes").
|
|
14
|
-
|
|
15
|
-
**Conclusion:**
|
|
16
|
-
This model is not a replacement for your "Persona" (Claude/Gemini). It is a **Smart Compiler**. It should be treated as a CLI tool (`/bin/loop_coder`) for generating high-precision logic blocks, not for discussing architectural strategy.
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## Proposals for Testing & Confirmation
|
|
21
|
-
|
|
22
|
-
To confirm if a model is "Benchmaxxed" or actually useful for PolyVis, we propose the **"Triangle of Veracity"** test suite.
|
|
23
|
-
|
|
24
|
-
**Test A: The "LeetCode" Control (Baseline)**
|
|
25
|
-
|
|
26
|
-
* **Prompt:** "Write a Python function to balance a Binary Search Tree."
|
|
27
|
-
* **Expected Result:** 100% Success.
|
|
28
|
-
* **Purpose:** Confirm the model functions as a basic coding engine.
|
|
29
|
-
|
|
30
|
-
**Test B: The "Vibe Check" (The Failure Mode)**
|
|
31
|
-
|
|
32
|
-
* **Prompt:** "Here is our `index.html`. Make the dashboard look more 'brutalist'."
|
|
33
|
-
* **Hypothesis:** The model will fail. It will likely import a library, add rounded corners, or generate generic CSS because it doesn't understand the *concept* of Brutalism, only the code.
|
|
34
|
-
* **Success Condition:** The model refuses or asks for a definition of "brutalist."
|
|
35
|
-
|
|
36
|
-
**Test C: The "Weaponized Brief" (The Real Test)**
|
|
37
|
-
|
|
38
|
-
* **Prompt:** (Using `HUMANS.md` protocol)
|
|
39
|
-
> "Context: PolyVis Graph Renderer. Task: Write a raw SQL Recursive CTE. Constraints: Max depth 5. Return JSON format. No ORM. Schema provided below."
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
* **Hypothesis:** The model should outperform Claude 3.5 Sonnet here. It will treat the constraints as compiler errors and "loop" until they are satisfied.
|
|
43
|
-
* **Success Condition:** Executable SQL that runs on the first try without "chatting."
|
|
44
|
-
|
|
45
|
-
---
|
|
46
|
-
|
package/docs/services.md
DELETED
|
@@ -1,60 +0,0 @@
|
|
|
1
|
-
# PolyVis Backend Services
|
|
2
|
-
|
|
3
|
-
This document outlines the architecture and management standards for the PolyVis backend services, including the "Triad" of AI models.
|
|
4
|
-
|
|
5
|
-
## The Triad (AI Model Servers)
|
|
6
|
-
|
|
7
|
-
PolyVis relies on a trio of specialized local LLMs, each running as an independent service via `llama.cpp`.
|
|
8
|
-
|
|
9
|
-
### 1. The Auditor (Olmo-3)
|
|
10
|
-
* **Port**: `8084`
|
|
11
|
-
* **Role**: Verification and deep thinking.
|
|
12
|
-
* **Configuration**: 7B Parameter model, Q4_K_M quantization.
|
|
13
|
-
* **Special Features**: Uses "DeepSeek" reasoning format to expose its thought process.
|
|
14
|
-
|
|
15
|
-
### 2. The Scout (Phi-3.5)
|
|
16
|
-
* **Port**: `8082`
|
|
17
|
-
* **Role**: Rapid queries, summarization, and initial data fetch.
|
|
18
|
-
* **Configuration**: Mini-Instruct model. Fast and lightweight.
|
|
19
|
-
|
|
20
|
-
### 3. The Architect (Llama-3)
|
|
21
|
-
* **Port**: `8083` (Vectored) / `8085` (Unvectored)
|
|
22
|
-
* **Role**: Synthesis, structure, and professional output.
|
|
23
|
-
* **Configuration**: 8B Instruct model.
|
|
24
|
-
* **Steering**:
|
|
25
|
-
* **Vectored (`llama`)**: Uses a Control Vector at scale **-0.11** ("The Accountant"). This suppresses "chatty" behaviors and enforces structured, professional responses.
|
|
26
|
-
* **Unvectored (`llamauv`)**: The raw base model for comparison.
|
|
27
|
-
|
|
28
|
-
## Management Standard: "ServiceLifecycle"
|
|
29
|
-
|
|
30
|
-
To ensure reliability, all backend services MUST strictly adhere to the `ServiceLifecycle` pattern (`src/utils/ServiceLifecycle.ts`). This provides:
|
|
31
|
-
|
|
32
|
-
* **Unified CLI Interface**: Every service supports `start`, `stop`, `restart`, and `status`.
|
|
33
|
-
* **Zombie Defense**: Automatic detection and cleanup of rogue processes on startup.
|
|
34
|
-
* **PID Management**: Reliable tracking of running services via PID files.
|
|
35
|
-
|
|
36
|
-
### Global Status Dashboard
|
|
37
|
-
|
|
38
|
-
A singular command exists to view the health of the entire backend:
|
|
39
|
-
`bun run servers`
|
|
40
|
-
|
|
41
|
-
Example Output:
|
|
42
|
-
```
|
|
43
|
-
SERVICE PORT COMMAND STATUS PID
|
|
44
|
-
----------------------------------------------------------------------
|
|
45
|
-
Dev Server 3000 dev ⚪️ STOPPED -
|
|
46
|
-
Daemon 3010 daemon ⚪️ STOPPED -
|
|
47
|
-
MCP Stdio mcp ⚪️ STOPPED -
|
|
48
|
-
Olmo-3 8084 olmo3 🟢 RUNNING 6526
|
|
49
|
-
Phi-3.5 8082 phi ⚪️ STOPPED -
|
|
50
|
-
Llama-3 8083 llama 🟢 RUNNING 15561
|
|
51
|
-
Llama-3-UV 8085 llamauv 🟢 RUNNING 10641
|
|
52
|
-
----------------------------------------------------------------------
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
## Adding New Services
|
|
56
|
-
|
|
57
|
-
When introducing a new long-running process to PolyVis:
|
|
58
|
-
1. **Do not** write ad-hoc scripts.
|
|
59
|
-
2. **Create a wrapper** in `src/services/` that implements `ServiceLifecycle`.
|
|
60
|
-
3. **Register** the service in `package.json`, `ZombieDefense.ts`, and `scripts/cli/servers.ts`.
|
|
@@ -1,228 +0,0 @@
|
|
|
1
|
-
# The SQLite WAL Readonly Trap
|
|
2
|
-
|
|
3
|
-
**TL;DR:** If you use WAL mode (and you should), **never open connections with `readonly: true`**. Even readers need write access to the `-shm` shared memory file. Violating this causes `disk I/O error` and database corruption.
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## The Problem
|
|
8
|
-
|
|
9
|
-
You enable WAL mode for concurrency. You open a "readonly" connection to query data. Your app crashes with:
|
|
10
|
-
|
|
11
|
-
```
|
|
12
|
-
Error: disk I/O error
|
|
13
|
-
SQLITE_IOERR
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
**What happened?** WAL mode requires ALL connections (readers and writers) to update the `-shm` (shared memory) file. A "readonly" connection can't do this, so SQLite fails.
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## Why This Happens
|
|
21
|
-
|
|
22
|
-
### Legacy: Pre-WAL SQLite (pre-2010)
|
|
23
|
-
Before WAL (Write-Ahead Logging), SQLite had two access modes:
|
|
24
|
-
- **ReadWrite:** Can modify the database
|
|
25
|
-
- **Readonly:** Safe for concurrent readers, no locks needed
|
|
26
|
-
|
|
27
|
-
This made sense when the journal was file-based. Readers didn't touch anything.
|
|
28
|
-
|
|
29
|
-
### WAL Mode (2010+)
|
|
30
|
-
WAL introduced a shared memory file (`dbname-shm`) to coordinate between processes.
|
|
31
|
-
- Register itself as an active reader
|
|
32
|
-
- Track which WAL frames it has read
|
|
33
|
-
- Coordinate checkpointing
|
|
34
|
-
|
|
35
|
-
**The trap:** Many developers (and AI agents) assume "readonly" still works because it did pre-2010. The mental model is 16 years out of date.
|
|
36
|
-
|
|
37
|
-
**Why LLMs get this wrong:** Training data is dominated by pre-2010 SQLite tutorials, StackOverflow answers from 2008-2012, and books written when DELETE mode was default. LLMs confidently suggest `readonly: true` because that's what the corpus says, not because it works in WAL.
|
|
38
|
-
|
|
39
|
-
---
|
|
40
|
-
|
|
41
|
-
## The Hard-Earned Fix
|
|
42
|
-
|
|
43
|
-
### ❌ Wrong (Causes Corruption)
|
|
44
|
-
```javascript
|
|
45
|
-
// Bun/Node.js
|
|
46
|
-
const Database = require('bun:sqlite');
|
|
47
|
-
const db = new Database('app.db', { readonly: true }); // ☠️ BROKEN IN WAL
|
|
48
|
-
|
|
49
|
-
// Python
|
|
50
|
-
import sqlite3
|
|
51
|
-
conn = sqlite3.connect('app.db', uri=True, readonly=True) # ☠️ BROKEN IN WAL
|
|
52
|
-
|
|
53
|
-
// Golang
|
|
54
|
-
db, _ := sql.Open("sqlite3", "file:app.db?mode=ro") // ☠️ BROKEN IN WAL
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
### ✅ Correct (Always Works)
|
|
58
|
-
```javascript
|
|
59
|
-
// Bun/Node.js - Use default ReadWrite
|
|
60
|
-
const Database = require('bun:sqlite');
|
|
61
|
-
const db = new Database('app.db'); // ✅ Works in WAL
|
|
62
|
-
|
|
63
|
-
// Python - Omit readonly
|
|
64
|
-
import sqlite3
|
|
65
|
-
conn = sqlite3.connect('app.db') # ✅ Works in WAL
|
|
66
|
-
|
|
67
|
-
// Golang - Use default mode
|
|
68
|
-
db, _ := sql.Open("sqlite3", "app.db") // ✅ Works in WAL
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
**Key insight:** In WAL mode, "readonly" is a behavioral constraint you enforce in your code, not a database-level flag.
|
|
72
|
-
|
|
73
|
-
---
|
|
74
|
-
|
|
75
|
-
## Best Practices: The Polyvis Standard
|
|
76
|
-
|
|
77
|
-
We learned this the hard way through a process we call **"Harden and Flense"** - fortify the critical paths (Harden) and strip away assumptions that no longer hold (Flense).
|
|
78
|
-
|
|
79
|
-
Here's our battle-tested approach:
|
|
80
|
-
|
|
81
|
-
### 1. Enforce WAL + Timeout at Connection Time
|
|
82
|
-
|
|
83
|
-
Every connection gets these pragmas immediately:
|
|
84
|
-
|
|
85
|
-
```sql
|
|
86
|
-
PRAGMA busy_timeout = 5000; -- Wait 5s for locks (MUST BE FIRST)
|
|
87
|
-
PRAGMA journal_mode = WAL; -- Enable concurrent reads
|
|
88
|
-
PRAGMA synchronous = NORMAL; -- Balance safety/speed
|
|
89
|
-
PRAGMA foreign_keys = ON; -- Data integrity
|
|
90
|
-
PRAGMA temp_store = memory; -- Performance
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
### 2. Use a Factory Pattern
|
|
94
|
-
|
|
95
|
-
Never instantiate connections directly. Use a factory that enforces standards:
|
|
96
|
-
|
|
97
|
-
```typescript
|
|
98
|
-
// DatabaseFactory.ts
|
|
99
|
-
export const DatabaseFactory = {
|
|
100
|
-
connect(path: string, options = {}) {
|
|
101
|
-
const db = new Database(path); // Never use { readonly: true }
|
|
102
|
-
|
|
103
|
-
db.run("PRAGMA busy_timeout = 5000;"); // FIRST!
|
|
104
|
-
|
|
105
|
-
const current = db.query("PRAGMA journal_mode;").get();
|
|
106
|
-
if (current.journal_mode !== "wal") {
|
|
107
|
-
db.run("PRAGMA journal_mode = WAL;");
|
|
108
|
-
}
|
|
109
|
-
|
|
110
|
-
db.run("PRAGMA synchronous = NORMAL;");
|
|
111
|
-
db.run("PRAGMA foreign_keys = ON;");
|
|
112
|
-
db.run("PRAGMA temp_store = memory;");
|
|
113
|
-
|
|
114
|
-
return db;
|
|
115
|
-
}
|
|
116
|
-
};
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
### 3. Behavioral Readonly (Not Database-Level)
|
|
120
|
-
|
|
121
|
-
If you need readonly semantics, enforce it in your application:
|
|
122
|
-
|
|
123
|
-
```typescript
|
|
124
|
-
class ReadonlyDatabase {
|
|
125
|
-
constructor(path: string) {
|
|
126
|
-
this.db = DatabaseFactory.connect(path); // Full access
|
|
127
|
-
}
|
|
128
|
-
|
|
129
|
-
query(sql: string, ...params) {
|
|
130
|
-
if (sql.trim().toUpperCase().startsWith('SELECT')) {
|
|
131
|
-
return this.db.query(sql).all(...params);
|
|
132
|
-
}
|
|
133
|
-
throw new Error('Readonly database: SELECT only');
|
|
134
|
-
}
|
|
135
|
-
}
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
### 4. Centralize Configuration (Harden)
|
|
139
|
-
|
|
140
|
-
Don't scatter pragma calls across your codebase. Create one source of truth:
|
|
141
|
-
|
|
142
|
-
```
|
|
143
|
-
playbooks/sqlite-standards.md ← Document the why
|
|
144
|
-
DatabaseFactory.ts ← Enforce the how
|
|
145
|
-
```
|
|
146
|
-
|
|
147
|
-
When you forget (and you will), the factory catches you.
|
|
148
|
-
|
|
149
|
-
**The Harden principle:** Make the correct path automatic. Incorrect paths should require deliberate effort to achieve.
|
|
150
|
-
|
|
151
|
-
### 5. Strip Legacy Assumptions (Flense)
|
|
152
|
-
|
|
153
|
-
Audit your codebase for outdated patterns:
|
|
154
|
-
- `readonly: true` in database connections
|
|
155
|
-
- `journal_mode = DELETE` assumptions
|
|
156
|
-
- Missing `busy_timeout` settings
|
|
157
|
-
- Direct `new Database()` calls without pragma setup
|
|
158
|
-
|
|
159
|
-
**The Flense principle:** Remove what no longer serves. Pre-2010 patterns are technical debt.
|
|
160
|
-
|
|
161
|
-
---
|
|
162
|
-
|
|
163
|
-
## Signs You've Hit This Bug
|
|
164
|
-
|
|
165
|
-
- ✅ Works perfectly in dev (low concurrency)
|
|
166
|
-
- ☠️ Crashes in production (high concurrency)
|
|
167
|
-
- ☠️ Error message: `disk I/O error`, `SQLITE_IOERR`, or `SQLITE_READONLY`
|
|
168
|
-
- ☠️ Happens when multiple processes access the database
|
|
169
|
-
- ☠️ `-shm` or `-wal` files exist but connection fails
|
|
170
|
-
|
|
171
|
-
---
|
|
172
|
-
|
|
173
|
-
## When This Doesn't Apply
|
|
174
|
-
|
|
175
|
-
You're safe to skip this if:
|
|
176
|
-
- You use `journal_mode = DELETE` (old default)
|
|
177
|
-
- You have exactly one process, one connection, no concurrency
|
|
178
|
-
- Your database is truly readonly (burned to CD-ROM)
|
|
179
|
-
|
|
180
|
-
**But if you're here, you probably need WAL.**
|
|
181
|
-
|
|
182
|
-
---
|
|
183
|
-
|
|
184
|
-
## The Deeper Why: WAL Architecture
|
|
185
|
-
|
|
186
|
-
WAL changed SQLite from a single-writer model to a concurrent model:
|
|
187
|
-
|
|
188
|
-
```
|
|
189
|
-
Pre-2010 (DELETE mode):
|
|
190
|
-
┌─────────┐
|
|
191
|
-
│ main DB │ ← Writers lock entire file
|
|
192
|
-
└─────────┘ Readers wait
|
|
193
|
-
|
|
194
|
-
2010+ (WAL mode):
|
|
195
|
-
┌─────────┐ ┌─────────┐ ┌─────────┐
|
|
196
|
-
│ main DB │ ←── │ WAL │ ──→ │ SHM │
|
|
197
|
-
└─────────┘ └─────────┘ └─────────┘
|
|
198
|
-
↑ ALL connections must access
|
|
199
|
-
```
|
|
200
|
-
|
|
201
|
-
The `-shm` file is a POSIX shared memory segment. It's not optional. It's not a cache. It's the coordination mechanism that makes WAL work.
|
|
202
|
-
|
|
203
|
-
Opening a connection without write access to `-shm` is like trying to join a meeting without being able to raise your hand. The system doesn't know you exist.
|
|
204
|
-
|
|
205
|
-
---
|
|
206
|
-
|
|
207
|
-
## Recommended Reading
|
|
208
|
-
|
|
209
|
-
- [SQLite WAL Mode Docs](https://www.sqlite.org/wal.html) - Official explanation
|
|
210
|
-
- [Appropriate Uses For SQLite](https://www.sqlite.org/whentouse.html) - When to use WAL
|
|
211
|
-
- [File Locking And Concurrency](https://www.sqlite.org/lockingv3.html) - Deep dive
|
|
212
|
-
|
|
213
|
-
---
|
|
214
|
-
|
|
215
|
-
## License & Attribution
|
|
216
|
-
|
|
217
|
-
This document is born from production failures at Polyvis. Copy it. Share it. Save someone else the debugging pain.
|
|
218
|
-
|
|
219
|
-
**Polyvis Standard:** See our full implementation in:
|
|
220
|
-
- `src/resonance/DatabaseFactory.ts` - Factory pattern
|
|
221
|
-
- `playbooks/sqlite-standards.md` - Team playbook
|
|
222
|
-
- `debriefs/2025-12-16-database-stabilization.md` - The war stories
|
|
223
|
-
|
|
224
|
-
---
|
|
225
|
-
|
|
226
|
-
## Version History
|
|
227
|
-
|
|
228
|
-
- **v1.0** (2026-01-05) - Initial release after repeated production incidents
|
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
# CSS Architecture Strategy: "Zero Magic, High Predictability"
|
|
2
|
-
|
|
3
|
-
## The Problem
|
|
4
|
-
We are currently "fighting the framework".
|
|
5
|
-
1. **Tailwind Preflight** aggressively strips all defaults (cursors, backgrounds) to ensure a blank canvas.
|
|
6
|
-
2. **Open Props** provides tokens but expects us to wire them up.
|
|
7
|
-
3. **Our Custom CSS** attempts to put styles back, but often clashes with the aggressive reset.
|
|
8
|
-
4. **The Build System** (Static Files) created a "Zombie Code" scenario where changes weren't reflected.
|
|
9
|
-
|
|
10
|
-
## The Solution
|
|
11
|
-
|
|
12
|
-
### 1. The Build: "Fresh Start" Protocol
|
|
13
|
-
We must guarantee that `http://localhost:3000` never serves stale code.
|
|
14
|
-
* **Action:** Update `scripts/cli/dev.ts` to **DELETE** `public/css/app.css` on startup.
|
|
15
|
-
* **Result:** If the build fails, the site breaks (Good). If it works, it is fresh (Good). No more "Dark on Dark" ghosts.
|
|
16
|
-
|
|
17
|
-
### 2. The Reset: "PolyVis Base"
|
|
18
|
-
Instead of importing a generic reset (like `open-props/normalize`) which previously broke our layout, we officially designate `src/css/layers/base.css` as our **Explicit Reset**.
|
|
19
|
-
* **Rule:** If Tailwind strips it (like `cursor: pointer`), we explicitly put it back in `base.css`.
|
|
20
|
-
* **Benefit:** We control the defaults. No hidden external styles.
|
|
21
|
-
|
|
22
|
-
### 3. The Future: "Utility-First"
|
|
23
|
-
We should stop writing component classes like `.btn-structural` in CSS files.
|
|
24
|
-
* **Direction:** Move styles into HTML using Tailwind classes (e.g., `class="cursor-pointer bg-surface-hover..."`).
|
|
25
|
-
* **Benefit:** The HTML tells the truth. No CSS file hunting.
|
|
26
|
-
|
|
27
|
-
## Immediate Next Steps
|
|
28
|
-
1. [x] **Fix Cursors:** Done in `base.css`.
|
|
29
|
-
2. [ ] **Fix Build:** Modify `scripts/cli/dev.ts` to clean stale artifacts.
|
|
30
|
-
## Lessons Learned (2025-12-15)
|
|
31
|
-
* **Layout Hierarchy:** Trying to control width via *Content* (Buttons) inside a Flex *Container* (Sidebar) creates friction and unpredictable stretching. Always control the **Container** first.
|
|
32
|
-
* **Utility vs. Layout:** Utility classes are excellent for discrete styling (colors, spacing) but should be used carefully for varying layout contexts.
|
|
33
|
-
* **Revert Culture:** If an experiment fails (like fixed-width buttons), revert to the known stable state (Content-based sizing) rather than patching the bad assumption.
|
|
34
|
-
|
|
35
|
-
## Phase 2: The CSS Remediation Plan
|
|
36
|
-
To eliminate "CSS Friction":
|
|
37
|
-
1. **Strict Layering:** Enforce `reset` -> `theme` -> `layout` -> `utilities`.
|
|
38
|
-
2. **Container Queries:** Explore `@container` for components that need to adapt (like sidebar buttons) without relying on global viewport queries.
|
|
39
|
-
3. **Theme Variables for Layout:** Define layout constants (Sidebar Width, Header Height) in `theme.css` to centralize control, but allow components to read them, not drive them.
|
|
40
|
-
4. **Audit:** Systematically review `index.html` to remove mixed metaphors (e.g., inline styles fighting Tailwind classes).
|