coaiajs 0.1.0 → 0.1.2

package/rispecs/09-mcp-server.spec.md

@@ -9,62 +9,60 @@ A single MCP server (`coaiajs-mcp`) that consolidates the 44 tools from coaia-na
 ## Structural Tension
 
 **Current Reality:**
-- MCP server entry defined in `package.json` as `"coaiajs-mcp": "./dist/mcp/server.js"` but `mcp/` directory is empty (only a `tools/` subdirectory stub)
-- `@modelcontextprotocol/sdk` v1.25.0 is installed as a dependency
-- Four separate MCP servers exist in parent projects:
-  - coaia-narrative: 27 tools (STC lifecycle, knowledge graph, narrative beats, MMOT)
-  - coaia-pde: 12 tools (decomposition, session management, STC transformation)
-  - coaia-planning: 5 tools (plan parsing, plan↔STC sync, PDE bridge)
-  - coaiapy: 0 MCP tools (Python library only, no MCP)
-- Each parent server has its own startup, tool registration, and transport handling
-- Users currently need 3 MCP server entries in their config to access all tools
+- [`package.json`](../package.json) defines `"coaiajs-mcp": "./dist/mcp/server.js"`.
+- [`mcp/server.ts`](../mcp/server.ts) implements stdio MCP transport using `@modelcontextprotocol/sdk`.
+- [`mcp/config.ts`](../mcp/config.ts) implements `MINIMAL`, `STANDARD`, `OBSERVABILITY`, and `FULL` feature sets via `COAIAJS_FEATURES` or `--features`.
+- [`mcp/tools/coaiapy-tools.ts`](../mcp/tools/coaiapy-tools.ts) defines coaiapy-compatible Redis and Langfuse tool schemas.
+- [`mcp/resources.ts`](../mcp/resources.ts) implements `coaia://templates/` resources backed by the pipeline template loader.
+- [`mcp/prompts.ts`](../mcp/prompts.ts) implements the three coaiapy-mcp prompt templates.
+- `npx coaiajs-mcp` starts successfully from an installed package tarball and reports `STANDARD: 64 tools, 3 prompts, 3 resources`.
+- Remaining gap: several desired package-native tool groups (audio and pipeline execution tools) are not exposed as standalone MCP tools yet; pipeline templates are currently exposed as resources.
 
 **Desired Outcome:**
 Single MCP server at `mcp/server.ts` providing:
-- All 44 parent tools with identical schemas and behavior
-- 20 new tools for previously-CLI-only functionality
-- Feature gating via `COAIAJS_MCP_MODE` environment variable
+- Parent and coaiapy-compatible tools through one stdio server
+- Template resources and workflow prompts from coaiapy-mcp
+- Feature gating via `COAIAJS_FEATURES` or `--features`
 - Single server entry in MCP client config
 
 ## Feature Gating
 
 ```typescript
-type McpMode = 'MINIMAL' | 'STANDARD' | 'FULL';
+type FeatureLevel = 'MINIMAL' | 'STANDARD' | 'OBSERVABILITY' | 'FULL';
 ```
 
 | Mode | Tools loaded | Use case |
 |------|-------------|----------|
-| **MINIMAL** | STC tools (11), KG tools (9), MMOT (2), narrative beats (3) = **25** | Memory-constrained agents, basic STC workflow |
-| **STANDARD** | MINIMAL + PDE (12) + planning (6) + Redis (5) = **48** | Standard development sessions |
-| **FULL** | STANDARD + Langfuse (8) + pipeline (3) + audio (2) + guidance (3) = **64** | Full-featured agent sessions |
+| **MINIMAL** | Redis + Langfuse observability core tools | Observability-only sessions |
+| **STANDARD** | MINIMAL + narrative + PDE + planning tools | Standard development sessions |
+| **OBSERVABILITY** | Alias of STANDARD in current implementation | Compatibility mode |
+| **FULL** | All registered tools, including media tools | Full-featured agent sessions |
 
-Default mode: `STANDARD`.
+Default feature level: `STANDARD`.
 
 ## Server Architecture
 
 ```typescript
 // mcp/server.ts
-import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
-import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
-
-const server = new McpServer({
+const featureConfig = new FeatureConfig(cliArgs.featureLevel);
+const server = new Server({
   name: 'coaiajs-mcp',
-  version: pkg.version
+  version: '0.1.1',
+}, {
+  capabilities: {
+    tools: {},
+    resources: {},
+    prompts: {},
+  },
 });
 
-// Register tools based on mode
-const mode = (process.env.COAIAJS_MCP_MODE ?? 'STANDARD') as McpMode;
-registerNarrativeTools(server);       // always
-if (mode !== 'MINIMAL') {
-  registerPdeTools(server);
-  registerPlanningTools(server);
-  registerRedisTools(server);
-}
-if (mode === 'FULL') {
-  registerLangfuseTools(server);
-  registerPipelineTools(server);
-  registerAudioTools(server);
-}
+server.setRequestHandler(ListToolsRequestSchema, async () => ({
+  tools: allTools.filter((tool) => featureConfig.isToolEnabled(tool.name)),
+}));
+
+server.setRequestHandler(CallToolRequestSchema, async (request) => {
+  return handleToolCall(request.params.name, request.params.arguments ?? {});
+});
 
 const transport = new StdioServerTransport();
 await server.connect(transport);
@@ -82,47 +80,44 @@ Narrative Beats: `create_narrative_beat`, `telescope_narrative_beat`, `list_narr
 
 MMOT: `perform_mmot_evaluation`, `init_llm_guidance`
 
-### PDE Engine (12 tools — STANDARD+)
+### PDE Engine (10 tools — STANDARD+)
 
-`pde_decompose`, `pde_parse_response`, `pde_get`, `pde_list`, `pde_export_markdown`, `pde_to_stc`, `pde_preview_stc`, `pde_create_session`, `pde_get_session`, `pde_list_sessions`, `pde_session_status`, `pde_bridge_plan`
+`import_pde_decomposition`, `create_stc_from_pde`, `list_pde_decompositions`, `get_session`, `list_sessions`, `pde_update_action_progress`, `pde_mark_action_complete`, `pde_add_action_step`, `pde_update_current_reality`, `complete_session`
 
 ### Planning Engine (6 tools — STANDARD+)
 
-`plan_parse`, `plan_to_stc`, `plan_from_stc`, `plan_sync`, `plan_diff`, `plan_bridge_pde`
+`parse_plan_structural`, `plan_to_stc`, `sync_plan_to_chart`, `sync_chart_to_plan`, `create_plan_trace`, `pde_to_plan`
 
-### Redis (5 tools — STANDARD+)
+### Redis
 
-`redis_tash`, `redis_fetch`, `redis_del`, `redis_keys`, `redis_exists`
+`coaia_tash`, `coaia_fetch`
 
-### Langfuse (8 tools — FULL only)
+### Langfuse
 
-`langfuse_list_traces`, `langfuse_get_trace`, `langfuse_list_prompts`, `langfuse_get_prompt`, `langfuse_list_datasets`, `langfuse_list_scores`, `langfuse_create_score`, `langfuse_list_score_configs`
+`coaia_fuse_trace_create`, `coaia_fuse_add_observation`, `coaia_fuse_trace_patch_output`, `coaia_fuse_trace_get`, `coaia_fuse_trace_view`, `coaia_fuse_observation_get`, `coaia_fuse_traces_list`, `coaia_fuse_traces_session_view`, comments, prompts, datasets, score configs, score application, and media tools.
 
-### Pipeline (3 tools — FULL only)
+### Resources
 
-`pipeline_list`, `pipeline_render`, `pipeline_execute`
+`coaia://templates/`, `coaia://templates/{name}`, `coaia://templates/{name}/variables`
 
-### Audio (2 tools — FULL only)
+### Prompts
 
-`audio_transcribe`, `audio_synthesize`
+`mia_miette_duo`, `create_observability_pipeline`, `analyze_audio_workflow`
 
 ## Tool Registration Pattern
 
-Each module provides a `registerXxxTools(server: McpServer)` function:
+The current server uses plain tool definition arrays plus dispatch handlers:
 
 ```typescript
-// mcp/tools/redis.ts
-export function registerRedisTools(server: McpServer) {
-  server.tool('redis_tash', 'Store a value in Redis with optional TTL', {
-    key: z.string().describe('Redis key'),
-    value: z.string().describe('Value to store'),
-    ttl: z.number().optional().describe('TTL in seconds'),
-  }, async ({ key, value, ttl }) => {
-    await tash(key, value, ttl);
-    return { content: [{ type: 'text', text: `Stored ${key}` }] };
-  });
-  // ... more tools
-}
+const allToolDefs = [
+  ...getCoaiapyToolDefinitions(featureConfig),
+  ...getNarrativeToolDefinitions(featureConfig),
+  ...getPdeToolDefinitions(featureConfig),
+  ...getPlanningToolDefinitions(featureConfig),
+];
+
+server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: allToolDefs }));
+server.setRequestHandler(CallToolRequestSchema, async (request) => routeToolCall(request.params));
 ```
 
 ## Transport
@@ -132,9 +127,9 @@ stdio transport only (standard MCP pattern). Server reads JSON-RPC from stdin, w
 ## Quality Criteria
 
 - ✅ Single MCP config entry replaces three separate servers
-- ✅ All 44 parent tools produce identical results to their parent implementations
+- ✅ Implemented parent tools route to TypeScript handlers instead of placeholders
 - ✅ Feature gating reduces tool count without breaking functionality
-- ✅ `COAIAJS_MCP_MODE=MINIMAL` loads ≤25 tools
-- ✅ Server starts in <500ms for MINIMAL mode
+- ✅ `COAIAJS_FEATURES` and `--features` select feature level
+- ✅ Server starts from installed tarball with `npx coaiajs-mcp`
 - ✅ Tool descriptions are self-documenting (LLMs understand usage without external docs)
 - ✅ Error responses include actionable information

package/rispecs/10-audio-module.spec.md

@@ -9,11 +9,11 @@ An audio module providing speech-to-text transcription via OpenAI Whisper and te
 ## Structural Tension
 
 **Current Reality:**
-- `src/audio.ts` is fully implemented (76 lines) with `synthesize()` function using AWS Polly
-- `src/llm.ts` includes `transcribeAudio()` using OpenAI Whisper
+- [`src/audio.ts`](../src/audio.ts) implements `synthesize()` with AWS Polly and lazy client initialization.
+- [`src/llm.ts`](../src/llm.ts) implements `transcribeAudio()` with OpenAI Whisper and exposes `llm()`, `generateImage()`, and `abstractProcess()`.
 - Both use lazy client initialization
 - coaiapy's `syntation.py` provides the same functionality via boto3 and openai Python packages
-- No MCP tools exist for audio in any parent project
+- Remaining gap: audio is available through the library and CLI transcription path, but standalone MCP audio tools are not wired yet.
 
 **Desired Outcome:**
 Audio module consolidating transcription and synthesis in one place:

package/rispecs/KINSHIP.md

@@ -4,7 +4,7 @@
 
 **Name:** coaiajs rispecs
 **Role:** RISE-based structural specifications for the coaiajs platform
-**Status:** Genesis (2026-03-11)
+**Status:** Maintained implementation-linked specs (updated 2026-05-07)
 
 ## Lineage
 
@@ -34,10 +34,10 @@ These specifications synthesize the creative intent, structural patterns, and be
 ## Accountabilities
 
 1. **Completeness** — These specs are sufficient for another LLM to re-implement the entire coaiajs platform from scratch
-2. **Accuracy** — Current reality assessments are honest and factual, reflecting actual implementation state as of 2026-03-11
+2. **Accuracy** — Current reality assessments are honest and factual, reflecting actual implementation state as of 2026-05-07
 3. **Creative orientation** — All desired outcomes use creation language, not problem-solving language
 4. **Backward compatibility** — Specs preserve behavioral compatibility with parent projects
-5. **Independence** — Specs are codebase-agnostic; they describe behavior, not implementation
+5. **Traceability** — Specs link to the code files that carry current behavior
 
 ## Consumers
 
@@ -53,4 +53,4 @@ These specifications synthesize the creative intent, structural patterns, and be
 
 **Desired Outcome:** These rispecs serve as the single source of truth for what coaiajs should become — complete enough for autonomous re-implementation, accurate enough for MMOT self-evaluation.
 
-**Current Reality:** 13 spec files covering the full platform. Parent project rispecs remain authoritative for deep behavioral details (especially coaia-narrative's 15 specs). These specs provide the consolidation vision and module boundaries.
+**Current Reality:** 13 spec files covering the full platform. Parent project rispecs remain authoritative for deep behavioral details (especially coaia-narrative's 15 specs). These specs now include implementation links for `src/`, `mcp/`, and package metadata so evaluators can trace desired behavior to current code.

package/rispecs/README.md

@@ -7,6 +7,15 @@
 
 These rispecs define the structural specifications for **coaiajs** — the unified TypeScript platform that consolidates `coaiapy`, `coaia-narrative`, `coaia-pde`, and `coaia-planning` into a single modern Node.js package.
 
+## RISE Framework Source
+
+These specs use the RISE Framework at `/src/llms/llms-rise-framework.txt` as the governing method:
+
+- **Reverse-engineer** parent package behavior from `coaiapy`, `coaia-narrative`, `coaia-pde`, and `coaia-planning`.
+- **Intent-extract** the creative outcome each module makes possible.
+- **Specify** the structural tension between current implementation and desired package behavior.
+- **Export** enough code-linked detail for an evaluator or implementation agent to act without rediscovery.
+
 ## Specification Index
 
 ### Platform
@@ -73,6 +82,22 @@ These rispecs define the structural specifications for **coaiajs** — the unifi
 └─────────────────────────────────────────────────────┘
 ```
 
+## Implementation Map
+
+| Spec | Primary implementation |
+|------|------------------------|
+| 00 platform | [`package.json`](../package.json), [`src/index.ts`](../src/index.ts) |
+| 01 config | [`src/config.ts`](../src/config.ts), [`src/types.ts`](../src/types.ts) |
+| 02 redis | [`src/redis.ts`](../src/redis.ts), [`mcp/tools/coaiapy-tools.ts`](../mcp/tools/coaiapy-tools.ts) |
+| 03 langfuse | [`src/langfuse/`](../src/langfuse/), [`mcp/server.ts`](../mcp/server.ts) |
+| 04 narrative | [`src/narrative/`](../src/narrative/), [`src/narrative/index.ts`](../src/narrative/index.ts) |
+| 05 PDE | [`src/pde/`](../src/pde/), [`src/pde/mcp-tools.ts`](../src/pde/mcp-tools.ts) |
+| 06 planning | [`src/planning/`](../src/planning/), [`src/planning/mcp-tools.ts`](../src/planning/mcp-tools.ts) |
+| 07 pipeline | [`src/pipeline/`](../src/pipeline/), [`mcp/resources.ts`](../mcp/resources.ts) |
+| 08 CLI | [`src/cli.ts`](../src/cli.ts), [`package.json`](../package.json) `bin.coaia` |
+| 09 MCP | [`mcp/server.ts`](../mcp/server.ts), [`mcp/config.ts`](../mcp/config.ts), [`mcp/prompts.ts`](../mcp/prompts.ts) |
+| 10 audio | [`src/audio.ts`](../src/audio.ts), [`src/llm.ts`](../src/llm.ts) |
+
 ## Conventions
 
 - **Creative orientation**: Desired Outcome describes what IS CREATED, not what is fixed
@@ -80,7 +105,7 @@ These rispecs define the structural specifications for **coaiajs** — the unifi
 - **Structural tension**: The gap between current and desired that drives advancement
 - **Variable detail**: Broad for obvious patterns, precise for critical behavior
 - **Naming**: `NN-kebab-case.spec.md`
-- **Codebase-agnostic**: Specs describe behavior, not implementation details
+- **Code-linked**: Specs describe behavior and link to the implementation files that currently carry it
 
 ## Relationship to Parent Rispecs
 
@@ -91,10 +116,11 @@ These rispecs define the structural specifications for **coaiajs** — the unifi
 | coaia-planning | — | No rispecs yet — spec 06 here is the first |
 | coaiapy | — | No rispecs — specs 01-03, 07, 10 here are the first |
 
-## RISE Framework Reference
+## Maintenance Protocol
+
+When code changes, update the relevant spec in the same pass:
 
-These specs follow the RISE Framework (`/src/llms/llms-rise-framework.txt`):
-- **R**everse-engineer: Extract creative intent from parent projects
-- **I**ntent-extract: Identify what each module enables users to create
-- **S**pecify: Define structural tension between current and desired
-- **E**xport: Produce specs sufficient for autonomous re-implementation
+1. Adjust **Current Reality** to match implemented files and verified behavior.
+2. Add or revise implementation links in the local spec and this README map.
+3. Keep **Desired Outcome** forward-looking only where a gap remains.
+4. Re-run `npm run build`, `npm test`, and `npm pack --dry-run` before handing off.

package/articles/academic/creative-orientation-vs-problem-solving.md

@@ -1,177 +0,0 @@
-# Creative Orientation vs. Problem-Solving in AI Agent Design
-
-> Academic brief for the CoAiA.js project — Why Robert Fritz's distinction between creative orientation (advancing toward desired outcomes) and problem-solving orientation (retreating from problems) is foundational for AI agent architecture.
-
-## Abstract
-
-AI agent architectures predominantly operate from what Robert Fritz terms "problem-solving orientation"—systems designed to detect undesirable states and take corrective action to eliminate them. This paper argues that problem-solving orientation produces structurally oscillatory agents: as problems diminish, the energy driving correction also diminishes, leading to cyclical regression. We present Fritz's alternative—creative orientation, where agents are structured to advance toward clearly articulated desired outcomes—and analyze its implications for AI agent design. Creative orientation produces advancing rather than oscillating dynamics by maintaining structural tension between a constant vision and an evolving reality. We examine how this distinction manifests in agent goal management, reward design, evaluation criteria, and long-term behavioral patterns. The analysis provides the theoretical foundation for CoAiA.js's creative-orientation-first architecture.
-
-## Introduction
-
-Consider two agents given the same objective—"improve code quality in this repository."
-
-**Agent A (Problem-Solving)** scans for problems: lint errors, security vulnerabilities, code smells, missing tests. It fixes each issue found. As the issue count drops from 200 to 20, its rate of productive activity slows—there are fewer problems to find. When a team member introduces new issues, Agent A reactivates. It oscillates between "fixing problems" and "waiting for problems."
-
-**Agent B (Creative Orientation)** begins by articulating a desired outcome: "A codebase with consistent architecture patterns, comprehensive test coverage, clear documentation, and proactive error handling." It then honestly assesses current reality. The structural tension between vision and reality drives sustained action—not just fixing what's wrong, but creating what's desired. When the issue count drops from 200 to 20, Agent B's energy does not diminish: the vision includes positive qualities (clarity, consistency, comprehensiveness) that have not yet been fully realized [1].
-
-This distinction—between agents that react to problems and agents that create toward visions—is the central contribution of Robert Fritz's work to AI agent design.
-
-## Background
-
-### Fritz's Structural Dynamics
-
-Robert Fritz, in *The Path of Least Resistance* (1989) and subsequent works, identified two fundamental orientations toward action:
-
-**Problem-Solving Orientation (Reactive-Responsive):**
-- Motivation comes from what you don't want
-- Energy is generated by the presence of a problem
-- As the problem diminishes, energy for action also diminishes
-- This creates oscillating structures: action → problem reduced → motivation reduced → problem returns → action resumes
-- The system perpetually returns to a baseline state
-
-**Creative Orientation (Generative):**
-- Motivation comes from what you want to create
-- Energy is generated by the vision of a desired outcome
-- As current reality advances toward the vision, new aspects of the vision become relevant
-- This creates advancing structures: action → reality improves → new creative possibilities emerge → further action
-- The system progressively builds upon prior achievements [2][3]
-
-Fritz demonstrated this pattern across domains: personal development, organizational leadership, artistic creation, and strategic planning. The key insight is that these are **structural** properties—not motivational or psychological ones. The underlying structure of the goal system determines whether it oscillates or advances, regardless of the effort or intention of the agents operating within it.
-
-### Problem-Solving in Traditional AI
-
-The dominant paradigms in AI agent design are structurally problem-solving:
-
-**Reinforcement Learning:** An agent receives negative reward (penalty) for undesirable states and learns to avoid them. The reward signal is fundamentally reactive—it measures distance from problems, not proximity to creative vision. Reward shaping can approximate creative orientation, but the underlying structure remains problem-reactive.
-
-**Loss Minimization:** Neural networks are trained to minimize a loss function—the gap between predicted and target outputs. This is structurally identical to problem-solving: the "problem" (high loss) drives action, and as loss approaches zero, gradient magnitude diminishes.
-
-**Error Correction:** Agent architectures that detect errors and trigger correction routines (retry logic, fallback strategies, error handlers) are purely reactive—they activate only when problems occur.
-
-**Goal Satisfaction:** STRIPS-style planning treats goals as conditions to satisfy. Once satisfied, no further action occurs—there is no mechanism for ongoing creative advancement [4].
-
-## Analysis
-
-### Structural Properties of Oscillation
-
-Oscillation in problem-solving systems follows a predictable pattern:
-
-```
-Problem Detected → High Energy → Action → Problem Reduced → 
-Low Energy → Inaction → Problem Returns → High Energy → Action...
-```
-
-In AI agent terms:
-1. Agent detects high error rate in generated code
-2. Agent focuses energy on fixing errors
-3. Error rate drops to acceptable level
-4. Agent shifts attention away (to other tasks or enters idle state)
-5. Without ongoing attention, new errors accumulate or latent issues surface
-6. Agent re-detects high error rate
-7. Cycle repeats
-
-The oscillation is not a failure of the agent's capabilities—it is a structural property of the problem-solving architecture. The more capable the agent, the faster it oscillates, but oscillation remains.
-
-### Structural Properties of Advancement
-
-Creative orientation produces a different dynamic:
-
-```
-Vision Articulated → Structural Tension → Action → Reality Advances → 
-New Aspects of Vision Become Relevant → Renewed Tension → Further Action...
-```
-
-In AI agent terms:
-1. Agent articulates desired outcome: "A robust, well-documented API with comprehensive error handling, 95% test coverage, and clear usage examples"
-2. Agent assesses current reality: "API has 12 endpoints, 4 undocumented, 62% test coverage, no error handling pattern"
-3. Structural tension drives action toward the vision
-4. After improving documentation and adding tests: current reality is now "12 endpoints, all documented, 78% test coverage, basic error handling"
-5. The vision hasn't changed—new aspects (usage examples, comprehensive error handling, 95% coverage) maintain tension
-6. Agent continues advancing rather than oscillating
-
-### Design Implications for AI Agents
-
-#### Goal Representation
-
-Problem-solving agents encode goals as conditions to eliminate:
-```json
-{"goal": "reduce_lint_errors", "target": 0, "current": 47}
-```
-
-Creative-orientation agents encode goals as outcomes to create:
-```json
-{
-  "desiredOutcome": "A codebase with consistent naming conventions, modular architecture, and clear documentation at every public interface",
-  "currentReality": "Mixed naming conventions across 3 historical styles. 40% of modules have circular dependencies. 12 of 45 public functions undocumented."
-}
-```
-
-The creative-orientation representation is richer, more nuanced, and maintains tension even when any single metric improves.
-
-#### Evaluation Criteria
-
-Problem-solving evaluation asks: "Is the problem gone?"
-Creative-orientation evaluation asks: "How does the current state compare to the desired creation?"
-
-The MMOT (Managerial Moment of Truth) cycle is inherently creative-orientation: it evaluates against Elements of Performance derived from the desired outcome, not against a problem checklist.
-
-#### Energy Management
-
-Problem-solving agents need external triggers (error events, threshold violations) to activate. Creative-orientation agents generate their own energy through structural tension—the gap between vision and reality is an intrinsic motivator that doesn't require external events.
-
-This has practical implications for agent scheduling: problem-solving agents need polling loops or event subscriptions; creative-orientation agents can self-schedule based on tension assessment.
-
-#### Long-Term Behavioral Patterns
-
-Over extended operation:
-- **Problem-solving agents** produce a sawtooth pattern: rapid improvement followed by plateau and regression
-- **Creative-orientation agents** produce a staircase pattern: sustained improvement with each step building on the previous one
-
-This is because creative-orientation agents "lock in" progress by updating their current reality assessment. Once a capability is created, it becomes part of the new baseline—the agent doesn't need to re-create it, and can build further upon it.
-
-### The Hybrid Case: Creative Orientation with Problem Awareness
-
-Fritz does not argue that problem-solving is always wrong—sometimes urgent problems require immediate reactive response. The key insight is that problem-solving should be *subordinate to* creative orientation, not the primary driver:
-
-1. **Primary:** Maintain structural tension toward the desired outcome
-2. **Secondary:** Address problems that arise as obstacles to advancement
-3. **Never:** Let problem-solving become the primary orientation
-
-In agent architecture terms: the structural tension chart defines the creative frame; error handlers and correction routines address specific problems within that frame; but the frame itself is creative, not reactive [5].
-
-### Fritz's Insight Applied to Reward Engineering
-
-In reinforcement learning, reward design is notoriously difficult—reward hacking, sparse rewards, and reward misspecification are persistent challenges. Fritz's creative/problem-solving distinction suggests a fundamental issue: most reward functions are problem-solving in structure (penalizing bad states) rather than creative (describing desired creations).
-
-A creative-orientation reward function would:
-- Describe the desired state in positive, compositional terms
-- Maintain its value even as the agent approaches the goal (because the description is rich enough to reveal new aspects)
-- Not diminish as specific problems are resolved
-- Be evaluated holistically (how does reality compare to the full vision?) rather than componentally (which problems remain?)
-
-## Implications for CoAiA.js
-
-Creative orientation is the architectural foundation of CoAiA.js:
-
-1. **All agent sessions begin with vision articulation.** Before any action, agents define what they want to create—not what problems they want to solve.
-2. **Structural tension charts, not task lists.** The primary data structure is the STC (desired outcome + current reality), not a problem backlog.
-3. **MMOT evaluates against creative criteria.** Self-evaluation measures advancement toward the desired outcome, not merely problem elimination.
-4. **Reality assessment is continuous and honest.** Agents regularly update their current reality assessment, maintaining the tension that drives creative advancement.
-5. **Problem-solving is subordinate.** Error handlers and correction routines operate within the creative frame, addressing obstacles to advancement without becoming the primary orientation.
-
-## Conclusion
-
-The distinction between creative orientation and problem-solving is not philosophical decoration—it is a structural property with measurable behavioral consequences. Agents designed around problem-solving oscillate; agents designed around creative orientation advance. CoAiA.js implements this insight at the architectural level, producing agents that create toward visions rather than merely react to problems. The result is sustained forward momentum, resistance to oscillation, and the capacity for genuine creative advancement in autonomous AI systems.
-
-## References
-
-1. Fritz, R. (1989). *The Path of Least Resistance: Learning to Become the Creative Force in Your Own Life*. Fawcett Columbine.
-2. Fritz, R. "The Creative Process." Robert Fritz Inc. https://www.robertfritz.com/wp/the-creative-process/
-3. Fritz, R. "Thinking in Structures." Robert Fritz Inc. https://www.robertfritz.com/wp/thinking-in-structures/
-4. Systems Field Book. "Creative Orientation." https://www.systemsfieldbook.org/creative-orientation
-5. jgwill/llms-txt. "Creative Orientation Documentation." https://github.com/jgwill/llms-txt/blob/main/docs/creative-orientation.md
-6. Fritz, R. "Writings." Robert Fritz Inc. https://www.robertfritz.com/wp/writings/
-7. Senge, P. (1990). *The Fifth Discipline: The Art & Practice of the Learning Organization*. Doubleday.
-8. Sutton, R. S., & Barto, A. G. (2018). *Reinforcement Learning: An Introduction*. 2nd ed. MIT Press.
-9. Think-2-Thrive. (2024). "Leading with Creative Tension." https://think-2-thrive.com/2024/12/02/blog-55-leading-with-creative-tension/
-10. Springer. (2024). "A TRIZ and Socratic AI-Based Problem-Solving Framework." https://link.springer.com/chapter/10.1007/978-3-032-08847-5_20

package/articles/academic/jsonl-knowledge-graphs-agent-memory.md

@@ -1,142 +0,0 @@
-# JSONL as Lightweight Knowledge Graph Storage for AI Agent Memory
-
-> Academic brief for the CoAiA.js project — Append-only line-delimited JSON as a knowledge graph substrate for agent state, compared with Neo4j, SQLite, and vector databases.
-
-## Abstract
-
-AI agent systems require persistent memory that balances expressiveness, queryability, and operational simplicity. This paper evaluates JSON Lines (JSONL)—an append-only, line-delimited JSON format—as a lightweight knowledge graph storage mechanism for agent memory. We compare JSONL entity-relation storage against established alternatives: Neo4j (property graphs), SQLite (relational), and vector databases (embedding-based retrieval). Our analysis demonstrates that JSONL offers unique advantages for agent memory workloads: zero-dependency deployment, natural append-only semantics aligned with temporal agent state, human-readable audit trails, and sufficient query performance for single-agent session contexts. We formalize the JSONL entity-relation model used in CoAiA.js, analyze its trade-offs against richer graph storage, and identify the operational sweet spot where JSONL-backed knowledge graphs outperform heavier alternatives.
-
-## Introduction
-
-The rise of autonomous AI agents—systems that maintain state across interactions, learn from observations, and pursue multi-step goals—has created an urgent need for agent memory architectures that are both expressive and operationally lightweight. An agent's memory must encode entities (concepts, observations, decisions), relations between entities (causal links, dependencies, temporal orderings), and evolving state (current reality assessments, progress markers) [1].
-
-The knowledge graph community has traditionally addressed such requirements through property graph databases (Neo4j, Memgraph), RDF triple stores (GraphDB, Fuseki), or more recently, vector databases (Pinecone, Weaviate, Chroma). Each brings significant capabilities but also operational overhead: server processes, schema management, connection pooling, and deployment complexity [2][3].
-
-For many agent memory workloads—particularly single-agent sessions, development environments, and edge deployments—this overhead is disproportionate to the actual storage and query requirements. This paper argues that JSONL, combined with in-memory indexing, provides a knowledge graph storage substrate that is adequate for the majority of agent memory use cases while dramatically reducing operational complexity.
-
-## Background
-
-### JSON Lines (JSONL) Format
-
-JSONL (also called newline-delimited JSON or NDJSON) is a text format where each line is a valid JSON object, separated by newline characters. The format has several properties relevant to knowledge graph storage:
-
-1. **Append-only by nature.** New records are written by appending lines; existing records are never modified in place.
-2. **Streamable.** Files can be processed line-by-line without loading the entire dataset into memory.
-3. **Human-readable.** Records are inspectable with standard text tools (cat, grep, head, tail).
-4. **Schema-flexible.** Each line can have a different structure, enabling heterogeneous entity types in a single file.
-5. **Zero-dependency.** No server process, no driver library, no connection management [4].
-
-### Knowledge Graph Storage Models
-
-The landscape of knowledge graph storage can be categorized into four primary models:
-
-- **Property Graphs (Neo4j, Memgraph, KuzuDB):** Nodes and edges with key-value properties; Cypher query language; optimized for traversals and path queries.
-- **RDF Triple Stores (GraphDB, Fuseki, Blazegraph):** Subject-predicate-object triples with URI-based identification; SPARQL query language; strong semantic reasoning.
-- **Relational (SQLite, PostgreSQL):** Tables with foreign key relationships; SQL query language; mature tooling but impedance mismatch with graph structures.
-- **Vector Databases (Pinecone, Weaviate, Chroma):** Embedding-based storage optimized for similarity search; excellent for semantic retrieval but lacking explicit relational structure [5][6].
-
-## Analysis
-
-### The JSONL Entity-Relation Model
-
-In the CoAiA.js implementation, knowledge graphs are stored as typed JSONL records with the following schema:
-
-```jsonl
-{"type":"entity","name":"mcp_server","entityType":"component","observations":["Handles tool dispatch","Uses stdio transport"]}
-{"type":"entity","name":"langfuse_integration","entityType":"service","observations":["Traces LLM generations","MIT licensed"]}
-{"type":"relation","from":"mcp_server","to":"langfuse_integration","relationType":"depends_on"}
-{"type":"observation","entityName":"mcp_server","content":"Added error handling for malformed JSON-RPC requests","timestamp":"2026-03-10T14:00:00Z"}
-```
-
-This model encodes:
-- **Entities** with typed classifications and evolving observation lists
-- **Relations** between entities with typed edges
-- **Temporal observations** that append new knowledge without modifying existing records
-
-The append-only semantics are not a limitation but a feature: they naturally encode the temporal evolution of agent knowledge, enabling queries like "what did the agent know at time T?" without requiring event sourcing or temporal tables.
-
-### Comparative Analysis
-
-#### JSONL vs. Neo4j
-
-Neo4j provides mature graph traversal capabilities, Cypher query language, and ACID transactions. For agent memory workloads, the key trade-offs are:
-
-| Dimension | JSONL | Neo4j |
-|-----------|-------|-------|
-| Deployment | Zero-config file | JVM server process |
-| Query Language | In-memory JavaScript/grep | Cypher |
-| Path Queries | O(n) scan or in-memory index | Optimized native |
-| Write Pattern | Append line | Transaction commit |
-| Concurrent Access | Single-writer safe | Multi-writer ACID |
-| Portability | Copy file | Export/import pipeline |
-| Cost | Zero | Community/Enterprise licensing |
-
-For single-agent sessions with knowledge graphs under 100,000 entities—which covers the vast majority of agent session workloads—JSONL with in-memory indexing provides sub-millisecond query times with zero operational overhead. Neo4j becomes necessary only when path query complexity or multi-user concurrency demands it.
-
-#### JSONL vs. SQLite
-
-SQLite offers a compelling middle ground: embedded, zero-config, yet supporting SQL queries and ACID transactions. However, SQLite's relational model creates impedance mismatch with graph structures:
-
-- Graph traversals require recursive CTEs, which are syntactically complex and performance-limited.
-- Schema changes (new entity types, new relation types) require ALTER TABLE operations.
-- The append-only temporal model must be explicitly implemented rather than arising naturally from the storage format.
-
-JSONL's schema flexibility and natural append semantics make it better suited to the evolving, heterogeneous nature of agent knowledge.
-
-#### JSONL vs. Vector Databases
-
-Vector databases excel at semantic similarity search—finding entities "similar to" a query embedding. However, they fundamentally lack explicit relational structure:
-
-- Relations between entities must be encoded implicitly through embedding proximity or metadata filters.
-- Temporal ordering is not native; timestamps must be stored as metadata and filtered externally.
-- The retrieval model (top-k nearest neighbors) does not naturally support the "desired outcome vs. current reality" structural tension queries that drive agent behavior.
-
-JSONL and vector databases are complementary rather than competing: JSONL provides the relational graph structure while vector embeddings (stored as entity properties) enable semantic retrieval within that structure.
-
-### Performance Characteristics
-
-For typical agent memory workloads (1,000–50,000 records per session), JSONL with in-memory indexing provides:
-
-- **Write latency:** <1ms (file append)
-- **Entity lookup:** <1ms (hash map)
-- **Relation traversal:** <5ms (adjacency list)
-- **Full scan:** <50ms for 50K records
-- **File size:** ~5MB for 50K records (highly compressible)
-
-These characteristics are more than adequate for interactive agent sessions where the bottleneck is LLM inference (typically 1-30 seconds per generation), not memory access.
-
-### Limitations and Mitigations
-
-JSONL-backed knowledge graphs have real limitations:
-
-1. **No native query language.** Queries must be implemented in application code or via streaming filters. *Mitigation:* CoAiA.js provides a query API that loads JSONL into an in-memory graph structure on startup.
-2. **No concurrent write safety.** Multiple writers can corrupt the file. *Mitigation:* Agent sessions are single-writer by design; file locking can be added for multi-agent scenarios.
-3. **Linear scan for complex queries.** Path queries and graph algorithms require full load into memory. *Mitigation:* Agent session graphs are small enough for full in-memory operation.
-4. **No built-in indexing.** Unlike databases, JSONL files have no persistent index structures. *Mitigation:* In-memory indexes are rebuilt on load, which takes <100ms for typical session sizes.
-
-## Implications for CoAiA.js
-
-CoAiA.js adopts JSONL as its primary knowledge graph storage format based on the analysis above. The implications are:
-
-1. **Zero-dependency memory.** Agent sessions can persist state without any external service, enabling deployment in constrained environments (edge, CI/CD, development workstations).
-2. **Full audit trail.** Every observation, entity, and relation ever created is preserved in temporal order, enabling complete reconstruction of agent reasoning.
-3. **Git-friendly storage.** JSONL files produce clean diffs, enabling version control of agent memory—a critical capability for reproducible agent behavior.
-4. **Composable with richer stores.** When workloads demand it, JSONL can be batch-imported into Neo4j or vector databases for advanced analysis without changing the write-time format.
-5. **Human-debuggable.** Developers can inspect agent memory with `cat`, `grep`, and `jq`—no specialized tooling required.
-
-## Conclusion
-
-JSONL-backed knowledge graphs represent an underappreciated sweet spot in the agent memory design space. For single-agent session workloads—the dominant pattern in current AI agent architectures—JSONL provides the expressiveness of a knowledge graph with the operational simplicity of a log file. By embracing append-only semantics as a feature rather than a limitation, and by leveraging in-memory indexing for query performance, CoAiA.js demonstrates that lightweight storage can be adequate without being simplistic.
-
-## References
-
-1. Graphlit Blog. (2024). "Survey of AI Agent Memory Frameworks." https://www.graphlit.com/blog/survey-of-ai-agent-memory-frameworks
-2. Neo4j Labs. (2025). "Agent Memory: A Graph-Native Memory System for AI." https://github.com/neo4j-labs/agent-memory
-3. Vela Partners. (2025). "KuzuDB for Production AI Agents." https://www.vela.partners/blog/kuzudb-ai-agent-memory-graph-database
-4. NDJSON Specification. https://github.com/ndjson/ndjson-spec
-5. Enterprise Knowledge. (2024). "Cutting Through the Noise: An Introduction to RDF & LPG Graphs." https://enterprise-knowledge.com/cutting-through-the-noise-an-introduction-to-rdf-lpg-graphs/
-6. Neo4j. (2024). "RDF Triple Stores vs. Property Graphs." https://neo4j.com/blog/knowledge-graph/rdf-vs-property-graphs-knowledge-graphs/
-7. FuturesSmart AI. (2024). "Building AI Knowledge Graph Using Graphiti and Neo4j." https://blog.futuresmart.ai/building-ai-knowledge-graph-using-graphiti-and-neo4j
-8. Memento MCP. (2024). "A Knowledge Graph Memory System for LLMs." https://github.com/gannonh/memento-mcp
-9. Ontotext. (2024). "Choosing A Graph Data Model to Best Serve Your Use Case." https://www.ontotext.com/blog/choosing-a-graph-data-model-to-best-serve-your-use-case/
-10. SAP Community. (2025). "Choosing Between Knowledge Graphs and Property Graphs in SAP HANA Cloud." https://community.sap.com/t5/technology-blog-posts-by-sap/choosing-between-knowledge-graphs-and-property-graphs-in-sap-hana-cloud-and/ba-p/14074575