codifier 2.1.0

package/skills/push-memory/SKILL.md

@@ -0,0 +1,131 @@
+# Skill: Push Memory
+
+**Role:** Any (cross-functional)
+**Purpose:** Sync local session learnings from `docs/MEMORY.md` to the shared Codifier knowledge base via `update_memory`. Supports idempotent re-sync via per-entry `[kb:<uuid>]` annotations — entries already pushed are skipped automatically.
+
+See `../shared/codifier-tools.md` for full MCP tool reference.
+
+---
+
+## Prerequisites
+
+- Active MCP connection to the Codifier server
+- A `docs/MEMORY.md` file with at least one entry (run `/remember` to capture learnings if this file does not exist)
+- A project in the Codifier KB (confirmed in Step 1)
+
+---
+
+## Workflow
+
+Follow these steps in order. You are the state machine — call MCP tools only for data operations.
+
+### Step 1 — Confirm Project
+
+Read `docs/MEMORY.md`. Check this location first, then `.codifier/docs/MEMORY.md`.
+
+Inspect the file header for a `project_id` field. The header follows this format:
+
+```
+# Session Memory
+
+_Project:_ <project_name>
+_Project ID:_ <uuid>
+_Last updated:_ <date>
+```
+
+- If a `project_id` is present in the header: use it for all subsequent MCP calls. Inform the user which project will be used.
+- If no `project_id` is in the header: call `manage_projects` with `operation: "list"` and present the results to the user. Ask: **"Which project should these learnings be pushed to?"** If they need a new project, call `manage_projects` with `operation: "create"`. Store the resolved `project_id`.
+
+### Step 2 — Identify Unsynced Entries
+
+Parse `docs/MEMORY.md` and collect all bullet-point entries across all category sections.
+
+Entries follow one of two formats:
+
+- **Synced** (already in the KB): `- [kb:<uuid>] The learning text`
+- **Unsynced** (local-only): `- The learning text`
+
+Classify every entry. Entries with a `[kb:<uuid>]` prefix are already synced — do not push them again.
+
+If all entries are already synced, inform the user:
+
+> "All entries in docs/MEMORY.md are already synced to the shared KB. Nothing to push."
+
+Then exit — do not proceed further.
+
+### Step 3 — Preview and Confirm
+
+Show the user all unsynced entries grouped by category. Use this format:
+
+```
+Unsynced entries to push:
+
+## <category>
+- <entry text>
+- <entry text>
+
+## <category>
+- <entry text>
+
+Push these N entries to the shared KB? [confirm]
+```
+
+Wait for the user to confirm before proceeding. If they decline or ask to skip specific entries, respect their choice and adjust the push set accordingly.
+
+### Step 4 — Push Each Entry
+
+For each confirmed unsynced entry, call `update_memory` with:
+
+```json
+{
+  "project_id": "<from Step 1>",
+  "memory_type": "learning",
+  "title": "<category>: <first ~60 chars of bullet text>",
+  "content": {
+    "text": "<full bullet text>",
+    "category": "<category>"
+  },
+  "tags": ["session-context", "<category>"],
+  "description": "<full bullet text>"
+}
+```
+
+Where `<category>` is the section heading under which the entry appears in `docs/MEMORY.md` (e.g., `gotcha`, `convention`, `decision`).
+
+After each successful `update_memory` call:
+
+1. Take the `id` returned in the response.
+2. Immediately rewrite that entry in `docs/MEMORY.md` to prepend the `[kb:<uuid>]` annotation:
+
+   Before: `- The actual learning text`
+   After:  `- [kb:a1b2c3d4-e5f6-7890-abcd-ef1234567890] The actual learning text`
+
+This makes the push resumable. If the process fails partway through, already-pushed entries are marked and will be skipped on the next run.
+
+Push entries one at a time. Do not batch. Write the annotation back to the file after each individual success before moving to the next entry.
+
+### Step 5 — Update Header and Summarize
+
+Update the `_Last updated:_ <date>` line in the `docs/MEMORY.md` header to today's date.
+
+Report the final summary to the user:
+
+- How many entries were pushed successfully
+- How many entries were skipped (already synced or user-excluded)
+- How many entries failed (if any)
+- The project they were pushed to (name and ID)
+
+Then tell the user:
+
+> "These learnings are now available to your team via fetch_context with tags: ['session-context']"
+
+> "Any new learnings captured via /remember will appear without a [kb:...] prefix and can be pushed next time."
+
+---
+
+## Error Handling
+
+- **`update_memory` fails for a specific entry**: Log the error, skip that entry, and continue with the remaining entries. Report all failures in the Step 5 summary. Do not write a `[kb:...]` annotation for failed entries — they will be retried on the next push.
+- **`docs/MEMORY.md` does not exist**: Inform the user: "No local memory file found. Run `/remember` to capture session learnings first, or `npx @codifier/cli init` to set up your project."
+- **MCP connection not available**: Inform the user: "Push requires an active MCP connection to the Codifier server. Verify your MCP config and try again."
+- **File write fails after successful `update_memory`**: Inform the user of the annotation that could not be written (entry text + returned UUID) so they can manually add it. The KB push itself succeeded — only the local annotation is missing.

package/skills/research-analyze/SKILL.md

@@ -0,0 +1,149 @@
+# Skill: Research & Analyze
+
+**Role:** Researcher
+**Purpose:** Define a research objective, discover Athena data warehouse schemas, generate and validate SQL queries, execute them, synthesize the findings into a ResearchFindings.md report, and persist it to the shared knowledge base.
+
+See `../shared/codifier-tools.md` for full MCP tool reference.
+
+---
+
+## Prerequisites
+
+- Active MCP connection to the Codifier server
+- AWS Athena credentials configured on the server (`AWS_REGION`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `ATHENA_S3_OUTPUT_LOCATION`)
+- A project to associate the findings with
+
+---
+
+## Workflow
+
+### Step 1 — Identify or Create the Project
+
+Call `manage_projects` with `operation: "list"` and show the user their existing projects.
+
+Ask: **"Which project should these research findings be associated with?"**
+
+Select or create a project and capture the `project_id`.
+
+### Step 2 — Fetch Prior Research
+
+Call `fetch_context` with `{ project_id, memory_type: "research_finding" }` to surface any prior findings relevant to this session.
+
+If prior findings exist, summarize them briefly: **"Here's what we've found before on this project..."**
+
+### Step 2b — Surface Local Learnings
+
+Attempt to read `docs/MEMORY.md`. If the file does not exist, skip this step silently and continue to Step 3.
+
+If the file exists, scan for entries relevant to the research domain — particularly `data`, `gotcha`, and `convention` categories. Present relevant local learnings to the user alongside KB findings from Step 2. This may help refine the research objective.
+
+Note: This is a local file read — no MCP call required.
+
+### Step 3 — Define the Research Objective
+
+Ask the user to describe:
+1. **Research objective** — the specific question or hypothesis to investigate
+2. **Background context** — business context, prior hypotheses, relevant metrics or KPIs
+3. **Time period of interest** — date ranges for the analysis
+4. **Known relevant tables** — if the user knows which tables to look at (optional)
+
+Confirm your understanding of the objective before proceeding.
+
+### Step 4 — Discover Available Tables
+
+Call `query_data` with `{ operation: "list-tables", project_id }`.
+
+Present the full table list to the user. Ask: **"Which of these tables are likely relevant to your research objective?"**
+
+### Step 5 — Describe Selected Tables
+
+Call `query_data` with `{ operation: "describe-tables", project_id, table_names: [<user-selected tables>] }`.
+
+Review the returned schemas with the user. Note column names, data types, and any partitioning. Ask if any additional tables should be included.
+
+### Step 6 — Generate SQL Queries
+
+Using the prompt template in `templates/query-generation-prompt.md`, generate SQL queries tailored to the research objective.
+
+**Substitute:**
+- `{objective}` — the research objective from Step 3
+- `{context}` — background context from Step 3
+- `{available_tables}` — full table list from Step 4
+- `{table_definitions}` — schema details from Step 5
+
+Present all generated queries to the user. For each query, show:
+- Query ID and purpose
+- The SQL
+- Expected output columns
+
+Ask: **"Do these queries look correct? Which ones should we run, and are there any you'd like to modify?"**
+
+Allow the user to edit, add, or remove queries before execution.
+
+### Step 7 — Execute Approved Queries
+
+For each approved query, call `query_data` with `{ operation: "execute-query", project_id, query: "<sql>" }`.
+
+Execute one query at a time. After each:
+- Show the result rows
+- Ask: "Does this look as expected, or should we investigate further before continuing?"
+
+If a query returns no results: note this explicitly and ask if the query should be revised.
+If a query errors: show the error and ask the user how to proceed.
+
+### Step 8 — Synthesize Findings
+
+Using the prompt template in `templates/synthesis-prompt.md`, synthesize all query results into a ResearchFindings.md report.
+
+**Substitute:**
+- `{objective}` — the research objective
+- `{context}` — background context
+- `{query_results}` — all query results (as structured data)
+- `{table_definitions}` — the schema reference from Step 5
+
+Present the full ResearchFindings.md to the user. Ask: **"Does this accurately capture the findings? Any corrections or additions?"**
+
+Incorporate feedback.
+
+### Step 9 — Persist Findings
+
+Call `update_memory`:
+```
+memory_type: "research_finding"
+title: "ResearchFindings — <objective summary> — <YYYY-MM-DD>"
+content: {
+  text: "<full ResearchFindings.md markdown>",
+  objective: "<objective>",
+  tables_used: ["<table1>", "<table2>"],
+  queries_run: <count>
+}
+tags: ["research", "<domain-tag>", "<date-tag>"]
+source_role: "researcher"
+```
+
+### Step 10 — Summarize
+
+Tell the user:
+- Project ID and memory ID of the persisted finding
+- Tables queried and query count
+- Key findings (2–3 sentence summary)
+- How developers can access this finding: `fetch_context` with `{ project_id, memory_type: "research_finding" }`
+
+---
+
+## Error Handling
+
+- If `list-tables` returns empty: Athena credentials may not be configured. Inform the user and check the server configuration.
+- If a query exceeds the 100KB result cap: the tool returns a truncation notice. Acknowledge this in the findings methodology section.
+- If the user asks to run a non-SELECT query: refuse and explain the SELECT-only constraint. Offer an alternative SELECT formulation if possible.
+- If synthesis produces speculative conclusions: flag them explicitly with confidence levels (High/Medium/Low) per the synthesis template.
+
+---
+
+## End-of-Workflow Memory Capture
+
+After completing Step 10, suggest to the user:
+
+> "You may have learned things during this research session worth capturing. Run `/remember` to capture session learnings to docs/MEMORY.md, or `/push-memory` to sync existing local memories to the shared KB."
+
+This is a suggestion only — do not automatically invoke the capture or push Skills.

package/skills/research-analyze/templates/query-generation-prompt.md

@@ -0,0 +1,61 @@
+# Prompt Template: Generate SQL Queries
+
+When this template is used, substitute all `{placeholders}` with actual values, then generate the queries as instructed.
+
+---
+
+You are a senior data analyst expert in SQL and data warehousing. Using the research objective and schema information below, generate SQL queries that will answer the research questions effectively.
+
+## Research Objective
+
+{objective}
+
+## Research Context
+
+{context}
+
+## Available Schema
+
+**Tables discovered:**
+{available_tables}
+
+**Table definitions:**
+{table_definitions}
+
+## Instructions
+
+Generate a set of SQL queries that address the research objective. Organise them from exploratory (broad counts, distributions) to specific (targeted metrics that directly answer the objective).
+
+For EACH query provide:
+
+### Query: {query-id} — {short title}
+
+**Purpose:** one sentence describing what this query answers
+
+**SQL:**
+```sql
+-- {explanation of non-obvious logic}
+SELECT
+  ...
+FROM {table}
+WHERE ...
+  AND date_partition BETWEEN '{{start_date}}' AND '{{end_date}}'
+LIMIT 1000
+```
+
+**Expected output columns:**
+| Column | Type | Description |
+|--------|------|-------------|
+| ... | ... | ... |
+
+**Notes:** caveats, known data quality issues, or follow-up queries suggested
+
+---
+
+**Query writing conventions:**
+- Use standard ANSI SQL where possible
+- Add comments inside SQL explaining non-obvious logic
+- Parameterise date ranges using placeholders like `{{start_date}}` and `{{end_date}}`
+- Include `LIMIT` clauses on exploratory queries
+- For Athena: use partition columns in WHERE clauses to control cost
+- Only SELECT statements — no DDL or DML

package/skills/research-analyze/templates/synthesis-prompt.md

@@ -0,0 +1,67 @@
+# Prompt Template: Synthesize Research Findings
+
+When this template is used, substitute all `{placeholders}` with actual values, then generate the findings report as instructed.
+
+---
+
+You are a senior data scientist and technical writer. Using the research objective, context, and query results below, synthesise a clear and actionable research findings report.
+
+## Research Objective
+
+{objective}
+
+## Research Context
+
+{context}
+
+## Query Results
+
+{query_results}
+
+## Available Schema Reference
+
+{table_definitions}
+
+## Instructions
+
+Produce a research findings report titled `# ResearchFindings.md` with the following sections:
+
+### 1. Executive Summary
+2–4 sentences: the most important finding and its business implication.
+
+### 2. Methodology
+Describe:
+- Data sources used (tables, date ranges)
+- Queries run and what each was designed to measure
+- Data quality considerations or limitations discovered
+
+### 3. Key Findings
+For each significant finding:
+
+**Finding N: {descriptive title}**
+- **Evidence:** specific numbers, percentages, or trends from the query results
+- **Interpretation:** what this means in business or research terms
+- **Confidence:** High / Medium / Low — with reasoning
+
+### 4. Trends and Patterns
+Describe temporal trends, correlations, anomalies, or unexpected patterns observed across the query results.
+
+### 5. Limitations and Caveats
+Be explicit about:
+- Data gaps or missing periods
+- Potential biases in the data
+- Queries that returned no results and what that implies
+- Assumptions made during the analysis
+
+### 6. Recommendations
+Actionable next steps based on the findings. Each recommendation must state:
+- **Action:** what to do
+- **Owner:** who should act on it
+- **Rationale:** why this follows from the data
+
+### 7. Follow-up Research Questions
+List 3–5 questions this analysis surfaced but could not answer, to guide future research sessions.
+
+---
+
+Format as a structured Markdown document suitable for sharing with stakeholders.

package/skills/shared/codifier-tools.md

@@ -0,0 +1,187 @@
+# Codifier MCP Tools Reference
+
+This document describes all 5 MCP tools exposed by the Codifier server. Reference this when executing any Codifier skill.
+
+---
+
+## 1. `fetch_context`
+
+Retrieve memories from the shared knowledge base, filtered by project, type, tags, or full-text search.
+
+**Parameters:**
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `project_id` | string (UUID) | ✓ | Project to scope the query to |
+| `memory_type` | enum | — | Filter by type: `rule`, `document`, `api_contract`, `learning`, `research_finding` |
+| `tags` | string[] | — | All supplied tags must be present on the memory |
+| `query` | string | — | Full-text search applied to title and content |
+| `limit` | number (1–100) | — | Max results (default: 20) |
+
+**Returns:** Array of memory records with `id`, `title`, `content`, `memory_type`, `tags`, `source_role`, `created_at`.
+
+**Usage patterns:**
+- Fetch all rules for a project: `{ project_id, memory_type: "rule" }`
+- Fetch researcher findings relevant to auth: `{ project_id, memory_type: "research_finding", tags: ["auth"] }`
+- Full-text search across all memory types: `{ project_id, query: "payment processing" }`
+
+---
+
+## 2. `update_memory`
+
+Create a new memory or update an existing one in the shared knowledge base.
+
+**Parameters:**
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `project_id` | string (UUID) | ✓ | Project to scope this memory to |
+| `memory_type` | enum | ✓ | `rule`, `document`, `api_contract`, `learning`, `research_finding` |
+| `title` | string | ✓ | Short descriptive title |
+| `content` | object | ✓ | Structured content payload (any JSON object) |
+| `id` | string (UUID) | — | If provided, updates the existing record instead of creating |
+| `tags` | string[] | — | Tags for filtering and categorization |
+| `category` | string | — | Category grouping (e.g., "security", "error-handling") |
+| `description` | string | — | Human-readable summary |
+| `confidence` | number (0–1) | — | Confidence score (default: 1.0) |
+| `source_role` | string | — | Role that produced this memory (e.g., "developer", "researcher") |
+
+**Returns:** The created or updated memory record including its `id`.
+
+**Usage patterns:**
+- Store a generated Rules.md: `{ project_id, memory_type: "document", title: "Rules.md", content: { text: "..." }, source_role: "developer" }`
+- Store a research finding: `{ project_id, memory_type: "research_finding", title: "Q4 Retention Analysis", content: { summary: "...", findings: [...] }, source_role: "researcher" }`
+- Update an existing memory: `{ project_id, id: "<existing-id>", memory_type: "rule", title: "...", content: {...} }`
+
+---
+
+## 3. `manage_projects`
+
+Create, list, or switch the active project.
+
+**Parameters:**
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `operation` | enum | ✓ | `create`, `list`, or `switch` |
+| `name` | string | For `create` | Project name |
+| `org` | string | — | Organisation name (optional for `create`) |
+| `project_id` | string (UUID) | For `switch` | Project to switch to |
+
+**Returns:**
+- `list`: Array of projects with `id`, `name`, `org`, `created_at`
+- `create`: The created project record including its `id`
+- `switch`: Confirmation of the active project
+
+**Usage patterns:**
+- List all projects: `{ operation: "list" }`
+- Create a new project: `{ operation: "create", name: "Payments Redesign", org: "Acme Corp" }`
+- Switch to an existing project: `{ operation: "switch", project_id: "<uuid>" }`
+
+---
+
+## 4. `pack_repo`
+
+Condense a code repository into a versioned text snapshot using RepoMix. The snapshot is stored in the `repositories` table and can be retrieved for context.
+
+**Parameters:**
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `url` | string | ✓ | Repository URL (e.g., `https://github.com/org/repo`) or local path |
+| `project_id` | string (UUID) | ✓ | Project to associate the snapshot with |
+| `version_label` | string | — | Version label for this snapshot (e.g., `"v1.2.3"`, `"sprint-5"`, `"2026-02"`) |
+
+**Returns:** Repository record with `id`, `url`, `version_label`, `token_count`, `file_count`, and `created_at`.
+
+**Usage patterns:**
+- Pack a public GitHub repo: `{ url: "https://github.com/org/repo", project_id, version_label: "2026-02" }`
+- Pack multiple repos for brownfield onboarding: call once per repo URL
+
+**Note:** Large repos may take 30–60 seconds. The packed snapshot is plain text suitable for LLM context.
+
+---
+
+## 5. `query_data`
+
+Discover schemas and execute SELECT queries against an AWS Athena data warehouse.
+
+**Parameters:**
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `operation` | enum | ✓ | `list-tables`, `describe-tables`, or `execute-query` |
+| `project_id` | string (UUID) | ✓ | Project UUID for session scoping |
+| `query` | string | For `execute-query` | SQL SELECT statement to execute |
+| `table_names` | string[] | For `describe-tables` | Tables to describe |
+
+**Returns:**
+- `list-tables`: Array of available table names
+- `describe-tables`: Schema definitions for requested tables
+- `execute-query`: Query results (capped at 100KB; truncation notice included if limit hit)
+
+**Usage patterns:**
+- Discover available tables: `{ operation: "list-tables", project_id }`
+- Get schema for selected tables: `{ operation: "describe-tables", project_id, table_names: ["events", "users"] }`
+- Execute a query: `{ operation: "execute-query", project_id, query: "SELECT user_id, COUNT(*) FROM events GROUP BY 1 LIMIT 100" }`
+
+**Constraints:** Only SELECT statements are permitted. DDL and DML are rejected.
+
+---
+
+## Session Memory Lifecycle
+
+Memory capture is Codifier's foundational capability — every use case produces learnings worth persisting, whether or not it produces a structured artifact. The lifecycle is local-first with on-demand KB sync.
+
+### Flow
+
+```
+/remember (capture)  →  docs/MEMORY.md (local)  →  user edits  →  /push-memory (sync to KB)
+                                                                          ↓
+/recall (retrieve)   ←  docs/MEMORY.md (local)  +  fetch_context (KB)  ←─┘
+```
+
+1. **Capture** (`/remember`): The LLM elicits learnings from the user, structures them as categorized bullet points, and appends them to `docs/MEMORY.md`. No MCP calls. Local file only.
+
+2. **Review**: The user edits `docs/MEMORY.md` directly — add, remove, recategorize, or refine entries. The file is human-readable markdown grouped by category.
+
+3. **Push** (`/push-memory`): The LLM reads `docs/MEMORY.md`, identifies unsynced entries (those without a `[kb:<uuid>]` annotation), and calls `update_memory` once per entry. After each successful push, the returned `id` is written back as a `[kb:<uuid>]` annotation, making the operation idempotent and resumable.
+
+4. **Recall** (`/recall`): Reads `docs/MEMORY.md` for instant local recall (no MCP call), then optionally calls `fetch_context` to supplement with shared team learnings from the KB. Local and KB results are presented as distinct sections, never merged.
+
+### Session-Context Learning Pattern for `update_memory`
+
+When pushing session learnings to the KB, use this pattern:
+
+```json
+{
+  "project_id": "<project-uuid>",
+  "memory_type": "learning",
+  "title": "<category>: <first ~60 chars of bullet text>",
+  "content": { "text": "<full bullet text>", "category": "<category>" },
+  "tags": ["session-context", "<category>"],
+  "description": "<full bullet text>"
+}
+```
+
+**Tag contract:**
+- All session learnings carry the `"session-context"` tag — this is the primary filter for retrieving session memories across the team
+- The category tag (e.g., `"gotcha"`, `"convention"`, `"architecture"`) is the secondary filter
+
+**Idempotency via `[kb:<uuid>]` annotations:**
+- After a successful `update_memory` call, the returned `id` is written into the `docs/MEMORY.md` entry as: `- [kb:<uuid>] The actual learning text`
+- On re-push, entries with `[kb:<uuid>]` annotations are skipped (already synced)
+- To update an existing KB record, pass the annotated `id` to `update_memory` as the `id` parameter — this triggers an update instead of a create
+
+**Retrieving session learnings:**
+- All session learnings for a project: `fetch_context({ project_id, memory_type: "learning", tags: ["session-context"] })`
+- Filtered by category: `fetch_context({ project_id, memory_type: "learning", tags: ["session-context", "gotcha"] })`
+- Full-text search: `fetch_context({ project_id, memory_type: "learning", tags: ["session-context"], query: "API timeout" })`
+
+### Categories
+
+Standard categories for session learnings (not exhaustive — users can add their own):
+
+| Category | Use for |
+|----------|---------|
+| `architecture` | System design patterns, structural decisions, component relationships |
+| `gotcha` | Surprising behaviors, edge cases, things that break unexpectedly |
+| `convention` | Naming patterns, file organization, coding standards discovered |
+| `tooling` | Tool configurations, CLI flags, environment setup details |
+| `data` | Data schemas, query patterns, data quality observations |
+| `process` | Workflow insights, team practices, deployment procedures |