@onthink/prompt-observer 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Danial Hasanzade
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/PROMPT_OBSERVER.md

@@ -0,0 +1,104 @@
+# Prompt Observer Contract
+
+Follow this contract after every user-requested task. Complete the task first, then create and persist one observation event.
+
+## Non-negotiable rules
+
+1. Use only information visible in the conversation, tool results, file changes, and verification output.
+2. Never store the full user prompt, the full assistant response, system instructions, private reasoning, chain-of-thought, credentials, access tokens, or other secrets.
+3. `prompt_summary` must be a short, redacted description of the request, not a quotation or close reproduction.
+4. Do not invent the model name, token counts, or cost. Use `null` values with `source: "unavailable"` unless the platform explicitly reports them. Use `source: "estimated"` only when a deterministic tokenizer or pricing tool produced the value, and name that tool in `estimation_method`.
+5. Base execution signals only on observed results. Do not claim a file changed or a test passed without evidence.
+6. Use schema version `1.0` and conform to `.prompt-observer/event.schema.json`.
+
+## Required workflow
+
+After completing a task:
+
+1. Evaluate the prompt on each 0–10 dimension in the event schema.
+2. Create a unique event ID and an ISO-8601 UTC timestamp.
+3. Write the event temporarily to `.prompt-observer/pending/<event_id>.json`.
+4. Run:
+
+   ```text
+   node .prompt-observer/prompt-observer.mjs log .prompt-observer/pending/<event_id>.json
+   ```
+
+5. After a successful log, remove only the temporary event file you created.
+6. End the user-facing answer with exactly three concise English lines:
+
+   ```text
+   Prompt Insight: <average>/10 — <short strength>
+   Main weakness: <highest-impact weakness, or "No material weakness detected">
+   Next time: <one actionable improvement>
+   ```
+
+Do not paste the complete JSON event into the user-facing response when persistence succeeds.
+
+## No-filesystem fallback
+
+If writing files or running the logger is unavailable, do not pretend the event was saved. Return the same three-line insight, followed by one fenced `json` block containing the complete valid event. This is the only fallback; do not retry with unsafe shell commands or external services.
+
+## Scoring guidance
+
+- `intent_clarity`: Is the desired outcome explicit and understandable?
+- `context_sufficiency`: Is enough relevant background provided for the task?
+- `scope_definition`: Are boundaries, affected areas, and exclusions clear?
+- `constraints_quality`: Are technical, safety, compatibility, and style constraints usable?
+- `acceptance_criteria`: Is completion objectively recognizable?
+- `verification_plan`: Are tests or other verification expectations stated or clearly inferable?
+- `ambiguity_risk`: Estimate the risk that a capable agent would choose the wrong interpretation.
+
+Score only prompt quality. Do not lower a score because implementation was difficult when the request itself was clear.
+
+## Event example
+
+```json
+{
+  "schema_version": "1.0",
+  "event_id": "evt-20260901-7f3a92c1",
+  "timestamp": "2026-09-01T12:00:00.000Z",
+  "task_type": "coding",
+  "prompt_summary": "Implement a dependency-free prompt observation kit with structured local logging.",
+  "intent_clarity": 9,
+  "context_sufficiency": 8,
+  "scope_definition": 9,
+  "constraints_quality": 9,
+  "acceptance_criteria": 8,
+  "verification_plan": 7,
+  "ambiguity_risk": "low",
+  "strengths": [
+    "The requested deliverables and runtime constraints are explicit."
+  ],
+  "weaknesses": [
+    {
+      "category": "missing_verification",
+      "severity": "low",
+      "message": "The exact expected report contents were not fully enumerated."
+    }
+  ],
+  "improvement_suggestions": [
+    "List the required report sections and one expected example."
+  ],
+  "execution_signals": {
+    "result_status": "completed",
+    "files_changed": [
+      "PROMPT_OBSERVER.md"
+    ],
+    "tests_run": [
+      {
+        "name": "node --test",
+        "status": "passed"
+      }
+    ]
+  },
+  "usage": {
+    "model": null,
+    "input_tokens": null,
+    "output_tokens": null,
+    "cost_usd": null,
+    "source": "unavailable",
+    "estimation_method": null
+  }
+}
+```

package/README.md

@@ -0,0 +1,104 @@
+# Prompt Observer
+
+Prompt Observer is a dependency-free observability kit for professional coding-agent workflows. It gives an agent a clear contract: after each task, evaluate the prompt, show a short insight, and save a privacy-safe structured event locally.
+
+It is designed for Vibe Coding workflows and works without a browser extension, a local model, or an additional AI API. The agent already completing the task creates the observation.
+
+## What it records
+
+- Prompt-quality dimensions: clarity, context, scope, constraints, acceptance criteria, and verification plan
+- Actionable weaknesses and improvement suggestions
+- Observed execution signals: result status, changed files, and test outcomes
+- Model, token, and cost information only when the platform explicitly provides it
+
+Raw prompts, raw responses, private reasoning, system instructions, and secrets are prohibited from the event format.
+
+## Requirements
+
+- Node.js 20 or newer
+- A coding agent that can read project instructions
+- Filesystem access by the agent for automatic event persistence
+
+## Quick start
+
+Clone this repository and initialize Prompt Observer in a target project:
+
+```powershell
+git clone <your-repository-url>
+cd prompt-observer
+node bin/prompt-observer.mjs init "E:\path\to\your-project"
+```
+
+Then add the following one-line instruction to the target project's existing agent-instruction file:
+
+```md
+Read and follow `.prompt-observer/PROMPT_OBSERVER.md` after every user-requested task.
+```
+
+Prompt Observer does not modify vendor-specific instruction files in v1. Add the line to whichever project-instruction mechanism your agent already uses.
+
+## How it works
+
+```text
+User request → Coding agent completes the task → Agent evaluates the prompt
+              → Validated JSONL event → Local Markdown report
+```
+
+Each initialized project receives:
+
+```text
+.prompt-observer/
+  PROMPT_OBSERVER.md       Agent contract
+  event.schema.json        Versioned event schema
+  prompt-observer.mjs      Portable local CLI
+  events.jsonl             Generated append-only event log
+  report.md                Generated analysis report
+```
+
+The generated log and report are excluded by the local `.prompt-observer/.gitignore`; the contract and schema can safely be committed.
+
+## Commands
+
+Run these commands inside an initialized project:
+
+```powershell
+# Check an event before saving it
+node .prompt-observer/prompt-observer.mjs validate .prompt-observer/pending/example.json
+
+# Validate and append an event to the local JSONL log
+node .prompt-observer/prompt-observer.mjs log .prompt-observer/pending/example.json
+
+# Generate the aggregate Markdown report
+node .prompt-observer/prompt-observer.mjs report .
+```
+
+The report includes average prompt health, recurring weaknesses, ambiguity risk, execution outcomes, verification trends, and platform-reported usage totals.
+
+## No-filesystem fallback
+
+When an agent cannot write files, the contract requires it to show the three-line insight and return a complete valid JSON event in a fenced `json` block. It must state that the event was not persisted. A later extension or integration can capture that output automatically.
+
+## npm package — coming soon
+
+The project is configured for npm packaging, but it is **not published yet**. The final package name will be chosen before release, preferably as a scoped name such as `@your-npm-username/prompt-observer`.
+
+After publication, the intended installation flow will be:
+
+```powershell
+npx @your-npm-username/prompt-observer init .
+```
+
+Before publishing, replace `@your-npm-username` with your real npm scope and configure the repository URL in `package.json`.
+
+## Development
+
+```powershell
+npm run check
+npm test
+```
+
+`prepublishOnly` runs both checks automatically before `npm publish`. The package uses only Node.js built-ins; JSONL is the source of truth for v1, while SQLite export is intentionally deferred to a later release.
+
+## License
+
+[MIT](LICENSE)

package/bin/prompt-observer.mjs

@@ -0,0 +1,603 @@
+#!/usr/bin/env node
+
+import {
+  appendFile,
+  copyFile,
+  mkdir,
+  open,
+  readFile,
+  stat,
+  writeFile,
+} from "node:fs/promises";
+import { dirname, basename, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const SCHEMA_VERSION = "1.0";
+const SCORE_FIELDS = [
+  "intent_clarity",
+  "context_sufficiency",
+  "scope_definition",
+  "constraints_quality",
+  "acceptance_criteria",
+  "verification_plan",
+];
+const TASK_TYPES = new Set([
+  "coding",
+  "debugging",
+  "planning",
+  "review",
+  "testing",
+  "documentation",
+  "refactoring",
+  "other",
+]);
+const RISK_LEVELS = new Set(["low", "medium", "high"]);
+const WEAKNESS_CATEGORIES = new Set([
+  "missing_context",
+  "ambiguous_goal",
+  "undefined_scope",
+  "missing_constraints",
+  "missing_acceptance_criteria",
+  "missing_verification",
+  "missing_output_format",
+  "conflicting_requirements",
+  "other",
+]);
+const SEVERITIES = new Set(["low", "medium", "high"]);
+const RESULT_STATUSES = new Set(["completed", "partial", "blocked"]);
+const TEST_STATUSES = new Set(["passed", "failed", "not_run", "unknown"]);
+const USAGE_SOURCES = new Set(["platform_reported", "estimated", "unavailable"]);
+const ROOT_KEYS = new Set([
+  "schema_version",
+  "event_id",
+  "timestamp",
+  "task_type",
+  "prompt_summary",
+  ...SCORE_FIELDS,
+  "ambiguity_risk",
+  "strengths",
+  "weaknesses",
+  "improvement_suggestions",
+  "execution_signals",
+  "usage",
+]);
+const BANNED_KEYS = new Set([
+  "prompt",
+  "raw_prompt",
+  "full_prompt",
+  "user_prompt",
+  "response",
+  "raw_response",
+  "full_response",
+  "assistant_response",
+  "reasoning",
+  "chain_of_thought",
+  "system_prompt",
+]);
+
+function isObject(value) {
+  return value !== null && typeof value === "object" && !Array.isArray(value);
+}
+
+function addUnknownKeyErrors(value, allowed, path, errors) {
+  for (const key of Object.keys(value)) {
+    if (!allowed.has(key)) errors.push(`${path}.${key} is not allowed`);
+  }
+}
+
+function validateShortText(value, path, errors, { max = 500 } = {}) {
+  if (typeof value !== "string" || value.trim().length === 0) {
+    errors.push(`${path} must be a non-empty string`);
+  } else if (value.length > max) {
+    errors.push(`${path} must be at most ${max} characters`);
+  }
+}
+
+function validateTextArray(value, path, errors, maxItems = 10) {
+  if (!Array.isArray(value)) {
+    errors.push(`${path} must be an array`);
+    return;
+  }
+  if (value.length > maxItems) errors.push(`${path} must contain at most ${maxItems} items`);
+  value.forEach((item, index) => validateShortText(item, `${path}[${index}]`, errors));
+}
+
+function findBannedKeys(value, path, errors) {
+  if (Array.isArray(value)) {
+    value.forEach((item, index) => findBannedKeys(item, `${path}[${index}]`, errors));
+    return;
+  }
+  if (!isObject(value)) return;
+  for (const [key, child] of Object.entries(value)) {
+    const normalized = key.toLowerCase().replaceAll("-", "_");
+    if (BANNED_KEYS.has(normalized)) {
+      errors.push(`${path}.${key} is prohibited because raw prompt, response, or private reasoning must not be stored`);
+    }
+    findBannedKeys(child, `${path}.${key}`, errors);
+  }
+}
+
+function findLikelySecrets(value, errors) {
+  const serialized = JSON.stringify(value);
+  const patterns = [
+    [/-----BEGIN [A-Z ]*PRIVATE KEY-----/i, "private key material"],
+    [/\bsk-[A-Za-z0-9_-]{16,}\b/, "an API key"],
+    [/\bgh[pousr]_[A-Za-z0-9]{20,}\b/, "a GitHub token"],
+    [/\bBearer\s+[A-Za-z0-9._~+/=-]{12,}/i, "a bearer token"],
+  ];
+  for (const [pattern, label] of patterns) {
+    if (pattern.test(serialized)) errors.push(`event appears to contain ${label}; redact it before logging`);
+  }
+}
+
+export function validateEvent(event) {
+  const errors = [];
+  if (!isObject(event)) return ["event must be a JSON object"];
+
+  addUnknownKeyErrors(event, ROOT_KEYS, "$", errors);
+  findBannedKeys(event, "$", errors);
+  findLikelySecrets(event, errors);
+
+  for (const key of ROOT_KEYS) {
+    if (!(key in event)) errors.push(`$.${key} is required`);
+  }
+  if (errors.some((error) => error.endsWith(" is required"))) return errors;
+
+  if (event.schema_version !== SCHEMA_VERSION) {
+    errors.push(`$.schema_version must equal ${SCHEMA_VERSION}`);
+  }
+  if (
+    typeof event.event_id !== "string" ||
+    !/^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$/.test(event.event_id)
+  ) {
+    errors.push("$.event_id must be 8-128 safe identifier characters");
+  }
+  if (
+    typeof event.timestamp !== "string" ||
+    !event.timestamp.includes("T") ||
+    Number.isNaN(Date.parse(event.timestamp))
+  ) {
+    errors.push("$.timestamp must be a valid ISO-8601 date-time");
+  }
+  if (!TASK_TYPES.has(event.task_type)) errors.push("$.task_type is invalid");
+  validateShortText(event.prompt_summary, "$.prompt_summary", errors);
+
+  for (const field of SCORE_FIELDS) {
+    if (!Number.isInteger(event[field]) || event[field] < 0 || event[field] > 10) {
+      errors.push(`$.${field} must be an integer from 0 to 10`);
+    }
+  }
+  if (!RISK_LEVELS.has(event.ambiguity_risk)) errors.push("$.ambiguity_risk is invalid");
+  validateTextArray(event.strengths, "$.strengths", errors);
+  validateTextArray(event.improvement_suggestions, "$.improvement_suggestions", errors);
+
+  if (!Array.isArray(event.weaknesses)) {
+    errors.push("$.weaknesses must be an array");
+  } else {
+    if (event.weaknesses.length > 10) errors.push("$.weaknesses must contain at most 10 items");
+    event.weaknesses.forEach((weakness, index) => {
+      const path = `$.weaknesses[${index}]`;
+      if (!isObject(weakness)) {
+        errors.push(`${path} must be an object`);
+        return;
+      }
+      addUnknownKeyErrors(weakness, new Set(["category", "severity", "message"]), path, errors);
+      if (!WEAKNESS_CATEGORIES.has(weakness.category)) errors.push(`${path}.category is invalid`);
+      if (!SEVERITIES.has(weakness.severity)) errors.push(`${path}.severity is invalid`);
+      validateShortText(weakness.message, `${path}.message`, errors);
+    });
+  }
+
+  const signals = event.execution_signals;
+  if (!isObject(signals)) {
+    errors.push("$.execution_signals must be an object");
+  } else {
+    addUnknownKeyErrors(signals, new Set(["result_status", "files_changed", "tests_run"]), "$.execution_signals", errors);
+    if (!RESULT_STATUSES.has(signals.result_status)) {
+      errors.push("$.execution_signals.result_status is invalid");
+    }
+    validateTextArray(signals.files_changed, "$.execution_signals.files_changed", errors, 500);
+    if (!Array.isArray(signals.tests_run)) {
+      errors.push("$.execution_signals.tests_run must be an array");
+    } else {
+      if (signals.tests_run.length > 100) {
+        errors.push("$.execution_signals.tests_run must contain at most 100 items");
+      }
+      signals.tests_run.forEach((test, index) => {
+        const path = `$.execution_signals.tests_run[${index}]`;
+        if (!isObject(test)) {
+          errors.push(`${path} must be an object`);
+          return;
+        }
+        addUnknownKeyErrors(test, new Set(["name", "status"]), path, errors);
+        validateShortText(test.name, `${path}.name`, errors);
+        if (!TEST_STATUSES.has(test.status)) errors.push(`${path}.status is invalid`);
+      });
+    }
+  }
+
+  const usage = event.usage;
+  if (!isObject(usage)) {
+    errors.push("$.usage must be an object");
+  } else {
+    const usageKeys = new Set([
+      "model",
+      "input_tokens",
+      "output_tokens",
+      "cost_usd",
+      "source",
+      "estimation_method",
+    ]);
+    addUnknownKeyErrors(usage, usageKeys, "$.usage", errors);
+    for (const key of usageKeys) {
+      if (!(key in usage)) errors.push(`$.usage.${key} is required`);
+    }
+    if (usage.model !== null && (typeof usage.model !== "string" || usage.model.length > 200)) {
+      errors.push("$.usage.model must be null or a string up to 200 characters");
+    }
+    for (const field of ["input_tokens", "output_tokens"]) {
+      if (usage[field] !== null && (!Number.isInteger(usage[field]) || usage[field] < 0)) {
+        errors.push(`$.usage.${field} must be null or a non-negative integer`);
+      }
+    }
+    if (usage.cost_usd !== null && (typeof usage.cost_usd !== "number" || usage.cost_usd < 0)) {
+      errors.push("$.usage.cost_usd must be null or a non-negative number");
+    }
+    if (!USAGE_SOURCES.has(usage.source)) errors.push("$.usage.source is invalid");
+    if (
+      usage.estimation_method !== null &&
+      (typeof usage.estimation_method !== "string" || usage.estimation_method.trim() === "" || usage.estimation_method.length > 300)
+    ) {
+      errors.push("$.usage.estimation_method must be null or a non-empty string up to 300 characters");
+    }
+    const metrics = [usage.model, usage.input_tokens, usage.output_tokens, usage.cost_usd];
+    if (usage.source === "unavailable" && metrics.some((value) => value !== null)) {
+      errors.push("$.usage metrics must all be null when source is unavailable");
+    }
+    if (usage.source === "unavailable" && usage.estimation_method !== null) {
+      errors.push("$.usage.estimation_method must be null when source is unavailable");
+    }
+    if (usage.source === "estimated") {
+      if (typeof usage.estimation_method !== "string" || usage.estimation_method.trim() === "") {
+        errors.push("$.usage.estimation_method is required when source is estimated");
+      }
+      if (usage.input_tokens === null && usage.output_tokens === null && usage.cost_usd === null) {
+        errors.push("$.usage estimated source requires at least one estimated numeric metric");
+      }
+    }
+    if (usage.source === "platform_reported" && usage.estimation_method !== null) {
+      errors.push("$.usage.estimation_method must be null when source is platform_reported");
+    }
+    if (usage.source === "platform_reported" && metrics.every((value) => value === null)) {
+      errors.push("$.usage platform_reported source requires at least one reported metric");
+    }
+  }
+
+  return errors;
+}
+
+async function exists(path) {
+  try {
+    await stat(path);
+    return true;
+  } catch (error) {
+    if (error.code === "ENOENT") return false;
+    throw error;
+  }
+}
+
+async function findResourceRoot() {
+  const selfDirectory = dirname(fileURLToPath(import.meta.url));
+  const candidates = [resolve(selfDirectory, ".."), selfDirectory];
+  for (const candidate of candidates) {
+    const contract = join(candidate, "PROMPT_OBSERVER.md");
+    const schema = (await exists(join(candidate, "schema", "event.schema.json")))
+      ? join(candidate, "schema", "event.schema.json")
+      : join(candidate, "event.schema.json");
+    if ((await exists(contract)) && (await exists(schema))) {
+      return { root: candidate, contract, schema };
+    }
+  }
+  throw new Error("Could not locate PROMPT_OBSERVER.md and event.schema.json next to the CLI.");
+}
+
+async function copyUnlessPresent(source, destination, actions) {
+  if (await exists(destination)) {
+    actions.push(`kept ${destination}`);
+    return;
+  }
+  await copyFile(source, destination);
+  actions.push(`created ${destination}`);
+}
+
+async function writeUnlessPresent(destination, content, actions) {
+  if (await exists(destination)) {
+    actions.push(`kept ${destination}`);
+    return;
+  }
+  await writeFile(destination, content, "utf8");
+  actions.push(`created ${destination}`);
+}
+
+export async function initProject(targetPath) {
+  const target = resolve(targetPath ?? ".");
+  const observerDirectory = join(target, ".prompt-observer");
+  const resources = await findResourceRoot();
+  const selfPath = fileURLToPath(import.meta.url);
+  const actions = [];
+
+  await mkdir(observerDirectory, { recursive: true });
+  await mkdir(join(observerDirectory, "pending"), { recursive: true });
+  await copyUnlessPresent(resources.contract, join(observerDirectory, "PROMPT_OBSERVER.md"), actions);
+  await copyUnlessPresent(resources.schema, join(observerDirectory, "event.schema.json"), actions);
+  await copyUnlessPresent(selfPath, join(observerDirectory, "prompt-observer.mjs"), actions);
+  await writeUnlessPresent(
+    join(observerDirectory, ".gitignore"),
+    ["events.jsonl", "report.md", "pending/", "*.tmp", ""].join("\n"),
+    actions,
+  );
+  const eventsFile = join(observerDirectory, "events.jsonl");
+  const file = await open(eventsFile, "a");
+  await file.close();
+  actions.push((await stat(eventsFile)).size === 0 ? `ready ${eventsFile}` : `kept ${eventsFile}`);
+
+  return { target, observerDirectory, actions };
+}
+
+async function resolveObserverDirectory(targetPath) {
+  if (targetPath) {
+    const resolved = resolve(targetPath);
+    const candidate = basename(resolved) === ".prompt-observer" ? resolved : join(resolved, ".prompt-observer");
+    if (await exists(candidate)) return candidate;
+    throw new Error(`Prompt Observer is not initialized at ${resolved}. Run init first.`);
+  }
+
+  let current = resolve(".");
+  while (true) {
+    const candidate = join(current, ".prompt-observer");
+    if (await exists(candidate)) return candidate;
+    const parent = dirname(current);
+    if (parent === current) break;
+    current = parent;
+  }
+  throw new Error("No .prompt-observer directory found from the current directory upward. Run init first.");
+}
+
+async function readEvents(eventsPath) {
+  if (!(await exists(eventsPath))) return [];
+  const contents = await readFile(eventsPath, "utf8");
+  const events = [];
+  for (const [index, line] of contents.split(/\r?\n/).entries()) {
+    if (line.trim() === "") continue;
+    let event;
+    try {
+      event = JSON.parse(line);
+    } catch (error) {
+      throw new Error(`Invalid JSON in ${eventsPath} at line ${index + 1}: ${error.message}`);
+    }
+    const validationErrors = validateEvent(event);
+    if (validationErrors.length > 0) {
+      throw new Error(`Invalid event in ${eventsPath} at line ${index + 1}:\n- ${validationErrors.join("\n- ")}`);
+    }
+    events.push(event);
+  }
+  return events;
+}
+
+export async function logEvent(eventFile, targetPath) {
+  if (!eventFile) throw new Error("An event JSON file is required.");
+  const eventPath = resolve(eventFile);
+  let event;
+  try {
+    event = JSON.parse(await readFile(eventPath, "utf8"));
+  } catch (error) {
+    throw new Error(`Could not read event JSON from ${eventPath}: ${error.message}`);
+  }
+
+  const validationErrors = validateEvent(event);
+  if (validationErrors.length > 0) {
+    throw new Error(`Event validation failed:\n- ${validationErrors.join("\n- ")}`);
+  }
+
+  const observerDirectory = await resolveObserverDirectory(targetPath);
+  const eventsPath = join(observerDirectory, "events.jsonl");
+  const existingEvents = await readEvents(eventsPath);
+  if (existingEvents.some((item) => item.event_id === event.event_id)) {
+    throw new Error(`Event ID ${event.event_id} already exists; refusing to create a duplicate.`);
+  }
+  await appendFile(eventsPath, `${JSON.stringify(event)}\n`, "utf8");
+  return { eventId: event.event_id, eventsPath };
+}
+
+function average(values) {
+  if (values.length === 0) return null;
+  return values.reduce((sum, value) => sum + value, 0) / values.length;
+}
+
+function formatAverage(value) {
+  return value === null ? "N/A" : value.toFixed(2);
+}
+
+function tableRows(counts, preferredOrder) {
+  return preferredOrder.map((key) => `| ${key} | ${counts.get(key) ?? 0} |`).join("\n");
+}
+
+export function buildReport(events) {
+  const dimensionAverages = new Map();
+  for (const field of SCORE_FIELDS) {
+    dimensionAverages.set(field, average(events.map((event) => event[field])));
+  }
+  const overallScores = events.map((event) => average(SCORE_FIELDS.map((field) => event[field])));
+  const overallAverage = average(overallScores.filter((value) => value !== null));
+
+  const weaknessCounts = new Map();
+  const severityCounts = new Map();
+  const riskCounts = new Map();
+  const resultCounts = new Map();
+  const testCounts = new Map();
+  let eventsWithTests = 0;
+  let unavailableUsage = 0;
+  let totalInputTokens = 0;
+  let totalOutputTokens = 0;
+  let totalCost = 0;
+  let hasInputTokens = false;
+  let hasOutputTokens = false;
+  let hasCost = false;
+
+  for (const event of events) {
+    riskCounts.set(event.ambiguity_risk, (riskCounts.get(event.ambiguity_risk) ?? 0) + 1);
+    const status = event.execution_signals.result_status;
+    resultCounts.set(status, (resultCounts.get(status) ?? 0) + 1);
+    if (event.execution_signals.tests_run.length > 0) eventsWithTests += 1;
+    for (const test of event.execution_signals.tests_run) {
+      testCounts.set(test.status, (testCounts.get(test.status) ?? 0) + 1);
+    }
+    for (const weakness of event.weaknesses) {
+      weaknessCounts.set(weakness.category, (weaknessCounts.get(weakness.category) ?? 0) + 1);
+      severityCounts.set(weakness.severity, (severityCounts.get(weakness.severity) ?? 0) + 1);
+    }
+    if (event.usage.source === "unavailable") unavailableUsage += 1;
+    if (event.usage.input_tokens !== null) {
+      totalInputTokens += event.usage.input_tokens;
+      hasInputTokens = true;
+    }
+    if (event.usage.output_tokens !== null) {
+      totalOutputTokens += event.usage.output_tokens;
+      hasOutputTokens = true;
+    }
+    if (event.usage.cost_usd !== null) {
+      totalCost += event.usage.cost_usd;
+      hasCost = true;
+    }
+  }
+
+  const topWeaknesses = [...weaknessCounts.entries()]
+    .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
+    .map(([category, count]) => `| ${category} | ${count} |`)
+    .join("\n");
+  const generatedAt = new Date().toISOString();
+
+  return `# Prompt Observer Report
+
+Generated: ${generatedAt}
+
+## Overview
+
+- Events: ${events.length}
+- Average prompt health: ${formatAverage(overallAverage)}/10
+- Events with tests: ${eventsWithTests}
+- Events with unavailable usage: ${unavailableUsage}
+
+## Quality dimensions
+
+| Dimension | Average |
+| --- | ---: |
+${SCORE_FIELDS.map((field) => `| ${field} | ${formatAverage(dimensionAverages.get(field))} |`).join("\n")}
+
+## Weaknesses
+
+| Category | Count |
+| --- | ---: |
+${topWeaknesses || "| None recorded | 0 |"}
+
+### Severity
+
+| Severity | Count |
+| --- | ---: |
+${tableRows(severityCounts, ["high", "medium", "low"])}
+
+## Ambiguity risk
+
+| Risk | Count |
+| --- | ---: |
+${tableRows(riskCounts, ["high", "medium", "low"])}
+
+## Execution outcomes
+
+| Status | Count |
+| --- | ---: |
+${tableRows(resultCounts, ["completed", "partial", "blocked"])}
+
+## Verification
+
+| Test status | Count |
+| --- | ---: |
+${tableRows(testCounts, ["passed", "failed", "not_run", "unknown"])}
+
+## Reported usage
+
+- Input tokens: ${hasInputTokens ? totalInputTokens : "N/A"}
+- Output tokens: ${hasOutputTokens ? totalOutputTokens : "N/A"}
+- Cost (USD): ${hasCost ? totalCost.toFixed(6) : "N/A"}
+`;
+}
+
+export async function generateReport(targetPath) {
+  const observerDirectory = await resolveObserverDirectory(targetPath ?? ".");
+  const eventsPath = join(observerDirectory, "events.jsonl");
+  const events = await readEvents(eventsPath);
+  const report = buildReport(events);
+  const reportPath = join(observerDirectory, "report.md");
+  await writeFile(reportPath, report, "utf8");
+  return { eventCount: events.length, reportPath, report };
+}
+
+function printHelp() {
+  process.stdout.write(`Prompt Observer ${SCHEMA_VERSION}
+
+Usage:
+  prompt-observer init <target-path>
+  prompt-observer log <event-file> [--target <target-path>]
+  prompt-observer report [target-path]
+  prompt-observer validate <event-file>
+  prompt-observer help
+`);
+}
+
+function optionValue(args, name) {
+  const index = args.indexOf(name);
+  if (index === -1) return undefined;
+  if (!args[index + 1]) throw new Error(`${name} requires a value.`);
+  return args[index + 1];
+}
+
+export async function runCli(args) {
+  const [command, ...rest] = args;
+  if (!command || command === "help" || command === "--help" || command === "-h") {
+    printHelp();
+    return;
+  }
+  if (command === "init") {
+    const result = await initProject(rest[0] ?? ".");
+    process.stdout.write(`Initialized Prompt Observer at ${result.observerDirectory}\n${result.actions.join("\n")}\n`);
+    return;
+  }
+  if (command === "validate") {
+    if (!rest[0]) throw new Error("validate requires an event JSON file.");
+    const event = JSON.parse(await readFile(resolve(rest[0]), "utf8"));
+    const errors = validateEvent(event);
+    if (errors.length > 0) throw new Error(`Event validation failed:\n- ${errors.join("\n- ")}`);
+    process.stdout.write(`Valid event: ${event.event_id}\n`);
+    return;
+  }
+  if (command === "log") {
+    const target = optionValue(rest, "--target");
+    const result = await logEvent(rest[0], target);
+    process.stdout.write(`Logged ${result.eventId} to ${result.eventsPath}\n`);
+    return;
+  }
+  if (command === "report") {
+    const result = await generateReport(rest[0] ?? ".");
+    process.stdout.write(`Generated ${result.reportPath} from ${result.eventCount} event(s).\n`);
+    return;
+  }
+  throw new Error(`Unknown command: ${command}`);
+}
+
+const isDirectExecution = process.argv[1] && resolve(process.argv[1]) === resolve(fileURLToPath(import.meta.url));
+if (isDirectExecution) {
+  runCli(process.argv.slice(2)).catch((error) => {
+    process.stderr.write(`Error: ${error.message}\n`);
+    process.exitCode = 1;
+  });
+}

package/package.json

@@ -0,0 +1,40 @@
+{
+  "name": "@onthink/prompt-observer",
+  "version": "0.1.0",
+  "description": "A dependency-free prompt observability contract and CLI for coding agents.",
+  "keywords": [
+    "ai",
+    "coding-agents",
+    "prompt-engineering",
+    "observability",
+    "vibe-coding"
+  ],
+  "type": "module",
+  "bin": {
+    "prompt-observer": "./bin/prompt-observer.mjs"
+  },
+  "scripts": {
+    "test": "node --test",
+    "check": "node --check bin/prompt-observer.mjs",
+    "prepublishOnly": "npm run check && npm test"
+  },
+  "engines": {
+    "node": ">=20"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/daniialHZ/prompt-observer.git"
+  },
+  "bugs": {
+    "url": "https://github.com/daniialHZ/prompt-observer/issues"
+  },
+  "homepage": "https://github.com/daniialHZ/prompt-observer#readme",
+  "license": "MIT",
+  "files": [
+    "bin/",
+    "schema/",
+    "PROMPT_OBSERVER.md",
+    "README.md",
+    "LICENSE"
+  ]
+}

package/schema/event.schema.json

@@ -0,0 +1,140 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://prompt-observer.local/schema/event-1.0.json",
+  "title": "Prompt Observer Event",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "schema_version",
+    "event_id",
+    "timestamp",
+    "task_type",
+    "prompt_summary",
+    "intent_clarity",
+    "context_sufficiency",
+    "scope_definition",
+    "constraints_quality",
+    "acceptance_criteria",
+    "verification_plan",
+    "ambiguity_risk",
+    "strengths",
+    "weaknesses",
+    "improvement_suggestions",
+    "execution_signals",
+    "usage"
+  ],
+  "properties": {
+    "schema_version": { "const": "1.0" },
+    "event_id": {
+      "type": "string",
+      "pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$"
+    },
+    "timestamp": { "type": "string", "format": "date-time" },
+    "task_type": {
+      "enum": [
+        "coding",
+        "debugging",
+        "planning",
+        "review",
+        "testing",
+        "documentation",
+        "refactoring",
+        "other"
+      ]
+    },
+    "prompt_summary": { "type": "string", "minLength": 1, "maxLength": 500 },
+    "intent_clarity": { "$ref": "#/$defs/score" },
+    "context_sufficiency": { "$ref": "#/$defs/score" },
+    "scope_definition": { "$ref": "#/$defs/score" },
+    "constraints_quality": { "$ref": "#/$defs/score" },
+    "acceptance_criteria": { "$ref": "#/$defs/score" },
+    "verification_plan": { "$ref": "#/$defs/score" },
+    "ambiguity_risk": { "enum": ["low", "medium", "high"] },
+    "strengths": {
+      "type": "array",
+      "items": { "$ref": "#/$defs/shortText" },
+      "maxItems": 10
+    },
+    "weaknesses": {
+      "type": "array",
+      "maxItems": 10,
+      "items": {
+        "type": "object",
+        "additionalProperties": false,
+        "required": ["category", "severity", "message"],
+        "properties": {
+          "category": {
+            "enum": [
+              "missing_context",
+              "ambiguous_goal",
+              "undefined_scope",
+              "missing_constraints",
+              "missing_acceptance_criteria",
+              "missing_verification",
+              "missing_output_format",
+              "conflicting_requirements",
+              "other"
+            ]
+          },
+          "severity": { "enum": ["low", "medium", "high"] },
+          "message": { "$ref": "#/$defs/shortText" }
+        }
+      }
+    },
+    "improvement_suggestions": {
+      "type": "array",
+      "items": { "$ref": "#/$defs/shortText" },
+      "maxItems": 10
+    },
+    "execution_signals": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["result_status", "files_changed", "tests_run"],
+      "properties": {
+        "result_status": { "enum": ["completed", "partial", "blocked"] },
+        "files_changed": {
+          "type": "array",
+          "items": { "type": "string", "minLength": 1, "maxLength": 500 },
+          "maxItems": 500
+        },
+        "tests_run": {
+          "type": "array",
+          "maxItems": 100,
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": ["name", "status"],
+            "properties": {
+              "name": { "type": "string", "minLength": 1, "maxLength": 500 },
+              "status": { "enum": ["passed", "failed", "not_run", "unknown"] }
+            }
+          }
+        }
+      }
+    },
+    "usage": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "model",
+        "input_tokens",
+        "output_tokens",
+        "cost_usd",
+        "source",
+        "estimation_method"
+      ],
+      "properties": {
+        "model": { "type": ["string", "null"], "maxLength": 200 },
+        "input_tokens": { "type": ["integer", "null"], "minimum": 0 },
+        "output_tokens": { "type": ["integer", "null"], "minimum": 0 },
+        "cost_usd": { "type": ["number", "null"], "minimum": 0 },
+        "source": { "enum": ["platform_reported", "estimated", "unavailable"] },
+        "estimation_method": { "type": ["string", "null"], "maxLength": 300 }
+      }
+    }
+  },
+  "$defs": {
+    "score": { "type": "integer", "minimum": 0, "maximum": 10 },
+    "shortText": { "type": "string", "minLength": 1, "maxLength": 500 }
+  }
+}