@onthink/prompt-observer 0.1.0 → 0.2.0

package/PROMPT_OBSERVER.md

@@ -7,9 +7,11 @@ Follow this contract after every user-requested task. Complete the task first, t
 1. Use only information visible in the conversation, tool results, file changes, and verification output.
 2. Never store the full user prompt, the full assistant response, system instructions, private reasoning, chain-of-thought, credentials, access tokens, or other secrets.
 3. `prompt_summary` must be a short, redacted description of the request, not a quotation or close reproduction.
-4. Do not invent the model name, token counts, or cost. Use `null` values with `source: "unavailable"` unless the platform explicitly reports them. Use `source: "estimated"` only when a deterministic tokenizer or pricing tool produced the value, and name that tool in `estimation_method`.
+4. Prefer platform-reported usage. When it is unavailable, estimate token counts only from the visible user input and final response with the deterministic heuristic below. Never guess a model name. The logger calculates estimated cost only for an exact model match in `.prompt-observer/pricing.json`.
 5. Base execution signals only on observed results. Do not claim a file changed or a test passed without evidence.
-6. Use schema version `1.0` and conform to `.prompt-observer/event.schema.json`.
+6. Use schema version `1.1` and conform to `.prompt-observer/event.schema.json`.
+7. Do not manufacture criticism. If the prompt is clear and sufficient for the task, keep `weaknesses` and `improvement_suggestions` empty.
+8. Evaluate against objective task requirements, not personal preferences about wording, tone, verbosity, formatting, workflow, or technology choices.
 
 ## Required workflow
 
@@ -30,7 +32,7 @@ After completing a task:
    ```text
    Prompt Insight: <average>/10 — <short strength>
    Main weakness: <highest-impact weakness, or "No material weakness detected">
-   Next time: <one actionable improvement>
+   Next time: <one actionable improvement, or "No change needed">
    ```
 
 Do not paste the complete JSON event into the user-facing response when persistence succeeds.
@@ -51,35 +53,64 @@ If writing files or running the logger is unavailable, do not pretend the event
 
 Score only prompt quality. Do not lower a score because implementation was difficult when the request itself was clear.
 
+## Structured insights
+
+In schema `1.1`, every strength and improvement suggestion is an object with:
+
+- `category`: one of `intent_clarity`, `context_sufficiency`, `scope_definition`, `constraints_quality`, `acceptance_criteria`, `verification_plan`, `output_format`, or `other`
+- `message`: one concise, evidence-based observation
+
+Legacy schema `1.0` events with string arrays remain readable, but all new events must use the structured `1.1` form.
+
+## Usage precedence and estimation
+
+1. If the platform reports any model, token, or cost metrics, record only those reported metrics, keep unreported metrics `null`, use `source: "platform_reported"`, and keep `estimation_method: null`.
+2. Otherwise, if the Agent can inspect the complete visible user input and its final response, estimate each side independently using `agent_text_heuristic_v1`:
+   - Ignore whitespace.
+   - Count ASCII letters and digits as `characters / 4`.
+   - Count non-ASCII letters, digits, and combining marks as `characters / 2`.
+   - Count punctuation and symbols as `characters / 2`.
+   - Add the three values and round up to the next integer.
+3. Store the two counts with `source: "estimated"` and `estimation_method: "agent_text_heuristic_v1"`.
+4. Set `model` only when the runtime explicitly identifies it. Never infer a model from the product name.
+5. Leave `cost_usd: null`. During `log`, the CLI fills it only when both token counts exist and `model` exactly matches `.prompt-observer/pricing.json`; the pricing snapshot identifier is then appended to `estimation_method`.
+6. If neither reported nor safely estimated usage is available, keep all metrics `null`, use `source: "unavailable"`, and keep `estimation_method: null`.
+
+Estimated usage is directional, not billing data. Never include hidden system instructions, tool payloads, cached-token adjustments, subscription fees, or guessed reasoning tokens in the estimate.
+
+## Evidence threshold and neutrality
+
+- Record a weakness only when a concrete omission, ambiguity, contradiction, or constraint creates a meaningful risk of wrong execution, wasted work, or unverifiable completion.
+- Do not require context, constraints, acceptance criteria, output formatting, or tests when they are unnecessary for the specific task.
+- Do not criticize a prompt merely because it could be longer, more formal, more structured, or written in a style you prefer.
+- Do not turn optional enhancements into weaknesses.
+- When no material weakness exists, use an empty `weaknesses` array, an empty `improvement_suggestions` array, `Main weakness: No material weakness detected`, and `Next time: No change needed`.
+- Positive scores must reflect the prompt as written; do not lower them just to create variation or appear critical.
+
 ## Event example
 
 ```json
 {
-  "schema_version": "1.0",
+  "schema_version": "1.1",
   "event_id": "evt-20260901-7f3a92c1",
   "timestamp": "2026-09-01T12:00:00.000Z",
   "task_type": "coding",
   "prompt_summary": "Implement a dependency-free prompt observation kit with structured local logging.",
-  "intent_clarity": 9,
-  "context_sufficiency": 8,
-  "scope_definition": 9,
-  "constraints_quality": 9,
-  "acceptance_criteria": 8,
-  "verification_plan": 7,
+  "intent_clarity": 10,
+  "context_sufficiency": 10,
+  "scope_definition": 10,
+  "constraints_quality": 10,
+  "acceptance_criteria": 9,
+  "verification_plan": 9,
   "ambiguity_risk": "low",
   "strengths": [
-    "The requested deliverables and runtime constraints are explicit."
-  ],
-  "weaknesses": [
     {
-      "category": "missing_verification",
-      "severity": "low",
-      "message": "The exact expected report contents were not fully enumerated."
+      "category": "constraints_quality",
+      "message": "The requested deliverables and runtime constraints are explicit."
     }
   ],
-  "improvement_suggestions": [
-    "List the required report sections and one expected example."
-  ],
+  "weaknesses": [],
+  "improvement_suggestions": [],
   "execution_signals": {
     "result_status": "completed",
     "files_changed": [
@@ -94,11 +125,11 @@ Score only prompt quality. Do not lower a score because implementation was diffi
   },
   "usage": {
     "model": null,
-    "input_tokens": null,
-    "output_tokens": null,
+    "input_tokens": 280,
+    "output_tokens": 640,
     "cost_usd": null,
-    "source": "unavailable",
-    "estimation_method": null
+    "source": "estimated",
+    "estimation_method": "agent_text_heuristic_v1"
   }
 }
 ```

package/README.md

@@ -4,15 +4,19 @@ Prompt Observer is a dependency-free observability kit for professional coding-a
 
 It is designed for Vibe Coding workflows and works without a browser extension, a local model, or an additional AI API. The agent already completing the task creates the observation.
 
+[View the package on npm](https://www.npmjs.com/package/@onthink/prompt-observer)
+
 ## What it records
 
 - Prompt-quality dimensions: clarity, context, scope, constraints, acceptance criteria, and verification plan
-- Actionable weaknesses and improvement suggestions
+- Structured strengths, actionable weaknesses, and improvement suggestions
 - Observed execution signals: result status, changed files, and test outcomes
-- Model, token, and cost information only when the platform explicitly provides it
+- Exact platform usage when available, otherwise clearly labeled token estimates and model-matched cost estimates
 
 Raw prompts, raw responses, private reasoning, system instructions, and secrets are prohibited from the event format.
 
+Prompt Observer does not force criticism. A clear, sufficient prompt receives no fabricated weakness or subjective style advice; its `weaknesses` and `improvement_suggestions` arrays remain empty.
+
 ## Requirements
 
 - Node.js 20 or newer
@@ -21,14 +25,15 @@ Raw prompts, raw responses, private reasoning, system instructions, and secrets
 
 ## Quick start
 
-Clone this repository and initialize Prompt Observer in a target project:
+Initialize Prompt Observer in any target project:
 
 ```powershell
-git clone <your-repository-url>
-cd prompt-observer
-node bin/prompt-observer.mjs init "E:\path\to\your-project"
+cd "E:\path\to\your-project"
+npx @onthink/prompt-observer init .
 ```
 
+`init` adds a portable local CLI under `.prompt-observer`, so the target project can log events and generate reports without a global installation.
+
 Then add the following one-line instruction to the target project's existing agent-instruction file:
 
 ```md
@@ -52,43 +57,75 @@ Each initialized project receives:
   event.schema.json        Versioned event schema
   prompt-observer.mjs      Portable local CLI
   events.jsonl             Generated append-only event log
-  report.md                Generated analysis report
+  pricing.json             Versioned model-pricing snapshot
+  report.md                Compact GitHub-friendly report
+  report.html              Interactive offline dashboard
 ```
 
-The generated log and report are excluded by the local `.prompt-observer/.gitignore`; the contract and schema can safely be committed.
+The generated log and reports are excluded by the local `.prompt-observer/.gitignore`; the contract, schema, and pricing snapshot can safely be committed.
 
-## Commands
+## Automatic behavior
 
-Run these commands inside an initialized project:
+After the project instruction is added, the coding agent handles event creation and logging after each task. You do **not** need to run `validate` or `log` yourself during normal use.
 
-```powershell
-# Check an event before saving it
-node .prompt-observer/prompt-observer.mjs validate .prompt-observer/pending/example.json
+The agent creates a temporary event, validates it, appends it to `.prompt-observer/events.jsonl`, and removes the temporary file after a successful save.
+
+## View reports
 
-# Validate and append an event to the local JSONL log
-node .prompt-observer/prompt-observer.mjs log .prompt-observer/pending/example.json
+Generate reports from the latest 50 events:
 
-# Generate the aggregate Markdown report
+```powershell
 node .prompt-observer/prompt-observer.mjs report .
 ```
 
-The report includes average prompt health, recurring weaknesses, ambiguity risk, execution outcomes, verification trends, and platform-reported usage totals.
+This creates `.prompt-observer/report.md` and `.prompt-observer/report.html`. The Markdown report is a compact repository-friendly summary. The self-contained HTML dashboard adds KPI cards, quality and trend charts, filters, recurring strengths, weaknesses, improvement suggestions, verification results, and usage coverage. It embeds only the selected window's already-redacted analytical fields, not changed-file lists or test names.
+
+Choose another bounded window or explicitly analyze all events:
+
+```powershell
+node .prompt-observer/prompt-observer.mjs report . --limit 100
+node .prompt-observer/prompt-observer.mjs report . --all
+```
+
+The default reader streams the JSONL history and retains only the latest 50 events plus the previous 50-event comparison window. The complete append-only log remains available without making the generated reports grow forever.
+
+## Usage accuracy and estimates
+
+Prompt Observer uses this precedence:
+
+1. Metrics explicitly reported by the Agent platform are stored as `platform_reported`.
+2. When the platform exposes no usage but the Agent can see the input and final response, token counts are estimated with the documented `agent_text_heuristic_v1` method.
+3. Estimated cost is calculated only when the runtime reports an exact model ID that exists in the local versioned `pricing.json` snapshot.
+4. Unknown values remain `null` and are reported as unavailable.
+
+The dashboard always separates exact, estimated, and unavailable values. Cost estimates exclude cached tokens, tools, subscription pricing, discounts, long-context premiums, and provider-specific charges, so they must not be treated as invoices.
+
+## Manual event troubleshooting (advanced)
+
+`validate` and `log` are diagnostic commands for a temporary event that an agent has already created. They are not part of the normal setup or daily workflow, and `example.json` is not created by `init`.
+
+Use them only when inspecting a real pending event before it is logged:
+
+```powershell
+node .prompt-observer/prompt-observer.mjs validate .prompt-observer/pending/<event-id>.json
+node .prompt-observer/prompt-observer.mjs log .prompt-observer/pending/<event-id>.json
+```
 
 ## No-filesystem fallback
 
 When an agent cannot write files, the contract requires it to show the three-line insight and return a complete valid JSON event in a fenced `json` block. It must state that the event was not persisted. A later extension or integration can capture that output automatically.
 
-## npm package — coming soon
+## npm package
 
-The project is configured for npm packaging, but it is **not published yet**. The final package name will be chosen before release, preferably as a scoped name such as `@your-npm-username/prompt-observer`.
+Prompt Observer is published as [`@onthink/prompt-observer`](https://www.npmjs.com/package/@onthink/prompt-observer).
 
-After publication, the intended installation flow will be:
+Use it without installing it globally:
 
 ```powershell
-npx @your-npm-username/prompt-observer init .
+npx @onthink/prompt-observer init .
 ```
 
-Before publishing, replace `@your-npm-username` with your real npm scope and configure the repository URL in `package.json`.
+Running `init` again is safe: it preserves existing Prompt Observer files and adds only missing files.
 
 ## Development
 
@@ -97,7 +134,7 @@ npm run check
 npm test
 ```
 
-`prepublishOnly` runs both checks automatically before `npm publish`. The package uses only Node.js built-ins; JSONL is the source of truth for v1, while SQLite export is intentionally deferred to a later release.
+`prepublishOnly` runs both checks automatically before `npm publish`. The package is published publicly under the `@onthink` scope. The project uses only Node.js built-ins; JSONL remains the source of truth, while SQLite export is intentionally deferred.
 
 ## License