freshcontext-mcp 0.3.17 → 0.3.18

package/docs/OPERATIONAL_DEMO_RUNBOOK.md

@@ -0,0 +1,458 @@
+# FreshContext Operational Demo Runbook
+
+This runbook is for a live call, screen share, buyer demo, agent-builder conversation, or handoff where you need to show what FreshContext can do today.
+
+It is intentionally practical. It does not describe future Operator mode as if it already exists.
+
+## The Short Explanation
+
+FreshContext is a context gateway between retrieval and reasoning.
+
+```text
+Raw candidate context goes in.
+FreshContext evaluates freshness, usefulness, traceability, and uncertainty.
+Decision-ready context comes out.
+```
+
+Use this sentence when someone asks what it is:
+
+```text
+FreshContext tells an agent or human whether a source should be used, cited, refreshed, verified, watched, treated as background, or excluded.
+```
+
+Use this sentence when someone asks why it matters:
+
+```text
+Search and retrieval find matching information; FreshContext decides what context deserves to reach the model first.
+```
+
+## Current Operational Shape
+
+FreshContext has four working lanes today:
+
+| Lane | What It Does Today | Demo Status |
+|---|---|---|
+| Core | Normalizes signals, scores freshness, ranks, explains, prepares envelopes/provenance, and interprets decisions | Working |
+| MCP package | Exposes 21 reference tools to MCP clients over stdio | Working |
+| BYOC local demos | Lets a user provide JSON source lists and get decision-first output | Working |
+| arXiv signal proof | Shows extracted adapter signals flowing into Core decisions | Working |
+
+Two companion systems support operational trust:
+
+| Companion | Role |
+|---|---|
+| Trust Scanner | Checks package, release, public-claim, and integrity risk |
+| Ops Pulse | Checks Cloudflare/runtime/D1/cron/observability health for deployed systems |
+
+## What FreshContext Is Not Yet
+
+Do not overclaim these in a live talk:
+
+- It is not full Operator mode yet.
+- It does not crawl the web by itself in the BYOC demo.
+- It does not silently read local folders.
+- It does not replace legal, medical, tax, employment, academic, or investment review.
+- It does not production-enforce Ha-Pri v2 yet.
+- It does not make `utility.score` replace default ranking.
+- It does not require publishing or deploying to demonstrate the current local flow.
+
+## Screen-Share Prep
+
+Open two terminals:
+
+1. FreshContext MCP repo:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-mcp"
+```
+
+2. Ops Pulse repo:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-ops-pulse"
+```
+
+Optional files to open beside the terminal:
+
+- `README.md`
+- `docs/OPERATIONAL_DEMO_RUNBOOK.md`
+- `examples/sources.academic.example.json`
+- `examples/sources.jobs.example.json`
+- `docs/SOURCE_PROFILES.md`
+- `docs/CORE_API.md`
+
+## Preflight Before A Call
+
+Run this before the call, not during the first minute of the call:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-mcp"
+
+git status --short --branch
+npm run build
+npm test
+npm run smoke:stdio
+npm run trust:scan:json
+```
+
+Expected healthy signs:
+
+```text
+working tree clean
+tests pass
+smoke: 21 tools, v0.3.18
+trust scan: 0 effective fail findings
+```
+
+For Ops Pulse:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-ops-pulse"
+
+git status --short --branch
+npm run check
+npm run build
+```
+
+Expected healthy signs:
+
+```text
+working tree clean
+typecheck passes
+build passes
+```
+
+## Best Live Demo Path
+
+Use the local demos first. They are deterministic and do not need live network access.
+
+### 1. Show Bring Your Own Context
+
+Academic/citation example:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-mcp"
+npm run demo:evaluate:file -- examples/sources.academic.example.json
+```
+
+Say:
+
+```text
+This is the simple user path. The user gives FreshContext candidate sources. FreshContext does not fetch or crawl here. It evaluates what was provided and returns decision-first context.
+```
+
+Point out:
+
+- `Decision`
+- `Meaning`
+- `Action`
+- `Warnings`
+- `Freshness`
+- `Rank score`
+- `Utility`
+- `Confidence`
+- `Why`
+
+The important visible result is not the number. It is the action label:
+
+```text
+Cite as primary
+Cite as supporting
+Needs verification
+Exclude
+```
+
+### 2. Show A Normal-User Example
+
+Jobs/opportunities example:
+
+```powershell
+npm run demo:evaluate:file -- examples/sources.jobs.example.json
+```
+
+Say:
+
+```text
+The same Core works for a different source profile and intent. Academic sources age slowly; jobs age quickly. FreshContext changes the decision meaning without needing a new product.
+```
+
+Point out decisions like:
+
+```text
+Use first
+Needs refresh
+Exclude
+```
+
+### 3. Show Adapter-To-Core Proof
+
+arXiv fixture-backed adapter proof:
+
+```powershell
+npm run demo:arxiv
+```
+
+Say:
+
+```text
+This closes the adapter loop. arXiv-style XML becomes FreshContext signals, then Core evaluates those signals, then the decision helper explains what to do with them.
+```
+
+Use this exact flow:
+
+```text
+arXiv XML
+-> searchArxivSignals(...)
+-> evaluateSignals(...)
+-> interpretEvaluations(...)
+-> decision-ready output
+```
+
+This is the bridge from reference MCP tools to source-aware adapter assets.
+
+### 4. Show MCP Runtime Health
+
+Only run this if network conditions are fine, because representative MCP smoke can touch live source paths:
+
+```powershell
+npm run smoke:stdio
+```
+
+Expected:
+
+```json
+{
+  "ok": true,
+  "package_version": "0.3.18",
+  "server_version": "0.3.18",
+  "tool_count": 21
+}
+```
+
+Say:
+
+```text
+This proves the MCP package still exposes the reference tool surface. But the product is not the number of tools. The tools are proof surfaces and adapter candidates over the Core judgment layer.
+```
+
+### 5. Show Trust Scanner
+
+```powershell
+npm run trust:scan:json
+```
+
+Say:
+
+```text
+This protects release and public-claim integrity. The line that matters for the gate is effective fail count: zero.
+```
+
+You do not need to read every raw finding live. The scanner intentionally reports review items and downgrades known non-blockers.
+
+### 6. Show Ops Pulse Health
+
+In the Ops Pulse repo:
+
+```powershell
+cd "C:\Users\Immanuel Gabriel\Downloads\freshcontext-ops-pulse"
+npm run check
+npm run build
+```
+
+Say:
+
+```text
+Ops Pulse is separate from FreshContext Core. It is the operational health companion for Cloudflare/runtime/D1/cron/observability workflows.
+```
+
+## Bring Your Own Context File Shape
+
+Minimal JSON shape:
+
+```json
+{
+  "profile": "academic_research",
+  "intent": "citation_check",
+  "signals": [
+    {
+      "title": "Example source",
+      "content": "Raw retrieved content or notes...",
+      "source": "https://example.com/source",
+      "source_type": "arxiv",
+      "published_at": "2026-05-24T12:00:00.000Z",
+      "retrieved_at": "2026-05-24T13:00:00.000Z",
+      "semantic_score": 0.92
+    }
+  ]
+}
+```
+
+Run a custom file:
+
+```powershell
+npm run demo:evaluate:file -- path\to\sources.json
+```
+
+Important boundary:
+
+```text
+The file demo evaluates candidate context you provide. It does not fetch URLs, crawl websites, read folders, or run Operator mode.
+```
+
+## How To Explain The Framework
+
+Use this architecture:
+
+```text
+Adapters / retrievers / databases / agents
+-> FreshContext Signal Contract
+-> Source Profile
+-> Core evaluation
+-> Decision Helper
+-> MCP / REST / SDK / CLI / agent output
+```
+
+The deeper system split is:
+
+```text
+Core = deterministic context intelligence
+Source Profiles = how source classes age and fail
+Adapters = source intake
+Hosts = MCP, REST, SDK, CLI
+Operator = future retrieval workflow over Core
+Trust Scanner = release/package/public-claim integrity
+Ops Pulse = runtime health diagnostics
+```
+
+## How Agents Would Use This
+
+For Codex, Claude, multi-agent workflows, or database-backed assistants, FreshContext should sit between retrieval and final reasoning:
+
+```text
+Agent retrieves 30 candidate items.
+FreshContext normalizes and evaluates them.
+FreshContext returns ranked decisions and warnings.
+Agent uses the best context first and avoids stale/failed/unknown sources.
+```
+
+That means FreshContext is not trying to be another chat agent. It is the layer an agent uses before deciding what to trust.
+
+In a multi-agent system:
+
+```text
+Research agent finds candidates.
+FreshContext scores and labels candidates.
+Writing agent receives only decision-ready context.
+Review agent sees warnings, provenance, and excluded sources.
+```
+
+For database workflows:
+
+```text
+Database query returns rows or documents.
+FreshContext evaluates dates, confidence, failure states, and source profile.
+Application stores or displays the ranked decision result.
+```
+
+## What To Say If Someone Says "Is This Just Docs?"
+
+Say:
+
+```text
+No. The docs describe the boundaries, but the local commands run the actual Core pipeline. The BYOC demo reads user-provided source JSON, evaluates it, ranks it, and prints decisions. The arXiv demo proves an adapter signal path reaches Core decisions. The MCP smoke proves the package exposes 21 working tools.
+```
+
+Then run:
+
+```powershell
+npm run demo:evaluate:file -- examples/sources.jobs.example.json
+```
+
+The output is the answer.
+
+## Current Demo Script
+
+Use this order in a live talk:
+
+1. One sentence:
+
+```text
+FreshContext decides what context deserves to reach the model.
+```
+
+2. Run academic BYOC:
+
+```powershell
+npm run demo:evaluate:file -- examples/sources.academic.example.json
+```
+
+3. Run jobs BYOC:
+
+```powershell
+npm run demo:evaluate:file -- examples/sources.jobs.example.json
+```
+
+4. Run arXiv adapter proof:
+
+```powershell
+npm run demo:arxiv
+```
+
+5. Run MCP smoke if the network is stable:
+
+```powershell
+npm run smoke:stdio
+```
+
+6. Close with:
+
+```text
+The product is the judgment layer. The tools are reference adapters. The next growth path is more adapters, CLI/SDK usability, and eventually Operator mode.
+```
+
+## Current Release Boundary
+
+Current prepared package version:
+
+```text
+freshcontext-mcp@0.3.18
+```
+
+Current state:
+
+```text
+release prep merged
+package ready for release execution gate
+npm publish not done
+deploy not done
+git tag not created
+```
+
+Do not publish, deploy, or tag during a demo.
+
+## Troubleshooting
+
+If the old ChatGPT handoff chat will not open:
+
+```text
+Do not depend on it. Use this runbook plus the local docs and repo state.
+```
+
+If `npm run smoke:stdio` fails during a live call:
+
+```text
+Switch to the offline demos: demo:evaluate:file and demo:arxiv.
+```
+
+If someone asks whether FreshContext gives advice:
+
+```text
+No. It gives context judgment: citation readiness, freshness, usefulness, traceability, and uncertainty. Humans and domain systems still own final decisions.
+```
+
+If someone asks what is next:
+
+```text
+Release execution gate, then CLI/SDK usability, then more adapter extraction, then Operator mode.
+```
+

package/docs/RELEASE_INTEGRITY.md

@@ -0,0 +1,53 @@
+# FreshContext Release Integrity Notes
+
+This document describes release hardening practices for future FreshContext package and archive releases. It is a plan and checklist, not an implemented signing or SBOM system.
+
+## Before Publishing or Sharing a Package
+
+- Start from a clean working tree.
+- Confirm the package version intentionally matches the release plan.
+- Run `npm run build`.
+- Run `npm test`.
+- Run `npm run smoke:stdio`.
+- Run `npm run example:ha-pri-v2`.
+- Run `cd worker && npx tsc --noEmit`.
+- Run `npm audit --omit=dev`.
+- Run `npm audit`.
+- Run `npm pack --dry-run --json`.
+- Smoke-test the packed tarball in a temporary install:
+  - Confirm `npm start` works from the installed package.
+  - Confirm the `freshcontext-mcp` binary works from the installed package.
+  - Confirm repo-only scripts print a source-checkout notice instead of failing when examples, tests, or scripts are intentionally excluded.
+  - Confirm `dist/server.js` is present and `dist/apify.js` is absent from the MCP npm package.
+  - Confirm fresh consumer `npm audit --omit=dev` is clean.
+- Run a stale-claim scan across public docs and package-facing files.
+- Run a secret scan before sharing archives, diligence folders, or package artifacts.
+
+## Package Exclusion Checks
+
+Confirm release artifacts do not include:
+
+- Local environment files.
+- Tokens or local credential files.
+- MCP registry local credential files.
+- Cloudflare local state.
+- Local database snapshots or SQL dumps.
+- Private sale, buyer, target, outreach, or diligence documents.
+- Private data-room folders.
+- Local logs.
+- Old package tarballs.
+
+## Release Notes and Integrity Artifacts
+
+- Current prepared release notes: [`RELEASE_NOTES.md`](./RELEASE_NOTES.md).
+
+Future release hardening may include:
+
+- GitHub release notes for tagged releases.
+- Signed git tags, if signing is configured.
+- `SHA256SUMS` files for release artifacts.
+- SBOM generation for buyer or enterprise diligence.
+- npm provenance and signature review.
+- A documented token-rotation checklist for any maintainer or ownership transfer.
+
+Do not publish from a dirty working tree. Do not publish from an environment that exposes secrets in logs or command output.

package/docs/RELEASE_NOTES.md

@@ -0,0 +1,38 @@
+# FreshContext Release Notes
+
+## 0.3.18
+
+FreshContext 0.3.18 prepares the next npm package release without changing the deployed Worker or publishing automatically.
+
+### Core and Context Evaluation
+
+- Added the Core signal evaluation pipeline for normalized, freshness-ranked context results.
+- Added Source Profiles as Core metadata for source-aware policy vocabulary.
+- Added the Context Decision Helper so evaluated context can be interpreted as use, cite, verify, refresh, watch, background, or exclude.
+- Preserved the ranking boundary: `ranked.final_score` controls default ordering, while `utility.score` remains sidecar intelligence.
+
+### Bring Your Own Context
+
+- Added decision-first local demos for user-provided JSON source lists.
+- Added academic citation and jobs/opportunity sample inputs.
+- Added `npm run demo:evaluate:file` for local source-list evaluation from a cloned source checkout.
+
+### Adapter Path
+
+- Added adapter registry metadata for the 21 existing MCP tools.
+- Added additive arXiv signal extraction without changing the existing MCP `extract_arxiv` behavior.
+- Added an arXiv signal-to-decision proof using a static fixture, Core evaluation, and decision output.
+
+### Package and Release Hygiene
+
+- Hardened the npm package file allowlist.
+- Added script guards so repo-only scripts show a source-checkout notice in packed installs.
+- Isolated Apify/Crawlee from the normal MCP npm runtime package while preserving source-checkout Apify Actor support.
+- Confirmed fresh consumer installs no longer install Apify/Crawlee/file-type through the default MCP package path.
+
+### Boundaries
+
+- No npm publish in this preparation pass.
+- No git tag in this preparation pass.
+- No deployment in this preparation pass.
+- No Worker, REST handler, Core runtime, MCP tool schema, or adapter behavior changes are included in the version bump itself.

package/docs/SIGNAL_CONTRACT.md

@@ -0,0 +1,89 @@
+# FreshContext Signal Contract v1
+
+FreshContext Signal Contract v1 defines the Core shape for a retrieved signal before it is ranked, wrapped, stored, or passed to an agent workflow.
+
+It is an additive Core API. It does not change MCP tool schemas, Worker runtime behavior, D1 schema, Store scoring, feeds, or deployment behavior.
+
+## Contract Version
+
+```ts
+type SignalContractVersion = "freshcontext.signal.v1";
+```
+
+Every normalized signal includes:
+
+```ts
+contract_version: "freshcontext.signal.v1"
+```
+
+## Input Shape
+
+`FreshContextSignalInput` accepts the common fields used by adapters, agents, ranking, and future Store wiring:
+
+```ts
+interface FreshContextSignalInput {
+  id?: string;
+  source: string;
+  source_type?: string;
+  title?: string;
+  content?: string;
+  published_at?: string | null;
+  content_date?: string | null;
+  retrieved_at?: string | null;
+  semantic_score?: number;
+  date_confidence?: "high" | "medium" | "low" | "unknown";
+  freshness_confidence?: "high" | "medium" | "low";
+  status?: "success" | "partial" | "stale" | "failed" | "unknown";
+  metadata?: Record<string, unknown>;
+}
+```
+
+`published_at` is the canonical signal timestamp. `content_date` is accepted as an adapter/envelope compatibility alias.
+
+## Normalized Output
+
+`normalizeSignal(input, options?)` returns a `FreshContextSignal`:
+
+```ts
+interface FreshContextSignal {
+  contract_version: "freshcontext.signal.v1";
+  id?: string;
+  source: string;
+  source_type: string;
+  title?: string;
+  content?: string;
+  published_at: string | null;
+  retrieved_at: string;
+  semantic_score: number;
+  date_confidence: "high" | "medium" | "low" | "unknown";
+  status: "success" | "partial" | "stale" | "failed" | "unknown";
+  metadata: Record<string, unknown>;
+  reasons: string[];
+}
+```
+
+## Normalization Rules
+
+- Missing or invalid `published_at` / `content_date` becomes `published_at: null`.
+- `content_date` maps to `published_at` when `published_at` is absent.
+- Meaningfully future-dated timestamps are cleared and receive `date_confidence: "unknown"`.
+- Small clock skew is tolerated by the same Core freshness policy used by envelope scoring.
+- Failed, empty, timeout, blocked, or error-looking content becomes `status: "failed"`.
+- Missing, invalid, negative, or oversized `semantic_score` is clamped into `0..1`.
+- `metadata` is shallow-copied so normalization does not mutate caller-owned objects.
+- `reasons` records meaningful normalization changes.
+
+## Relationship to Existing Core Types
+
+The signal contract does not replace existing Core types:
+
+- `AdapterResult` remains the adapter-to-envelope input shape.
+- `FreshContext` remains the envelope output shape.
+- `FreshSignal` and `RankedSignal` remain the ranking input/output shapes.
+- `ContextUtilityInput` remains the pure context-conditioned utility primitive.
+
+The contract gives these surfaces a shared signal vocabulary without requiring Store, Worker, or MCP schema changes.
+
+## Boundary
+
+Signal Contract v1 does not determine truth, certify data, or provide legal, medical, tax, or financial advice. It provides normalized context metadata for freshness, provenance, relevance, and workflow review.