freshcontext-mcp 0.3.18 → 0.3.20

package/METHODOLOGY.md

@@ -0,0 +1,381 @@
+# FreshContext Data Intelligence Methodology
+**Version 1.2 — May 2026**
+*Authored by Immanuel Gabriel (Prince Gabriel) — Grootfontein, Namibia*
+
+---
+
+## What This Document Is
+
+This document formally describes the data collection, scoring, ranking, storage, and provenance methodology underlying FreshContext.
+
+It exists for four audiences:
+
+1. **Technical integrators** — teams embedding FreshContext into their agent infrastructure who need to understand what the data represents and how it is scored.
+2. **Agent/retrieval system builders** — teams designing retrieval pipelines that need temporal relevance instead of undated context.
+3. **Auditors and reviewers** — people verifying that timestamped AI context is represented honestly and reproducibly.
+4. **Future licensing or platform partners** — entities evaluating FreshContext as infrastructure, who need to audit the methodology that makes the data defensible.
+
+---
+
+## Section 1: Core Methodology and Source Collection
+
+### 1.1 Architecture
+
+FreshContext Core methodology describes the signal contract and temporal scoring primitives that can be used by MCP servers, APIs, CLIs, dashboards, agents, or internal retrieval systems.
+
+The Core methodology covers:
+
+- **Signal schema** — source, content, timestamps, confidence, adapter identity
+- **Source/provenance** — where the observation came from and how it was retrieved
+- **Published/content date** — when the source claims the content became true or available
+- **Retrieved timestamp** — when FreshContext observed the content
+- **Confidence** — how reliable the timestamp extraction is
+- **Decay-Adjusted Relevancy (DAR)** — temporal utility after source-specific decay
+- **Failure honesty** — failed adapters must not be promoted as fresh successful context
+- **Ranking/explain primitives** — fields that let agents and systems explain why a signal ranked where it did
+
+FreshContext also supports a Store/Ledger methodology for systems that persist recurring signals over time. The production Worker implementation uses Cloudflare runtime pieces for MCP transport, KV cache policy, rate limiting, D1 persistence, feeds, cron collection, and deployment concerns. Those runtime concerns are implementation layers, not requirements for every FreshContext-compatible system.
+
+### 1.2 Store / Ledger Collection Layer
+
+The Store/Ledger methodology describes a continuous data collection pipeline that can run on Cloudflare's global edge infrastructure. A deployment may execute scheduled collection via cron and query watched definitions stored in D1 or another durable store.
+
+Each watched query specifies:
+- **Adapter** — the data source to query (e.g., `hackernews`, `jobs`, `reposearch`)
+- **Query** — the search term or URL
+- **User ID** — the profile this query serves
+- **Filters** — optional parameters (location, exclusion terms, etc.)
+
+This D1 cron ledger is one implementation layer and future Store direction. It is not required for every FreshContext-compatible envelope implementation.
+
+### 1.3 Example Adapter / Source Classes
+
+FreshContext currently has:
+
+- A reference MCP implementation with `evaluate_context` and 21 read-only reference adapters
+- Separate feed products such as Fresh HN Feed and Fresh Jobs Feed
+- A Store/Ledger methodology for systems that collect recurring signals over time
+
+The following table describes example source classes used by FreshContext implementations. Not every source class is necessarily collected by every cron/feed deployment.
+
+| Adapter class | Source | Auth Required | Typical Update Frequency |
+|---|---|---|---|
+| `hackernews` | Hacker News Algolia API | None | Real-time |
+| `jobs` | Remotive API | None | Continuous |
+| `reposearch` | GitHub Search API | Optional (rate limit) | Real-time |
+| `github` | GitHub Repository API | Optional | Real-time |
+| `reddit` | Reddit JSON API | None | Real-time |
+| `yc` | YC Open Source API | None | Per batch cycle |
+| `packagetrends` | npm Registry + npm Downloads API | None | Per publish |
+| `finance` | Stooq quote API | None | Market hours / quote feed cadence |
+| `producthunt` | Product Hunt launch data | Token when API-backed | Launch cadence |
+| `changelog` | GitHub Releases / npm package metadata | Optional | Per release |
+| `arxiv` / `scholar` | Academic sources | None | Publication cadence |
+| `gdelt` | GDELT global news | None | 15-minute feed cadence |
+| `govcontracts` / `gebiz` | Government procurement datasets | None | Dataset cadence |
+| `sec_filings` | SEC EDGAR filings | None | Filing cadence |
+
+FreshContext adapters operate on publicly accessible or publicly documented data sources. Most reference adapters require no credentials. Some APIs may optionally use tokens for rate limits or official API access, but FreshContext-compatible adapters should not require private user data unless explicitly documented by the implementation. All fetch requests include a `User-Agent` header identifying the FreshContext crawler where the runtime/source supports it.
+
+### 1.4 Content Hash Deduplication
+
+Before any signal is stored, the platform computes a 32-bit rolling hash of the raw content. If the most recent stored result for a given watched query carries an identical hash, the current result is discarded. This prevents storing unchanged content across cron cycles.
+
+### 1.5 Semantic Deduplication
+
+Beyond exact-match deduplication, FreshContext implements semantic deduplication to prevent the same underlying story appearing as multiple signals because it was covered by multiple sources (e.g., the same GitHub release appearing in both HN and Reddit).
+
+The semantic fingerprint is computed as follows:
+
+1. Extract the first canonical URL from the raw content
+2. Extract the first ISO 8601 publication date from the raw content
+3. Extract and normalise the first substantive line (title) — lowercased, punctuation stripped, truncated to 80 characters
+4. Concatenate: `normalised_title|canonical_url|publication_date`
+5. Compute SHA-256 of the concatenated string
+6. Retain the first 16 hex characters as the fingerprint
+
+If any signal stored within the preceding 48 hours carries an identical fingerprint, the new result is discarded. The 48-hour window is configurable.
+
+---
+
+## Section 2: Temporal Scoring — The DAR Engine
+
+### 2.1 Overview
+
+The Decay-Adjusted Relevancy (DAR) engine scores every collected signal on two axes:
+
+- **R_0 (Base Score)** — semantic relevancy of the content against the user's profile, independent of time
+- **R_t (Decay-Adjusted Score)** — R_0 adjusted for how much time has elapsed since the content was published
+
+The final stored `rt_score` is what drives signal ranking in briefings and the intelligence feed.
+
+FreshContext measures temporal utility, not truth. A source can be valid and still have low utility for the current query if it is stale. A source can be fresh but low-confidence if its timestamp is missing, malformed, inferred, or contradicted.
+
+### 2.2 Base Score Calculation (R_0)
+
+R_0 is the starting relevance or utility before temporal decay. In the Store/Feed implementation, R_0 is computed by matching content against the user profile:
+
+```
+R_0 = baseline (40)
+    + vital_keyword_matches × 15   [capped at +35]
+    + skill_keyword_matches × 3    [capped at +15]
+    + location_accessibility_bonus  [+8 if remote/accessible]
+    - error_penalty                 [−40 if content is empty/error]
+```
+
+Vital keywords are drawn from the `targets` field of the user profile — job titles, company names, and technology domains the user is specifically tracking.
+
+Skill keywords are drawn from the `skills` field — the user's technical competencies. A match here adds relevancy signal but at lower weight than a direct target match.
+
+The location accessibility bonus is applied when the content explicitly mentions "remote", "worldwide", "anywhere", or the user's stated location. This is not a geographic filter — it is a signal boost for content that is accessible to the user regardless of their physical location.
+
+**Hard exclusions:** If any term from the `exclusion_terms` list appears in the content, R_0 is forced to zero. The result is still stored (for audit purposes) but marked `is_relevant = 0`.
+
+This profile formula is a Store/Feed implementation example, not the only possible way to produce base relevance. For Core/MCP envelope scoring, R_0 may be normalised to 100. For feed/ranking systems, R_0 may come from semantic relevance, profile relevance, adapter-specific relevance, or another documented scoring layer.
+
+### 2.3 Context-Conditioned Utility
+
+FreshContext scoring is context-conditioned. The same signal can have different utility depending on the user, query, agent, platform, or workflow requesting it.
+
+In the Store/Feed implementation, this context is represented by `R_0`, the base relevance or utility score before temporal decay. `R_0` may be computed from profile targets, query terms, semantic relevance, adapter-specific relevance, or another documented scoring layer.
+
+The DAR function then applies temporal pressure:
+
+```
+R_t = R_0 · e^(-λt)
+```
+
+This means FreshContext does not treat freshness as a standalone ranking signal. A fresh but irrelevant signal should not outrank an older but highly relevant signal unless the source policy and use case justify it.
+
+FreshContext Core exposes a pure context utility primitive for this direction:
+
+```
+U(q, s, t) = R(q, s) · e^(-λt) · C_date · C_status
+```
+
+Where:
+- `q` is the requester context: user, query, agent, platform, or workflow
+- `s` is the signal or database record
+- `R(q, s)` is contextual relevance between the request and the signal
+- `λ` is the source-specific decay constant
+- `t` is signal age
+- `C_date` is a timestamp-confidence factor
+- `C_status` is a failure/partial/success factor
+
+This is an extension of the DAR methodology, not a replacement for it. The purpose is to support systems where FreshContext runs over databases, feeds, retrieved documents, or agent memory and ranks information by both relevance and temporal utility. It does not imply vector search, multi-agent orchestration, or a hosted context store.
+
+### 2.4 Decay Function (R_t)
+
+```
+R_t = R_0 · e^(-λt)
+```
+
+Where:
+- `λ` = source-specific decay constant (per hour)
+- `t` = hours elapsed since `published_at` / `content_date`
+- `R_t` = current temporal utility score
+
+If `published_at` / `content_date` cannot be extracted, the system must not pretend the signal is fresh. Core-compatible envelope scoring SHOULD use `freshness_score: null` and low confidence. Store/feed systems MAY apply a conservative fallback assumption, such as one source half-life, but must mark confidence low and explain the assumption.
+
+### 2.5 Source Decay Constants (λ)
+
+These constants are reference/default calibration values for how quickly signals from each source class lose temporal utility:
+
+| Source | λ (per hour) | Half-life |
+|---|---|---|
+| Hacker News | 0.050 | ~14 hours |
+| Reddit | 0.010 | ~3 days |
+| Product Hunt | 0.010 | ~3 days |
+| Job listings | 0.005 | ~6 days |
+| Financial data | 0.001 | ~29 days |
+| YC companies | 0.001 | ~29 days |
+| Package trends | 0.0005 | ~58 days |
+| GitHub repositories | 0.0002 | ~5 months |
+| Academic papers | 0.00005 | ~1.6 years |
+
+These constants are reference defaults used by the FreshContext methodology and may be tuned by implementation. Hosted or private deployments may use calibrated variants per source, query type, or user profile. The calibration process and production tuning may be proprietary, even when public reference defaults are documented.
+
+### 2.6 Entropy Classification
+
+Each signal is classified into one of three entropy states based on its position on the decay curve:
+
+| State | Condition | Interpretation |
+|---|---|---|
+| `low` | `t < half_life / 2` | Signal near peak value — act now |
+| `stable` | `t < 1.5 × half_life` | Usable signal — monitor |
+| `high` | `t ≥ 1.5 × half_life` | Significantly degraded — verify before acting |
+
+Entropy labels describe signal decay state, not confidence level. A high-entropy signal may still be factually accurate, but it has lost temporal utility for current retrieval unless reinforced by newer evidence.
+
+### 2.7 Relevancy Threshold
+
+Signals with `rt_score < 35` are stored with `is_relevant = 0`. They remain in the database for audit and historical analysis but are excluded from briefings and the intelligence feed by default. The threshold is configurable per profile.
+
+### 2.8 Failure Honesty
+
+Failed adapters must not be promoted by freshness scoring. Empty, blocked, timeout, malformed, rate-limited, access-denied, or error-only outputs reduce R_0 or mark the signal status as failed/unknown.
+
+A failed result should not receive high confidence. A failed result should not produce `Score: 100/100`. Partial composites should preserve successful upstream results while marking failures explicitly.
+
+---
+
+## Section 3: FreshContext Store / Ledger Methodology
+
+### 3.1 The Ha-Pri Audit Signature
+
+Every signal stored in a FreshContext Store/Ledger deployment carries a `ha_pri_sig` — a SHA-256 audit signature computed as:
+
+```
+SHA-256( result_id + ":" + content_hash + ":" + "FRESHCONTEXT_DAR_V1" )
+```
+
+In Ha-Pri v1, this signature is a provenance stamp and audit reference for stored signals. It binds the result ID, the current content hash, and the engine version. It is not yet a full tamper-enforcement system: the current `content_hash` source is the existing rolling `result_hash`, and signatures are not recomputed on read to reject modified rows.
+
+Ha-Pri v1 serves three purposes:
+
+1. **Provenance reference** — the signature binds the result ID, current rolling content hash, and engine/version marker so the stored signal can be audited against the v1 formula.
+2. **Scoring lineage** — the signature records the scoring/signature formula used when the row was written.
+3. **Licensing / audit reference** — when FreshContext data is provided to a third party under licence, the `ha_pri_sig` column gives a stable reference for what was stored and delivered.
+
+Ha-Pri v1 is not hard tamper enforcement. It is not recomputed on read, it signs the existing rolling result_hash (`result_hash`) rather than canonical content SHA-256, and it does not reject rows. Ha-Pri v2 is the planned/additive path for stronger verification.
+
+Future Ha-Pri v2 may add canonical content SHA-256, stronger canonicalization, and explicit verification/rejection on read. That hardening is separate from the current v1 provenance stamp.
+
+Ha-Pri v1 is the provenance layer and the foundation for a stronger integrity layer, while DAR and context-conditioned utility are the ranking/scoring layer.
+
+### 3.2 D1 Historical Ledger
+
+The `scrape_results` table functions as a **Contextual Ledger** — not merely a cache, but a time-series record of intelligence signals with full provenance.
+
+This Store/Ledger methodology is not required for basic FreshContext-compatible envelope implementations. It is the methodology for systems that persist recurring signals and want auditability over time.
+
+Key properties of the ledger:
+- Scored signal material is treated as immutable once written; consumption metadata such as `is_new` may be updated
+- Every row carries a `scraped_at` timestamp with second precision
+- Every row carries a `published_at` date extracted from content (where available)
+- The ledger accumulates continuously at 6-hour intervals regardless of active user sessions
+- The ledger enables time-travel queries: "what was the intelligence landscape for topic X at date Y?"
+
+### 3.3 Schema Reference
+
+```sql
+scrape_results (
+  id                  TEXT PRIMARY KEY,    -- sr_{timestamp}_{random}
+  watched_query_id    TEXT,                -- FK → watched_queries.id
+  adapter             TEXT,                -- source adapter name
+  query               TEXT,                -- the search term used
+  raw_content         TEXT,                -- scraped content (max 8000 chars)
+  result_hash         TEXT,                -- 32-bit rolling hash of raw_content
+  semantic_fingerprint TEXT,               -- 16-char SHA-256 of normalised title|url|date
+  is_new              INTEGER,             -- 1 until consumed by briefing
+  scraped_at          TEXT,                -- ISO 8601 UTC timestamp
+  published_at        TEXT,                -- extracted content publication date
+  relevancy_score     INTEGER,             -- = round(rt_score), 0-100
+  is_relevant         INTEGER,             -- 1 if rt_score >= 35, else 0
+  base_score          INTEGER,             -- R_0 semantic score, 0-100
+  rt_score            REAL,                -- R_t decay-adjusted score, 0-100
+  ha_pri_sig          TEXT,                -- SHA-256 audit signature (64 hex chars)
+  entropy_level       TEXT                 -- 'low' | 'stable' | 'high'
+)
+```
+
+---
+
+## Section 4: The Intelligence Feed
+
+### 4.1 Endpoint
+
+```
+GET /v1/intel/feed/{profile_id}
+```
+
+Optional parameters:
+- `limit` — maximum signals to return (default: 20)
+- `min_rt` — minimum rt_score filter (default: 0)
+
+### 4.2 Response Structure
+
+```json
+{
+  "feed_metadata": {
+    "profile_id": "default",
+    "generated_at": "2026-04-14T09:00:00Z",
+    "signal_count": 18,
+    "version": "freshcontext-1.2"
+  },
+  "signals": [
+    {
+      "signal_id": "sr_1744628412_a3f7b",
+      "source": "hackernews",
+      "label": "HN: MCP Servers",
+      "content": {
+        "preview": "...",
+        "url": "mcp server 2026"
+      },
+      "intelligence_stamps": {
+        "scraped_at": "2026-04-14T08:12:00Z",
+        "published_at": "2026-04-14",
+        "base_score": 78,
+        "rt_score": 61.4,
+        "entropy_level": "stable",
+        "ha_pri_sig": "a3f7b2c1d4e5f6a7b8c9d0e1f2a3b4c5..."
+      }
+    }
+  ]
+}
+```
+
+### 4.3 LLM Integration
+
+The intelligence feed is designed to be consumed directly by any language model or AI agent without modification. The `intelligence_stamps` block gives the agent everything it needs to reason about data freshness:
+
+- `rt_score` — a single number representing current signal value
+- `entropy_level` — human-readable decay state
+- `published_at` — the actual content date (not the retrieval date)
+- `ha_pri_sig` — provenance reference the agent can cite
+
+This is the core value proposition: **AI agents get grounded, timestamped, scored intelligence rather than undated web content of unknown age.**
+
+MCP is one interface over this methodology, not the whole system. The same scoring, timestamp, confidence, and provenance primitives can support APIs, CLIs, npm packages, dashboards, agents, and internal services.
+
+---
+
+## Section 5: Asset Summary
+
+For technical integrators, auditors, and future platform partners:
+
+**What FreshContext owns:**
+
+1. **The FreshContext Specification v1.2** (MIT licence, open standard) — defines the envelope format, confidence levels, structured JSON form, freshness score behavior, and failure-honesty requirements. Timestamped in the public GitHub repository.
+
+2. **The DAR Engine** — the exponential decay scoring methodology with source-specific λ reference defaults and calibrated production tuning.
+
+3. **The Semantic Fingerprinting Method** — the three-field normalisation and SHA-256 fingerprinting approach for cross-adapter deduplication.
+
+4. **The Ha-Pri Audit Signature scheme** — the provenance stamp and audit reference that binds stored row material to the current v1 formula; stronger tamper-evidence is the future additive v2 path.
+
+5. **The Store / Ledger design** — support for recurring watched queries, historical signal accumulation, D1-backed storage, and time-series auditability.
+
+6. **The Reference Implementation** — `freshcontext-mcp@0.3.19`, the `evaluate_context` MCP interface, and 21 read-only reference adapters, listed on npm and the MCP Registry. The hosted Worker endpoint is a separate deployment surface.
+
+---
+
+## Changelog
+
+### Version 1.2 — May 2026
+- Clarified Core methodology vs Store/Ledger methodology.
+- Preserved DAR as the mathematical scoring backbone.
+- Updated reference implementation language for `evaluate_context` plus 21 MCP reference adapters.
+- Reframed source decay constants as reference defaults/calibration values.
+- Added failure-honesty methodology.
+- Added context-conditioned utility as a Core scoring primitive.
+- Clarified missing timestamp behavior.
+- Clarified MCP as one interface, not the whole system.
+
+### Version 1.1 — April 2026
+- Existing methodology version.
+
+---
+
+*"The work isn't gone. It's just waiting to be continued."*
+*— Prince Gabriel, Grootfontein, Namibia*

package/README.md

@@ -6,7 +6,7 @@ Claude had no idea. It presented everything with the same confidence.
 
 That's the problem freshcontext fixes.
 
-This repository is the integrated FreshContext MCP/Core package. Core is the context-integrity layer that scores, ranks, explains, and wraps retrieved context before it reaches an LLM or agent; MCP is the primary reference/interface implementation.
+This repository is the integrated FreshContext Core/MCP package. FreshContext is the context judgment layer between retrieval and reasoning. Core is the reusable engine that scores, ranks, explains, and turns candidate context into decision-ready context; MCP is the first live host/interface over that engine.
 
 [![npm version](https://img.shields.io/npm/v/freshcontext-mcp)](https://www.npmjs.com/package/freshcontext-mcp)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
@@ -30,7 +30,16 @@ That's not hallucination. That's correct summarization of corrupted retrieval.
 
 ## The layer
 
-FreshContext is a **temporal correction layer for retrieval systems**. One math correction applied before context reaches the LLM:
+FreshContext is **context integrity infrastructure for AI agents and retrieval systems**. It sits between retrieval and reasoning:
+
+```text
+candidate context
+  -> FreshContext Core
+  -> decision-ready context
+  -> model / agent / app
+```
+
+FreshContext evaluates freshness, source profile, confidence, utility, provenance material, and failure honesty before context reaches the LLM. The temporal core uses Decay-Adjusted Relevancy:
 
 ```
 R_t = R_0 · e^(−λt)
@@ -41,9 +50,9 @@ R_t = R_0 · e^(−λt)
 - `t` — hours elapsed since publication
 - `R_t` — decay-adjusted relevancy at query time
 
-That's the whole fix. No model swap. No re-embedding. No re-indexing. The layer drops onto whatever retrieval pipeline you already have.
+That's the core correction. No model swap. No re-embedding. No re-indexing. The layer drops onto whatever retrieval pipeline you already have.
 
-**The layer is the product.** The 21 tools shipped with this repo are reference adapters demonstrating compatibility — useful, but commodity. The DAR engine, the freshness envelope, and the FreshContext Specification are the moat.
+**The layer is the product.** The named adapters shipped with this repo demonstrate compatibility across different source classes. The DAR engine, the freshness envelope, Source Profiles, and the FreshContext Specification are the moat.
 
 ---
 
@@ -70,17 +79,62 @@ The FreshContext Specification v1.2 is published as an open standard under MIT l
 
 ## Architecture boundary
 
-FreshContext Core is the reusable center of the current integrated package. It owns freshness scoring, envelope formatting, failure guards, shared types, rank/explain primitives, and the context-conditioned utility primitive.
+FreshContext Core is the reusable center of the current integrated package. It owns signal normalization, freshness scoring, Source Profiles, decision output, envelope formatting, failure guards, shared types, rank/explain primitives, and the context-conditioned utility primitive.
 
-MCP is the primary reference/interface implementation over Core. Claude Desktop is supported, but not required. The 21 MCP tools in this repo are reference adapters and a live interface for using the system.
+MCP is the primary reference/interface implementation over Core. Claude Desktop is supported, but not required. The MCP tool surface exposes named reference adapters and a live interface for using the system.
 
 The production Cloudflare Worker now uses Core-backed envelope generation. Worker-specific concerns remain outside Core: MCP transport, runtime guards, KV cache policy, cache metadata injection, JSON parse/replace cache helpers, D1 feeds, cron, rate limiting, and Store/feed scoring/provenance.
 
+See [Core / MCP Boundary](./docs/CORE_MCP_BOUNDARY.md) for the current package boundary and the staged path toward a future standalone Core package.
+
 ---
 
-## The intelligence feed
+## Primary MCP interface
 
-Beyond the per-call envelope, the production FreshContext deployment exposes a continuous, decay-scored, deduplicated feed:
+The clearest MCP path is `evaluate_context`.
+
+It accepts candidate context from any retriever, agent, database, local script, note parser, or adapter output:
+
+```json
+{
+  "profile": "academic_research",
+  "intent": "citation_check",
+  "signals": [
+    {
+      "title": "Example source",
+      "content": "Candidate context text...",
+      "source": "https://example.com/source",
+      "source_type": "arxiv",
+      "published_at": "2026-05-24T12:00:00.000Z",
+      "retrieved_at": "2026-05-24T13:00:00.000Z",
+      "semantic_score": 0.92
+    }
+  ]
+}
+```
+
+FreshContext returns decision-first output:
+
+- Decision
+- Meaning
+- Action
+- Warnings
+- Source
+- Freshness
+- Rank score
+- Utility
+- Confidence
+- Why
+
+`evaluate_context` does not fetch URLs, crawl, scrape, browse, read folders, or call adapters. It only evaluates candidate context the caller provides.
+
+Current boundary: `evaluate_context` is part of the published npm/local stdio MCP server and has been verified on the hosted Cloudflare Worker MCP endpoint at `0.3.19 / 22 tools`. The Worker remains a separate deployment surface, so future package interfaces should be re-verified remotely before being claimed live.
+
+---
+
+## Advanced Worker/feed surface
+
+Beyond the per-call Core/MCP paths, the production Worker deployment exposes a continuous, decay-scored, deduplicated feed. This is an advanced deployment surface, not the required way to use FreshContext Core:
 
 ```
 GET /v1/intel/feed/:profile_id?limit=20&min_rt=0
@@ -94,7 +148,7 @@ Production endpoint: `https://freshcontext-mcp.gimmanuel73.workers.dev`
 
 ## Reference adapters
 
-The repo ships 21 tools demonstrating how to make any data source FreshContext-compatible. Useful as drop-in tools, but the value is the layer above them.
+The repo ships named reference adapters that demonstrate how different source classes can become FreshContext-compatible. Each adapter keeps its own name because it represents a source boundary; the adapter count is operational proof, not the product headline.
 
 ### Intelligence
 | Adapter | What it returns |
@@ -128,7 +182,7 @@ The repo ships 21 tools demonstrating how to make any data source FreshContext-c
 | `extract_finance_landscape` | 5 | Finance + HN + Reddit + GitHub + changelog |
 | `extract_company_landscape` | 5 | The full picture on any company |
 
-### Unique — not available in any other MCP server
+### Official, regulatory, and procurement sources
 | Adapter | Source | What it returns |
 |---|---|---|
 | `extract_changelog` | GitHub Releases / npm / auto-discover | Update history from any repo, package, or website |
@@ -141,6 +195,8 @@ The repo ships 21 tools demonstrating how to make any data source FreshContext-c
 
 ## Quick start
 
+For Claude Desktop, Codex, `npx`, global npm, and source-checkout setup, see the concise [client setup guide](./docs/CLIENT_SETUP.md).
+
 ### Cloud (no install)
 
 Add to your Claude Desktop config and restart:
@@ -290,6 +346,14 @@ Minimal shape:
 
 This local demo does not fetch URLs, crawl, or read folders. It evaluates candidate context you provide and returns decision-first output: Decision, Meaning, Action, Warnings, and supporting metrics.
 
+In an MCP client, use `evaluate_context` when you already have candidate context from another retriever, database, agent, or script:
+
+```text
+Use evaluate_context with profile "academic_research", intent "citation_check", and these candidate signals: [...]
+```
+
+Use the named reference adapters when you want FreshContext's current MCP package to fetch public source examples for you.
+
 **Should I build this idea?**
 ```
 Use extract_idea_landscape with idea "procurement intelligence saas"
@@ -325,7 +389,7 @@ The reference implementation runs on Cloudflare's global edge:
 | `/` | GET | Service info + endpoint list |
 | `/health` | GET | Liveness check |
 | `/mcp` | POST | MCP JSON-RPC transport |
-| `/demo` | GET | Live before/after demo (no API key required) |
+| `/demo` | GET | Live before/after demo (no auth token required) |
 | `/briefing` | GET | Latest stored briefing |
 | `/v1/intel/feed/:profile_id` | GET | DAR-scored intelligence feed |
 | `/watched-queries` | GET | List all watched queries |
@@ -343,26 +407,31 @@ Production: `https://freshcontext-mcp.gimmanuel73.workers.dev`
 ## Roadmap
 
 - [x] FreshContext Specification v1.2 published (MIT, open standard)
-- [x] DAR engine with proprietary λ constants (v0.3.18)
+- [x] DAR engine with source-specific lambda constants (v0.3.19)
 - [x] Ha-Pri v1 provenance signatures on stored signals
 - [x] Semantic deduplication via fingerprinting
 - [x] Live before/after demo at `/demo`
-- [x] METHODOLOGY.md — formal IP and engineering documentation
-- [x] 21 reference tools across intelligence, competitive research, market data, and composites
+- [x] METHODOLOGY.md — methodology and engineering documentation
+- [x] Named reference adapters across intelligence, competitive research, market data, and composites
+- [x] Generic MCP `evaluate_context` tool for caller-provided candidate context
 - [x] Core-backed envelope generation shared by npm/MCP and the Cloudflare Worker
 - [x] Cloudflare Workers deployment — global edge, KV cache, KV rate limiting
-- [x] Listed on official MCP Registry, Apify Store, npm
-- [ ] Ha-Pri v2 hardened canonical content hash verification
+- [x] Published on npm and listed for MCP usage; Apify/feed assets are separated from the normal MCP runtime package
+- [x] Ha-Pri v2 Core helper and deterministic golden vectors
+- [x] Ha-Pri v2 production-enforcement design document
+- [ ] Ha-Pri v2 Worker/D1 production enforcement
 - [x] GitHub Actions release workflow — manual or `v*` tag-triggered npm publish path
 - [ ] Webhook triggers — push high-entropy signals on threshold
 - [ ] Dashboard — React frontend for the D1 intelligence pipeline
 - [ ] GKG upgrade for `extract_gdelt` — tone scores, goldstein scale, event codes
 
+Future work is organized in [FreshContext Future Lanes](./docs/FUTURE_LANES.md). Roadmap items are not live product claims until implemented and validated.
+
 ---
 
 ## Contributing
 
-PRs welcome. New adapters are the highest-value contribution — see `src/adapters/` for the pattern and [`FRESHCONTEXT_SPEC.md`](./FRESHCONTEXT_SPEC.md) for the contract any adapter must fulfil.
+PRs welcome. The highest-value contributions improve the caller-provided context path, decision output, host integrations, and FreshContext-compatible signal quality. New reference adapters are useful when they preserve source boundaries and emit timestamped, failure-honest context — see `src/adapters/` for examples and [`FRESHCONTEXT_SPEC.md`](./FRESHCONTEXT_SPEC.md) for the compatibility contract.
 
 If you're building something FreshContext-compatible, open an issue and we'll add you to the ecosystem list.
 
@@ -377,7 +446,6 @@ If you're building something FreshContext-compatible, open an issue and we'll ad
 - [Dependency diligence notes](./docs/DEPENDENCY_DILIGENCE.md)
 - [Release integrity notes](./docs/RELEASE_INTEGRITY.md)
 - [Release notes](./docs/RELEASE_NOTES.md)
-- [Operational demo runbook](./docs/OPERATIONAL_DEMO_RUNBOOK.md)
 
 ---
 
@@ -392,4 +460,4 @@ MIT
 
 ---
 
-**Also on:** [Apify Store](https://apify.com/prince_gabriel/freshcontext-mcp) · [MCP Registry](https://registry.modelcontextprotocol.io) · [npm](https://www.npmjs.com/package/freshcontext-mcp)
+**Also on:** [MCP Registry](https://registry.modelcontextprotocol.io) · [npm](https://www.npmjs.com/package/freshcontext-mcp)

package/dist/adapters/arxiv.js

@@ -1,3 +1,4 @@
+import { validateUrl } from "../security.js";
 const USER_AGENT = "freshcontext-mcp/0.1.7 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)";
 const DEFAULT_ARXIV_SIGNAL_SCORE = 0.8;
 function buildArxivApiUrl(input, maxResults = 10) {
@@ -6,7 +7,7 @@ function buildArxivApiUrl(input, maxResults = 10) {
         ? Math.max(1, Math.min(Math.trunc(maxResults), 50))
         : 10;
     return trimmed.startsWith("http")
-        ? trimmed
+        ? validateUrl(trimmed, "arxiv")
         : `https://export.arxiv.org/api/query?search_query=all:${encodeURIComponent(trimmed)}&start=0&max_results=${safeMaxResults}&sortBy=relevance&sortOrder=descending`;
 }
 async function fetchArxivXml(apiUrl) {

package/dist/adapters/changelog.js

@@ -1,3 +1,4 @@
+import { validateUrl } from "../security.js";
 /**
  * Changelog adapter — extracts update history from any product or repo.
  *
@@ -193,7 +194,8 @@ export async function changelogAdapter(options) {
         return fetchNpmChangelog(input, maxLength);
     }
     // GitHub repo URL → use releases API
-    const ghMatch = input.match(/github\.com\/([^/]+)\/([^/?\s]+)/);
+    const safeInput = validateUrl(input, "changelog");
+    const ghMatch = safeInput.match(/github\.com\/([^/]+)\/([^/?\s]+)/);
     if (ghMatch) {
         try {
             return await fetchGitHubReleases(ghMatch[1], ghMatch[2], maxLength);
@@ -203,5 +205,5 @@ export async function changelogAdapter(options) {
         }
     }
     // Any other URL → discover changelog
-    return discoverChangelog(input, maxLength);
+    return discoverChangelog(safeInput, maxLength);
 }

package/dist/adapters/finance.js

@@ -30,7 +30,7 @@ async function fetchStooqQuote(ticker) {
     const url = `https://stooq.com/q/l/?s=${encodeURIComponent(stooqSymbol.toLowerCase())}&f=sd2t2ohlcv&h&e=json`;
     const res = await fetch(url, {
         headers: {
-            "User-Agent": "freshcontext-mcp/0.3.17",
+            "User-Agent": "freshcontext-mcp/0.3.19",
             "Accept": "application/json",
         },
     });

package/dist/adapters/gdelt.js

@@ -11,7 +11,7 @@
  */
 const HEADERS = {
     "Accept": "application/json",
-    "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+    "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
 };
 function parseGdeltDate(raw) {
     if (!raw)

package/dist/adapters/gebiz.js

@@ -20,7 +20,7 @@ const DATASET_ID = "d_acde1106003906a75c3fa052592f2fcb";
 const BASE_URL = "https://data.gov.sg/api/action/datastore_search";
 const HEADERS = {
     "Accept": "application/json",
-    "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+    "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
 };
 function formatDate(raw) {
     if (!raw)