freshcontext-mcp 0.3.19 → 0.3.20

package/METHODOLOGY.md

@@ -0,0 +1,381 @@
+# FreshContext Data Intelligence Methodology
+**Version 1.2 — May 2026**
+*Authored by Immanuel Gabriel (Prince Gabriel) — Grootfontein, Namibia*
+
+---
+
+## What This Document Is
+
+This document formally describes the data collection, scoring, ranking, storage, and provenance methodology underlying FreshContext.
+
+It exists for four audiences:
+
+1. **Technical integrators** — teams embedding FreshContext into their agent infrastructure who need to understand what the data represents and how it is scored.
+2. **Agent/retrieval system builders** — teams designing retrieval pipelines that need temporal relevance instead of undated context.
+3. **Auditors and reviewers** — people verifying that timestamped AI context is represented honestly and reproducibly.
+4. **Future licensing or platform partners** — entities evaluating FreshContext as infrastructure, who need to audit the methodology that makes the data defensible.
+
+---
+
+## Section 1: Core Methodology and Source Collection
+
+### 1.1 Architecture
+
+FreshContext Core methodology describes the signal contract and temporal scoring primitives that can be used by MCP servers, APIs, CLIs, dashboards, agents, or internal retrieval systems.
+
+The Core methodology covers:
+
+- **Signal schema** — source, content, timestamps, confidence, adapter identity
+- **Source/provenance** — where the observation came from and how it was retrieved
+- **Published/content date** — when the source claims the content became true or available
+- **Retrieved timestamp** — when FreshContext observed the content
+- **Confidence** — how reliable the timestamp extraction is
+- **Decay-Adjusted Relevancy (DAR)** — temporal utility after source-specific decay
+- **Failure honesty** — failed adapters must not be promoted as fresh successful context
+- **Ranking/explain primitives** — fields that let agents and systems explain why a signal ranked where it did
+
+FreshContext also supports a Store/Ledger methodology for systems that persist recurring signals over time. The production Worker implementation uses Cloudflare runtime pieces for MCP transport, KV cache policy, rate limiting, D1 persistence, feeds, cron collection, and deployment concerns. Those runtime concerns are implementation layers, not requirements for every FreshContext-compatible system.
+
+### 1.2 Store / Ledger Collection Layer
+
+The Store/Ledger methodology describes a continuous data collection pipeline that can run on Cloudflare's global edge infrastructure. A deployment may execute scheduled collection via cron and query watched definitions stored in D1 or another durable store.
+
+Each watched query specifies:
+- **Adapter** — the data source to query (e.g., `hackernews`, `jobs`, `reposearch`)
+- **Query** — the search term or URL
+- **User ID** — the profile this query serves
+- **Filters** — optional parameters (location, exclusion terms, etc.)
+
+This D1 cron ledger is one implementation layer and future Store direction. It is not required for every FreshContext-compatible envelope implementation.
+
+### 1.3 Example Adapter / Source Classes
+
+FreshContext currently has:
+
+- A reference MCP implementation with `evaluate_context` and 21 read-only reference adapters
+- Separate feed products such as Fresh HN Feed and Fresh Jobs Feed
+- A Store/Ledger methodology for systems that collect recurring signals over time
+
+The following table describes example source classes used by FreshContext implementations. Not every source class is necessarily collected by every cron/feed deployment.
+
+| Adapter class | Source | Auth Required | Typical Update Frequency |
+|---|---|---|---|
+| `hackernews` | Hacker News Algolia API | None | Real-time |
+| `jobs` | Remotive API | None | Continuous |
+| `reposearch` | GitHub Search API | Optional (rate limit) | Real-time |
+| `github` | GitHub Repository API | Optional | Real-time |
+| `reddit` | Reddit JSON API | None | Real-time |
+| `yc` | YC Open Source API | None | Per batch cycle |
+| `packagetrends` | npm Registry + npm Downloads API | None | Per publish |
+| `finance` | Stooq quote API | None | Market hours / quote feed cadence |
+| `producthunt` | Product Hunt launch data | Token when API-backed | Launch cadence |
+| `changelog` | GitHub Releases / npm package metadata | Optional | Per release |
+| `arxiv` / `scholar` | Academic sources | None | Publication cadence |
+| `gdelt` | GDELT global news | None | 15-minute feed cadence |
+| `govcontracts` / `gebiz` | Government procurement datasets | None | Dataset cadence |
+| `sec_filings` | SEC EDGAR filings | None | Filing cadence |
+
+FreshContext adapters operate on publicly accessible or publicly documented data sources. Most reference adapters require no credentials. Some APIs may optionally use tokens for rate limits or official API access, but FreshContext-compatible adapters should not require private user data unless explicitly documented by the implementation. All fetch requests include a `User-Agent` header identifying the FreshContext crawler where the runtime/source supports it.
+
+### 1.4 Content Hash Deduplication
+
+Before any signal is stored, the platform computes a 32-bit rolling hash of the raw content. If the most recent stored result for a given watched query carries an identical hash, the current result is discarded. This prevents storing unchanged content across cron cycles.
+
+### 1.5 Semantic Deduplication
+
+Beyond exact-match deduplication, FreshContext implements semantic deduplication to prevent the same underlying story appearing as multiple signals because it was covered by multiple sources (e.g., the same GitHub release appearing in both HN and Reddit).
+
+The semantic fingerprint is computed as follows:
+
+1. Extract the first canonical URL from the raw content
+2. Extract the first ISO 8601 publication date from the raw content
+3. Extract and normalise the first substantive line (title) — lowercased, punctuation stripped, truncated to 80 characters
+4. Concatenate: `normalised_title|canonical_url|publication_date`
+5. Compute SHA-256 of the concatenated string
+6. Retain the first 16 hex characters as the fingerprint
+
+If any signal stored within the preceding 48 hours carries an identical fingerprint, the new result is discarded. The 48-hour window is configurable.
+
+---
+
+## Section 2: Temporal Scoring — The DAR Engine
+
+### 2.1 Overview
+
+The Decay-Adjusted Relevancy (DAR) engine scores every collected signal on two axes:
+
+- **R_0 (Base Score)** — semantic relevancy of the content against the user's profile, independent of time
+- **R_t (Decay-Adjusted Score)** — R_0 adjusted for how much time has elapsed since the content was published
+
+The final stored `rt_score` is what drives signal ranking in briefings and the intelligence feed.
+
+FreshContext measures temporal utility, not truth. A source can be valid and still have low utility for the current query if it is stale. A source can be fresh but low-confidence if its timestamp is missing, malformed, inferred, or contradicted.
+
+### 2.2 Base Score Calculation (R_0)
+
+R_0 is the starting relevance or utility before temporal decay. In the Store/Feed implementation, R_0 is computed by matching content against the user profile:
+
+```
+R_0 = baseline (40)
+    + vital_keyword_matches × 15   [capped at +35]
+    + skill_keyword_matches × 3    [capped at +15]
+    + location_accessibility_bonus  [+8 if remote/accessible]
+    - error_penalty                 [−40 if content is empty/error]
+```
+
+Vital keywords are drawn from the `targets` field of the user profile — job titles, company names, and technology domains the user is specifically tracking.
+
+Skill keywords are drawn from the `skills` field — the user's technical competencies. A match here adds relevancy signal but at lower weight than a direct target match.
+
+The location accessibility bonus is applied when the content explicitly mentions "remote", "worldwide", "anywhere", or the user's stated location. This is not a geographic filter — it is a signal boost for content that is accessible to the user regardless of their physical location.
+
+**Hard exclusions:** If any term from the `exclusion_terms` list appears in the content, R_0 is forced to zero. The result is still stored (for audit purposes) but marked `is_relevant = 0`.
+
+This profile formula is a Store/Feed implementation example, not the only possible way to produce base relevance. For Core/MCP envelope scoring, R_0 may be normalised to 100. For feed/ranking systems, R_0 may come from semantic relevance, profile relevance, adapter-specific relevance, or another documented scoring layer.
+
+### 2.3 Context-Conditioned Utility
+
+FreshContext scoring is context-conditioned. The same signal can have different utility depending on the user, query, agent, platform, or workflow requesting it.
+
+In the Store/Feed implementation, this context is represented by `R_0`, the base relevance or utility score before temporal decay. `R_0` may be computed from profile targets, query terms, semantic relevance, adapter-specific relevance, or another documented scoring layer.
+
+The DAR function then applies temporal pressure:
+
+```
+R_t = R_0 · e^(-λt)
+```
+
+This means FreshContext does not treat freshness as a standalone ranking signal. A fresh but irrelevant signal should not outrank an older but highly relevant signal unless the source policy and use case justify it.
+
+FreshContext Core exposes a pure context utility primitive for this direction:
+
+```
+U(q, s, t) = R(q, s) · e^(-λt) · C_date · C_status
+```
+
+Where:
+- `q` is the requester context: user, query, agent, platform, or workflow
+- `s` is the signal or database record
+- `R(q, s)` is contextual relevance between the request and the signal
+- `λ` is the source-specific decay constant
+- `t` is signal age
+- `C_date` is a timestamp-confidence factor
+- `C_status` is a failure/partial/success factor
+
+This is an extension of the DAR methodology, not a replacement for it. The purpose is to support systems where FreshContext runs over databases, feeds, retrieved documents, or agent memory and ranks information by both relevance and temporal utility. It does not imply vector search, multi-agent orchestration, or a hosted context store.
+
+### 2.4 Decay Function (R_t)
+
+```
+R_t = R_0 · e^(-λt)
+```
+
+Where:
+- `λ` = source-specific decay constant (per hour)
+- `t` = hours elapsed since `published_at` / `content_date`
+- `R_t` = current temporal utility score
+
+If `published_at` / `content_date` cannot be extracted, the system must not pretend the signal is fresh. Core-compatible envelope scoring SHOULD use `freshness_score: null` and low confidence. Store/feed systems MAY apply a conservative fallback assumption, such as one source half-life, but must mark confidence low and explain the assumption.
+
+### 2.5 Source Decay Constants (λ)
+
+These constants are reference/default calibration values for how quickly signals from each source class lose temporal utility:
+
+| Source | λ (per hour) | Half-life |
+|---|---|---|
+| Hacker News | 0.050 | ~14 hours |
+| Reddit | 0.010 | ~3 days |
+| Product Hunt | 0.010 | ~3 days |
+| Job listings | 0.005 | ~6 days |
+| Financial data | 0.001 | ~29 days |
+| YC companies | 0.001 | ~29 days |
+| Package trends | 0.0005 | ~58 days |
+| GitHub repositories | 0.0002 | ~5 months |
+| Academic papers | 0.00005 | ~1.6 years |
+
+These constants are reference defaults used by the FreshContext methodology and may be tuned by implementation. Hosted or private deployments may use calibrated variants per source, query type, or user profile. The calibration process and production tuning may be proprietary, even when public reference defaults are documented.
+
+### 2.6 Entropy Classification
+
+Each signal is classified into one of three entropy states based on its position on the decay curve:
+
+| State | Condition | Interpretation |
+|---|---|---|
+| `low` | `t < half_life / 2` | Signal near peak value — act now |
+| `stable` | `t < 1.5 × half_life` | Usable signal — monitor |
+| `high` | `t ≥ 1.5 × half_life` | Significantly degraded — verify before acting |
+
+Entropy labels describe signal decay state, not confidence level. A high-entropy signal may still be factually accurate, but it has lost temporal utility for current retrieval unless reinforced by newer evidence.
+
+### 2.7 Relevancy Threshold
+
+Signals with `rt_score < 35` are stored with `is_relevant = 0`. They remain in the database for audit and historical analysis but are excluded from briefings and the intelligence feed by default. The threshold is configurable per profile.
+
+### 2.8 Failure Honesty
+
+Failed adapters must not be promoted by freshness scoring. Empty, blocked, timeout, malformed, rate-limited, access-denied, or error-only outputs reduce R_0 or mark the signal status as failed/unknown.
+
+A failed result should not receive high confidence. A failed result should not produce `Score: 100/100`. Partial composites should preserve successful upstream results while marking failures explicitly.
+
+---
+
+## Section 3: FreshContext Store / Ledger Methodology
+
+### 3.1 The Ha-Pri Audit Signature
+
+Every signal stored in a FreshContext Store/Ledger deployment carries a `ha_pri_sig` — a SHA-256 audit signature computed as:
+
+```
+SHA-256( result_id + ":" + content_hash + ":" + "FRESHCONTEXT_DAR_V1" )
+```
+
+In Ha-Pri v1, this signature is a provenance stamp and audit reference for stored signals. It binds the result ID, the current content hash, and the engine version. It is not yet a full tamper-enforcement system: the current `content_hash` source is the existing rolling `result_hash`, and signatures are not recomputed on read to reject modified rows.
+
+Ha-Pri v1 serves three purposes:
+
+1. **Provenance reference** — the signature binds the result ID, current rolling content hash, and engine/version marker so the stored signal can be audited against the v1 formula.
+2. **Scoring lineage** — the signature records the scoring/signature formula used when the row was written.
+3. **Licensing / audit reference** — when FreshContext data is provided to a third party under licence, the `ha_pri_sig` column gives a stable reference for what was stored and delivered.
+
+Ha-Pri v1 is not hard tamper enforcement. It is not recomputed on read, it signs the existing rolling result_hash (`result_hash`) rather than canonical content SHA-256, and it does not reject rows. Ha-Pri v2 is the planned/additive path for stronger verification.
+
+Future Ha-Pri v2 may add canonical content SHA-256, stronger canonicalization, and explicit verification/rejection on read. That hardening is separate from the current v1 provenance stamp.
+
+Ha-Pri v1 is the provenance layer and the foundation for a stronger integrity layer, while DAR and context-conditioned utility are the ranking/scoring layer.
+
+### 3.2 D1 Historical Ledger
+
+The `scrape_results` table functions as a **Contextual Ledger** — not merely a cache, but a time-series record of intelligence signals with full provenance.
+
+This Store/Ledger methodology is not required for basic FreshContext-compatible envelope implementations. It is the methodology for systems that persist recurring signals and want auditability over time.
+
+Key properties of the ledger:
+- Scored signal material is treated as immutable once written; consumption metadata such as `is_new` may be updated
+- Every row carries a `scraped_at` timestamp with second precision
+- Every row carries a `published_at` date extracted from content (where available)
+- The ledger accumulates continuously at 6-hour intervals regardless of active user sessions
+- The ledger enables time-travel queries: "what was the intelligence landscape for topic X at date Y?"
+
+### 3.3 Schema Reference
+
+```sql
+scrape_results (
+  id                  TEXT PRIMARY KEY,    -- sr_{timestamp}_{random}
+  watched_query_id    TEXT,                -- FK → watched_queries.id
+  adapter             TEXT,                -- source adapter name
+  query               TEXT,                -- the search term used
+  raw_content         TEXT,                -- scraped content (max 8000 chars)
+  result_hash         TEXT,                -- 32-bit rolling hash of raw_content
+  semantic_fingerprint TEXT,               -- 16-char SHA-256 of normalised title|url|date
+  is_new              INTEGER,             -- 1 until consumed by briefing
+  scraped_at          TEXT,                -- ISO 8601 UTC timestamp
+  published_at        TEXT,                -- extracted content publication date
+  relevancy_score     INTEGER,             -- = round(rt_score), 0-100
+  is_relevant         INTEGER,             -- 1 if rt_score >= 35, else 0
+  base_score          INTEGER,             -- R_0 semantic score, 0-100
+  rt_score            REAL,                -- R_t decay-adjusted score, 0-100
+  ha_pri_sig          TEXT,                -- SHA-256 audit signature (64 hex chars)
+  entropy_level       TEXT                 -- 'low' | 'stable' | 'high'
+)
+```
+
+---
+
+## Section 4: The Intelligence Feed
+
+### 4.1 Endpoint
+
+```
+GET /v1/intel/feed/{profile_id}
+```
+
+Optional parameters:
+- `limit` — maximum signals to return (default: 20)
+- `min_rt` — minimum rt_score filter (default: 0)
+
+### 4.2 Response Structure
+
+```json
+{
+  "feed_metadata": {
+    "profile_id": "default",
+    "generated_at": "2026-04-14T09:00:00Z",
+    "signal_count": 18,
+    "version": "freshcontext-1.2"
+  },
+  "signals": [
+    {
+      "signal_id": "sr_1744628412_a3f7b",
+      "source": "hackernews",
+      "label": "HN: MCP Servers",
+      "content": {
+        "preview": "...",
+        "url": "mcp server 2026"
+      },
+      "intelligence_stamps": {
+        "scraped_at": "2026-04-14T08:12:00Z",
+        "published_at": "2026-04-14",
+        "base_score": 78,
+        "rt_score": 61.4,
+        "entropy_level": "stable",
+        "ha_pri_sig": "a3f7b2c1d4e5f6a7b8c9d0e1f2a3b4c5..."
+      }
+    }
+  ]
+}
+```
+
+### 4.3 LLM Integration
+
+The intelligence feed is designed to be consumed directly by any language model or AI agent without modification. The `intelligence_stamps` block gives the agent everything it needs to reason about data freshness:
+
+- `rt_score` — a single number representing current signal value
+- `entropy_level` — human-readable decay state
+- `published_at` — the actual content date (not the retrieval date)
+- `ha_pri_sig` — provenance reference the agent can cite
+
+This is the core value proposition: **AI agents get grounded, timestamped, scored intelligence rather than undated web content of unknown age.**
+
+MCP is one interface over this methodology, not the whole system. The same scoring, timestamp, confidence, and provenance primitives can support APIs, CLIs, npm packages, dashboards, agents, and internal services.
+
+---
+
+## Section 5: Asset Summary
+
+For technical integrators, auditors, and future platform partners:
+
+**What FreshContext owns:**
+
+1. **The FreshContext Specification v1.2** (MIT licence, open standard) — defines the envelope format, confidence levels, structured JSON form, freshness score behavior, and failure-honesty requirements. Timestamped in the public GitHub repository.
+
+2. **The DAR Engine** — the exponential decay scoring methodology with source-specific λ reference defaults and calibrated production tuning.
+
+3. **The Semantic Fingerprinting Method** — the three-field normalisation and SHA-256 fingerprinting approach for cross-adapter deduplication.
+
+4. **The Ha-Pri Audit Signature scheme** — the provenance stamp and audit reference that binds stored row material to the current v1 formula; stronger tamper-evidence is the future additive v2 path.
+
+5. **The Store / Ledger design** — support for recurring watched queries, historical signal accumulation, D1-backed storage, and time-series auditability.
+
+6. **The Reference Implementation** — `freshcontext-mcp@0.3.19`, the `evaluate_context` MCP interface, and 21 read-only reference adapters, listed on npm and the MCP Registry. The hosted Worker endpoint is a separate deployment surface.
+
+---
+
+## Changelog
+
+### Version 1.2 — May 2026
+- Clarified Core methodology vs Store/Ledger methodology.
+- Preserved DAR as the mathematical scoring backbone.
+- Updated reference implementation language for `evaluate_context` plus 21 MCP reference adapters.
+- Reframed source decay constants as reference defaults/calibration values.
+- Added failure-honesty methodology.
+- Added context-conditioned utility as a Core scoring primitive.
+- Clarified missing timestamp behavior.
+- Clarified MCP as one interface, not the whole system.
+
+### Version 1.1 — April 2026
+- Existing methodology version.
+
+---
+
+*"The work isn't gone. It's just waiting to be continued."*
+*— Prince Gabriel, Grootfontein, Namibia*

package/README.md

@@ -126,9 +126,9 @@ FreshContext returns decision-first output:
 - Confidence
 - Why
 
-`evaluate_context` does not fetch URLs, crawl, scrape, browse, read folders, or call adapters. It only evaluates candidate context the caller provides.
-
-Current boundary: `evaluate_context` is part of the published npm/local stdio MCP server. The hosted Cloudflare Worker endpoint is a separate deployment surface and may lag npm package interfaces until a dedicated Worker sync pass.
+`evaluate_context` does not fetch URLs, crawl, scrape, browse, read folders, or call adapters. It only evaluates candidate context the caller provides.
+
+Current boundary: `evaluate_context` is part of the published npm/local stdio MCP server and has been verified on the hosted Cloudflare Worker MCP endpoint at `0.3.19 / 22 tools`. The Worker remains a separate deployment surface, so future package interfaces should be re-verified remotely before being claimed live.
 
 ---
 
@@ -195,6 +195,8 @@ The repo ships named reference adapters that demonstrate how different source cl
 
 ## Quick start
 
+For Claude Desktop, Codex, `npx`, global npm, and source-checkout setup, see the concise [client setup guide](./docs/CLIENT_SETUP.md).
+
 ### Cloud (no install)
 
 Add to your Claude Desktop config and restart:
@@ -387,7 +389,7 @@ The reference implementation runs on Cloudflare's global edge:
 | `/` | GET | Service info + endpoint list |
 | `/health` | GET | Liveness check |
 | `/mcp` | POST | MCP JSON-RPC transport |
-| `/demo` | GET | Live before/after demo (no API key required) |
+| `/demo` | GET | Live before/after demo (no auth token required) |
 | `/briefing` | GET | Latest stored briefing |
 | `/v1/intel/feed/:profile_id` | GET | DAR-scored intelligence feed |
 | `/watched-queries` | GET | List all watched queries |
@@ -405,17 +407,19 @@ Production: `https://freshcontext-mcp.gimmanuel73.workers.dev`
 ## Roadmap
 
 - [x] FreshContext Specification v1.2 published (MIT, open standard)
-- [x] DAR engine with proprietary λ constants (v0.3.19)
+- [x] DAR engine with source-specific lambda constants (v0.3.19)
 - [x] Ha-Pri v1 provenance signatures on stored signals
 - [x] Semantic deduplication via fingerprinting
 - [x] Live before/after demo at `/demo`
-- [x] METHODOLOGY.md — formal IP and engineering documentation
+- [x] METHODOLOGY.md — methodology and engineering documentation
 - [x] Named reference adapters across intelligence, competitive research, market data, and composites
 - [x] Generic MCP `evaluate_context` tool for caller-provided candidate context
 - [x] Core-backed envelope generation shared by npm/MCP and the Cloudflare Worker
 - [x] Cloudflare Workers deployment — global edge, KV cache, KV rate limiting
 - [x] Published on npm and listed for MCP usage; Apify/feed assets are separated from the normal MCP runtime package
-- [ ] Ha-Pri v2 hardened canonical content hash verification
+- [x] Ha-Pri v2 Core helper and deterministic golden vectors
+- [x] Ha-Pri v2 production-enforcement design document
+- [ ] Ha-Pri v2 Worker/D1 production enforcement
 - [x] GitHub Actions release workflow — manual or `v*` tag-triggered npm publish path
 - [ ] Webhook triggers — push high-entropy signals on threshold
 - [ ] Dashboard — React frontend for the D1 intelligence pipeline

package/dist/adapters/arxiv.js

@@ -1,3 +1,4 @@
+import { validateUrl } from "../security.js";
 const USER_AGENT = "freshcontext-mcp/0.1.7 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)";
 const DEFAULT_ARXIV_SIGNAL_SCORE = 0.8;
 function buildArxivApiUrl(input, maxResults = 10) {
@@ -6,7 +7,7 @@ function buildArxivApiUrl(input, maxResults = 10) {
         ? Math.max(1, Math.min(Math.trunc(maxResults), 50))
         : 10;
     return trimmed.startsWith("http")
-        ? trimmed
+        ? validateUrl(trimmed, "arxiv")
         : `https://export.arxiv.org/api/query?search_query=all:${encodeURIComponent(trimmed)}&start=0&max_results=${safeMaxResults}&sortBy=relevance&sortOrder=descending`;
 }
 async function fetchArxivXml(apiUrl) {

package/dist/adapters/changelog.js

@@ -1,3 +1,4 @@
+import { validateUrl } from "../security.js";
 /**
  * Changelog adapter — extracts update history from any product or repo.
  *
@@ -193,7 +194,8 @@ export async function changelogAdapter(options) {
         return fetchNpmChangelog(input, maxLength);
     }
     // GitHub repo URL → use releases API
-    const ghMatch = input.match(/github\.com\/([^/]+)\/([^/?\s]+)/);
+    const safeInput = validateUrl(input, "changelog");
+    const ghMatch = safeInput.match(/github\.com\/([^/]+)\/([^/?\s]+)/);
     if (ghMatch) {
         try {
             return await fetchGitHubReleases(ghMatch[1], ghMatch[2], maxLength);
@@ -203,5 +205,5 @@ export async function changelogAdapter(options) {
         }
     }
     // Any other URL → discover changelog
-    return discoverChangelog(input, maxLength);
+    return discoverChangelog(safeInput, maxLength);
 }

package/dist/adapters/finance.js

@@ -30,7 +30,7 @@ async function fetchStooqQuote(ticker) {
     const url = `https://stooq.com/q/l/?s=${encodeURIComponent(stooqSymbol.toLowerCase())}&f=sd2t2ohlcv&h&e=json`;
     const res = await fetch(url, {
         headers: {
-            "User-Agent": "freshcontext-mcp/0.3.17",
+            "User-Agent": "freshcontext-mcp/0.3.19",
             "Accept": "application/json",
         },
     });

package/dist/adapters/gdelt.js

@@ -11,7 +11,7 @@
  */
 const HEADERS = {
     "Accept": "application/json",
-    "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+    "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
 };
 function parseGdeltDate(raw) {
     if (!raw)

package/dist/adapters/gebiz.js

@@ -20,7 +20,7 @@ const DATASET_ID = "d_acde1106003906a75c3fa052592f2fcb";
 const BASE_URL = "https://data.gov.sg/api/action/datastore_search";
 const HEADERS = {
     "Accept": "application/json",
-    "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+    "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
 };
 function formatDate(raw) {
     if (!raw)

package/dist/adapters/reddit.js

@@ -1,3 +1,4 @@
+import { sanitizeQuery, validateUrl } from "../security.js";
 /**
  * Reddit adapter — public JSON API, no auth required.
  * Accepts subreddit URLs or search queries.
@@ -6,10 +7,15 @@
  */
 export async function redditAdapter(options) {
     let apiUrl = options.url;
-    // If they pass a plain subreddit name like "r/MachineLearning", build the URL
+    // If they pass a plain subreddit or search query, build a Reddit JSON URL.
     if (!apiUrl.startsWith("http")) {
-        const clean = apiUrl.replace(/^r\//, "");
-        apiUrl = `https://www.reddit.com/r/${clean}/.json?limit=25&sort=hot`;
+        const clean = sanitizeQuery(apiUrl, 120).replace(/^r\//, "").replace(/^\/+|\/+$/g, "");
+        if (/^[A-Za-z0-9_]{2,21}$/.test(clean)) {
+            apiUrl = `https://www.reddit.com/r/${clean}/.json?limit=25&sort=hot`;
+        }
+        else {
+            apiUrl = `https://www.reddit.com/search.json?q=${encodeURIComponent(clean)}&sort=new&limit=25`;
+        }
     }
     // Ensure we hit the JSON endpoint
     if (!apiUrl.includes(".json")) {
@@ -19,7 +25,8 @@ export async function redditAdapter(options) {
     if (!apiUrl.includes("limit=")) {
         apiUrl += (apiUrl.includes("?") ? "&" : "?") + "limit=25";
     }
-    const res = await fetch(apiUrl, {
+    const safeUrl = validateUrl(apiUrl, "reddit");
+    const res = await fetch(safeUrl, {
         headers: {
             "User-Agent": "freshcontext-mcp/0.1.5 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
             "Accept": "application/json",

package/dist/adapters/repoSearch.js

@@ -22,7 +22,7 @@ export async function repoSearchAdapter(options) {
     const res = await fetch(apiUrl, {
         headers: {
             Accept: "application/vnd.github.v3+json",
-            "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+            "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
         },
     });
     if (!res.ok) {

package/dist/adapters/secFilings.js

@@ -12,7 +12,7 @@
  */
 const HEADERS = {
     "Accept": "application/json",
-    "User-Agent": "freshcontext-mcp/0.3.17 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
+    "User-Agent": "freshcontext-mcp/0.3.19 (https://github.com/PrinceGabriel-lgtm/freshcontext-mcp)",
 };
 async function fetchSecFilings(query, maxResults = 10) {
     const today = new Date().toISOString().slice(0, 10);

package/dist/core/envelope.js

@@ -1,5 +1,13 @@
 import { calculateFreshnessScore, isMeaningfullyFutureDate, scoreLabel } from "./decay.js";
 import { looksLikeFailedAdapterContent } from "./guards.js";
+export const MAX_ENVELOPE_CONTENT_LENGTH = 20000;
+function clampEnvelopeMaxLength(maxLength) {
+    if (maxLength === 0)
+        return 0;
+    if (maxLength === undefined || !Number.isFinite(maxLength))
+        return 8000;
+    return Math.min(MAX_ENVELOPE_CONTENT_LENGTH, Math.max(1, Math.floor(maxLength)));
+}
 export function stampFreshness(result, options, adapter) {
     const retrieved_at = new Date().toISOString();
     const failedContent = looksLikeFailedAdapterContent(result.raw);
@@ -8,7 +16,7 @@ export function stampFreshness(result, options, adapter) {
     const freshness_confidence = failedContent || futureDated ? "low" : result.freshness_confidence;
     const freshness_score = calculateFreshnessScore(content_date, retrieved_at, adapter);
     return {
-        content: result.raw.slice(0, options.maxLength ?? 8000),
+        content: result.raw.slice(0, clampEnvelopeMaxLength(options.maxLength)),
         source_url: options.url,
         content_date,
         retrieved_at,

package/dist/security.js

@@ -10,9 +10,11 @@ export const ALLOWED_DOMAINS = {
     yc: ["www.ycombinator.com", "ycombinator.com"],
     repoSearch: [], // uses GitHub API directly, no browser
     packageTrends: [], // uses npm/PyPI APIs directly, no browser
-    reddit: [], // uses public Reddit JSON API, no browser
+    reddit: ["www.reddit.com", "reddit.com", "old.reddit.com"],
     finance: [], // uses Stooq quote API, no browser
+    arxiv: ["export.arxiv.org", "arxiv.org"],
     productHunt: ["www.producthunt.com", "producthunt.com"],
+    changelog: [], // accepts public changelog URLs but blocks private/internal targets
 };
 // ─── Blocked IP ranges and internal hostnames ────────────────────────────────
 const BLOCKED_PATTERNS = [

package/dist/server.js

@@ -23,7 +23,7 @@ import { EvaluateContextInputError, evaluateContextInput, formatEvaluateContextR
 import { formatSecurityError } from "./security.js";
 const server = new McpServer({
     name: "freshcontext-mcp",
-    version: "0.3.19",
+    version: "0.3.20",
 });
 const signalInputSchema = z.object({
     id: z.string().optional(),
@@ -46,7 +46,7 @@ server.registerTool("evaluate_context", {
     inputSchema: z.object({
         profile: z.string().min(1).describe("Source Profile id, e.g. academic_research, jobs_opportunities, market_finance, official_docs, local_custom."),
         intent: z.string().min(1).describe("Intent Profile id, e.g. citation_check, student_research, developer_adoption, job_search, market_watch, business_due_diligence, medical_literature_triage."),
-        signals: z.array(signalInputSchema).min(1).describe("Candidate context items provided by the caller. FreshContext evaluates these; it does not retrieve them."),
+        signals: z.array(signalInputSchema).min(1).max(100).describe("Candidate context items provided by the caller. FreshContext evaluates these; it does not retrieve them."),
         now: z.string().optional().describe("Optional ISO timestamp for deterministic evaluation."),
     }),
     annotations: { readOnlyHint: true, openWorldHint: false },