canli-validation-mcp 0.6.0 → 0.7.1

package/README.md

@@ -2,8 +2,37 @@
 
 [![npm](https://img.shields.io/npm/v/canli-validation-mcp)](https://www.npmjs.com/package/canli-validation-mcp)
 [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/arhancanli/canli-validation-mcp/badge)](https://scorecard.dev/viewer/?uri=github.com/arhancanli/canli-validation-mcp)
+[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/14954/badge)](https://www.bestpractices.dev/projects/14954)
 [![Glama score](https://glama.ai/mcp/servers/arhancanli/canli-validation-mcp/badges/score.svg)](https://glama.ai/mcp/servers/arhancanli/canli-validation-mcp)
 
+**Is your best backtest real, or just the luckiest of the variants you tried?** This MCP server
+lets Claude, Cursor or any MCP client answer that with the standard corrections: the deflated
+Sharpe ratio, the CSCV probability of backtest overfitting, the minimum track record length, the
+haircut Sharpe ratio, and luck-equivalent trials. Free and MIT-licensed.
+
+## Quick start
+
+```bash
+# Claude Code, computed on your machine: no key, nothing sent anywhere
+claude mcp add canli-local --env CANLI_LOCAL=1 -- npx -y canli-validation-mcp
+
+# the same tools with signed, stored receipts from the free API (a free key is issued on first use)
+claude mcp add canli -- npx -y canli-validation-mcp
+
+# nothing to install: the hosted endpoint
+claude mcp add --transport http canli https://canlicapital.com/mcp
+```
+
+Then ask, for example: *"I tried 229 variants and kept the best: an annualised Sharpe of
+1.5 over 730 daily returns (365 a year), skew -0.5, kurtosis 5, and the variants'
+Sharpe ratios spread by 0.57. Is it real?"* The assistant calls `validate_deflated_sharpe`: the
+best of 229 skill-less variants would reach 1.60 by luck alone, so the probability that the
+Sharpe is above zero falls from 98.1% to 44.4% once the search is counted. Claude Desktop and any
+other stdio client (Cursor, VS Code) run the same `npx` command; see "Claude Desktop" and
+"Generic stdio client" below.
+
+## What it does
+
 An MCP (Model Context Protocol) server over canlicapital.com's free, keyed validation API. It
 gives a coding agent fourteen tools: issue a free key, run the eight validators (deflated Sharpe,
 CSCV overfitting, paper-evidence conformance, breadth ceiling, minimum track record length,
@@ -133,6 +162,7 @@ an array of objects.
 | `CANLI_API_BASE` | `https://canlicapital.com` | Where the API lives. Point it at a preview deployment for testing. |
 | `CANLI_KEY` | unset | A key already issued from `POST /api/v1/keys`. When set, `get_key` sends no request and reports the key is already configured; every other tool sends it as `Authorization: Bearer <key>`. |
 | `CANLI_FULL_ENVELOPE` | unset | `1` or `true` returns each validation's full API envelope instead of the compact result (below). |
+| `CANLI_TOOLSETS` | all | Which tools to list: a comma-separated choice of `validate`, `receipts`, `company` and `status`, or `all`. An unknown name is refused at startup. See "Toolsets" below. |
 | `CANLI_LOCAL` | unset | `1` or `true` runs the eight validators on this machine (private local mode, below): no key, no network, no receipt. |
 
 If `CANLI_KEY` is not set and local mode is off, call `get_key` once per session before the validators. The key it
@@ -163,13 +193,18 @@ claude mcp add --transport http canli https://canlicapital.com/mcp
 ```
 
 Without a key, requests run under a shared anonymous key, so the daily validation quota is shared
-by every hosted caller. For your own quota, issue a free key (see
+by every hosted caller. When that shared quota is used up for the day, validations are still
+answered, computed by the same code on the hosted endpoint, but without a stored receipt; the
+result says so. For your own quota, issue a free key (see
 [/developers](https://canlicapital.com/developers#quickstart)) and send it as a header:
 
 ```bash
 claude mcp add --transport http canli https://canlicapital.com/mcp --header "Authorization: Bearer $CANLI_KEY"
 ```
 
+Add `?toolsets=` to the URL to list only some tools (see "Toolsets" below), for example
+`https://canlicapital.com/mcp?toolsets=company`.
+
 The endpoint is stateless. On it, `get_key` issues nothing and says which key is in use, because a
 key issued there would not reach the next request. A malformed Authorization header is refused
 rather than replaced with the shared key.
@@ -279,6 +314,35 @@ describes the service rather than the answer, and an agent pays for every token
 call. It stays in the stored receipt, which `get_receipt` returns in full, and in `service_status`.
 On a breadth result this is about half the text. Set `CANLI_FULL_ENVELOPE=1` to receive every field.
 
+## Toolsets (tokens)
+
+A client sends the model the whole tool list on every turn, and it is most of each turn's prompt:
+a validation result is a few hundred tokens, the list of all fourteen tools several thousand. A
+client that needs one kind of tool can list only that kind, with `CANLI_TOOLSETS` (stdio) or
+`?toolsets=` (hosted endpoint). The default is every tool.
+
+| toolset | tools |
+|---|---|
+| `validate` | `get_key`, the eight validators, `audit_backtest` |
+| `receipts` | `get_receipt`, `verify_receipt` |
+| `company` | `company_financial_history` |
+| `status` | `service_status` |
+
+Measured with `bench/tool_list_tokens.py` (tokenizer: tiktoken `o200k_base`; other models'
+tokenizers give different absolute counts), in the shape an OpenAI-style client sends the list:
+
+| CANLI_TOOLSETS | tools | tokens per turn | of all |
+|---|---|---|---|
+| `all` | 14 | 3,888 | 100% |
+| `validate` | 10 | 3,154 | 81% |
+| `receipts` | 2 | 340 | 9% |
+| `company` | 1 | 302 | 8% |
+| `status` | 1 | 98 | 3% |
+
+Providers cache a tool list that is identical from turn to turn and bill the cached part at a
+fraction of the price (`test/tool-list-stable.test.mjs` keeps each list byte-stable); a smaller list
+costs less either way.
+
 ## Local checkout
 
 Only needed to develop or test this package itself, not to run the published one.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "canli-validation-mcp",
-  "version": "0.6.0",
+  "version": "0.7.1",
   "description": "MCP server for canlicapital.com's free validation API: deflated Sharpe, CSCV overfitting, paper-evidence conformance, and breadth ceiling, each returned as the full API envelope so the boundary language cannot be dropped.",
   "private": false,
   "type": "module",

package/src/server.mjs

@@ -66,7 +66,29 @@ export function configuredLocal(value) {
   return v === "1" || v?.toLowerCase() === "true";
 }
 
-export function createSession({ base, fetchImpl, envKey, timeoutMs = REQUEST_TIMEOUT_MS, hosted, local, fullEnvelope, receiptKeys } = {}) {
+// Toolsets: which tools the server lists. The tool list is re-sent to the model on every turn and
+// is most of each turn's prompt (README, "Toolsets"; bench/tool_list_tokens.py measures it), so a
+// client that needs one kind of tool can load only that kind. Default: all.
+export const TOOLSETS = Object.freeze({
+  validate: Object.freeze(["get_key", "validate_deflated_sharpe", "validate_overfitting", "validate_paper_evidence", "validate_breadth", "validate_track_record", "validate_backtest_length", "validate_haircut_sharpe", "validate_luck_trials", "audit_backtest"]),
+  receipts: Object.freeze(["get_receipt", "verify_receipt"]),
+  company: Object.freeze(["company_financial_history"]),
+  status: Object.freeze(["service_status"]),
+});
+
+// CANLI_TOOLSETS=validate,company (or ?toolsets= on the hosted endpoint): a comma-separated list of
+// TOOLSETS names, or "all". Empty or unsubstituted means all; an unknown name is refused, so a typo
+// never silently leaves a client without the tools it asked for.
+export function configuredToolsets(value) {
+  const v = configuredKey(value);
+  if (!v || v.trim().toLowerCase() === "all") return Object.keys(TOOLSETS);
+  const names = [...new Set(v.split(",").map((s) => s.trim().toLowerCase()).filter(Boolean))];
+  const unknown = names.filter((n) => !Object.hasOwn(TOOLSETS, n));
+  if (unknown.length || !names.length) throw new Error(`Unknown toolset ${unknown.join(", ") || "(none)"}; choose from ${Object.keys(TOOLSETS).join(", ")} or all`);
+  return names;
+}
+
+export function createSession({ base, fetchImpl, envKey, timeoutMs = REQUEST_TIMEOUT_MS, hosted, local, fullEnvelope, receiptKeys, toolsets } = {}) {
   if (!Number.isSafeInteger(timeoutMs) || timeoutMs <= 0) throw new Error("Request timeout must be a positive integer");
   return {
     base: base ?? process.env.CANLI_API_BASE ?? DEFAULT_BASE,
@@ -81,6 +103,7 @@ export function createSession({ base, fetchImpl, envKey, timeoutMs = REQUEST_TIM
     fullEnvelope: fullEnvelope ?? configuredFullEnvelope(process.env.CANLI_FULL_ENVELOPE),
     // Tests pass their own keys; everyone else verifies against the bundled published keys.
     receiptKeys: receiptKeys ?? undefined,
+    toolsets: toolsets ?? configuredToolsets(process.env.CANLI_TOOLSETS),
   };
 }
 
@@ -109,6 +132,29 @@ async function callApi(session, { path, method = "GET", body }) {
   return { envelope, failed: res.status >= 400 || Boolean(envelope?.error) };
 }
 
+// The hosted endpoint's shared anonymous key has one daily quota for every caller who connects
+// without a key of their own. When it is used up, the answer is still computed, by the same code
+// the API runs (src/local, byte for byte), on the hosted endpoint; what the caller loses is the
+// stored, signed receipt, and the note says how to get one. A caller's own key, or a missing
+// shared key, still gets the API's refusal unchanged.
+export const SHARED_QUOTA_NOTE = "The shared anonymous quota of this hosted endpoint is used up for today (it resets at 00:00 UTC), so this result was computed by the same code on the hosted endpoint and no receipt was stored. For a receipt and a quota of your own, get a free key at https://canlicapital.com/developers#quickstart and send it as 'Authorization: Bearer <key>'. To run with no quota at all, on your own machine: npx -y canli-validation-mcp with CANLI_LOCAL=1.";
+
+async function validateRemote(session, tool, path, body) {
+  let response = await callApi(session, { path, method: "POST", body });
+  // stdio without CANLI_KEY: the first validation used to come back 401 and the model had to work
+  // out that get_key comes first. Issue the free key once, as get_key would, and retry once.
+  if (response.failed && response.envelope?.error?.code === "unauthorized" && !session.hosted && !session.key && !session.envKey) {
+    const issued = await callApi(session, { path: "/api/v1/keys", method: "POST", body: { label: "auto" } });
+    if (!issued.failed && issued.envelope?.data?.key) {
+      session.key = issued.envelope.data.key;
+      response = await callApi(session, { path, method: "POST", body });
+    }
+  }
+  if (!response.failed || session.hosted?.keySource !== "shared" || response.envelope?.error?.code !== "quota_exhausted") return response;
+  const fallback = computeLocally(tool, body);
+  return { ...fallback, envelope: { ...fallback.envelope, computed: "hosted_without_receipt", note: SHARED_QUOTA_NOTE } };
+}
+
 // Compact context (0.3.0): minified JSON. Indentation is whitespace an agent pays for in tokens and
 // never reads; every field, boundary sentence and provenance value is kept. Measured on the live
 // Apple StockholdersEquity record (README, "Compact context"): 2,214 -> 1,560 tokens minified,
@@ -207,56 +253,56 @@ export async function toolValidateDeflatedSharpe(session, args) {
     );
   }
   if (session.local) { const local = computeLocally("validate_deflated_sharpe", parsed.data); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/deflated-sharpe", method: "POST", body: parsed.data });
+  const response = await validateRemote(session, "validate_deflated_sharpe", "/api/v1/validate/deflated-sharpe", parsed.data);
   return validationText(session, response);
 }
 
 export async function toolValidateOverfitting(session, args) {
   const body = parseOrThrow(overfittingInput, args, "validate_overfitting");
   if (session.local) { const local = computeLocally("validate_overfitting", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/overfitting", method: "POST", body });
+  const response = await validateRemote(session, "validate_overfitting", "/api/v1/validate/overfitting", body);
   return validationText(session, response);
 }
 
 export async function toolValidatePaperEvidence(session, args) {
   const body = parseOrThrow(paperEvidenceInput, args, "validate_paper_evidence");
   if (session.local) { const local = computeLocally("validate_paper_evidence", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/paper-evidence", method: "POST", body });
+  const response = await validateRemote(session, "validate_paper_evidence", "/api/v1/validate/paper-evidence", body);
   return validationText(session, response);
 }
 
 export async function toolValidateBreadth(session, args) {
   const body = parseOrThrow(breadthInput, args, "validate_breadth");
   if (session.local) { const local = computeLocally("validate_breadth", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/breadth", method: "POST", body });
+  const response = await validateRemote(session, "validate_breadth", "/api/v1/validate/breadth", body);
   return validationText(session, response);
 }
 
 export async function toolValidateTrackRecord(session, args) {
   const body = parseOrThrow(trackRecordInput, args, "validate_track_record");
   if (session.local) { const local = computeLocally("validate_track_record", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/track-record", method: "POST", body });
+  const response = await validateRemote(session, "validate_track_record", "/api/v1/validate/track-record", body);
   return validationText(session, response);
 }
 
 export async function toolValidateBacktestLength(session, args) {
   const body = parseOrThrow(backtestLengthInput, args, "validate_backtest_length");
   if (session.local) { const local = computeLocally("validate_backtest_length", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/backtest-length", method: "POST", body });
+  const response = await validateRemote(session, "validate_backtest_length", "/api/v1/validate/backtest-length", body);
   return validationText(session, response);
 }
 
 export async function toolValidateHaircutSharpe(session, args) {
   const body = parseOrThrow(haircutSharpeInput, args, "validate_haircut_sharpe");
   if (session.local) { const local = computeLocally("validate_haircut_sharpe", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/haircut-sharpe", method: "POST", body });
+  const response = await validateRemote(session, "validate_haircut_sharpe", "/api/v1/validate/haircut-sharpe", body);
   return validationText(session, response);
 }
 
 export async function toolValidateLuckTrials(session, args) {
   const body = parseOrThrow(luckTrialsInput, args, "validate_luck_trials");
   if (session.local) { const local = computeLocally("validate_luck_trials", body); return validationText(session, local); }
-  const response = await callApi(session, { path: "/api/v1/validate/luck-trials", method: "POST", body });
+  const response = await validateRemote(session, "validate_luck_trials", "/api/v1/validate/luck-trials", body);
   return validationText(session, response);
 }
 
@@ -264,7 +310,7 @@ export async function toolValidateLuckTrials(session, args) {
 // through the API (one validation of quota, one receipt).
 async function runValidator(session, tool, path, body) {
   if (session.local) return computeLocally(tool, body);
-  return callApi(session, { path, method: "POST", body });
+  return validateRemote(session, tool, path, body);
 }
 
 const sameJson = (a, b) => JSON.stringify(a) === JSON.stringify(b);
@@ -476,72 +522,74 @@ const READ_ONLY = { readOnlyHint: true, destructiveHint: false, openWorldHint: t
 const WRITES_RECEIPT = { readOnlyHint: false, destructiveHint: false, openWorldHint: true };
 
 export function registerTools(server, session) {
-  server.registerTool(
+  const enabled = new Set((session.toolsets ?? Object.keys(TOOLSETS)).flatMap((name) => TOOLSETS[name]));
+  const register = (name, ...rest) => { if (enabled.has(name)) server.registerTool(name, ...rest); };
+  register(
     "get_key",
     { title: "Get a free validation key", annotations: { title: "Get a free validation key", readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: true }, description: TOOL_DESCRIPTIONS.get_key, inputSchema: getKeyInput },
     (args) => toolGetKey(session, args),
   );
-  server.registerTool(
+  register(
     "validate_deflated_sharpe",
     { title: "Validate deflated Sharpe", annotations: { title: "Validate deflated Sharpe", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_deflated_sharpe, inputSchema: deflatedSharpeToolShape },
     (args) => toolValidateDeflatedSharpe(session, args),
   );
-  server.registerTool(
+  register(
     "validate_overfitting",
     { title: "Validate overfitting (CSCV)", annotations: { title: "Validate overfitting (CSCV)", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_overfitting, inputSchema: overfittingInput },
     (args) => toolValidateOverfitting(session, args),
   );
-  server.registerTool(
+  register(
     "validate_paper_evidence",
     { title: "Validate paper evidence", annotations: { title: "Validate paper evidence", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_paper_evidence, inputSchema: paperEvidenceInput },
     (args) => toolValidatePaperEvidence(session, args),
   );
-  server.registerTool(
+  register(
     "validate_breadth",
     { title: "Validate breadth ceiling", annotations: { title: "Validate breadth ceiling", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_breadth, inputSchema: breadthInput },
     (args) => toolValidateBreadth(session, args),
   );
-  server.registerTool(
+  register(
     "validate_track_record",
     { title: "Minimum track record length", annotations: { title: "Minimum track record length", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_track_record, inputSchema: trackRecordInput },
     (args) => toolValidateTrackRecord(session, args),
   );
-  server.registerTool(
+  register(
     "validate_backtest_length",
     { title: "Minimum backtest length", annotations: { title: "Minimum backtest length", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_backtest_length, inputSchema: backtestLengthInput },
     (args) => toolValidateBacktestLength(session, args),
   );
-  server.registerTool(
+  register(
     "validate_haircut_sharpe",
     { title: "Haircut Sharpe ratio", annotations: { title: "Haircut Sharpe ratio", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_haircut_sharpe, inputSchema: haircutSharpeInput },
     (args) => toolValidateHaircutSharpe(session, args),
   );
-  server.registerTool(
+  register(
     "validate_luck_trials",
     { title: "Luck-equivalent trials", annotations: { title: "Luck-equivalent trials", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.validate_luck_trials, inputSchema: luckTrialsInput },
     (args) => toolValidateLuckTrials(session, args),
   );
-  server.registerTool(
+  register(
     "audit_backtest",
     { title: "Audit a backtest", annotations: { title: "Audit a backtest", ...WRITES_RECEIPT }, description: TOOL_DESCRIPTIONS.audit_backtest, inputSchema: auditBacktestToolShape },
     (args) => toolAuditBacktest(session, args),
   );
-  server.registerTool(
+  register(
     "get_receipt",
     { title: "Get a receipt", annotations: { title: "Get a receipt", ...READ_ONLY }, description: TOOL_DESCRIPTIONS.get_receipt, inputSchema: getReceiptInput },
     (args) => toolGetReceipt(session, args),
   );
-  server.registerTool(
+  register(
     "verify_receipt",
     { title: "Verify a receipt", annotations: { title: "Verify a receipt", ...READ_ONLY }, description: TOOL_DESCRIPTIONS.verify_receipt, inputSchema: verifyReceiptToolShape },
     (args) => toolVerifyReceipt(session, args),
   );
-  server.registerTool(
+  register(
     "service_status",
     { title: "Service status", annotations: { title: "Service status", ...READ_ONLY }, description: TOOL_DESCRIPTIONS.service_status, inputSchema: emptyInput },
     () => toolServiceStatus(session),
   );
-  server.registerTool(
+  register(
     "company_financial_history",
     { title: "Company financial history (SEC)", annotations: { title: "Company financial history (SEC)", ...READ_ONLY }, description: TOOL_DESCRIPTIONS.company_financial_history, inputSchema: companyHistoryToolShape },
     (args) => toolCompanyFinancialHistory(session, args),