@bbradar/mcp 0.1.20 → 0.2.0

package/README.md

@@ -1,6 +1,6 @@
 # @bbradar/mcp
 
-Local STDIO MCP server for [bbradar.io](https://bbradar.io) Pro subscribers. It lets MCP clients query [bbradar.io](https://bbradar.io) Pro program, target, opportunity, and recent-change data through a local npm package.
+Local STDIO MCP server for [bbradar.io](https://bbradar.io) Pro subscribers. It lets MCP clients query [bbradar.io](https://bbradar.io) platform and Self-Hosted program profiles, active targets, target-level opportunity intelligence, recent target changes, and repository-change events through a local npm package.
 
 ## Requirements
 
@@ -106,15 +106,32 @@ claude mcp add bbradar_io --scope user --env BBRADAR_API_KEY="your_pro_mcp_api_k
 
 After adding the MCP config, restart your client and call `get_mcp_status`. A working setup returns server status, timing, and rate-limit metadata.
 
+Every server process negotiates once against `GET /api/v1/pro/capabilities` and retains the contract only in memory. `get_mcp_status` reports the negotiated version and endpoint set. Negotiation is best-effort by default; when startup validation is enabled, authentication or entitlement failures stop startup. Older API deployments without the capability route fall back to the program-list validation probe.
+
 For simple tool selection, call `get_bbradar_guide`. It returns the recommended entry points, common filter meanings, and example calls without contacting the BBRadar API.
 
 Most tools that accept `program_id` also accept a human program name or handle. The server resolves names automatically when there is one clear match; for ambiguous names, call `resolve_program` first and retry with the returned `platform:handle` id.
 
+Self-Hosted programs use IDs such as `Self-Hosted:program-slug`. They are available through program search, exact program detail, and program-target tools. Their `data_capabilities` field identifies unavailable source signals; the MCP server does not invent opportunity scores, public report counts, target-change history, submission costs, or repository changes for that catalog.
+
+`search_programs` accepts a `search` value and sends it to the API for indexed server-side filtering. Name resolution uses the same path automatically, avoiding broad page scans for exact or partial program names.
+
 `find_programs_by_target` defaults to flexible target matching. It accepts bare domains, full URLs, wildcard scopes, target names, package/repo-style identifiers, or short phrases like `known api target`.
+It uses the Pro API's direct reverse-target-search route first, including for Self-Hosted targets, and only falls back to sampled target lists when an older API deployment does not expose that route.
+
+`export_targets` uses the bounded bearer `POST /api/v1/pro/targets/export` route directly. JSON exports preserve `program_id` on every target; CSV previews are capped in the tool result while the complete payload remains available for ten minutes through the returned `bbradar://exports` resource. The local resource store is capped at 25 entries and 64 MiB in aggregate, and CSV formula/control-byte neutralization is applied independently of the API.
+
+`get_program_targets` forwards text, target type, eligibility, minimum Target Opportunity Score, Target Opportunity Label, and competition-risk filters to the API. Compact output retains the score, label, competition risk, and Why This Target; full output also includes the score breakdown and reason fields.
+
+`get_repository_changes` exposes the read-only repository event feed. It defaults to `include_targets=false` and compact output, never contains source code or diffs, and supports `after_id` cursor polling. Both change-feed tools keep at most 500 cursors keyed separately by endpoint and normalized filters. Seed a stream once with `after_id`, then use `resume_cursor=true`; the server stores `next_after_id` even for an empty page. Cursor reads continue sequentially while `has_more=true`, bounded by `max_cursor_pages` (default `2`) and `cursor_time_budget_ms` (default `3000`). Cursor requests bypass the response cache so an empty page cannot hide newly arriving changes.
+
+Cursors are memory-only by default. Set `BBRADAR_MCP_CURSOR_STATE_PATH` to opt into restart-safe cursor persistence. The state file contains only endpoint-prefixed SHA-256 filter hashes and non-negative numeric IDs—never raw filters, API responses, program or target records, credentials, or authorization headers.
+
+Large target and change collections return at most 50 inline records by default. The remainder is held in a bounded, ten-minute in-memory continuation and exposed through a `bbradar://continuations` resource link. This keeps model context small without losing access to the complete bounded result.
 
 ## Performance Tuning
 
-The defaults expose the complete tool surface and full text results while enabling bounded concurrency, positive response caching, in-flight request coalescing, and short negative caching for repeated 404/405 lookups.
+The defaults expose the 16-tool core surface and compact text results while enabling bounded concurrency, positive response caching, in-flight request coalescing, stale-while-revalidate program metadata, endpoint circuit breakers, and short negative caching for repeated 404/405 lookups. Set the tool profile to `full` when all specialized tools are needed.
 
 | Environment variable | Default | Purpose |
 | --- | ---: | --- |
@@ -123,14 +140,31 @@ The defaults expose the complete tool surface and full text results while enabli
 | `BBRADAR_MCP_CACHE_MAX_ENTRIES` | `500` | Maximum entries in each local response cache. |
 | `BBRADAR_MCP_MAX_CONCURRENT_REQUESTS` | `6` | Maximum simultaneous upstream BBRadar API requests. |
 | `BBRADAR_MCP_REQUEST_TIMEOUT_SECONDS` | `30` | Timeout for each active upstream request; queue time is excluded. |
-| `BBRADAR_MCP_COMPACT_TOOL_TEXT` | `false` | Set to `true` to avoid duplicating results for clients that consume MCP `structuredContent`. |
-| `BBRADAR_MCP_TOOL_PROFILE` | `full` | Complete specialized tool surface. Set to `core` only when a smaller 15-tool discovery payload is preferred. |
+| `BBRADAR_MCP_COMPACT_TOOL_TEXT` | `true` | Avoid duplicating full results for clients that consume MCP `structuredContent`. Set to `false` for legacy text-only clients. |
+| `BBRADAR_MCP_TOOL_PROFILE` | `core` | Compact 16-tool discovery surface. Set to `full` for every specialized tool. |
+| `BBRADAR_MCP_PROGRAM_STALE_WHILE_REVALIDATE_SECONDS` | `300` | Maximum stale window for exact program metadata while one background refresh runs. |
+| `BBRADAR_MCP_CIRCUIT_BREAKER_FAILURE_THRESHOLD` | `3` | Consecutive retryable endpoint failures before its circuit opens. |
+| `BBRADAR_MCP_CIRCUIT_BREAKER_COOLDOWN_SECONDS` | `30` | Endpoint circuit cooldown before a new request is admitted. |
+| `BBRADAR_MCP_CURSOR_STATE_PATH` | unset | Optional path for numeric cursor-only restart persistence. API responses are never written there. |
+| `BBRADAR_MCP_CONTINUATION_INLINE_LIMIT` | `50` | Maximum records kept inline before a local continuation resource is returned. |
+| `BBRADAR_MCP_CONTINUATION_TTL_SECONDS` | `600` | Lifetime of in-memory continuation results. |
+| `BBRADAR_MCP_ALLOW_INSECURE_HTTP` | `false` | Explicitly allow an HTTP API base URL outside loopback for a trusted private development network. Never enable for internet traffic. |
+
+No performance variables are required. Capability-derived page limits, caching, request coalescing, server-side name resolution, reverse-target search, early stopping, bounded concurrency, aggregate-call parallelism, a 64 KiB request-body ceiling, a 16 MiB upstream response ceiling, automatic HTTP compression handling, connection reuse, and upstream quota protection are always active. Unsupported capability-gated tools are removed from discovery. Target responses expose upstream pagination state so an AI client does not mistake a capped page for a complete inventory.
+
+The client learns the API's `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset` headers per endpoint group. Once an observed group reaches zero, it blocks additional requests to that group locally until reset instead of repeatedly hitting the API, while unrelated endpoint groups remain usable. Target and repository changes share one quota state and one conservative two-request concurrency pool. A rolling 60-second guard also enforces the documented shared key-and-IP ceiling across uncached upstream requests. Real `423`, `429`, and `503` responses honor `Retry-After` as a local cooldown, capped at ten minutes so a malformed upstream header cannot lock the MCP process indefinitely. Requests are not automatically retried, so one MCP action cannot amplify an admission or rate-limit response into more API calls. The global upstream concurrency default is six and bulk exports are capped at two; cursor page walking is strictly sequential.
+
+The bearer API destination must use HTTPS. Loopback HTTP remains available for local development; non-loopback HTTP requires the explicit `BBRADAR_MCP_ALLOW_INSECURE_HTTP=true` opt-in. API base URLs containing embedded credentials, query strings, fragments, or non-HTTP schemes are rejected before the key can be sent.
+
+Call `get_mcp_status` to inspect cache and stale-hit rates, circuit states, cursor persistence, concurrency, negotiated capabilities, upstream quotas, actual request counts, per-endpoint status totals, and timing.
+
+Run `npm run benchmark` for the deterministic local performance benchmark. It reports request and byte reductions for reverse target discovery, exact-program cache efficiency, and context reduction from bounded continuations.
 
-No performance variables are required. Caching, request coalescing, name-resolution improvements, early stopping, bounded concurrency, aggregate-call parallelism, and upstream quota protection are always active.
+### Flexible multi-signal discovery
 
-The client learns the API's `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset` headers per endpoint group. Once an observed group reaches zero, it blocks additional requests to that group locally until reset instead of repeatedly hitting the API, while unrelated endpoint groups remain usable. A rolling 60-second guard also enforces the documented shared key-and-IP ceiling across uncached upstream requests. A real `429` honors `Retry-After` as a global cooldown; requests are not automatically retried, so one MCP action cannot amplify a rate-limit response into more API calls. The global upstream concurrency default is six, with the heavier target-changes endpoint capped at three.
+Use `find_programs` as the generic ranked path for combinations of platform, reward, opportunity, scope breadth, technology, competition, and freshness signals. It supports both inclusion and exclusion through `platforms`, `exclude_platforms`, `web3`, `exclude_web3`, and the remaining scope/reward/activity filters. When a user describes an exclusion without naming exact platform values, call `list_filters` first and reuse the observed values. If an intentionally strict combination returns no candidates, relax one local threshold at a time; the upstream program pages are cached, so these fallback passes normally add no API requests.
 
-Call `get_mcp_status` to inspect cache hit rates, negative-cache hits, concurrency state, the latest upstream quota state, actual upstream request counts, per-endpoint status totals, and timing.
+For repository-aware hunting, call `get_repository_changes` and `find_programs` in parallel with the same requested platform set. The repository feed supplies recent events and scored targets while `find_programs` supplies reward and ranking data. Intersect their program ids, discard non-requested programs that merely share the same repository, and treat targets as event-level associations when the feed omits `program_id`. This remains generic across platforms and technology stacks and avoids a specialized tool for every prompt variation.
 
 ### Optimized wildcard discovery
 
@@ -149,7 +183,7 @@ For the latest added wildcard targets from one platform with reward and report-c
 
 This path uses one recent-changes request. Use `find_wildcard_programs` for a ranked current-program shortlist, or `find_recently_added_wildcards` when recent wildcard activity matters. Both accept platform, minimum reward, report-count, and bounded target-sample filters. Wildcard shortlist workflows automatically send the `wildcard` target-type tag upstream before applying reward and report filters locally, so their page budget is spent only on relevant programs. `find_low_competition_high_reward` also accepts `min_wildcards`.
 
-## Tools
+## Full-profile tools
 
 - `search_programs`
 - `get_program`
@@ -157,6 +191,7 @@ This path uses one recent-changes request. Use `find_wildcard_programs` for a ra
 - `run_program_name_action`
 - `get_program_targets`
 - `get_recent_changes`
+- `get_repository_changes`
 - `get_opportunities`
 - `find_programs`
 - `find_wildcard_programs`

package/benchmark/mcp-benchmark.mjs

@@ -0,0 +1,167 @@
+import http from "node:http";
+import { performance } from "node:perf_hooks";
+
+import { BBRadarClient } from "../dist/bbradarClient.js";
+
+const programCount = 100;
+const targetsPerProgram = 20;
+const programs = Array.from({ length: programCount }, (_, index) => ({
+  id: `Platform:program-${index}`,
+  name: `Program ${index}`,
+  platform: "Platform"
+}));
+const targets = new Map(programs.map((program, programIndex) => [
+  program.id,
+  Array.from({ length: targetsPerProgram }, (_, targetIndex) => ({
+    identifier: `api-${targetIndex}.program-${programIndex}.example`,
+    target_type: "domain",
+    in_scope: true,
+    eligible_for_bounty: true
+  }))
+]));
+const counters = { requests: 0, responseBytes: 0 };
+
+const api = http.createServer((req, res) => {
+  counters.requests += 1;
+  const url = new URL(req.url ?? "/", "http://127.0.0.1");
+  let payload;
+
+  if (url.pathname === "/api/v1/pro/programs") {
+    payload = { programs, meta: { total: programs.length, total_pages: 1 } };
+  } else if (url.pathname === "/api/v1/pro/targets/search") {
+    payload = {
+      results: [{ program: programs[42], target: targets.get(programs[42].id)[0] }],
+      meta: { total: 1, returned: 1 }
+    };
+  } else if (url.pathname === "/api/v1/pro/programs/Platform:program-42") {
+    payload = programs[42];
+  } else {
+    const match = url.pathname.match(/^\/api\/v1\/pro\/programs\/(.+)\/targets$/);
+    payload = match
+      ? { targets: targets.get(decodeURIComponent(match[1])) ?? [], meta: { total: targetsPerProgram } }
+      : { detail: "not found" };
+    if (!match) {
+      res.statusCode = 404;
+    }
+  }
+
+  const body = JSON.stringify(payload);
+  counters.responseBytes += Buffer.byteLength(body);
+  res.setHeader("content-type", "application/json");
+  res.end(body);
+});
+
+await new Promise((resolve) => api.listen(0, "127.0.0.1", resolve));
+const { port } = api.address();
+const baseUrl = `http://127.0.0.1:${port}/api/v1`;
+const headers = { authorization: "Bearer benchmark" };
+
+try {
+  resetCounters();
+  const baselineStarted = performance.now();
+  const listed = await getJson(`${baseUrl}/pro/programs`, headers);
+  let baselineMatch;
+  for (const program of listed.programs) {
+    const response = await getJson(`${baseUrl}/pro/programs/${encodeURIComponent(program.id)}/targets`, headers);
+    baselineMatch = response.targets.find((target) => target.identifier === "api-0.program-42.example") ?? baselineMatch;
+  }
+  const baseline = snapshot(performance.now() - baselineStarted);
+
+  resetCounters();
+  const optimizedStarted = performance.now();
+  const optimizedMatch = await getJson(
+    `${baseUrl}/pro/targets/search?q=api-0.program-42.example&match_mode=exact&page=1&page_size=1`,
+    headers
+  );
+  const optimized = snapshot(performance.now() - optimizedStarted);
+
+  const client = new BBRadarClient({
+    apiKey: "benchmark",
+    apiBaseUrl: baseUrl,
+    webBaseUrl: "https://bbradar.io",
+    cacheTtlMs: 30_000,
+    negativeCacheTtlMs: 15_000,
+    cacheMaxEntries: 500,
+    defaultRateLimitPerMinute: 60,
+    exportRateLimitPerMinute: 15,
+    maxConcurrentRequests: 6,
+    compactToolText: true,
+    toolProfile: "core",
+    requestTimeoutMs: 5_000,
+    validateOnStartup: false
+  });
+  resetCounters();
+  const cacheStarted = performance.now();
+  for (let index = 0; index < 50; index += 1) {
+    await client.getProgram("Platform:program-42");
+  }
+  const cachedMetadata = snapshot(performance.now() - cacheStarted);
+
+  const largeResult = Array.from({ length: 500 }, (_, index) => ({
+    identifier: `target-${index}.example`,
+    target_opportunity_score: 70 + (index % 20),
+    why_this_target: "Compact benchmark fixture"
+  }));
+  const fullContextBytes = Buffer.byteLength(JSON.stringify(largeResult));
+  const inlineContextBytes = Buffer.byteLength(JSON.stringify(largeResult.slice(0, 50)));
+  const requestReduction = percentReduction(baseline.requests, optimized.requests);
+  const byteReduction = percentReduction(baseline.response_bytes, optimized.response_bytes);
+  const contextReduction = percentReduction(fullContextBytes, inlineContextBytes);
+
+  if (
+    requestReduction < 90 ||
+    byteReduction < 90 ||
+    cachedMetadata.requests !== 1 ||
+    contextReduction < 80
+  ) {
+    throw new Error("Performance regression threshold failed.");
+  }
+
+  process.stdout.write(`${JSON.stringify({
+    fixture: { program_count: programCount, targets_per_program: targetsPerProgram },
+    reverse_target_discovery: {
+      baseline,
+      optimized,
+      request_reduction_percent: requestReduction,
+      byte_reduction_percent: byteReduction,
+      match_found: Boolean(baselineMatch && optimizedMatch.results?.length === 1)
+    },
+    repeated_exact_program_metadata: {
+      calls: 50,
+      upstream: cachedMetadata,
+      cache_hit_ratio: client.diagnostics().response_cache.hits / 50
+    },
+    bounded_model_context: {
+      full_500_record_bytes: fullContextBytes,
+      inline_50_record_bytes: inlineContextBytes,
+      byte_reduction_percent: contextReduction
+    }
+  }, null, 2)}\n`);
+} finally {
+  api.close();
+}
+
+async function getJson(url, requestHeaders) {
+  const response = await fetch(url, { headers: requestHeaders });
+  if (!response.ok) {
+    throw new Error(`Benchmark request failed: ${response.status}`);
+  }
+  return response.json();
+}
+
+function resetCounters() {
+  counters.requests = 0;
+  counters.responseBytes = 0;
+}
+
+function snapshot(durationMs) {
+  return {
+    requests: counters.requests,
+    response_bytes: counters.responseBytes,
+    duration_ms: Math.round(durationMs * 100) / 100
+  };
+}
+
+function percentReduction(before, after) {
+  return before === 0 ? 0 : Math.round((1 - after / before) * 10_000) / 100;
+}

package/dist/bbradarClient.d.ts

@@ -7,6 +7,7 @@ export type ApiRequestOptions = {
     body?: unknown;
     cache?: boolean;
     coalesce?: boolean;
+    staleWhileRevalidateMs?: number;
 };
 export type ApiResult<T> = {
     data: T;
@@ -17,6 +18,13 @@ export type ApiResult<T> = {
     status: number;
     fetchedAt: string;
     cacheExpiresAt?: string | undefined;
+    stale?: boolean | undefined;
+};
+type CapabilityNegotiationState = {
+    status: "not_started" | "negotiated" | "legacy" | "failed";
+    negotiatedAt?: string | undefined;
+    contract?: unknown;
+    error?: string | undefined;
 };
 export declare class BBRadarApiError extends Error {
     readonly requestId: string;
@@ -50,9 +58,12 @@ export declare class BBRadarClient {
     private readonly endpointRequestQueues;
     private readonly endpointActiveRequests;
     private readonly upstreamRateLimits;
+    private readonly circuitBreakers;
     private readonly upstreamRequestTimestampsMs;
     private activeRequests;
     private globalBlockedUntilMs;
+    private capabilityNegotiation;
+    private capabilityState;
     constructor(config: ServerConfig);
     diagnostics(): {
         response_cache: {
@@ -64,6 +75,7 @@ export declare class BBRadarClient {
             sets: number;
             evictions: number;
             expirations: number;
+            stale_hits: number;
             enabled: boolean;
         };
         negative_response_cache: {
@@ -75,15 +87,36 @@ export declare class BBRadarClient {
             sets: number;
             evictions: number;
             expirations: number;
+            stale_hits: number;
             enabled: boolean;
         };
+        capability_negotiation: CapabilityNegotiationState;
+        circuit_breakers: {
+            failure_threshold: number;
+            cooldown_ms: number;
+            by_endpoint: {
+                [k: string]: {
+                    state: string;
+                    consecutive_failures: number;
+                    open_until: string | undefined;
+                    last_failure_at: string | undefined;
+                    last_success_at: string | undefined;
+                    last_status: number | undefined;
+                };
+            };
+        };
         in_flight_requests: number;
         request_concurrency: {
             limit: number;
             active: number;
             queued: number;
             endpoint_limits: {
-                target_changes: {
+                change_feeds: {
+                    limit: number;
+                    active: number;
+                    queued: number;
+                };
+                target_export: {
                     limit: number;
                     active: number;
                     queued: number;
@@ -141,27 +174,43 @@ export declare class BBRadarClient {
         };
     };
     validateKey(): Promise<void>;
+    negotiateCapabilities(): Promise<void>;
+    capabilitySnapshot(): CapabilityNegotiationState;
+    supportsEndpoint(endpoint: string): boolean;
+    endpointCapability(endpoint: string): Record<string, unknown> | undefined;
+    endpointNumber(endpoint: string, field: string, fallback: number): number;
+    endpointStringArray(endpoint: string, field: string, fallback: readonly string[]): string[];
+    private fetchCapabilityContract;
     listPrograms(query: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
+    getCapabilities(options?: Pick<ApiRequestOptions, "cache" | "coalesce">): Promise<ApiResult<unknown>>;
     getProgram(programId: string): Promise<ApiResult<unknown>>;
-    getProgramTargets(programId: string): Promise<ApiResult<unknown>>;
+    getProgramTargets(programId: string, query?: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
     getRecentChanges(query: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
+    getRepositoryChanges(query: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
     searchTargets(query: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
     getOpportunities(level: string, query: Record<string, QueryValue>): Promise<ApiResult<unknown>>;
     exportTargets(body: Record<string, unknown>): Promise<ApiResult<unknown>>;
     request<T = unknown>(options: ApiRequestOptions): Promise<ApiResult<T>>;
     private fetchFromApi;
+    private refreshInBackground;
     private withRequestSlot;
     private withGlobalRequestSlot;
     private withEndpointRequestSlot;
     private recordEndpointMetric;
     private reserveUpstreamRequest;
     private observeUpstreamRateLimit;
+    private observeUpstreamErrorBody;
     private rateLimitState;
     private refreshUpstreamRequestWindow;
     private upstreamRequestWindowDiagnostics;
     private shouldEnforceSharedRequestWindow;
     private refreshUpstreamRateLimitWindow;
     private upstreamRateLimitDiagnostics;
+    private assertCircuitClosed;
+    private observeCircuitResponse;
+    private observeCircuitFailure;
+    private circuitBreakerDiagnostics;
     private buildUrl;
     private headers;
 }
+export {};