@cyanheads/secedgar-mcp-server 0.15.6 → 0.15.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +35 -23
- package/CLAUDE.md +35 -23
- package/README.md +112 -159
- package/changelog/0.15.x/0.15.7.md +36 -0
- package/changelog/0.15.x/0.15.8.md +29 -0
- package/changelog/template.md +7 -7
- package/dist/config/server-config.d.ts +1 -0
- package/dist/config/server-config.d.ts.map +1 -1
- package/dist/config/server-config.js +8 -1
- package/dist/config/server-config.js.map +1 -1
- package/dist/index.js +5 -3
- package/dist/index.js.map +1 -1
- package/dist/mcp-server/resources/definitions/filing-types.resource.js +2 -2
- package/dist/mcp-server/resources/definitions/filing-types.resource.js.map +1 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.d.ts +8 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.js +39 -34
- package/dist/mcp-server/tools/definitions/company-search.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/compare-companies.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/compare-companies.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/compare-companies.tool.js +8 -0
- package/dist/mcp-server/tools/definitions/compare-companies.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/dataframe-query.tool.d.ts +31 -2
- package/dist/mcp-server/tools/definitions/dataframe-query.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/dataframe-query.tool.js +53 -5
- package/dist/mcp-server/tools/definitions/dataframe-query.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/fetch-frames.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/fetch-frames.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/fetch-frames.tool.js +8 -0
- package/dist/mcp-server/tools/definitions/fetch-frames.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/find-holders.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/find-holders.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/find-holders.tool.js +9 -1
- package/dist/mcp-server/tools/definitions/find-holders.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.js +9 -1
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-filing.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/get-filing.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-filing.tool.js +24 -11
- package/dist/mcp-server/tools/definitions/get-filing.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-financials.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/get-financials.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-financials.tool.js +10 -0
- package/dist/mcp-server/tools/definitions/get-financials.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.d.ts +8 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.js +9 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.d.ts +14 -4
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.js +167 -34
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-institutional-holdings.tool.d.ts +11 -4
- package/dist/mcp-server/tools/definitions/get-institutional-holdings.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-institutional-holdings.tool.js +113 -55
- package/dist/mcp-server/tools/definitions/get-institutional-holdings.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-material-events.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/get-material-events.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-material-events.tool.js +47 -33
- package/dist/mcp-server/tools/definitions/get-material-events.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-snapshot.tool.d.ts +7 -0
- package/dist/mcp-server/tools/definitions/get-snapshot.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-snapshot.tool.js +10 -0
- package/dist/mcp-server/tools/definitions/get-snapshot.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/index.d.ts +141 -15
- package/dist/mcp-server/tools/definitions/index.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/search-filings.tool.d.ts +14 -4
- package/dist/mcp-server/tools/definitions/search-filings.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/search-filings.tool.js +49 -38
- package/dist/mcp-server/tools/definitions/search-filings.tool.js.map +1 -1
- package/dist/services/canvas-bridge/canvas-bridge.d.ts +4 -1
- package/dist/services/canvas-bridge/canvas-bridge.d.ts.map +1 -1
- package/dist/services/canvas-bridge/canvas-bridge.js +4 -1
- package/dist/services/canvas-bridge/canvas-bridge.js.map +1 -1
- package/dist/services/edgar/edgar-api-service.d.ts +82 -51
- package/dist/services/edgar/edgar-api-service.d.ts.map +1 -1
- package/dist/services/edgar/edgar-api-service.js +332 -167
- package/dist/services/edgar/edgar-api-service.js.map +1 -1
- package/dist/services/edgar/filing-headers.d.ts +37 -9
- package/dist/services/edgar/filing-headers.d.ts.map +1 -1
- package/dist/services/edgar/filing-headers.js +40 -11
- package/dist/services/edgar/filing-headers.js.map +1 -1
- package/dist/services/edgar/filing-to-text.d.ts.map +1 -1
- package/dist/services/edgar/filing-to-text.js +24 -2
- package/dist/services/edgar/filing-to-text.js.map +1 -1
- package/dist/services/edgar/mirror/tickers-sync.d.ts +4 -2
- package/dist/services/edgar/mirror/tickers-sync.d.ts.map +1 -1
- package/dist/services/edgar/mirror/tickers-sync.js +4 -2
- package/dist/services/edgar/mirror/tickers-sync.js.map +1 -1
- package/dist/services/edgar/mirror/types.d.ts +6 -2
- package/dist/services/edgar/mirror/types.d.ts.map +1 -1
- package/dist/services/edgar/ownership-parser.d.ts +16 -0
- package/dist/services/edgar/ownership-parser.d.ts.map +1 -1
- package/dist/services/edgar/ownership-parser.js +19 -0
- package/dist/services/edgar/ownership-parser.js.map +1 -1
- package/dist/services/edgar/submissions-archive.d.ts +78 -0
- package/dist/services/edgar/submissions-archive.d.ts.map +1 -0
- package/dist/services/edgar/submissions-archive.js +87 -0
- package/dist/services/edgar/submissions-archive.js.map +1 -0
- package/package.json +7 -7
- package/server.json +17 -3
package/AGENTS.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Agent Protocol
|
|
2
2
|
|
|
3
3
|
**Server:** secedgar-mcp-server
|
|
4
|
-
**Version:** 0.15.
|
|
5
|
-
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.
|
|
4
|
+
**Version:** 0.15.8
|
|
5
|
+
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.6`
|
|
6
6
|
**Engines:** Bun ≥1.4.0, Node ≥24.0.0
|
|
7
7
|
|
|
8
8
|
Query SEC EDGAR filings, XBRL financials, and company data through MCP. Read-only, no API keys required. Full design: `docs/sec-edgar-mcp-design.md`.
|
|
@@ -44,14 +44,14 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
44
44
|
|
|
45
45
|
| Name | Description | Key Inputs |
|
|
46
46
|
|:-----|:------------|:-----------|
|
|
47
|
-
| `secedgar_company_search` | Find companies and retrieve entity info with optional recent filings | `query`, `include_filings?`, `
|
|
48
|
-
| `secedgar_search_filings` | Search EDGAR filings since 1993 — full-text (2001+) plus archive-backed browse for pre-2001 ranges | `query?`, `forms?`, `
|
|
47
|
+
| `secedgar_company_search` | Find companies and retrieve entity info with optional recent filings | `query`, `include_filings?`, `forms?`, `filing_limit?`, `filed_after?`, `filed_before?` |
|
|
48
|
+
| `secedgar_search_filings` | Search EDGAR filings since 1993 — full-text (2001+) plus archive-backed browse for pre-2001 ranges | `query?`, `forms?`, `filed_after?`, `filed_before?`, `limit?` |
|
|
49
49
|
| `secedgar_get_filing` | Fetch a specific filing's metadata and document content | `accession_number`, `cik?`, `content_limit?`, `document?` |
|
|
50
50
|
| `secedgar_get_financials` | Get historical XBRL financial data for a company | `company`, `concept`, `taxonomy?`, `period_type?`, `limit?` |
|
|
51
51
|
| `secedgar_get_snapshot` | One-call financial profile — latest value of every supported concept from a single companyfacts read | `company`, `taxonomy?`, `period_type?` |
|
|
52
52
|
| `secedgar_get_material_events` | 8-K filings with item codes decoded, filterable by item | `company`, `items?`, `filed_after?`, `filed_before?`, `limit?` |
|
|
53
|
-
| `secedgar_get_insider_transactions` | Form 4 / 4-A insider transactions parsed from ownership XML | `
|
|
54
|
-
| `secedgar_get_institutional_holdings` | 13F-HR quarterly institutional holdings parsed from the information table | `
|
|
53
|
+
| `secedgar_get_insider_transactions` | Form 4 / 4-A insider transactions parsed from ownership XML | `company`, `transaction_type?`, `limit?`, `filed_after?`, `filed_before?` |
|
|
54
|
+
| `secedgar_get_institutional_holdings` | 13F-HR quarterly institutional holdings parsed from the information table | `company`, `quarter?`, `limit?`, `offset?`, `consolidate?` |
|
|
55
55
|
| `secedgar_find_holders` | Reverse 13F lookup — which institutional managers reported holding an issuer | `issuer`, `cusip?`, `quarter?`, `limit?` |
|
|
56
56
|
| `secedgar_get_beneficial_owners` | 5%+ blockholders of an issuer from structured SCHEDULE 13D/13G XML | `issuer`, `form_kind?`, `include_amendments?`, `limit?` |
|
|
57
57
|
| `secedgar_get_fund_holdings` | Fund portfolio holdings from the quarterly NPORT-P report | `fund`, `series_id?`, `report_date?`, `limit?`, `offset?` |
|
|
@@ -79,10 +79,11 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
79
79
|
|
|
80
80
|
## Domain Notes
|
|
81
81
|
|
|
82
|
-
- **Rate limit:** 10 req/s per IP
|
|
82
|
+
- **Rate limit:** 10 req/s per IP. Every SEC request goes through one process-wide `createPacer` queue in `EdgarApiService` that spaces request *starts* `1000 / EDGAR_RATE_LIMIT_RPS` ms apart, with no concurrency cap and no wait budget — an ordinary burst drains at the configured rate and is never shed, and this server's own traffic can never exceed it, so `rateLimitRps` stays at 10 and headroom for a co-tenant on the outbound IP is the operator's call via `EDGAR_RATE_LIMIT_RPS`. A 429 is **not** retried: SEC holds the block until the request rate has stayed under the threshold for ten minutes and every request sent meanwhile restarts that clock, so the ~3s retry budget (`MAX_RETRIES` 3 × `BASE_BACKOFF_MS` 1000, exponential) cannot outlast it (#112). Instead a 429 closes the service's **block gate** for `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` (default 600). While it is closed nothing is sent: a call fails locally as `RateLimited` with `retryAfter` set to the whole seconds remaining (never below 1), and the gate is re-read at dispatch, so a call already queued in the pacer when the 429 landed — or a 5xx retry sleeping through its backoff — is refused too. The first call after the cool-down is sent alone as a probe while every other call waits on it: any HTTP answer but a 429 reopens normal pacing, a probe 429 closes the gate for the full cool-down again, and a probe that gets no HTTP answer (a network error) proves nothing, so the next call probes in its place. **The gate lives in the service, not in the pacer's `cooldown` option** — that gate holds queued calls until it reopens and then releases them all at the pacing rate, so it can neither refuse locally nor keep the reopening to one request. The upstream 429 and the local refusal carry one `data.reason: 'rate_limited'`, declared (`thrownBy: 'service'`, `retryable: true`) on every tool that reaches SEC, plus `data.retryable: true` — `ctx.fail` copies a contract entry's `retryable` onto `data`, a service throw does not, so both throw sites set it themselves (#122) — and one recovery hint naming SEC's ten-minute window and the seconds to wait; SEC's HTML block page stays out of the payload (`captureBody: false`). Mirror-served reads (the ticker base behind `resolveCik`, and `tryGetCompanyFacts` / `tryGetCompanyConcept` / `tryGetFrames` through `mirrorOrLive`) never reach `rawFetch`, so they keep answering during a block. 500/502/503/504 still retry with the backoff. `createApp({ teardown })` disposes the pacer through `disposeEdgarApiService()` (#116)
|
|
83
|
+
- **Parameter names:** one name per concept across tools — `company` (the entity a tool reads; on `get_institutional_holdings` that is the 13F filer), `filed_after` / `filed_before` (inclusive filing-date bounds), `forms`. Every tool carrying one of those keys declares `inputAliases` for the other spellings in use — `ticker`, `cik`, `ticker_or_cik` → `company`; `start_date`, `date_from` → `filed_after`; `end_date`, `date_to` → `filed_before`; `form_types` → `forms` — so a caller carrying a name from one tool to the next is not rejected (#115). An alias is declared only where it maps to exactly one key of that tool and its value shape fits: a singular `form_type` cannot fill the `forms` array, so it is left out rather than widening the schema. `issuer`, `fund`, `companies`, and `query` name different roles and are not aliased. `search_filings` still filters by date only with both bounds; `company_search`, `get_material_events`, and `get_insider_transactions` accept either alone
|
|
83
84
|
- **User-Agent required:** `"AppName contact@email.com"` on every SEC request or IP gets blocked
|
|
84
85
|
- **CIK zero-padding:** URLs require 10-digit zero-padded CIK (`String(cik).padStart(10, '0')`)
|
|
85
|
-
- **ETF/mutual-fund tickers:** `loadTickerCache` fetches `company_tickers_mf.json` alongside `company_tickers.json` and merges fund symbols (VOO, SCHD, JEPI…) into `byTicker`. Fund CIKs are registrant trusts (1:many with series), so
|
|
86
|
+
- **ETF/mutual-fund tickers:** `loadTickerCache` fetches `company_tickers_mf.json` alongside `company_tickers.json` and merges fund symbols (VOO, SCHD, JEPI…) into `byTicker`. Fund CIKs are registrant trusts (1:many with series), so fund rows go into `byTicker` only — never `byCik`, whichever source they come from. Resolved fund matches carry `seriesId`/`classId` from the live MF file, the only source of either. On a mirror-served ticker base the live MF fetch is merged in when `EDGAR_MIRROR_FALLBACK_LIVE` is on, so fund tickers resolve even if the mirror predates MF ingestion (#43); strict mirror-only (`FALLBACK_LIVE=false`) uses the mirror's own MF rows (ticker→CIK, no series/class). The mirror stores a fund symbol with an empty name — its schema has no series column — and `mirrorIndexRow` reads that empty name as the fund marker; a symbol claims `byTicker` in the order equity → live fund → mirror fund, so a live entry supersedes the mirror's bare row and keeps its series (#135). **The live fund slice has its own failure window:** a failed MF load (any HTTP failure, a malformed body, the block gate's refusal) is not cached for the ticker-cache TTL — `loadMfTickers` returns no rows and a `retryAt` `FUND_SLICE_RETRY_MS` (60 s) out, or the error's `retryAfter` when longer, so nothing is retried while the gate is closed. The first resolution after it refetches the file once, single-flight, onto the cached base (`reloadFundSlice`), which keeps its own load time: the equity index is not refetched and its TTL does not move. A per-resolution retry was rejected — against a persistent 5xx it adds three requests and ~3 s of backoff to every lookup (#119). Each failed load logs one warning through the framework's global `logger` (no request context to attach to) carrying the URL, the HTTP status or `malformed_body`, and the retry delay (#43).
|
|
86
87
|
- **Name and ticker resolution:** `resolveCik` walks numeric CIK → the `/^[A-Z]{1,5}$/` ticker fast path → the name passes (exact → prefix → substring over `allEntries`) → a catch-all `byTicker` lookup for symbols whose shape skipped the fast path. Two normalizations hang off that chain. The name passes' **exact** tier also compares suffix-normalized forms (`normalizeCompanySuffix`): the terminal whitespace-delimited token only, a trailing `.`/`,` stripped as part of recognizing it, expanded to one of four canonical long forms — `corp`→`corporation`, `inc`→`incorporated`, `co`→`company`, `ltd`→`limited`. **The four buckets are never merged.** The registry carries distinct registrants differing only in which suffix they use (`TORO CO` 0000737758 vs `TORO CORP.` 0001941131, and the same shape on Blue Owl and Fluent), so a single "has a corporate suffix" marker would compare two companies equal. Only the terminal token is rewritten — the same words sit mid-name in titles ending in a different suffix (`American Water Works Company, Inc.`) — and a terminal token that merely contains one (`Ltd./ADR`) is not a suffix. Mid-name punctuation is out of scope and still blocks a match (`Fluent Inc` does not reach `Fluent, Inc.`); so are `llc`/`plc` and the `lp`/limited-partnership family, which must never fold into the `ltd` bucket (#107). The **catch-all ticker lookup** retries a dotted symbol with `.`→`-` on a miss — brokers and market data write `BRK.B` where `company_tickers.json` lists `BRK-B`, and that file carries exactly one dotted ticker (a placeholder), so the substitution cannot land on the wrong registrant. The early gate stays untouched: it is a fast path, not the only ticker lookup, and `search_filings`' `ticker:` token inherits the fix through `resolveCik` with no change of its own (#110)
|
|
87
88
|
- **Former-name resolution:** `src/services/edgar/data/former-names.json` is a committed asset (generated offline via `bun run gen:former-names`) containing `[lowercasedFormerName, zeroPaddedCIK]` tuples. `buildTickerCache` folds them into `allEntries` for name search and the trigram pass. Regenerate on demand after major M&A waves — former names are immutable, so the file rarely needs updating.
|
|
88
89
|
- **Near-match suggestions:** `suggestCompanies(query, allEntries)` runs a Dice-coefficient trigram pass on the zero-hit search path, returning up to 3 candidates above a threshold. Each entry is scored on its **name and its ticker**, keeping the higher of the two — one ranked, deduped, top-N list rather than two disjoint ones, so a strong ticker match can outrank a weak name match and vice versa, and a ticker-shaped miss gets help instead of a bare `no_match` (#111). An entry missing either field is simply not scored on it: former-name entries carry a name and no ticker, and MF rows carry neither a name nor a place in `allEntries`, so fund tickers stay structurally unreachable here regardless of what the scorer does. Suggestions appear in the `no_match` error `data.suggestions` and in the error message. Never auto-resolves.
|
|
@@ -93,14 +94,16 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
93
94
|
- **`get_financials` period default:** with `period_type` unset the series defaults to `annual` (clean FY series); if the annual filter empties a non-empty series whose frames are all instant (`CY####Q#I` — balance-sheet, shares-outstanding, raw instant tags), it falls back to the full series so the first call returns data (#48). An explicit `period_type` is honored as-is and still errors when it excludes everything.
|
|
94
95
|
- **8-K item codes:** SEC replaced the original single-integer item numbering with the current `x.xx` scheme in Release 33-8400, effective **2004-08-23**, so EDGAR's `items` field carries two disjoint vocabularies. `eight-k-items.ts` decodes by the code's **shape**, never the filing date — dotted is current, a bare integer is legacy — because filers reported pre-changeover events under the old numbers for months after the effective date, and a date-keyed table would mislabel exactly those. The two vocabularies reuse integers for unrelated events (legacy `5` is Other Events, current `5.02` is officer departures; legacy `12` is the ancestor of `2.02`), so a dotted `items` filter never matches a pre-2004 filing — `get_material_events`' zero-hit notice says so and echoes the codes actually present. The tables back three surfaces: the tool's `items` enum, the decoded per-filing rows, and the `secedgar://filing-types` resource. A code matching a shape but absent from that regime's table comes back with no label rather than a guess.
|
|
95
96
|
- **EFTS quirks:** `dateRange=custom` must be set when using date params; the singular `entity` param is ignored — `cik:`/`ticker:` targeting passes the resolved CIK via the plural `ciks` param (server-side scope, independent of the filing's name text, so former-name filings on the same CIK are matched); `size` is ignored entirely and every request answers with 100 hits, so `find_holders` pages with `from` alone and expresses its budget in pages (5 → 500 rows)
|
|
96
|
-
- **
|
|
97
|
+
- **Submissions archive walk:** the submissions feed's `filings.recent` holds the last year or 1,000 filings, whichever is more — so 26,000+ rows for JPMorgan, and a filing day is never split (Microsoft's window runs to 1,002 rows to finish its oldest day); everything older lives on `filings.files[]` archive pages of 200–360 KB each. Every tool that reads past `recent` goes through `SubmissionsArchiveWalk` (`submissions-archive.ts`): one `ARCHIVE_PAGE_SCAN_CAP` (10) for all, pages read one at a time through the service's pacer and page cache, and no page selected when the lower bound falls on or after `recent`'s oldest date — so a call answered by `recent` sends no archive request. The caller picks the order — `newest-first` (`company_search`, `search_filings`' pre-2001 arm, `get_material_events`, `get_insider_transactions`' window), or `oldest-first` forward from a date (`get_institutional_holdings`' quarter lookup, since newest-first cannot reach JPMorgan's 2020-Q4 13F-HR on page 41 of 70 under the cap) — and can stop after any page; `scannedThrough` (the oldest page start read) and `truncated` (selected pages left unread, whether the cap, an early stop, or a caller that never started the walk left them) are reported the same way everywhere. Worst case per call is the submissions document plus 10 pages plus the tool's own document reads (`get_insider_transactions`: 111). `get_material_events` without a date window walks only to fill `limit`, stopping on the page that fills it with rows passing the `items` filter, and its `dataset.truncated` is true whenever selected pages went unread — including when `recent` alone fills `limit` and no page is read, since the dataframe then holds only the recent window (#134); `get_institutional_holdings` stops on the first page holding a 13F-HR or 13F-NT for the quarter, and runs the #86 operating-company routing only when neither `recent` nor the pages read hold any 13F-HR or 13F-NT — a notice-only manager asked for a quarter it filed nothing for gets the plain quarter miss (#117, #133); without a `quarter`, a manager whose `recent` holds 13F-NT notices and no 13F-HR gets the notice message for its newest notice rather than the #86 classification (#133). **Page selection widens both date bounds by a day:** a manifest page's `filingFrom` / `filingTo` can sit a day off its own rows in either direction (JPMorgan's page 041 is listed to 2021-02-17 but holds 42 filings dated 2021-02-18; Apple's page 001 is listed a day past its newest row), so an exact overlap test skipped the page holding a window's edge day. The slack costs at most one extra page per bound, and every caller filters rows by their own filing dates (#137)
|
|
98
|
+
- **Filing metadata outside `recent`:** `get_filing` takes `form`, `filing_date`, and `period_ending` from `recent` when the accession is there, else from the filing's own SEC header — the raw `<TYPE>` / `<FILING-DATE>` / `<PERIOD>` in `index-headers.html`, which the tool already fetches to type the documents, and only when that page 404s (common before 2014) the sub-1 KB `<accession>.hdr.sgml`. `FILING-DATE` is the filing date, not the acceptance timestamp, and matches the submissions feed; an S-8 or Form 4 header has no `PERIOD`. `company_name` stays the current registrant name (#126)
|
|
99
|
+
- **Filing content:** HTML → text via `html-to-text` library; pre-2005 filings produce noisier output. The library walks the DOM recursively, so `CONVERT_OPTIONS` caps it at `limits.maxDepth: 512` with a `[…]` ellipsis where the cut falls — without it deep markup overflowed the stack and one such candidate failed `search_filings`' whole pre-2001 text scan (#118). The bound must hold on both runtimes the server ships under: a fresh Node process overflows near 1,360 levels of nested lists (its most stack-hungry shape), Bun near 13,000; real HTML filings measure under 25 levels. Legacy SGML `<PAGE>` markers are replaced with a line break before parsing (`neutralizePageMarkers`) — the parser never closes them, so a text document otherwise nests one level per page and a long one would be cut at the cap; with them gone the deepest measured `.txt` sits under 100 (an EX-27 schedule's unclosed tags). Both changes leave every sampled filing's text byte-identical. `limits` deep-merges with the library defaults, so its 16,777,216-character `maxInputLength` (silent truncation, no ellipsis) still applies
|
|
97
100
|
- **Heading detection:** `detectHeadings` runs three line-anchored patterns — the all-caps forms, the mixed-case same-line `Item N. Title` / `Part II` forms (#71), and a bare `Item N` marker whose title lands on a later line. Every whitespace match is horizontal-only: `\s` matches `\n`, which fused runs of consecutive all-caps lines into composite entries (`"TABLE OF CONTENTS\n\nPART I"`) (#105). Horizontal-only is not space-or-tab — the all-caps arm alternates `[^\S\n]` in rather than listing literal members, because filings put NBSP inside all-caps headings too (`MICROSOFT<NBSP>CORPORATION`). The Item suffix spans `16A`–`16K` and the marker's period is optional — 20-F filers write the lettered markers unpunctuated. **A bare marker is admitted on the shape of what follows, not just on a title being there:** a blank line between marker and title admits the pair; without one, the pair is admitted only when no bare page-number line sits under the title, since marker/title/page-number on three tight lines is the TOC-cell artifact #71's guard exists to reject. That test is one-way — it rejects a TOC row, it does not certify a heading — so a marker line anywhere else (a page footer, an unwrapped paragraph) is admitted too, taking the next line as its title; the entry carries the marker's own offset, so `section` still lands in the right place. A heading text repeated `RUNNING_HEADER_MIN_OCCURRENCES` (5) times at distinct offsets is page furniture — C3is' FY2025 20-F repeats `TABLE OF CONTENTS` 129 times and a 10-Q stamps `PART I` on all 47 of its pages, while a heading printed once in the TOC and once in the body occurs twice. **Furniture is then handled by shape, two ways:** a bare marker (`Item 1`, `PART I`) carries no navigable text, so no occurrence of it earns an entry and every one is dropped; anything else keeps its FIRST occurrence and drops the rest, inverting the keep-later rule — a filing that runs a section's own heading across every page of that section (`NOTES TO THE CONSOLIDATED FINANCIAL STATEMENTS` through an S-1's financial statements) prints it first at the section start, and keep-later would land it on the section's last page instead. So the outline carries no fused composite and at most one entry per running header, at its first occurrence. **A bare marker is counted on its own line, never on the joined heading:** a page footer's marker takes its title from whatever prose the next page opens with, so every composite it produces is a different string and only the marker line repeats — count composites instead and a 10-Q's per-page `Item 1` footer yields one bogus entry per page, crowding the real Items past `maxEntries`. That count is per rendered form (`Item 1` and `Item 1.` are separate), since a filing stamping the bare form on every page still writes the punctuated one for the real heading. Dedup (keep the later, i.e. body, occurrence) keys on `foldForHeadingMatch` plus a terminal-punctuation strip, never raw lowercase: the same filing prints one marker with different space characters and a trailing period in its TOC and its body (`Item 3.` with U+0020 vs U+2009; `…Policies` vs `…Policies.`), and a raw key leaves both in the outline. Trailing bare page numbers are stripped from Item headings for the same reason. Lettered sub-item markers (a bare `A.`/`D.` inside an Item) are deliberately not detected — single letters recur throughout filing prose.
|
|
98
101
|
- **Section matching:** `get_filing`'s `section` compares through `foldForHeadingMatch` (`filing-to-text.ts`) on **both** operands — Unicode whitespace runs collapse to one space, U+2018/U+2019 → `'`, U+201C/U+201D → `"`, then lowercase. Comparison-time only: `FilingHeading.heading`, the `outline` output, `content[]`, and every character offset keep the document's own bytes. EDGAR headings carry NBSP runs and typographic quotes, so a caller re-sending an outline heading through a UI that normalizes either one never byte-matched the heading it came from — which broke the `section_not_found` recovery loop the error's own hint points at (#106). Still a plain substring test, not fuzzy matching — `"item 8"` resolves only to Item 8 — but run on folded operands, so a single-spaced needle now also reaches a heading whose marker carries an NBSP run or a double space (`Item 9.` in a 2004 10-K). Folding only one operand does not fix the round trip.
|
|
99
102
|
- **Binary filing documents:** a filing index routinely carries scanned pages, PDF exhibits, `Financial_Report.xlsx`, and `-xbrl.zip` — `filingToExtract` runs `html-to-text` over whatever bytes arrive and never inspects a content type, so requesting one used to return decoded binary as `content` with no error (#96). `get-filing.tool.ts` detects them by filename extension (`binaryTypeFromName`), because that is the only signal available before the body is fetched: the canonical `GRAPHIC` type lives in the submission header, which the archive path fetches *after* the document, and requiring it would trade today's parallel header/submissions fetch for a sequential one. Two halves, both needed — `documents[]` entries carry `binary: true` (`isBinaryDocument` also honors a header `GRAPHIC` type), and `resolveFilingArchive` throws `binary_document` before `tryGetFilingDocument` runs, since the `document` input is an unconstrained string and a catalog marker alone cannot stop a caller. Extension-driven and bucket-independent on purpose: a PDF exhibit is typed `EX-99.*` and files under `exhibits`, not among the graphics.
|
|
100
|
-
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and insider rows on `cusip`.
|
|
103
|
+
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints. `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and insider rows on `cusip`.
|
|
101
104
|
- **Beneficial ownership (13D/13G):** SEC replaced the legacy `SC 13D` / `SC 13G` text filings with structured XML under the `SCHEDULE 13D` / `SCHEDULE 13G` names on **2024-12-18**; the changeover is total, so a legacy-name filter matches nothing recent and a current-name filter matches nothing older. The two schedules are **separate schemas, not one shape with optional fields** — 13D repeats `reportingPersons/reportingPersonInfo` with the power fields flat on each person and the percentage in `percentOfClass`; 13G repeats `coverPageHeaderReportingPersonDetails` with the powers nested inside `reportingPersonBeneficiallyOwnedNumberOfShares`, the percentage in `classPercent`, no reporting-person CIK anywhere, and no purpose-of-transaction field at all (its item 4 is structured ownership data, which is what makes it the passive form). `beneficial-ownership-parser.ts` therefore branches per schema and dispatches on the submission type, falling back to whichever repeated reporting-person element the document carries. **Ownership powers and percent of class are per reporting person, including on joint filings** — a fund, its adviser, and its controlling principal each report the same underlying shares, so the parsed owner list is never collapsed to a group total and the percentages must not be summed. Discovery runs off the issuer's own `getSubmissions()`: SEC cross-lists a schedule under every CIK party to it, so no entity-scoped EFTS step is needed.
|
|
102
105
|
- **NPORT-P series routing:** a fund's public portfolio report covers **exactly one series**, a registrant trust holds many (iShares Trust: 390 listed series, 200+ reports for a single period), and the series identity appears **only inside the document** — neither the submissions feed nor EFTS carries `seriesId` on an NPORT-P row. Routing is therefore series-first: EDGAR's company browse accepts a series ID in place of a CIK (`getFundSeriesFilings`), which is the one SEC surface mapping a series to its accessions and which also names the registrant, so a bare series ID resolves in one request. The browse feed's registrant outranks any registrant the `fund` input implied, since `series_id` can name a series of an entirely different trust and the archive path is keyed on the registrant. The bounded header scan (`tryGetFilingDocumentHead` + `parseNportHeader`, cap `SERIES_SCAN_LIMIT`) is the fallback and the disambiguator for a trust whose series carry no listed ticker; SEC's archive host **ignores `Range`** (a ranged request answers 200 with the whole body), so the saving is client-side stream cancellation only — and the cutoff cannot cut below one read chunk, which Bun sets at 262,144 bytes, so `HEADER_MAX_BYTES` binds only on a document arriving in several chunks. **A registrant's own NPORT-P list is never the candidate set for a series-structured trust**: series fiscal quarters are staggered, so the list interleaves funds and one report at the newest period does not mean one fund — it means the fund whose quarter ends latest. The scan therefore runs on the newest batch even when that batch holds one report, and a series found there re-routes through the browse feed; only a report naming no series at all (a closed-end fund, which also carries no `classId`) is answered from the registrant's list. Candidates are ordered by **reported period, not filing date**: an `NPORT-P/A` restating an old period is filed after every report of the periods that followed it. Reports publish on a ~60-day lag, so `report_period_date` and `publication_lag_days` are first-class output and the `as_of` enrichment states both.
|
|
103
|
-
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (
|
|
106
|
+
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (superseding the mirror's bare fund rows with series/class, and covering a mirror synced before MF ingestion), so funds resolve there too (#43, #135).
|
|
104
107
|
|
|
105
108
|
---
|
|
106
109
|
|
|
@@ -110,9 +113,10 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
110
113
|
|:--------|:---------|:--------|:------------|
|
|
111
114
|
| `EDGAR_USER_AGENT` | **Yes** | — | User-Agent for SEC compliance. Format: `"AppName contact@email.com"` |
|
|
112
115
|
| `EDGAR_RATE_LIMIT_RPS` | No | `10` | Max requests/second to SEC APIs. Do not exceed 10. |
|
|
113
|
-
| `
|
|
116
|
+
| `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` | No | `600` | Seconds to stop sending to SEC after a 429. Calls are refused locally for this long, then the first one goes out alone as a probe. SEC lifts its block only after ten quiet minutes, so a shorter value just probes into it. |
|
|
117
|
+
| `EDGAR_TICKER_CACHE_TTL` | No | `3600` | Seconds to cache the ticker index (company_tickers.json + company_tickers_mf.json). A failed fund-file load is retried after 60 s (or the rate-limit cool-down) instead of standing for the TTL. |
|
|
114
118
|
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes. Sliding window touched on every dataframe op. |
|
|
115
|
-
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` on mcp-ts-core 0.13.
|
|
119
|
+
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` on mcp-ts-core 0.13.6 carries no per-tool definitions at all, so nothing changes there (#103). |
|
|
116
120
|
| `EDGAR_MIRROR_ENABLED` | No | `false` | Enable the local SQLite mirror of company_tickers + XBRL company-facts. Node/Bun only (skipped on Workers). Bootstrap once with `bun run mirror:init`. |
|
|
117
121
|
| `EDGAR_MIRROR_PATH` | No | `./data/edgar-mirror` | Directory holding the mirror SQLite databases (tickers + companyfacts). |
|
|
118
122
|
| `EDGAR_MIRROR_REFRESH_CRON` | No | — | In-process nightly refresh cron (HTTP transport only). Recommended `0 9 * * *`. Omit to refresh out-of-band via `bun run mirror:refresh`. |
|
|
@@ -125,12 +129,15 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
125
129
|
|
|
126
130
|
| Module | Path | Purpose |
|
|
127
131
|
|:-------|:-----|:--------|
|
|
128
|
-
| `EdgarApiService` | `src/services/edgar/edgar-api-service.ts` |
|
|
132
|
+
| `EdgarApiService` | `src/services/edgar/edgar-api-service.ts` | Paced HTTP client with the rate-limit block gate, CIK resolution, all SEC API calls |
|
|
133
|
+
| `submissions-archive` | `src/services/edgar/submissions-archive.ts` | `SubmissionsArchiveWalk` — the one walk over `filings.files[]` archive pages: shared page cap, caller-chosen order, early stop, scan depth and truncation |
|
|
134
|
+
| `filing-headers` | `src/services/edgar/filing-headers.ts` | SEC submission-header parsers: per-document types and the submission's form / filing date / period |
|
|
129
135
|
| `concept-map` | `src/services/edgar/concept-map.ts` | Static friendly name → XBRL tag mapping; `resolveConceptTarget` picks the taxonomy + tag list |
|
|
130
136
|
| `concept-series` | `src/services/edgar/concept-series.ts` | Shared frame dedup + tag selection (priority or per-filer coverage); one value per standard calendar period; series-staleness caveats |
|
|
131
137
|
| `fiscal-periods` | `src/services/edgar/fiscal-periods.ts` | Fiscal-Q4 frame caveat and per-filer missing-quarter detection |
|
|
132
138
|
| `eight-k-items` | `src/services/edgar/eight-k-items.ts` | 8-K item-code decode tables for both numbering regimes; shape-driven `decodeEightKItem` |
|
|
133
139
|
| `xml-nodes` | `src/services/edgar/xml-nodes.ts` | Prefix-insensitive tag lookup + value coercion shared by the SEC XML parsers |
|
|
140
|
+
| `ownership-parser` | `src/services/edgar/ownership-parser.ts` | Form 4 ownership XML, 13F information tables, and the 13F cover page (filer name, period) |
|
|
134
141
|
| `beneficial-ownership-parser` | `src/services/edgar/beneficial-ownership-parser.ts` | Sibling SCHEDULE 13D / 13G parsers with a submission-type dispatcher |
|
|
135
142
|
| `nport-parser` | `src/services/edgar/nport-parser.ts` | NPORT-P identity block (cheap, for series routing) and full portfolio report |
|
|
136
143
|
| `filing-to-text` | `src/services/edgar/filing-to-text.ts` | HTML → readable plain text conversion; heading detection and the comparison fold `section` matches through |
|
|
@@ -162,7 +169,7 @@ Handlers receive a unified `ctx` object. Key properties:
|
|
|
162
169
|
|
|
163
170
|
Handlers throw — the framework catches, classifies, and formats.
|
|
164
171
|
|
|
165
|
-
**Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable? }]` on `tool()`. The handler then receives `ctx.fail(reason, msg?, data?)` typed against the reason union, and `data.reason` is auto-populated for observability. The `recovery` field is required (≥5 words, lint-validated). Use `ctx.recoveryFor('reason')` to spread the contract recovery onto the wire; pass an explicit `{ recovery: { hint } }` when runtime context matters. Baseline codes (`InternalError`, `ServiceUnavailable`, `Timeout`, `ValidationError`, `SerializationError`, `RequestCancelled`) bubble freely without declaration. **All EDGAR tools that have known failure modes use this pattern.**
|
|
172
|
+
**Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable?, severity?, thrownBy? }]` on `tool()`. The handler then receives `ctx.fail(reason, msg?, data?)` typed against the reason union, and `data.reason` is auto-populated for observability. The `recovery` field is required (≥5 words, lint-validated). Use `ctx.recoveryFor('reason')` to spread the contract recovery onto the wire (mirrored into `content[]` unless the message already contains it verbatim); pass an explicit `{ recovery: { hint } }` when runtime context matters. Forwarding it is lint-enforced per throw site (`error-contract-recovery-unforwarded`). Mark an entry raised below the handler — a module-level helper, a service, the canvas bridge — with `thrownBy: 'service'` so `error-contract-unthrown` skips it; lint-only metadata, nothing at runtime reads it. Baseline codes (`InternalError`, `ServiceUnavailable`, `Timeout`, `ValidationError`, `SerializationError`, `RequestCancelled`) bubble freely without declaration. **All EDGAR tools that have known failure modes use this pattern.**
|
|
166
173
|
|
|
167
174
|
```ts
|
|
168
175
|
errors: [
|
|
@@ -177,7 +184,7 @@ async handler(input, ctx) {
|
|
|
177
184
|
}
|
|
178
185
|
```
|
|
179
186
|
|
|
180
|
-
**Service-layer pattern (no `ctx`).** Throw an `McpError` with `data: { reason, recovery: { hint } }`. The auto-classifier preserves `data` on the wire so clients see the same `error.data.reason` they'd see from `ctx.fail`.
|
|
187
|
+
**Service-layer pattern (no `ctx`).** Throw an `McpError` with `data: { reason, recovery: { hint } }`, and mark the tool's matching `errors[]` entry `thrownBy: 'service'`. The auto-classifier preserves `data` on the wire so clients see the same `error.data.reason` they'd see from `ctx.fail`. The tool error text closes with a term line rendering only what `data` carries — `(reason <reason>)` for a reason alone, `· retryable` / `· not retryable` appended when `data.retryable` is a boolean — so tests assert that `content[0].text` contains the diagnostic rather than pinning it exactly. A contract entry's `retryable` reaches `data` only through `ctx.fail`; a service throw sets `data.retryable` itself (#122).
|
|
181
188
|
|
|
182
189
|
**Fallback for ad-hoc throws** (no contract entry fits, prototype code): use error factories or plain `Error`.
|
|
183
190
|
|
|
@@ -197,17 +204,20 @@ See framework CLAUDE.md and the `api-errors` skill for the full reference.
|
|
|
197
204
|
|
|
198
205
|
```text
|
|
199
206
|
src/
|
|
200
|
-
index.ts # createApp() entry point — stateless sessionMode, teardown closes the mirror
|
|
207
|
+
index.ts # createApp() entry point — stateless sessionMode, teardown stops the SEC pacer and closes the mirror
|
|
201
208
|
config/
|
|
202
209
|
server-config.ts # EDGAR env vars (Zod schema)
|
|
203
210
|
services/
|
|
204
211
|
edgar/
|
|
205
|
-
edgar-api-service.ts #
|
|
212
|
+
edgar-api-service.ts # Paced HTTP client + rate-limit block gate, CIK resolution
|
|
213
|
+
submissions-archive.ts # Shared archive-page walk (page cap, order, early stop)
|
|
214
|
+
filing-headers.ts # SEC submission-header parsers
|
|
206
215
|
concept-map.ts # Friendly name → XBRL tag mapping + taxonomy/tag resolution
|
|
207
216
|
concept-series.ts # Shared frame dedup, tag selection, staleness caveats
|
|
208
217
|
fiscal-periods.ts # Fiscal Q4 / missing-quarter caveats
|
|
209
218
|
eight-k-items.ts # 8-K item-code decode tables (current + legacy regimes)
|
|
210
219
|
xml-nodes.ts # Shared XML tag lookup + value coercion helpers
|
|
220
|
+
ownership-parser.ts # Form 4, 13F information table + cover page parsers
|
|
211
221
|
beneficial-ownership-parser.ts # SCHEDULE 13D / 13G parsers + dispatcher
|
|
212
222
|
nport-parser.ts # NPORT-P header (series routing) + full report
|
|
213
223
|
filing-to-text.ts # HTML → plain text
|
|
@@ -274,7 +284,7 @@ Available skills:
|
|
|
274
284
|
| `setup` | Post-init project orientation |
|
|
275
285
|
| `design-mcp-server` | Design tool surface, resources, and services for a new server |
|
|
276
286
|
| `add-tool` | Scaffold a new tool definition |
|
|
277
|
-
| `add-app-tool` | Scaffold an MCP App tool + UI resource
|
|
287
|
+
| `add-app-tool` | Scaffold an MCP App tool + paired UI resource |
|
|
278
288
|
| `add-resource` | Scaffold a new resource definition |
|
|
279
289
|
| `add-prompt` | Scaffold a new prompt definition |
|
|
280
290
|
| `add-service` | Scaffold a new service integration |
|
|
@@ -285,7 +295,7 @@ Available skills:
|
|
|
285
295
|
| `code-simplifier` | Post-session cleanup against `git diff` — modernize syntax, consolidate duplication, align with the codebase |
|
|
286
296
|
| `polish-docs-meta` | Finalize docs, README, metadata, and agent protocol for shipping |
|
|
287
297
|
| `git-wrapup` | Land working-tree changes as a commit stack — version bump, changelog, verify, commit by concern, release commit on top. No tag, no push to main; opens the release PR when the project declares release PR mode |
|
|
288
|
-
| `release-pr-review` | Review pass on an open release PR — simplifier + correctness review,
|
|
298
|
+
| `release-pr-review` | Review pass on an open release PR — simplifier + correctness review, fixes as ordinary commits on top of the stack, PR body kept in sync. Release PR mode only |
|
|
289
299
|
| `release-and-publish` | Fast-forward merge (release PR mode) + tag + push + npm + MCP Registry + GH Release + Docker. Picks up from `git-wrapup` |
|
|
290
300
|
| `report-issue-local` | File a bug or feature request against this server's own repo via `gh` CLI |
|
|
291
301
|
| `report-issue-framework` | File a bug or feature request against `@cyanheads/mcp-ts-core` via `gh` CLI |
|
|
@@ -295,10 +305,10 @@ Available skills:
|
|
|
295
305
|
| `api-auth` | Auth modes, scopes, JWT/OAuth |
|
|
296
306
|
| `api-config` | AppConfig, parseConfig, env vars |
|
|
297
307
|
| `api-canvas` | DataCanvas: register tabular data, run SQL, export, plus the `spillover()` helper for big result sets — Tier 3 opt-in |
|
|
298
|
-
| `api-context` | Context interface, logger, state,
|
|
308
|
+
| `api-context` | Context interface, RequestContext, logger, state, multi-round-trip input |
|
|
299
309
|
| `api-errors` | McpError, JsonRpcErrorCode, error patterns |
|
|
300
310
|
| `api-linter` | Definition linter rule catalog — invoked by `bun run lint:mcp` and `devcheck` |
|
|
301
|
-
| `api-mirror` | MirrorService: persistent local mirror of a bulk upstream dataset
|
|
311
|
+
| `api-mirror` | MirrorService: persistent self-refreshing local mirror (embedded SQLite + FTS5) of a bulk upstream dataset — Tier 3 opt-in |
|
|
302
312
|
| `api-services` | LLM, Speech, Graph services |
|
|
303
313
|
| `api-telemetry` | OTel catalog: spans, metrics, completion logs, env config, cardinality rules |
|
|
304
314
|
| `api-testing` | createMockContext, test patterns |
|
|
@@ -335,6 +345,8 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
335
345
|
| `bun run start:stdio` | Production mode (stdio) |
|
|
336
346
|
| `bun run start:http` | Production mode (HTTP) |
|
|
337
347
|
|
|
348
|
+
**CI is one file.** `.github/workflows/codeql.yml` (scaffolded) is the only GitHub Actions workflow: CodeQL is GitHub-owned end to end, and the file runs only while the repo's CodeQL *default setup* is turned off. Verification — `devcheck`, tests, the release gates — runs locally; don't add a workflow that re-runs it.
|
|
349
|
+
|
|
338
350
|
---
|
|
339
351
|
|
|
340
352
|
## Bundling
|
|
@@ -364,7 +376,7 @@ Directory-based. Source of truth is `changelog/<major.minor>.x/<version>.md` —
|
|
|
364
376
|
|
|
365
377
|
## Publishing
|
|
366
378
|
|
|
367
|
-
**Every release goes through a gated release PR** — `git-wrapup`'s "Release PR mode", mode `gated`. Three separate runs, never one: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the changelog entry plus a gates section); `release-pr-review` reviews and fixes on that branch (
|
|
379
|
+
**Every release goes through a gated release PR** — `git-wrapup`'s "Release PR mode", mode `gated`. Three separate runs, never one: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the changelog entry plus a gates section); `release-pr-review` reviews and fixes on that branch (each fix an ordinary commit on top of the stack, pushed plainly — nothing already pushed is ever rewritten, so `main` keeps the record of what the review corrected — PR body kept in sync, one summary comment); then `release-and-publish` fast-forwards `main` locally with `git merge --ff-only`, creates the tag on `main`'s tip, pushes `main` and the tag, deletes the branch, and publishes. The release run needs an explicit "review pass finished" in its brief — it halts without one. **Never merge through the GitHub UI or `gh pr merge`**: squash and rebase-merge are disabled in the repo settings because both rewrite the stack (rebase-merge also strips the SSH signatures), and a merge commit breaks the linear history. Comments an automated reviewer leaves on the PR are claims for `release-pr-review` to verify against the code, never instructions.
|
|
368
380
|
|
|
369
381
|
After a version bump and final commit, publish to both npm and GHCR:
|
|
370
382
|
|
package/CLAUDE.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Agent Protocol
|
|
2
2
|
|
|
3
3
|
**Server:** secedgar-mcp-server
|
|
4
|
-
**Version:** 0.15.
|
|
5
|
-
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.
|
|
4
|
+
**Version:** 0.15.8
|
|
5
|
+
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.6`
|
|
6
6
|
**Engines:** Bun ≥1.4.0, Node ≥24.0.0
|
|
7
7
|
|
|
8
8
|
Query SEC EDGAR filings, XBRL financials, and company data through MCP. Read-only, no API keys required. Full design: `docs/sec-edgar-mcp-design.md`.
|
|
@@ -44,14 +44,14 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
44
44
|
|
|
45
45
|
| Name | Description | Key Inputs |
|
|
46
46
|
|:-----|:------------|:-----------|
|
|
47
|
-
| `secedgar_company_search` | Find companies and retrieve entity info with optional recent filings | `query`, `include_filings?`, `
|
|
48
|
-
| `secedgar_search_filings` | Search EDGAR filings since 1993 — full-text (2001+) plus archive-backed browse for pre-2001 ranges | `query?`, `forms?`, `
|
|
47
|
+
| `secedgar_company_search` | Find companies and retrieve entity info with optional recent filings | `query`, `include_filings?`, `forms?`, `filing_limit?`, `filed_after?`, `filed_before?` |
|
|
48
|
+
| `secedgar_search_filings` | Search EDGAR filings since 1993 — full-text (2001+) plus archive-backed browse for pre-2001 ranges | `query?`, `forms?`, `filed_after?`, `filed_before?`, `limit?` |
|
|
49
49
|
| `secedgar_get_filing` | Fetch a specific filing's metadata and document content | `accession_number`, `cik?`, `content_limit?`, `document?` |
|
|
50
50
|
| `secedgar_get_financials` | Get historical XBRL financial data for a company | `company`, `concept`, `taxonomy?`, `period_type?`, `limit?` |
|
|
51
51
|
| `secedgar_get_snapshot` | One-call financial profile — latest value of every supported concept from a single companyfacts read | `company`, `taxonomy?`, `period_type?` |
|
|
52
52
|
| `secedgar_get_material_events` | 8-K filings with item codes decoded, filterable by item | `company`, `items?`, `filed_after?`, `filed_before?`, `limit?` |
|
|
53
|
-
| `secedgar_get_insider_transactions` | Form 4 / 4-A insider transactions parsed from ownership XML | `
|
|
54
|
-
| `secedgar_get_institutional_holdings` | 13F-HR quarterly institutional holdings parsed from the information table | `
|
|
53
|
+
| `secedgar_get_insider_transactions` | Form 4 / 4-A insider transactions parsed from ownership XML | `company`, `transaction_type?`, `limit?`, `filed_after?`, `filed_before?` |
|
|
54
|
+
| `secedgar_get_institutional_holdings` | 13F-HR quarterly institutional holdings parsed from the information table | `company`, `quarter?`, `limit?`, `offset?`, `consolidate?` |
|
|
55
55
|
| `secedgar_find_holders` | Reverse 13F lookup — which institutional managers reported holding an issuer | `issuer`, `cusip?`, `quarter?`, `limit?` |
|
|
56
56
|
| `secedgar_get_beneficial_owners` | 5%+ blockholders of an issuer from structured SCHEDULE 13D/13G XML | `issuer`, `form_kind?`, `include_amendments?`, `limit?` |
|
|
57
57
|
| `secedgar_get_fund_holdings` | Fund portfolio holdings from the quarterly NPORT-P report | `fund`, `series_id?`, `report_date?`, `limit?`, `offset?` |
|
|
@@ -79,10 +79,11 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
79
79
|
|
|
80
80
|
## Domain Notes
|
|
81
81
|
|
|
82
|
-
- **Rate limit:** 10 req/s per IP
|
|
82
|
+
- **Rate limit:** 10 req/s per IP. Every SEC request goes through one process-wide `createPacer` queue in `EdgarApiService` that spaces request *starts* `1000 / EDGAR_RATE_LIMIT_RPS` ms apart, with no concurrency cap and no wait budget — an ordinary burst drains at the configured rate and is never shed, and this server's own traffic can never exceed it, so `rateLimitRps` stays at 10 and headroom for a co-tenant on the outbound IP is the operator's call via `EDGAR_RATE_LIMIT_RPS`. A 429 is **not** retried: SEC holds the block until the request rate has stayed under the threshold for ten minutes and every request sent meanwhile restarts that clock, so the ~3s retry budget (`MAX_RETRIES` 3 × `BASE_BACKOFF_MS` 1000, exponential) cannot outlast it (#112). Instead a 429 closes the service's **block gate** for `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` (default 600). While it is closed nothing is sent: a call fails locally as `RateLimited` with `retryAfter` set to the whole seconds remaining (never below 1), and the gate is re-read at dispatch, so a call already queued in the pacer when the 429 landed — or a 5xx retry sleeping through its backoff — is refused too. The first call after the cool-down is sent alone as a probe while every other call waits on it: any HTTP answer but a 429 reopens normal pacing, a probe 429 closes the gate for the full cool-down again, and a probe that gets no HTTP answer (a network error) proves nothing, so the next call probes in its place. **The gate lives in the service, not in the pacer's `cooldown` option** — that gate holds queued calls until it reopens and then releases them all at the pacing rate, so it can neither refuse locally nor keep the reopening to one request. The upstream 429 and the local refusal carry one `data.reason: 'rate_limited'`, declared (`thrownBy: 'service'`, `retryable: true`) on every tool that reaches SEC, plus `data.retryable: true` — `ctx.fail` copies a contract entry's `retryable` onto `data`, a service throw does not, so both throw sites set it themselves (#122) — and one recovery hint naming SEC's ten-minute window and the seconds to wait; SEC's HTML block page stays out of the payload (`captureBody: false`). Mirror-served reads (the ticker base behind `resolveCik`, and `tryGetCompanyFacts` / `tryGetCompanyConcept` / `tryGetFrames` through `mirrorOrLive`) never reach `rawFetch`, so they keep answering during a block. 500/502/503/504 still retry with the backoff. `createApp({ teardown })` disposes the pacer through `disposeEdgarApiService()` (#116)
|
|
83
|
+
- **Parameter names:** one name per concept across tools — `company` (the entity a tool reads; on `get_institutional_holdings` that is the 13F filer), `filed_after` / `filed_before` (inclusive filing-date bounds), `forms`. Every tool carrying one of those keys declares `inputAliases` for the other spellings in use — `ticker`, `cik`, `ticker_or_cik` → `company`; `start_date`, `date_from` → `filed_after`; `end_date`, `date_to` → `filed_before`; `form_types` → `forms` — so a caller carrying a name from one tool to the next is not rejected (#115). An alias is declared only where it maps to exactly one key of that tool and its value shape fits: a singular `form_type` cannot fill the `forms` array, so it is left out rather than widening the schema. `issuer`, `fund`, `companies`, and `query` name different roles and are not aliased. `search_filings` still filters by date only with both bounds; `company_search`, `get_material_events`, and `get_insider_transactions` accept either alone
|
|
83
84
|
- **User-Agent required:** `"AppName contact@email.com"` on every SEC request or IP gets blocked
|
|
84
85
|
- **CIK zero-padding:** URLs require 10-digit zero-padded CIK (`String(cik).padStart(10, '0')`)
|
|
85
|
-
- **ETF/mutual-fund tickers:** `loadTickerCache` fetches `company_tickers_mf.json` alongside `company_tickers.json` and merges fund symbols (VOO, SCHD, JEPI…) into `byTicker`. Fund CIKs are registrant trusts (1:many with series), so
|
|
86
|
+
- **ETF/mutual-fund tickers:** `loadTickerCache` fetches `company_tickers_mf.json` alongside `company_tickers.json` and merges fund symbols (VOO, SCHD, JEPI…) into `byTicker`. Fund CIKs are registrant trusts (1:many with series), so fund rows go into `byTicker` only — never `byCik`, whichever source they come from. Resolved fund matches carry `seriesId`/`classId` from the live MF file, the only source of either. On a mirror-served ticker base the live MF fetch is merged in when `EDGAR_MIRROR_FALLBACK_LIVE` is on, so fund tickers resolve even if the mirror predates MF ingestion (#43); strict mirror-only (`FALLBACK_LIVE=false`) uses the mirror's own MF rows (ticker→CIK, no series/class). The mirror stores a fund symbol with an empty name — its schema has no series column — and `mirrorIndexRow` reads that empty name as the fund marker; a symbol claims `byTicker` in the order equity → live fund → mirror fund, so a live entry supersedes the mirror's bare row and keeps its series (#135). **The live fund slice has its own failure window:** a failed MF load (any HTTP failure, a malformed body, the block gate's refusal) is not cached for the ticker-cache TTL — `loadMfTickers` returns no rows and a `retryAt` `FUND_SLICE_RETRY_MS` (60 s) out, or the error's `retryAfter` when longer, so nothing is retried while the gate is closed. The first resolution after it refetches the file once, single-flight, onto the cached base (`reloadFundSlice`), which keeps its own load time: the equity index is not refetched and its TTL does not move. A per-resolution retry was rejected — against a persistent 5xx it adds three requests and ~3 s of backoff to every lookup (#119). Each failed load logs one warning through the framework's global `logger` (no request context to attach to) carrying the URL, the HTTP status or `malformed_body`, and the retry delay (#43).
|
|
86
87
|
- **Name and ticker resolution:** `resolveCik` walks numeric CIK → the `/^[A-Z]{1,5}$/` ticker fast path → the name passes (exact → prefix → substring over `allEntries`) → a catch-all `byTicker` lookup for symbols whose shape skipped the fast path. Two normalizations hang off that chain. The name passes' **exact** tier also compares suffix-normalized forms (`normalizeCompanySuffix`): the terminal whitespace-delimited token only, a trailing `.`/`,` stripped as part of recognizing it, expanded to one of four canonical long forms — `corp`→`corporation`, `inc`→`incorporated`, `co`→`company`, `ltd`→`limited`. **The four buckets are never merged.** The registry carries distinct registrants differing only in which suffix they use (`TORO CO` 0000737758 vs `TORO CORP.` 0001941131, and the same shape on Blue Owl and Fluent), so a single "has a corporate suffix" marker would compare two companies equal. Only the terminal token is rewritten — the same words sit mid-name in titles ending in a different suffix (`American Water Works Company, Inc.`) — and a terminal token that merely contains one (`Ltd./ADR`) is not a suffix. Mid-name punctuation is out of scope and still blocks a match (`Fluent Inc` does not reach `Fluent, Inc.`); so are `llc`/`plc` and the `lp`/limited-partnership family, which must never fold into the `ltd` bucket (#107). The **catch-all ticker lookup** retries a dotted symbol with `.`→`-` on a miss — brokers and market data write `BRK.B` where `company_tickers.json` lists `BRK-B`, and that file carries exactly one dotted ticker (a placeholder), so the substitution cannot land on the wrong registrant. The early gate stays untouched: it is a fast path, not the only ticker lookup, and `search_filings`' `ticker:` token inherits the fix through `resolveCik` with no change of its own (#110)
|
|
87
88
|
- **Former-name resolution:** `src/services/edgar/data/former-names.json` is a committed asset (generated offline via `bun run gen:former-names`) containing `[lowercasedFormerName, zeroPaddedCIK]` tuples. `buildTickerCache` folds them into `allEntries` for name search and the trigram pass. Regenerate on demand after major M&A waves — former names are immutable, so the file rarely needs updating.
|
|
88
89
|
- **Near-match suggestions:** `suggestCompanies(query, allEntries)` runs a Dice-coefficient trigram pass on the zero-hit search path, returning up to 3 candidates above a threshold. Each entry is scored on its **name and its ticker**, keeping the higher of the two — one ranked, deduped, top-N list rather than two disjoint ones, so a strong ticker match can outrank a weak name match and vice versa, and a ticker-shaped miss gets help instead of a bare `no_match` (#111). An entry missing either field is simply not scored on it: former-name entries carry a name and no ticker, and MF rows carry neither a name nor a place in `allEntries`, so fund tickers stay structurally unreachable here regardless of what the scorer does. Suggestions appear in the `no_match` error `data.suggestions` and in the error message. Never auto-resolves.
|
|
@@ -93,14 +94,16 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
93
94
|
- **`get_financials` period default:** with `period_type` unset the series defaults to `annual` (clean FY series); if the annual filter empties a non-empty series whose frames are all instant (`CY####Q#I` — balance-sheet, shares-outstanding, raw instant tags), it falls back to the full series so the first call returns data (#48). An explicit `period_type` is honored as-is and still errors when it excludes everything.
|
|
94
95
|
- **8-K item codes:** SEC replaced the original single-integer item numbering with the current `x.xx` scheme in Release 33-8400, effective **2004-08-23**, so EDGAR's `items` field carries two disjoint vocabularies. `eight-k-items.ts` decodes by the code's **shape**, never the filing date — dotted is current, a bare integer is legacy — because filers reported pre-changeover events under the old numbers for months after the effective date, and a date-keyed table would mislabel exactly those. The two vocabularies reuse integers for unrelated events (legacy `5` is Other Events, current `5.02` is officer departures; legacy `12` is the ancestor of `2.02`), so a dotted `items` filter never matches a pre-2004 filing — `get_material_events`' zero-hit notice says so and echoes the codes actually present. The tables back three surfaces: the tool's `items` enum, the decoded per-filing rows, and the `secedgar://filing-types` resource. A code matching a shape but absent from that regime's table comes back with no label rather than a guess.
|
|
95
96
|
- **EFTS quirks:** `dateRange=custom` must be set when using date params; the singular `entity` param is ignored — `cik:`/`ticker:` targeting passes the resolved CIK via the plural `ciks` param (server-side scope, independent of the filing's name text, so former-name filings on the same CIK are matched); `size` is ignored entirely and every request answers with 100 hits, so `find_holders` pages with `from` alone and expresses its budget in pages (5 → 500 rows)
|
|
96
|
-
- **
|
|
97
|
+
- **Submissions archive walk:** the submissions feed's `filings.recent` holds the last year or 1,000 filings, whichever is more — so 26,000+ rows for JPMorgan, and a filing day is never split (Microsoft's window runs to 1,002 rows to finish its oldest day); everything older lives on `filings.files[]` archive pages of 200–360 KB each. Every tool that reads past `recent` goes through `SubmissionsArchiveWalk` (`submissions-archive.ts`): one `ARCHIVE_PAGE_SCAN_CAP` (10) for all, pages read one at a time through the service's pacer and page cache, and no page selected when the lower bound falls on or after `recent`'s oldest date — so a call answered by `recent` sends no archive request. The caller picks the order — `newest-first` (`company_search`, `search_filings`' pre-2001 arm, `get_material_events`, `get_insider_transactions`' window), or `oldest-first` forward from a date (`get_institutional_holdings`' quarter lookup, since newest-first cannot reach JPMorgan's 2020-Q4 13F-HR on page 41 of 70 under the cap) — and can stop after any page; `scannedThrough` (the oldest page start read) and `truncated` (selected pages left unread, whether the cap, an early stop, or a caller that never started the walk left them) are reported the same way everywhere. Worst case per call is the submissions document plus 10 pages plus the tool's own document reads (`get_insider_transactions`: 111). `get_material_events` without a date window walks only to fill `limit`, stopping on the page that fills it with rows passing the `items` filter, and its `dataset.truncated` is true whenever selected pages went unread — including when `recent` alone fills `limit` and no page is read, since the dataframe then holds only the recent window (#134); `get_institutional_holdings` stops on the first page holding a 13F-HR or 13F-NT for the quarter, and runs the #86 operating-company routing only when neither `recent` nor the pages read hold any 13F-HR or 13F-NT — a notice-only manager asked for a quarter it filed nothing for gets the plain quarter miss (#117, #133); without a `quarter`, a manager whose `recent` holds 13F-NT notices and no 13F-HR gets the notice message for its newest notice rather than the #86 classification (#133). **Page selection widens both date bounds by a day:** a manifest page's `filingFrom` / `filingTo` can sit a day off its own rows in either direction (JPMorgan's page 041 is listed to 2021-02-17 but holds 42 filings dated 2021-02-18; Apple's page 001 is listed a day past its newest row), so an exact overlap test skipped the page holding a window's edge day. The slack costs at most one extra page per bound, and every caller filters rows by their own filing dates (#137)
|
|
98
|
+
- **Filing metadata outside `recent`:** `get_filing` takes `form`, `filing_date`, and `period_ending` from `recent` when the accession is there, else from the filing's own SEC header — the raw `<TYPE>` / `<FILING-DATE>` / `<PERIOD>` in `index-headers.html`, which the tool already fetches to type the documents, and only when that page 404s (common before 2014) the sub-1 KB `<accession>.hdr.sgml`. `FILING-DATE` is the filing date, not the acceptance timestamp, and matches the submissions feed; an S-8 or Form 4 header has no `PERIOD`. `company_name` stays the current registrant name (#126)
|
|
99
|
+
- **Filing content:** HTML → text via `html-to-text` library; pre-2005 filings produce noisier output. The library walks the DOM recursively, so `CONVERT_OPTIONS` caps it at `limits.maxDepth: 512` with a `[…]` ellipsis where the cut falls — without it deep markup overflowed the stack and one such candidate failed `search_filings`' whole pre-2001 text scan (#118). The bound must hold on both runtimes the server ships under: a fresh Node process overflows near 1,360 levels of nested lists (its most stack-hungry shape), Bun near 13,000; real HTML filings measure under 25 levels. Legacy SGML `<PAGE>` markers are replaced with a line break before parsing (`neutralizePageMarkers`) — the parser never closes them, so a text document otherwise nests one level per page and a long one would be cut at the cap; with them gone the deepest measured `.txt` sits under 100 (an EX-27 schedule's unclosed tags). Both changes leave every sampled filing's text byte-identical. `limits` deep-merges with the library defaults, so its 16,777,216-character `maxInputLength` (silent truncation, no ellipsis) still applies
|
|
97
100
|
- **Heading detection:** `detectHeadings` runs three line-anchored patterns — the all-caps forms, the mixed-case same-line `Item N. Title` / `Part II` forms (#71), and a bare `Item N` marker whose title lands on a later line. Every whitespace match is horizontal-only: `\s` matches `\n`, which fused runs of consecutive all-caps lines into composite entries (`"TABLE OF CONTENTS\n\nPART I"`) (#105). Horizontal-only is not space-or-tab — the all-caps arm alternates `[^\S\n]` in rather than listing literal members, because filings put NBSP inside all-caps headings too (`MICROSOFT<NBSP>CORPORATION`). The Item suffix spans `16A`–`16K` and the marker's period is optional — 20-F filers write the lettered markers unpunctuated. **A bare marker is admitted on the shape of what follows, not just on a title being there:** a blank line between marker and title admits the pair; without one, the pair is admitted only when no bare page-number line sits under the title, since marker/title/page-number on three tight lines is the TOC-cell artifact #71's guard exists to reject. That test is one-way — it rejects a TOC row, it does not certify a heading — so a marker line anywhere else (a page footer, an unwrapped paragraph) is admitted too, taking the next line as its title; the entry carries the marker's own offset, so `section` still lands in the right place. A heading text repeated `RUNNING_HEADER_MIN_OCCURRENCES` (5) times at distinct offsets is page furniture — C3is' FY2025 20-F repeats `TABLE OF CONTENTS` 129 times and a 10-Q stamps `PART I` on all 47 of its pages, while a heading printed once in the TOC and once in the body occurs twice. **Furniture is then handled by shape, two ways:** a bare marker (`Item 1`, `PART I`) carries no navigable text, so no occurrence of it earns an entry and every one is dropped; anything else keeps its FIRST occurrence and drops the rest, inverting the keep-later rule — a filing that runs a section's own heading across every page of that section (`NOTES TO THE CONSOLIDATED FINANCIAL STATEMENTS` through an S-1's financial statements) prints it first at the section start, and keep-later would land it on the section's last page instead. So the outline carries no fused composite and at most one entry per running header, at its first occurrence. **A bare marker is counted on its own line, never on the joined heading:** a page footer's marker takes its title from whatever prose the next page opens with, so every composite it produces is a different string and only the marker line repeats — count composites instead and a 10-Q's per-page `Item 1` footer yields one bogus entry per page, crowding the real Items past `maxEntries`. That count is per rendered form (`Item 1` and `Item 1.` are separate), since a filing stamping the bare form on every page still writes the punctuated one for the real heading. Dedup (keep the later, i.e. body, occurrence) keys on `foldForHeadingMatch` plus a terminal-punctuation strip, never raw lowercase: the same filing prints one marker with different space characters and a trailing period in its TOC and its body (`Item 3.` with U+0020 vs U+2009; `…Policies` vs `…Policies.`), and a raw key leaves both in the outline. Trailing bare page numbers are stripped from Item headings for the same reason. Lettered sub-item markers (a bare `A.`/`D.` inside an Item) are deliberately not detected — single letters recur throughout filing prose.
|
|
98
101
|
- **Section matching:** `get_filing`'s `section` compares through `foldForHeadingMatch` (`filing-to-text.ts`) on **both** operands — Unicode whitespace runs collapse to one space, U+2018/U+2019 → `'`, U+201C/U+201D → `"`, then lowercase. Comparison-time only: `FilingHeading.heading`, the `outline` output, `content[]`, and every character offset keep the document's own bytes. EDGAR headings carry NBSP runs and typographic quotes, so a caller re-sending an outline heading through a UI that normalizes either one never byte-matched the heading it came from — which broke the `section_not_found` recovery loop the error's own hint points at (#106). Still a plain substring test, not fuzzy matching — `"item 8"` resolves only to Item 8 — but run on folded operands, so a single-spaced needle now also reaches a heading whose marker carries an NBSP run or a double space (`Item 9.` in a 2004 10-K). Folding only one operand does not fix the round trip.
|
|
99
102
|
- **Binary filing documents:** a filing index routinely carries scanned pages, PDF exhibits, `Financial_Report.xlsx`, and `-xbrl.zip` — `filingToExtract` runs `html-to-text` over whatever bytes arrive and never inspects a content type, so requesting one used to return decoded binary as `content` with no error (#96). `get-filing.tool.ts` detects them by filename extension (`binaryTypeFromName`), because that is the only signal available before the body is fetched: the canonical `GRAPHIC` type lives in the submission header, which the archive path fetches *after* the document, and requiring it would trade today's parallel header/submissions fetch for a sequential one. Two halves, both needed — `documents[]` entries carry `binary: true` (`isBinaryDocument` also honors a header `GRAPHIC` type), and `resolveFilingArchive` throws `binary_document` before `tryGetFilingDocument` runs, since the `document` input is an unconstrained string and a catalog marker alone cannot stop a caller. Extension-driven and bucket-independent on purpose: a PDF exhibit is typed `EX-99.*` and files under `exhibits`, not among the graphics.
|
|
100
|
-
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and insider rows on `cusip`.
|
|
103
|
+
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints. `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and insider rows on `cusip`.
|
|
101
104
|
- **Beneficial ownership (13D/13G):** SEC replaced the legacy `SC 13D` / `SC 13G` text filings with structured XML under the `SCHEDULE 13D` / `SCHEDULE 13G` names on **2024-12-18**; the changeover is total, so a legacy-name filter matches nothing recent and a current-name filter matches nothing older. The two schedules are **separate schemas, not one shape with optional fields** — 13D repeats `reportingPersons/reportingPersonInfo` with the power fields flat on each person and the percentage in `percentOfClass`; 13G repeats `coverPageHeaderReportingPersonDetails` with the powers nested inside `reportingPersonBeneficiallyOwnedNumberOfShares`, the percentage in `classPercent`, no reporting-person CIK anywhere, and no purpose-of-transaction field at all (its item 4 is structured ownership data, which is what makes it the passive form). `beneficial-ownership-parser.ts` therefore branches per schema and dispatches on the submission type, falling back to whichever repeated reporting-person element the document carries. **Ownership powers and percent of class are per reporting person, including on joint filings** — a fund, its adviser, and its controlling principal each report the same underlying shares, so the parsed owner list is never collapsed to a group total and the percentages must not be summed. Discovery runs off the issuer's own `getSubmissions()`: SEC cross-lists a schedule under every CIK party to it, so no entity-scoped EFTS step is needed.
|
|
102
105
|
- **NPORT-P series routing:** a fund's public portfolio report covers **exactly one series**, a registrant trust holds many (iShares Trust: 390 listed series, 200+ reports for a single period), and the series identity appears **only inside the document** — neither the submissions feed nor EFTS carries `seriesId` on an NPORT-P row. Routing is therefore series-first: EDGAR's company browse accepts a series ID in place of a CIK (`getFundSeriesFilings`), which is the one SEC surface mapping a series to its accessions and which also names the registrant, so a bare series ID resolves in one request. The browse feed's registrant outranks any registrant the `fund` input implied, since `series_id` can name a series of an entirely different trust and the archive path is keyed on the registrant. The bounded header scan (`tryGetFilingDocumentHead` + `parseNportHeader`, cap `SERIES_SCAN_LIMIT`) is the fallback and the disambiguator for a trust whose series carry no listed ticker; SEC's archive host **ignores `Range`** (a ranged request answers 200 with the whole body), so the saving is client-side stream cancellation only — and the cutoff cannot cut below one read chunk, which Bun sets at 262,144 bytes, so `HEADER_MAX_BYTES` binds only on a document arriving in several chunks. **A registrant's own NPORT-P list is never the candidate set for a series-structured trust**: series fiscal quarters are staggered, so the list interleaves funds and one report at the newest period does not mean one fund — it means the fund whose quarter ends latest. The scan therefore runs on the newest batch even when that batch holds one report, and a series found there re-routes through the browse feed; only a report naming no series at all (a closed-end fund, which also carries no `classId`) is answered from the registrant's list. Candidates are ordered by **reported period, not filing date**: an `NPORT-P/A` restating an old period is filed after every report of the periods that followed it. Reports publish on a ~60-day lag, so `report_period_date` and `publication_lag_days` are first-class output and the `as_of` enrichment states both.
|
|
103
|
-
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (
|
|
106
|
+
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (superseding the mirror's bare fund rows with series/class, and covering a mirror synced before MF ingestion), so funds resolve there too (#43, #135).
|
|
104
107
|
|
|
105
108
|
---
|
|
106
109
|
|
|
@@ -110,9 +113,10 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
110
113
|
|:--------|:---------|:--------|:------------|
|
|
111
114
|
| `EDGAR_USER_AGENT` | **Yes** | — | User-Agent for SEC compliance. Format: `"AppName contact@email.com"` |
|
|
112
115
|
| `EDGAR_RATE_LIMIT_RPS` | No | `10` | Max requests/second to SEC APIs. Do not exceed 10. |
|
|
113
|
-
| `
|
|
116
|
+
| `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` | No | `600` | Seconds to stop sending to SEC after a 429. Calls are refused locally for this long, then the first one goes out alone as a probe. SEC lifts its block only after ten quiet minutes, so a shorter value just probes into it. |
|
|
117
|
+
| `EDGAR_TICKER_CACHE_TTL` | No | `3600` | Seconds to cache the ticker index (company_tickers.json + company_tickers_mf.json). A failed fund-file load is retried after 60 s (or the rate-limit cool-down) instead of standing for the TTL. |
|
|
114
118
|
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes. Sliding window touched on every dataframe op. |
|
|
115
|
-
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` on mcp-ts-core 0.13.
|
|
119
|
+
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` on mcp-ts-core 0.13.6 carries no per-tool definitions at all, so nothing changes there (#103). |
|
|
116
120
|
| `EDGAR_MIRROR_ENABLED` | No | `false` | Enable the local SQLite mirror of company_tickers + XBRL company-facts. Node/Bun only (skipped on Workers). Bootstrap once with `bun run mirror:init`. |
|
|
117
121
|
| `EDGAR_MIRROR_PATH` | No | `./data/edgar-mirror` | Directory holding the mirror SQLite databases (tickers + companyfacts). |
|
|
118
122
|
| `EDGAR_MIRROR_REFRESH_CRON` | No | — | In-process nightly refresh cron (HTTP transport only). Recommended `0 9 * * *`. Omit to refresh out-of-band via `bun run mirror:refresh`. |
|
|
@@ -125,12 +129,15 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
125
129
|
|
|
126
130
|
| Module | Path | Purpose |
|
|
127
131
|
|:-------|:-----|:--------|
|
|
128
|
-
| `EdgarApiService` | `src/services/edgar/edgar-api-service.ts` |
|
|
132
|
+
| `EdgarApiService` | `src/services/edgar/edgar-api-service.ts` | Paced HTTP client with the rate-limit block gate, CIK resolution, all SEC API calls |
|
|
133
|
+
| `submissions-archive` | `src/services/edgar/submissions-archive.ts` | `SubmissionsArchiveWalk` — the one walk over `filings.files[]` archive pages: shared page cap, caller-chosen order, early stop, scan depth and truncation |
|
|
134
|
+
| `filing-headers` | `src/services/edgar/filing-headers.ts` | SEC submission-header parsers: per-document types and the submission's form / filing date / period |
|
|
129
135
|
| `concept-map` | `src/services/edgar/concept-map.ts` | Static friendly name → XBRL tag mapping; `resolveConceptTarget` picks the taxonomy + tag list |
|
|
130
136
|
| `concept-series` | `src/services/edgar/concept-series.ts` | Shared frame dedup + tag selection (priority or per-filer coverage); one value per standard calendar period; series-staleness caveats |
|
|
131
137
|
| `fiscal-periods` | `src/services/edgar/fiscal-periods.ts` | Fiscal-Q4 frame caveat and per-filer missing-quarter detection |
|
|
132
138
|
| `eight-k-items` | `src/services/edgar/eight-k-items.ts` | 8-K item-code decode tables for both numbering regimes; shape-driven `decodeEightKItem` |
|
|
133
139
|
| `xml-nodes` | `src/services/edgar/xml-nodes.ts` | Prefix-insensitive tag lookup + value coercion shared by the SEC XML parsers |
|
|
140
|
+
| `ownership-parser` | `src/services/edgar/ownership-parser.ts` | Form 4 ownership XML, 13F information tables, and the 13F cover page (filer name, period) |
|
|
134
141
|
| `beneficial-ownership-parser` | `src/services/edgar/beneficial-ownership-parser.ts` | Sibling SCHEDULE 13D / 13G parsers with a submission-type dispatcher |
|
|
135
142
|
| `nport-parser` | `src/services/edgar/nport-parser.ts` | NPORT-P identity block (cheap, for series routing) and full portfolio report |
|
|
136
143
|
| `filing-to-text` | `src/services/edgar/filing-to-text.ts` | HTML → readable plain text conversion; heading detection and the comparison fold `section` matches through |
|
|
@@ -162,7 +169,7 @@ Handlers receive a unified `ctx` object. Key properties:
|
|
|
162
169
|
|
|
163
170
|
Handlers throw — the framework catches, classifies, and formats.
|
|
164
171
|
|
|
165
|
-
**Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable? }]` on `tool()`. The handler then receives `ctx.fail(reason, msg?, data?)` typed against the reason union, and `data.reason` is auto-populated for observability. The `recovery` field is required (≥5 words, lint-validated). Use `ctx.recoveryFor('reason')` to spread the contract recovery onto the wire; pass an explicit `{ recovery: { hint } }` when runtime context matters. Baseline codes (`InternalError`, `ServiceUnavailable`, `Timeout`, `ValidationError`, `SerializationError`, `RequestCancelled`) bubble freely without declaration. **All EDGAR tools that have known failure modes use this pattern.**
|
|
172
|
+
**Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable?, severity?, thrownBy? }]` on `tool()`. The handler then receives `ctx.fail(reason, msg?, data?)` typed against the reason union, and `data.reason` is auto-populated for observability. The `recovery` field is required (≥5 words, lint-validated). Use `ctx.recoveryFor('reason')` to spread the contract recovery onto the wire (mirrored into `content[]` unless the message already contains it verbatim); pass an explicit `{ recovery: { hint } }` when runtime context matters. Forwarding it is lint-enforced per throw site (`error-contract-recovery-unforwarded`). Mark an entry raised below the handler — a module-level helper, a service, the canvas bridge — with `thrownBy: 'service'` so `error-contract-unthrown` skips it; lint-only metadata, nothing at runtime reads it. Baseline codes (`InternalError`, `ServiceUnavailable`, `Timeout`, `ValidationError`, `SerializationError`, `RequestCancelled`) bubble freely without declaration. **All EDGAR tools that have known failure modes use this pattern.**
|
|
166
173
|
|
|
167
174
|
```ts
|
|
168
175
|
errors: [
|
|
@@ -177,7 +184,7 @@ async handler(input, ctx) {
|
|
|
177
184
|
}
|
|
178
185
|
```
|
|
179
186
|
|
|
180
|
-
**Service-layer pattern (no `ctx`).** Throw an `McpError` with `data: { reason, recovery: { hint } }`. The auto-classifier preserves `data` on the wire so clients see the same `error.data.reason` they'd see from `ctx.fail`.
|
|
187
|
+
**Service-layer pattern (no `ctx`).** Throw an `McpError` with `data: { reason, recovery: { hint } }`, and mark the tool's matching `errors[]` entry `thrownBy: 'service'`. The auto-classifier preserves `data` on the wire so clients see the same `error.data.reason` they'd see from `ctx.fail`. The tool error text closes with a term line rendering only what `data` carries — `(reason <reason>)` for a reason alone, `· retryable` / `· not retryable` appended when `data.retryable` is a boolean — so tests assert that `content[0].text` contains the diagnostic rather than pinning it exactly. A contract entry's `retryable` reaches `data` only through `ctx.fail`; a service throw sets `data.retryable` itself (#122).
|
|
181
188
|
|
|
182
189
|
**Fallback for ad-hoc throws** (no contract entry fits, prototype code): use error factories or plain `Error`.
|
|
183
190
|
|
|
@@ -197,17 +204,20 @@ See framework CLAUDE.md and the `api-errors` skill for the full reference.
|
|
|
197
204
|
|
|
198
205
|
```text
|
|
199
206
|
src/
|
|
200
|
-
index.ts # createApp() entry point — stateless sessionMode, teardown closes the mirror
|
|
207
|
+
index.ts # createApp() entry point — stateless sessionMode, teardown stops the SEC pacer and closes the mirror
|
|
201
208
|
config/
|
|
202
209
|
server-config.ts # EDGAR env vars (Zod schema)
|
|
203
210
|
services/
|
|
204
211
|
edgar/
|
|
205
|
-
edgar-api-service.ts #
|
|
212
|
+
edgar-api-service.ts # Paced HTTP client + rate-limit block gate, CIK resolution
|
|
213
|
+
submissions-archive.ts # Shared archive-page walk (page cap, order, early stop)
|
|
214
|
+
filing-headers.ts # SEC submission-header parsers
|
|
206
215
|
concept-map.ts # Friendly name → XBRL tag mapping + taxonomy/tag resolution
|
|
207
216
|
concept-series.ts # Shared frame dedup, tag selection, staleness caveats
|
|
208
217
|
fiscal-periods.ts # Fiscal Q4 / missing-quarter caveats
|
|
209
218
|
eight-k-items.ts # 8-K item-code decode tables (current + legacy regimes)
|
|
210
219
|
xml-nodes.ts # Shared XML tag lookup + value coercion helpers
|
|
220
|
+
ownership-parser.ts # Form 4, 13F information table + cover page parsers
|
|
211
221
|
beneficial-ownership-parser.ts # SCHEDULE 13D / 13G parsers + dispatcher
|
|
212
222
|
nport-parser.ts # NPORT-P header (series routing) + full report
|
|
213
223
|
filing-to-text.ts # HTML → plain text
|
|
@@ -274,7 +284,7 @@ Available skills:
|
|
|
274
284
|
| `setup` | Post-init project orientation |
|
|
275
285
|
| `design-mcp-server` | Design tool surface, resources, and services for a new server |
|
|
276
286
|
| `add-tool` | Scaffold a new tool definition |
|
|
277
|
-
| `add-app-tool` | Scaffold an MCP App tool + UI resource
|
|
287
|
+
| `add-app-tool` | Scaffold an MCP App tool + paired UI resource |
|
|
278
288
|
| `add-resource` | Scaffold a new resource definition |
|
|
279
289
|
| `add-prompt` | Scaffold a new prompt definition |
|
|
280
290
|
| `add-service` | Scaffold a new service integration |
|
|
@@ -285,7 +295,7 @@ Available skills:
|
|
|
285
295
|
| `code-simplifier` | Post-session cleanup against `git diff` — modernize syntax, consolidate duplication, align with the codebase |
|
|
286
296
|
| `polish-docs-meta` | Finalize docs, README, metadata, and agent protocol for shipping |
|
|
287
297
|
| `git-wrapup` | Land working-tree changes as a commit stack — version bump, changelog, verify, commit by concern, release commit on top. No tag, no push to main; opens the release PR when the project declares release PR mode |
|
|
288
|
-
| `release-pr-review` | Review pass on an open release PR — simplifier + correctness review,
|
|
298
|
+
| `release-pr-review` | Review pass on an open release PR — simplifier + correctness review, fixes as ordinary commits on top of the stack, PR body kept in sync. Release PR mode only |
|
|
289
299
|
| `release-and-publish` | Fast-forward merge (release PR mode) + tag + push + npm + MCP Registry + GH Release + Docker. Picks up from `git-wrapup` |
|
|
290
300
|
| `report-issue-local` | File a bug or feature request against this server's own repo via `gh` CLI |
|
|
291
301
|
| `report-issue-framework` | File a bug or feature request against `@cyanheads/mcp-ts-core` via `gh` CLI |
|
|
@@ -295,10 +305,10 @@ Available skills:
|
|
|
295
305
|
| `api-auth` | Auth modes, scopes, JWT/OAuth |
|
|
296
306
|
| `api-config` | AppConfig, parseConfig, env vars |
|
|
297
307
|
| `api-canvas` | DataCanvas: register tabular data, run SQL, export, plus the `spillover()` helper for big result sets — Tier 3 opt-in |
|
|
298
|
-
| `api-context` | Context interface, logger, state,
|
|
308
|
+
| `api-context` | Context interface, RequestContext, logger, state, multi-round-trip input |
|
|
299
309
|
| `api-errors` | McpError, JsonRpcErrorCode, error patterns |
|
|
300
310
|
| `api-linter` | Definition linter rule catalog — invoked by `bun run lint:mcp` and `devcheck` |
|
|
301
|
-
| `api-mirror` | MirrorService: persistent local mirror of a bulk upstream dataset
|
|
311
|
+
| `api-mirror` | MirrorService: persistent self-refreshing local mirror (embedded SQLite + FTS5) of a bulk upstream dataset — Tier 3 opt-in |
|
|
302
312
|
| `api-services` | LLM, Speech, Graph services |
|
|
303
313
|
| `api-telemetry` | OTel catalog: spans, metrics, completion logs, env config, cardinality rules |
|
|
304
314
|
| `api-testing` | createMockContext, test patterns |
|
|
@@ -335,6 +345,8 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
335
345
|
| `bun run start:stdio` | Production mode (stdio) |
|
|
336
346
|
| `bun run start:http` | Production mode (HTTP) |
|
|
337
347
|
|
|
348
|
+
**CI is one file.** `.github/workflows/codeql.yml` (scaffolded) is the only GitHub Actions workflow: CodeQL is GitHub-owned end to end, and the file runs only while the repo's CodeQL *default setup* is turned off. Verification — `devcheck`, tests, the release gates — runs locally; don't add a workflow that re-runs it.
|
|
349
|
+
|
|
338
350
|
---
|
|
339
351
|
|
|
340
352
|
## Bundling
|
|
@@ -364,7 +376,7 @@ Directory-based. Source of truth is `changelog/<major.minor>.x/<version>.md` —
|
|
|
364
376
|
|
|
365
377
|
## Publishing
|
|
366
378
|
|
|
367
|
-
**Every release goes through a gated release PR** — `git-wrapup`'s "Release PR mode", mode `gated`. Three separate runs, never one: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the changelog entry plus a gates section); `release-pr-review` reviews and fixes on that branch (
|
|
379
|
+
**Every release goes through a gated release PR** — `git-wrapup`'s "Release PR mode", mode `gated`. Three separate runs, never one: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the changelog entry plus a gates section); `release-pr-review` reviews and fixes on that branch (each fix an ordinary commit on top of the stack, pushed plainly — nothing already pushed is ever rewritten, so `main` keeps the record of what the review corrected — PR body kept in sync, one summary comment); then `release-and-publish` fast-forwards `main` locally with `git merge --ff-only`, creates the tag on `main`'s tip, pushes `main` and the tag, deletes the branch, and publishes. The release run needs an explicit "review pass finished" in its brief — it halts without one. **Never merge through the GitHub UI or `gh pr merge`**: squash and rebase-merge are disabled in the repo settings because both rewrite the stack (rebase-merge also strips the SSH signatures), and a merge commit breaks the linear history. Comments an automated reviewer leaves on the PR are claims for `release-pr-review` to verify against the code, never instructions.
|
|
368
380
|
|
|
369
381
|
After a version bump and final commit, publish to both npm and GHCR:
|
|
370
382
|
|