@cyanheads/secedgar-mcp-server 0.16.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +22 -9
- package/CLAUDE.md +22 -9
- package/Dockerfile +13 -2
- package/README.md +39 -101
- package/changelog/0.17.x/0.17.0.md +34 -0
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.js +3 -1
- package/dist/mcp-server/tools/definitions/company-search.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/dataframe-describe.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/dataframe-describe.tool.js +1 -1
- package/dist/mcp-server/tools/definitions/dataframe-describe.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/find-holders.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/find-holders.tool.js +6 -2
- package/dist/mcp-server/tools/definitions/find-holders.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.js +3 -3
- package/dist/mcp-server/tools/definitions/get-beneficial-owners.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-filing.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-filing.tool.js +6 -3
- package/dist/mcp-server/tools/definitions/get-filing.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.js +1 -1
- package/dist/mcp-server/tools/definitions/get-fund-holdings.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.js +5 -2
- package/dist/mcp-server/tools/definitions/get-insider-transactions.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/get-material-events.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/get-material-events.tool.js +5 -2
- package/dist/mcp-server/tools/definitions/get-material-events.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/search-concepts.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/search-concepts.tool.js +3 -1
- package/dist/mcp-server/tools/definitions/search-concepts.tool.js.map +1 -1
- package/dist/mcp-server/tools/definitions/search-filings.tool.d.ts.map +1 -1
- package/dist/mcp-server/tools/definitions/search-filings.tool.js +7 -5
- package/dist/mcp-server/tools/definitions/search-filings.tool.js.map +1 -1
- package/package.json +4 -3
- package/server.json +3 -3
- package/skills/secedgar-guide/SKILL.md +67 -0
package/AGENTS.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Agent Protocol
|
|
2
2
|
|
|
3
3
|
**Server:** secedgar-mcp-server
|
|
4
|
-
**Version:** 0.
|
|
5
|
-
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.
|
|
4
|
+
**Version:** 0.17.0
|
|
5
|
+
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.18`
|
|
6
6
|
**Engines:** Bun ≥1.4.0, Node ≥24.0.0
|
|
7
|
-
**MCP SDK:** `@modelcontextprotocol/server` ^2.
|
|
7
|
+
**MCP SDK:** `@modelcontextprotocol/server` ^2.4.0
|
|
8
8
|
**Zod:** ^4.6.7
|
|
9
9
|
|
|
10
10
|
Query SEC EDGAR filings, XBRL financials, and company data through MCP. Read-only, no API keys required. Full design: `docs/sec-edgar-mcp-design.md`.
|
|
@@ -107,7 +107,7 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
107
107
|
- **Heading detection:** `detectHeadings` runs three line-anchored patterns — the all-caps forms, the mixed-case same-line `Item N. Title` / `Part II` forms (#71), and a bare `Item N` marker whose title lands on a later line. **One NBSP may open a same-line Item line (#176):** both Item arms admit a single U+00A0 ahead of the marker, since an ` ` before a styled span left Horizon Space's 8-K line `<NBSP>Item 1.01. Entry into a Material Definitive Agreement.` (0001929980-26-000542) out of an outline that listed Items 7.01 and 9.01. A lookahead holds the NBSP to an Item marker; the entry's offset is the marker's (1960, not the NBSP's 1959), `isWrappedProse` and `followsMidSentence` still read the line from its real start, and a line that is a bare marker once its page cell is stripped (`<NBSP>ITEM 2.02`, `<NBSP>Item 1. 3`) yields nothing (`itemEntryOffset`). Nothing wider: ASCII-space or tab indentation, two NBSPs, and NBSP-led Part lines stay out — Microsoft's 2004 10-K (0001193125-04-150689) stamps an NBSP-led `Part I` on every page, and admitting those turns its body `PART I` (3832), `PART II` (64076), and `PART IV` (308565) into running-header furniture. Across the 67-filing corpus the change adds that one entry and removes none. Every whitespace match is horizontal-only: `\s` matches `\n`, which fused runs of consecutive all-caps lines into composite entries (`"TABLE OF CONTENTS\n\nPART I"`) (#105). Horizontal-only is not space-or-tab — filings put NBSP inside all-caps headings too (`MICROSOFT<NBSP>CORPORATION`), so the all-caps arm nests the class-set difference `[\s--\n]` in its class (the `v` flag). It is one class, not the class alternated with `[^\S\n]`: the quantified alternation slows sharply on very long lines, while a single class scans in linear time. The all-caps line admits `-`, `–`, `—`, digits, and apostrophes, and `isAllCapsHeadingText` bounds it and narrows the last two. Its heading text runs at most `MAX_TITLE_CHARS` (140): past it the line is a paragraph set in capitals, which the widened class otherwise admitted (Alternative Ballistics' 308-character risk-factor coda took `section: "business"` by substring from `OUR BUSINESS`; EuroSPAC's 860-character tax legend; GT Biopharma's 288- and 326-character voting notices). A digit is admitted only inside a whitespace-delimited `\d{1,2}[A-Z]?\.?` token judged on the whole line, a TOC page cell included (`NOTE 10`, `SCHEDULE 14A`; `USD 2,929`, `FMR LLC(4)`, `WASHINGTON, D.C. 20549`, and `…ACT OF 1934` stay out), and an apostrophe only inside a word or after a word-final `S` (`MANAGEMENT’S`, `STOCKHOLDERS’ EQUITY`), never opening or closing the heading. The all-caps `ITEM …` arm takes the rest of its line whatever it holds, and neither Item arm admits a line that runs a table of contents together: a second Item marker — after whitespace, or abutting the period, digit, or parenthesis that ends the cell before it (`Item 2.Item 3.`, `Item 9.01Item 9.02`, #180), while an opening quote or parenthesis before it is a title citing another Item (`(Item 408 of Regulation S-K)`) — or a page cell mid-line followed by two or more spaces and the next cell (`isRunTogetherToc`). Without it a 238-character 8-K table of contents (`ITEM 8.01. OTHER EVENTS. 3 ITEM 9.01. …`, Amazon's 0001104659-26-107526) was one heading and took `item 8`, `item 9`, and `exhibits`. The test is on shape, not length: a 140-character bound on that arm dropped a proxy's one proposal heading, Modine's 185-character `ITEM 1 – TO APPROVE THE PROPOSAL TO AMEND THE Amended and Restated Articles…` (DEF 14A 0001104659-26-115309), and turned `item 1` and `proposal` into misses. Across 104 cached filings (244 all-caps and 568 mixed-case Item lines) the shape test flags only the Amazon line, and no all-caps Item line runs past 240 characters, so the arm carries no outer length bound. The mixed-case same-line arm keeps its title bound as its prose guard. A colon stays out: every colon line it admitted across 26 sampled documents was a statement caption (`CURRENT ASSETS:`). A Part line may carry a title — whitespace, an optional `-`/`–`/`—`/`:`/`.`, then an uppercase letter, at most 140 characters — and is a cross-reference rather than a heading (`isPartCrossReference`), on the mixed-case and the all-caps arm alike, when it sits under a line ending on a comma or semicolon like an unpunctuated Item, or when its title opens on another marker (`Part II — Item 5. Market for Registrant’s Common Equity`, a bank 10-K's cross-reference index row), which would otherwise also displace the bare body Part it names. After dedup, a bare Part entry gives way to a later titled heading for the same numeral (`dropBarePartsTitledLater`, which applies the same marker-title test): a 10-Q's TOC row `Part II` and its body `PART II — OTHER INFORMATION` have different keys, so without the merge `section: "part ii"` landed on the TOC row 50,045 characters early (Apple Q3 FY2026). Over the 26-document corpus these rules add 72 headings and remove 4 TOC Part rows: 66 section headings (all 18 of Microsoft FY2026's `NOTE N — TITLE` lines, statement titles, titled Parts) and 6 furniture (statement captions, `SCHEDULE 14A`, a `PAGE` stamp, a wrapped legend line), no numeric table row (#170). The Item code spans `16A`–`16K` and the current 8-K's dotted `N.NN` (`ITEM_CODE`, shared by all four Item patterns), and the marker's period is optional — 20-F filers write the lettered markers unpunctuated. A dotted code may also abut its title or carry a paragraph suffix (`Item 9.01Financial Statements…`, `Item 9.01(d). Financial…`, the shapes html-to-text leaves when the marker and title are separate spans), and then the title must start with a letter, so a bare `Item 5.02.` line never matches on its own. **Any other code may abut its title too (#180):** past its period (`Item 2.Management’s Discussion…`, `Item 1A.Risk Factors`, `Item 16A.Audit…`), the title again opening on a letter, while a digit after the period stays part of the code (`Item 2.02` and the dotted forms above read as before; `Item 2.0 something` is no heading). With no period, a code or Part numeral runs into a title only where a capital followed by a lowercase letter opens it (`TITLE_RUN_IN`, a case-sensitive `(?-i:)` lookahead inside the case-insensitive patterns): `Item 1Financial Statements` is Item `1` and `Part IFinancial Information` Part `I`, while a lettered code or a longer numeral reads whole (`Item 16KCybersecurity`, `Part IVExhibits`) and an all-caps run-in (`Item 1ARISK FACTORS`) matches nothing. New Peoples Bankshares' 10-Q (0001903596-26-000307) prints every body marker this way and had no Item in its outline; it now lists all eleven Items and Part I (28 → 40 entries; the period form alone 28 → 38), `item 1` lands on Part I's `Item 1Financial Statements` (4167) instead of Part II's `Item 1.Legal Proceedings` (113660), and `risk factors` and `management's discussion` land at 114156 and 73945 instead of missing. Across the other 25 corpus filings no outline entry or landing moved. Every new alternative is a fixed-width lookahead, so the scans stay linear. A code in `CURRENT_EIGHT_K_ITEMS` takes a 160-character title bound (`EIGHT_K_MAX_TITLE_CHARS`) on the same-line arm and in `usableTitle` — Item 5.02's 146-character title is the longest across the 53 sampled 8-Ks, and the 14-character margin admits a short trailing note while a paragraph opening on the marker line (`Item 5.02(b) On June 2, 2026, the Board…`) runs to hundreds of characters and stays out — and every other marker keeps the 140-character prose guard, a dotted code outside the table (`Item 7.09`) included. Across 53 current 8-Ks the outline holds all 112 item codes EDGAR lists for them (24 before); without the bound exemption it holds 89, without the abutting/suffix form 103 (#165). When an all-caps marker line (`ITEM 2.02`) both matches the all-caps arm alone and joins the title below it, and both entries survive dedup at one offset, the bare entry is dropped — one heading, listed once. Because the period is optional, an unpunctuated same-line Item is dropped as wrapped prose (`isWrappedProse`) when its next word starts lowercase or the line above ends on a comma or semicolon, a title run straight into its code included (`…in Part II,` over `Item 8Financial Statements of this report`, #180). A line ending on a lowercase word is no evidence: a running page header (`Table of Contents`) sits directly above an unpunctuated body heading (`Item 5 — Other information`) at the top of a page. A plain-text 10-K's cross-reference that wraps to start a line with `Item 8 on this Form 10-K…` otherwise takes the `item 8` landing ahead of the real heading (#136). **A bare marker is admitted on the shape of what follows, not just on a title being there:** a blank line between marker and title admits the pair; without one, the pair is admitted only when no bare page-number line sits under the title, since marker/title/page-number on three tight lines is the TOC-cell artifact #71's guard exists to reject. That test is one-way — it rejects a TOC row, it does not certify a heading — so a marker line anywhere else (a page footer, an unwrapped paragraph) is admitted too, taking the next line as its title; the entry carries the marker's own offset, so `section` still lands in the right place. A heading text repeated `RUNNING_HEADER_MIN_OCCURRENCES` (5) times at distinct offsets is page furniture — C3is' FY2025 20-F repeats `TABLE OF CONTENTS` 129 times and a 10-Q stamps `PART I` on all 47 of its pages, while a heading printed once in the TOC and once in the body occurs twice. **Furniture is then handled by shape, two ways:** a bare marker (`Item 1`, `PART I`) carries no navigable text, so no occurrence of it earns an entry and every one is dropped; anything else keeps its FIRST occurrence and drops the rest, inverting the keep-later rule — a filing that runs a section's own heading across every page of that section (`NOTES TO THE CONSOLIDATED FINANCIAL STATEMENTS` through an S-1's financial statements) prints it first at the section start, and keep-later would land it on the section's last page instead. So the outline carries no fused composite and at most one entry per running header, at its first occurrence. **A bare marker is counted on its own line, never on the joined heading:** a page footer's marker takes its title from whatever prose the next page opens with, so every composite it produces is a different string and only the marker line repeats — count composites instead and a 10-Q's per-page `Item 1` footer yields one bogus entry per page, crowding the real Items past `maxEntries`. That count is per rendered form (`Item 1` and `Item 1.` are separate), since a filing stamping the bare form on every page still writes the punctuated one for the real heading. Dedup (keep the later, i.e. body, occurrence) keys on `headingKey` — `foldForHeadingMatch`, a terminal-punctuation strip, and the separator between an Item/Part marker and its title folded to one space — never raw lowercase: the same filing prints one marker with different space characters and a trailing period in its TOC and its body (`Item 3.` with U+0020 vs U+2009; `…Policies` vs `…Policies.`), and a raw key leaves both in the outline. The separator is the third difference: Worthington Steel's 10-Q (0001968487-26-000036) prints `Item 1A. Risk Factors` in its TOC and `Item 1A. – Risk Factors` in its body, so both kept an entry and `risk factors` landed on the TOC row at 4292 instead of 250397. A marker key is rebuilt as the marker, one space, and the title (`item 1a risk factors`), whatever period, dash, or colon stood between them (`MARKER_TITLE_RE`, exported for the section matcher); across the 67-filing corpus that drops the 10-Q's 11 TOC copies (34 → 23 entries) and moves no other outline entry (#174). A title run straight into its marker gets its space where detection ends the marker, before the fold lowercases the capital that marks it (`RUN_IN_MARKER_RE`): `Item 1Financial Statements` keys as `item 1 financial statements`, not `item 1f inancial statements`, so it dedups against a TOC row's `Item 1. Financial Statements`, `dropBarePartsTitledLater` reads `Part IFinancial Information` as a titled Part I, and `section: "part i"` lands on it as Part I's heading (#180). `section` reads the same space in through `spaceRunInMarker`: the Item/Part test (`isStructuralHeading`, which the outline cut shares) and `sectionMatches`' operands (`foldForSectionMatch`, heading and value alike), so the numeral ends there too — `part i` reaches New Peoples' `Part IFinancial Information` (4136) in tier 2 instead of through the line tier with a line notice, and a run-in Part past the 50th heading stays in a cut outline, while `part i` never reaches `Part IIOther Information` or `Part IVExhibits` and `item 1` never `Item 12Security Ownership`. A run-in line keys one character longer than it prints, so the line scan's length short-circuit allows that one character. The furniture count, and the copy furniture keeps, key on the printed form instead — `headingKey` short of the separator fold (`printedKey`) — so a TOC row and a body heading never pool with a running header printed with another separator and lose the body heading to the TOC row. A TOC row's trailing page cell — a page number or a lettered page (`F-3`), optionally after a `Page` word (`Page 1`), set off by a tab or two or more spaces (`TOC_PAGE_TAIL_RE`) — is stripped from every heading for the same reason. Without the `Page` form, First Bancorp's 8-K TOC row `Item 8.01 Other Events Page 1` (0000765207-26-000110) kept its cell, never deduped against the body `Item 8.01 Other Events.`, and took `item 8` at 1998 instead of 2227. A number after a single space is the heading's own (`PROPOSAL NO. 1`, `ARTICLE 5`, `…Securities Exchange Act of 1934`): the single-space strip cut it off, and dedup then merged GT Biopharma's three `PROPOSAL NO. N` headings into one `PROPOSAL NO.` at the last (#172). Across 81 cached filings the change moved nothing else in any outline. Lettered sub-item markers (a bare `A.`/`D.` inside an Item) are deliberately not detected — single letters recur throughout filing prose — and neither is a digit-led sub-item line (`3D. RISK FACTORS`; the all-caps arm needs a leading letter): `section` reaches a labelled line by stripping the label at comparison time instead (#175, see Section matching), so no detected heading is added.
|
|
108
108
|
- **Section matching:** `get_filing`'s `section` compares through `foldForHeadingMatch` (`filing-to-text.ts`) on **both** operands — Unicode whitespace runs collapse to one space, U+2018/U+2019 → `'`, U+201C/U+201D → `"`, then lowercase. Comparison-time only: `FilingHeading.heading`, the `outline` output, `content[]`, and every character offset keep the document's own bytes. EDGAR headings carry NBSP runs and typographic quotes, so a caller re-sending an outline heading through a UI that normalizes either one never byte-matched the heading it came from — which broke the `section_not_found` recovery loop the error's own hint points at (#106). No fuzzy matching — `"item 8"` resolves only to Item 8 — but every tier runs on folded operands, so a single-spaced needle also reaches a heading whose marker carries an NBSP run or a double space (`Item 9.` in a 2004 10-K). Folding only one operand does not fix the round trip. One boundary (`sectionMatches`): a needle ending in a number never continues into a longer one. A trailing digit never runs on into another digit, so `item 1` matches `Item 1.` and `Item 1A` but never `Item 12`, and `item 5` reaches an 8-K's `Item 5.02` while `item 5.0` does not. A trailing roman numeral (a final word of `i`, `v`, `x`) must end at a space, punctuation, or the heading's end, so `part i` never matches `PART II`, `PART III`, `PART IV`, or `PART II, III`: Sono-Tek's 10-Q (0001171520-26-000305), which prints Part I only inside a one-line TOC, landed `part i` on `PART II - OTHER INFORMATION`, and Microsoft's FY2026 10-K on `PART II, III` (348204). `part i` now falls through to the line tier (Microsoft: the body `PART I` at 6900) or misses with the outline (Sono-Tek; an S-1, whose Part I is the unlabeled prospectus). Plain-text outlines keep TOC rows worded differently from their body heading, and a TOC row for Item 10–16 precedes the body Item 1 — a bare substring test landed `item 1` on another Item in 6 of 37 plain 10-K/10-Q outlines and 1 of 270 HTML landings (#136). **Resolution order (#169, #166, #177)**, first hit wins, over every detected heading (`detectHeadings(text)`, which takes no cap, run once per call — the 50-entry cap belongs to the rendered outline alone; with the first 50 only, a heavy 10-K's financial-statement captions put Items 9B–16 out of reach) and the document's own lines: (1) **exact** (`resolveSection`) — a heading whose `headingKey` (the dedup key: the fold, a terminal-punctuation strip, and the marker separator folded to one space) equals the needle's, or whose title after its Item/Part marker does — failing that, the same with a leading sub-item label removed (below); a bare marker (`ITEM 1.`, `Part II`) has no title and is never a candidate, or GSBD's `item 1` jumps to a bare TOC `ITEM 1.`. The tier exists for #170's widened all-caps class: CCC's `SPECIAL NOTE REGARDING FORWARD-LOOKING STATEMENTS AND RISK FACTORS` (7562) otherwise takes `risk factors` from `Item 1A. Risk Factors.` (67351). (2) **Item/Part substring** — the first Item or Part heading (`isStructuralHeading`, bare and run-in markers included) that `sectionMatches`: a value inside an Item title names that Item, while a line equal to it is often a table-row label (CCC's `compensation` → `Item 11. Executive Compensation` at 493087, not the `Compensation` cell at 418658). (3) **an equal line** and (5) **a table-row label, else a prefix line** — a document line (`matchSectionLine`, `filing-to-text.ts`, which reports whether its line ranks in tier 3; `findSectionLine` is its offset alone), for a mixed-case subheading the outline never lists (`Liquidity and Capital Resources`): a line whose `headingKey` equals the needle's and that labels no table row (3), else — only when no such line exists — the first equal line that labels one (5, below), else one that starts with it (5) at a word boundary within `MAX_TITLE_CHARS` (a date-suffixed title: `Consolidated Schedule of Investments as of June 30, 2026`), unless a one-word needle runs on into a longer Title Case title (`continuesIntoLongerTitle`: the next word opens on a capital in a line that is not all caps). One word heads many unrelated titles — `business` landed on `Business Combinations.` (C3is), `Business Seasonality and Product Introductions` (Apple 10-Q), and a cross-reference line `Business Overview—Governmental regulations…` (DRDGOLD); a needle of two or more words keeps its continuation (`Human Capital` → `Human Capital Management`), and a suffix opening lowercase or on punctuation (`(continued)`, `—Parent Company Only`) always does. The cost: `forward-looking` no longer reaches `Forward-Looking Statements`, nor `revenue` `Revenue Recognition`; both miss with the outline. Equality goes first because a prefix match alone lands `Competition` on an earlier line that merely begins with the word. **A table-row label ranks in tier 5, ahead of a prefix line (#184):** an equal line labels a row when its next line with content (spacer lines passed over) is a figure row, or is a label — at most 60 characters, opening uppercase, ending on no `.`, `:`, `;`, or `,`, closing quotes and brackets passed over (a cross-reference `See “Item 18. Financial Statements.”` over its page number is no label) — whose next line with content is one (`labelsTableRow`; Microsoft's segment table `Revenue` / `Client` / `$ 9,350 …`). A figure row is a lone currency or nil cell (`$`, `-`, `–`, `—`), or a line whose text after its last letter holds a digit and, when the line has a letter, opens on a dotted leader, a tab, or two or more spaces behind a label of at most 60 characters (`isFigureRow`: `Product sales....... $4,759`; ``$ 7,746 $ `7,835``, a stray backtick in a 2004 cell); one space never qualifies (`The effective rate was 21%.`), and past the label bound the line is a table set on one line, which a heading may sit over (AMC Robotics' S-1 0001493152-26-047958: MD&A `Revenue` at 177926 over `Three months ended June 30, 2026 2025 $ Change …`). All 13 `Revenue` lines in Microsoft's 2004 10-K (0001193125-04-150689) label rows, and as tier-3 lines took `revenue` from `SEGMENT PRODUCT REVENUE/OPERATING INCOME (LOSS)` (81018) to the selected-data row at 66987; Aware's `Revenue:` over `Product sales....... $4,759` (112947) took it from `CONTRACT REVENUE` (49489). Demoted rather than skipped, a row label still lands when no heading contains the value (`total revenue` keeps its income-statement row), under the plain line notice. The test reads at most two content lines past an equal line. Over 26 corpus filings × 104 section values it labels 124 lines on the resolution path, none a detected heading, and moves 22 landings and no outline entry: 15 from a row label onto a heading or prose subheading for the value (Microsoft FY2026 `revenue` → its MD&A `Revenue` subheading at 217873; `earnings per share` → the EPS note in both Microsoft 10-Ks), 2 onto a table's caption line, 1 onto another row label (BAB's `goodwill`, from its balance-sheet row to a note table's), and 4 onto a heading that only contains the words (Arxis `net income` → `NOTE 16. NET INCOME (LOSS) PER SHARE`, and Nutanix's → `NOTE 11. NET INCOME (LOSS) PER SHARE` past a `Net income` row of `—` cells; CCC `net income` → the all-caps row label `NET INCOME (LOSS) INCLUDING NON-CONTROLLING INTEREST`, a detected heading; CCC `revenues` → `COST OF REVENUES` at 332106, since all four of its `Revenues` lines label rows). In both line tiers a line is skipped when it opens lowercase (a wrapped sentence or bullet); when it sits under a line ending on a comma or semicolon, or opens on an unpunctuated Item marker followed by a lowercase word (a wrapped cross-reference, the `followsMidSentence` / `isWrappedProse` guards heading detection applies: `…in Part II,\nItem 8 on this Form 10-K…`); or when it is a TOC row (`isTocRow`). A TOC row ends in a page cell set off by **two or more** spaces or a tab (`TOC_PAGE_TAIL_RE`; one space would read the year in `as of June 30, 2026` as a page number), or its next line with content is a page cell (`PAGE_CELL_LINE_RE`: a number, `F-3`, or a lowercase roman `ii`, which counts only on its own line — `Part iv` is not a page cell); either form may put a `Page` word before the number (`Page 3`), spacer lines passed over (an HTML TOC table puts NBSP rows between its cells). A line that is not a marker, followed directly by an Item/Part marker, is a TOC title cell with no page column; a marker run straight into its title counts (`MARKER_LINE_RE` takes `TITLE_RUN_IN`, #180: `Item 2Properties`). A marker line — a Part over its first Item — is a TOC row when that Item row is: the Item line ends in a page cell, the line below it holds or ends in one (a title cell, or Aware's wrapped `of Equity Securities....... 12`), or, for a bare marker, the line below its title holds one. Microsoft's FY2026 10-K prints body `PART IV` only as page furniture, so `part iv` reaches the line tier, where `PART IV / Item 15. / Exhibit and Financial Statement Schedules / 93` was taken; it now lands on the body's first `PART IV` (352884). A body Part sits over its Item too, but prose follows, not a page cell. The first remaining line wins, at its first non-blank character; one linear pass serves tiers 3 and 5, run only after tiers 1 and 2 miss, so a value landing on a tier-4 heading now pays that pass (7–11 ms on a 1 MB S-1). A cut page's guidance after a `section` landing says to pass `next_offset` as `offset` without `section`, which outranks `offset` — kept, it returns the same page. A line landing says so in the notice, composed into the truncation guidance when the page is cut (`notice` is last-wins): `No detected heading matches section '<value>', so this page starts at the first document line that does.` — no new output field. An equal line that outranked a heading merely containing the value says `No detected heading equals section '<value>', so this page starts at the first document line that does, ahead of headings that only contain it.` instead, since that heading does match by substring. Every #166 acceptance offset holds. (4) **other substring** — the first other heading that `sectionMatches`, after the equal line: a heading that merely holds the word took Arxis' S-1 `business` to `NOTE 3. BUSINESS COMBINATIONS` (544705) past its `Business` heading line (323578), and CCC's `revenue` to `COST OF REVENUES` (332106) past the MD&A `Revenue` subheading (256790). An exact or Item/Part heading still beats an equal line (Reddit's `use of proceeds` → the `USE OF PROCEEDS` heading); every other heading ranks after one. Over 67 corpus filings × 103 section values the order moved 24 landings, each one the validation predicted: 17 onto the section named (Microsoft FY2026 `general` → `GENERAL` at 8720, not `GENERAL RISKS`; DRDGOLD's `risk`, `directors`, `management`, and `employees` → body Items instead of TOC rows), and 7 onto a different place that is defensible but that a reader may like less, since an Item heading or a line equal to the value now outranks a heading that only holds it — JPM's 10-Q `risk` lands on Item 3, a one-sentence cross-reference, not `FIRMWIDE RISK MANAGEMENT`; `revenue` in Microsoft's 2004 and FY2026 10-Ks and Aware's on a selected-data, MD&A-table, or income-statement row label rather than a heading containing the word — the three landings tier 5's row labels move (#184, above); `summary` on Item 16 `Form 10-K Summary`; Intel's 2000 `consolidated balance sheets` on an Item 14 index row. (6) `section_not_found`. **A proxy statement keeps its headings ahead of the lines:** for a proxy or information statement form (`PROXY_FORM_RE`: DEF 14A, DEFA14A, DEFM14A, PRE 14A, DEF 14C, …), tier 1 is followed by the first heading of any kind that `sectionMatches`, then the line tiers — its ITEM headings are ballot proposals, not the Regulation S-K item taxonomy, and its lines equal to a value are mostly table cells. Under tiers 2–4 Goldman Sachs' DEF 14A (0001193125-26-117433) took `compensation` from the CD&A (123680) to `ITEM 2. … (SAY ON PAY)` (274744) and `summary` from `EXECUTIVE SUMMARY` (16953) to a Pay versus Performance column header (282286). The form is read before `section` resolves, so the `.hdr.sgml` fallback request (#126) now precedes a `section_not_found`. **Inside each heading tier a body heading ranks ahead of a table-of-contents row (#174):** a heading is a TOC row when its title line — its own line, or the next line with content after a bare or joined marker — is one by `isTocRow` (the line tiers' test), or the next line with content below it ends in a page cell (a TOC title wrapped onto a second line, Aware's `of Equity Securities....... 12`), or the title wraps once with its page cell on its own line under the wrap — the title line, one continuation line, and a `PAGE_CELL_LINE_RE` line on three consecutive lines, the continuation in the title line's case: no lowercase letter under an all-caps title line, an opening lowercase one under a mixed-case one (`of Equity Securities`) (#183). DRDGOLD's TOC row `ITEM 14.` / `MATERIAL MODIFICATIONS TO THE RIGHTS OF SECURITY HOLDERS AND USE OF` / `PROCEEDS` / `79` took `item 14` and `material modifications to the rights of security holders` at 8146 from the body heading at 433144. A body heading over one line of body text and a page number has the same three lines: across 26 filings the step without the consecutive-line and case conditions flags 47 body headings (`Item 4. Mine Safety Disclosures` / `Not applicable.` / `31`, spacer lines between or not, and DRDGOLD's `NOTE 22 ASSET HELD FOR SALE` over its page footer and `F-50`); with them it flags only that TOC row (8146) and its title cell (8155), moves those two landings, and changes no outline. An all-caps body heading over a one-word all-caps line and a page number on consecutive lines (`ITEM 4. MINE SAFETY DISCLOSURES` / `NONE` / `31`) still reads as a TOC row and lands as its tier's only match; `isTocHeading` in `filing-to-text.ts` owns the test, finding the line start from the newline before the offset since an NBSP-led Item's offset sits one past it. A TOC row still lands when it is its tier's only match. Three TOC shapes survive dedup: a row worded apart from its body heading (First Bancorp's 8-K `Item 9.01 Financial Statements and Exhibits. Page 1` took `item 9` at 2041 from the body Item at 2402), a bare all-caps marker over its title and page cells (DRDGOLD's FY2026 20-F `ITEM 3.` / `KEY INFORMATION` / `6` took `item 3` at 6174 from `ITEM 3. KEY INFORMATION` at 34346, and nearly every other `item N` the same way), and the title cell itself (`key information`). TOC rows are ranked, never dropped or pushed behind the line tiers: the test also reads a body heading over a page number (CCC's `Item 6. Reserved.` at 234199, Reddit's S-1 `PART II` at 1021631), and behind the line tiers CCC's and Microsoft's `item 6` left the body heading for a bare TOC marker line. With the dedup-key fold, the ranking moved 83 landings over the corpus, each from a TOC row onto a body heading (DRDGOLD 35, Worthington Steel 20, GSBD 9, Amazon's 2003 10-K 8, Aware 6, First Bancorp 4, CCC 1), and no other: 78 onto the body heading the row copies, 5 onto another the value names (Amazon 2003's `part ii`–`part iv` onto its body Parts, Aware's `management` from its TOC Item 12 to body Item 7, Worthington Steel's `other information` from its TOC Item 5 to body `PART II. OTHER INFORMATION`). **A leading 20-F sub-item label is ignored (#175):** 20-F filers label sub-items (`3D. RISK FACTORS`, `B. Business Overview`), so a plain-phrase value never equalled the line. Tier 1 and both line tiers also compare the heading or line key with a leading label removed (`withoutSubItemLabel`: `/^(?:\d{1,2}[a-z]|[a-z])\. /` on the folded key, anchored, so constant-time), and the one-word longer-title rule judges the line after its label — Regencell's `B. Business Overview` keeps `business` a miss while `business overview` lands (151573). The value itself is never stripped, so `3D. RISK FACTORS` still matches its own line; a line opening on an initial (`L. John Doerr`) strips the same way and so matches only a value equal to the rest of the line. The landing is the line's first character, label included (DRDGOLD `risk factors` → 34503). Tier 1 strips only once no heading equals the value as printed, so an earlier `A. RISK FACTORS` never takes `risk factors` from a later `Item 1A. Risk Factors`; a labelled body heading still ranks ahead of a TOC row. The line tiers keep the first line equal either way: across the corpus the labelled line they land on is the 20-F sub-item itself and a later unlabelled one is narrower (DRDGOLD `business overview` → `4B. BUSINESS OVERVIEW` at 176975, not `Business overview` at 269017). Over the corpus the strip moved 23 landings, all in the three 20-Fs: 19 misses now land on the labelled sub-item, and 4 earlier landings move onto one (DRDGOLD `compensation` from the `Compensation of key management` subheading inside Item 6B, 328266, to `6B. COMPENSATION` at 324690); no filing without labelled sub-item lines changed. **The outline cut (#169):** past 50 detected headings the outline is every Item/Part heading (`isStructuralHeading`, bare and run-in markers included) plus the earliest other headings up to 50, in offset order (`selectOutline`) — shared by the first-page `outline` and the `section_not_found` message and data. When the Item/Part headings alone pass 50, each bare marker that a later titled heading for the same Item or Part repeats is left out first (`outlineStructuralHeadings`): DRDGOLD's FY2026 20-F renders all 31 TOC Items as bare markers (`ITEM 3.`), which filled the cut and pushed body Items 16A–19 out; its outline now runs `ITEM 1.` through `ITEM 19. EXHIBITS` from the body. A bare marker no titled heading repeats stays, and `section` still matches every heading. The miss message heads a cut outline `Outline (50 of N detected headings):` (else `Outline:`), and a cut first page appends `The outline lists 50 of N detected headings, every Item and Part heading among them; section matches all of them.` to the truncation guidance — `…among them, bare table-of-contents copies of titled ones aside; …` when bare markers were left out (DRDGOLD: `The outline lists 50 of 142 detected headings, every Item and Part heading among them, bare table-of-contents copies of titled ones aside; section matches all of them.`), and no Item/Part clause when the kept Item/Part headings alone pass 50, which reaches `structuredContent.notice` and `content[]` — no new output field, since the outline is emitted only on a truncated first page. A document of 50 or fewer headings renders as before.
|
|
109
109
|
- **Binary filing documents:** a filing index routinely carries scanned pages, PDF exhibits, `Financial_Report.xlsx`, and `-xbrl.zip` — `filingToExtract` runs `html-to-text` over whatever bytes arrive and never inspects a content type, so requesting one used to return decoded binary as `content` with no error (#96). `get-filing.tool.ts` detects them with one rule, `binaryType`: the filename extension (`binaryTypeFromName`), the one signal every filing carries, plus a header-page `GRAPHIC` type for a scan named oddly — the header page is read before the body (#161). Two halves, both needed — `documents[]` entries carry `binary: true`, and `resolveFilingArchive` throws `binary_document` before `tryGetFilingDocument` runs, since the `document` input is an unconstrained string and a catalog marker alone cannot stop a caller. Extension-driven and bucket-independent on purpose: a PDF exhibit is typed `EX-99.*` and files under `exhibits`, not among the graphics.
|
|
110
|
-
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); a company's submissions feed also lists the Form 4s it filed as a reporting owner of another issuer (JPMorgan's 10% stake in a BlackRock fund), so a parsed document whose `issuerCik` differs from the resolved CIK, compared with zero-padding stripped, adds no row inline or to the dataframe and is counted in `filings_other_issuer` — it still counts toward `filings_scanned`, the scan floor, and `history_scanned_through`, and the no-canvas stop rule is unchanged — though a skipped filing adds no transactions, so a no-canvas call reads on through such filings until `limit` transactions are in hand, within the same scan cap (#157); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). In an annual matrix an instant enters only on one of the filer's fiscal-year ends and takes the key of the annual duration ending that day. The ends come from the companyfacts payload already in hand (`filerFiscalYearEndsWithComparatives` in `concept-series.ts`, shared with `get_financials`' annual view of an instant): each annual report's own year (the #148 rule) plus the comparative years it restates — its year-long facts ending a whole number of years earlier, within a week — so the first XBRL 10-K's prior-year balance keys too, while a year-long figure ending off the report's calendar (a December filer's trailing year to 09-30) stays out. The #148 test of a 10-Q-held annual frame keeps `filerFiscalYearEnds` (each report's own year only): it matches day of year, where a comparative year adds nothing but a wider tolerance. The key is the first `CY####` frame on any concept in the payload ending then, so a balance lands beside its fiscal year's income lines even when none is requested; with no framed duration ending on it, the calendar year in which that fiscal year's second half starts (SEC's own framing: a year ending 06-30 is that year, one ending 06-27 or 05-31 the year before — Seagate's and Western Digital's years ending 2025-06-27 are `CY2024`; SEC departs from it for the year both ended 2012-06-29, framed `CY2012`, which is why a framed key wins). Interim and cover-date instants stay out of the annual matrix, and a company-concept left with none gets a gap plus a caveat pointing at quarterly; when that empties the whole matrix, `no_comparable_data` says the values were all off fiscal-year ends and its hint names `period_type: "quarterly"` (omitting `period_type` keeps annual). Quarterly keys are unchanged. The year ends are read lazily, only when an annual matrix meets an instant (#171). The period-ends caveat checks every inline column — the newest can hold one company alone — spelling out the newest column whose ends differ and naming the others; the inline window is the newest periods of the union across companies, so a company-concept pair whose values all predate it is named in `caveats` — one line per concept, each company with its newest period and the dataframe pointer — and never in `gaps`, which keeps meaning no value in any period (#144). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, `sql_read_only`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints — since mcp-ts-core 0.13.15 the gate also denies engine-setting and SQL-from-string functions (`current_setting`, `query_table`) as `denied_function` and a file named as a table (`FROM 'data.csv'`) as `sql_read_only`, and since 0.13.16 it refuses any unqualified `current_schema`, since the parse tree cannot tell the bare function from a column of that name — a staged or aliased column so named is read qualified (`c.current_schema`). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and
|
|
110
|
+
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); a company's submissions feed also lists the Form 4s it filed as a reporting owner of another issuer (JPMorgan's 10% stake in a BlackRock fund), so a parsed document whose `issuerCik` differs from the resolved CIK, compared with zero-padding stripped, adds no row inline or to the dataframe and is counted in `filings_other_issuer` — it still counts toward `filings_scanned`, the scan floor, and `history_scanned_through`, and the no-canvas stop rule is unchanged — though a skipped filing adds no transactions, so a no-canvas call reads on through such filings until `limit` transactions are in hand, within the same scan cap (#157); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). In an annual matrix an instant enters only on one of the filer's fiscal-year ends and takes the key of the annual duration ending that day. The ends come from the companyfacts payload already in hand (`filerFiscalYearEndsWithComparatives` in `concept-series.ts`, shared with `get_financials`' annual view of an instant): each annual report's own year (the #148 rule) plus the comparative years it restates — its year-long facts ending a whole number of years earlier, within a week — so the first XBRL 10-K's prior-year balance keys too, while a year-long figure ending off the report's calendar (a December filer's trailing year to 09-30) stays out. The #148 test of a 10-Q-held annual frame keeps `filerFiscalYearEnds` (each report's own year only): it matches day of year, where a comparative year adds nothing but a wider tolerance. The key is the first `CY####` frame on any concept in the payload ending then, so a balance lands beside its fiscal year's income lines even when none is requested; with no framed duration ending on it, the calendar year in which that fiscal year's second half starts (SEC's own framing: a year ending 06-30 is that year, one ending 06-27 or 05-31 the year before — Seagate's and Western Digital's years ending 2025-06-27 are `CY2024`; SEC departs from it for the year both ended 2012-06-29, framed `CY2012`, which is why a framed key wins). Interim and cover-date instants stay out of the annual matrix, and a company-concept left with none gets a gap plus a caveat pointing at quarterly; when that empties the whole matrix, `no_comparable_data` says the values were all off fiscal-year ends and its hint names `period_type: "quarterly"` (omitting `period_type` keeps annual). Quarterly keys are unchanged. The year ends are read lazily, only when an annual matrix meets an instant (#171). The period-ends caveat checks every inline column — the newest can hold one company alone — spelling out the newest column whose ends differ and naming the others; the inline window is the newest periods of the union across companies, so a company-concept pair whose values all predate it is named in `caveats` — one line per concept, each company with its newest period and the dataframe pointer — and never in `gaps`, which keeps meaning no value in any period (#144). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, `sql_read_only`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints — since mcp-ts-core 0.13.15 the gate also denies engine-setting and SQL-from-string functions (`current_setting`, `query_table`) as `denied_function` and a file named as a table (`FROM 'data.csv'`) as `sql_read_only`, and since 0.13.16 it refuses any unqualified `current_schema`, since the parse tree cannot tell the bare function from a column of that name — a staged or aliased column so named is read qualified (`c.current_schema`). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and 13D/G rows on `cusip`. Insider rows carry `issuer_cik` and no `cusip`, 13F rows the reverse; 13D/G rows carry both.
|
|
111
111
|
- **Beneficial ownership (13D/13G):** SEC replaced the legacy `SC 13D` / `SC 13G` text filings with structured XML under the `SCHEDULE 13D` / `SCHEDULE 13G` names on **2024-12-18**; the changeover is total, so a legacy-name filter matches nothing recent and a current-name filter matches nothing older. The two schedules are **separate schemas, not one shape with optional fields** — 13D repeats `reportingPersons/reportingPersonInfo` with the power fields flat on each person and the percentage in `percentOfClass`; 13G repeats `coverPageHeaderReportingPersonDetails` with the powers nested inside `reportingPersonBeneficiallyOwnedNumberOfShares`, the percentage in `classPercent`, no reporting-person CIK anywhere, and no purpose-of-transaction field at all (its item 4 is structured ownership data, which is what makes it the passive form). `beneficial-ownership-parser.ts` therefore branches per schema and dispatches on the submission type, falling back to whichever repeated reporting-person element the document carries. **Ownership powers and percent of class are per reporting person, including on joint filings** — a fund, its adviser, and its controlling principal each report the same underlying shares, so the parsed owner list is never collapsed to a group total and the percentages must not be summed. Discovery runs off the issuer's own `getSubmissions()`: SEC cross-lists a schedule under every CIK party to it, so no entity-scoped EFTS step is needed.
|
|
112
112
|
- **NPORT-P series routing:** a fund's public portfolio report covers **exactly one series**, a registrant trust holds many (iShares Trust: 390 listed series, 200+ reports for a single period), and the series identity appears **only inside the document** — neither the submissions feed nor EFTS carries `seriesId` on an NPORT-P row. Routing is therefore series-first: EDGAR's company browse accepts a series ID in place of a CIK (`getFundSeriesFilings`), which is the one SEC surface mapping a series to its accessions and which also names the registrant, so a bare series ID resolves in one request. The browse feed's registrant outranks any registrant the `fund` input implied, since `series_id` can name a series of an entirely different trust and the archive path is keyed on the registrant. The bounded header scan (`tryGetFilingDocumentHead` + `parseNportHeader`, cap `SERIES_SCAN_LIMIT`) is the fallback and the disambiguator for a trust whose series carry no listed ticker; SEC's archive host **ignores `Range`** (a ranged request answers 200 with the whole body), so the saving is client-side stream cancellation only — and the cutoff cannot cut below one read chunk, which Bun sets at 262,144 bytes, so `HEADER_MAX_BYTES` binds only on a document arriving in several chunks. **A registrant's own NPORT-P list is never the candidate set for a series-structured trust**: series fiscal quarters are staggered, so the list interleaves funds and one report at the newest period does not mean one fund — it means the fund whose quarter ends latest. The scan therefore runs on the newest batch even when that batch holds one report, and a series found there re-routes through the browse feed; only a report naming no series at all (a closed-end fund, which also carries no `classId`) is answered from the registrant's list. Candidates are ordered by **reported period, not filing date**: an `NPORT-P/A` restating an old period is filed after every report of the periods that followed it. Reports publish on a ~60-day lag, so `report_period_date` and `publication_lag_days` are first-class output and the `as_of` enrichment states both.
|
|
113
113
|
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. The ingester stores a tag's own name when SEC serves it with no label (`InterestExpenseNonoperating`), so every read path treats a stored label equal to a camel-case compound tag as no label (`storedLabel`) — callers fall back to the concept's label, and `getFrames` answers an empty `label` as the live frames API does; a one-word element's label is the word itself (`Revenues`) and is kept. Read-side, so stored rows need no migration. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (superseding the mirror's bare fund rows with series/class, and covering a mirror synced before MF ingestion), so funds resolve there too (#43, #135).
|
|
@@ -122,7 +122,7 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
122
122
|
| `EDGAR_RATE_LIMIT_RPS` | No | `10` | Max requests/second to SEC APIs. Do not exceed 10. |
|
|
123
123
|
| `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` | No | `600` | Seconds to stop sending to SEC after a 429. Calls are refused locally for this long, then the first one goes out alone as a probe. SEC lifts its block only after ten quiet minutes, so a shorter value just probes into it. |
|
|
124
124
|
| `EDGAR_TICKER_CACHE_TTL` | No | `3600` | Seconds to cache the ticker index (company_tickers.json + company_tickers_mf.json). A failed fund-file load is retried after 60 s (or the rate-limit cool-down) instead of standing for the TTL. |
|
|
125
|
-
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes
|
|
125
|
+
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes, counted from when the table is staged; queries do not extend it. |
|
|
126
126
|
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` carries no per-tool definitions at all, only capability flags, so nothing changes there (#103). |
|
|
127
127
|
| `EDGAR_CLAMP_OVERSIZED_LIMITS` | No | `true` | Serve a count or size input above the cap its tool applies at that cap (up to each field's rejection bound) instead of rejecting it, so the call returns what the cap returns. Set `false` to advertise each cap as the field's maximum and reject anything larger. Read before `createApp()` from its own variable, like `EDGAR_DATAFRAME_DROP_ENABLED`; every other server variable is validated in `setup()` behind the configuration banner (#197). |
|
|
128
128
|
| `EDGAR_MIRROR_ENABLED` | No | `false` | Enable the local SQLite mirror of company_tickers + XBRL company-facts. Node/Bun only (skipped on Workers). Bootstrap once with `bun run mirror:init`. |
|
|
@@ -287,9 +287,18 @@ src/
|
|
|
287
287
|
|
|
288
288
|
## Skills
|
|
289
289
|
|
|
290
|
-
|
|
290
|
+
Two skill trees live at the project root, written for different readers:
|
|
291
291
|
|
|
292
|
-
|
|
292
|
+
| Tree | For | Written by |
|
|
293
|
+
|:-----|:----|:-----------|
|
|
294
|
+
| `framework-skills/` | The agent developing this server — framework skills (`metadata.audience: external`) plus any development-time skills of the server's own | `init` and the `maintenance` sync (framework skills); you (your own) |
|
|
295
|
+
| `skills/` | The agents that install the server and call its tools — tool chains, identifier sources, gates | You, deliberately, with the `add-skill` skill. Absent by default; usually none is needed |
|
|
296
|
+
|
|
297
|
+
Development skills are modular instructions. Read them directly when a task matches — e.g., `framework-skills/add-tool/SKILL.md` when adding a tool. `bun run list-skills` prints the full registry. Their directory is deliberately not `skills/`: Claude Code and Codex auto-load a plugin's root `skills/`, so a server that ships `.claude-plugin/` or `.codex-plugin/` would hand them to every agent that installs it.
|
|
298
|
+
|
|
299
|
+
A published skill in root `skills/` reaches plugin installs and the `.mcpb` bundle as-is, ships in the npm package and the Docker image, and reaches MCP clients when the server opts in with `createApp({ skills: 'skills' })`. `lint:packaging` validates the tree.
|
|
300
|
+
|
|
301
|
+
**Agent skill directory:** Copy `framework-skills/` into the directory your agent discovers (Claude Code: `.claude/skills/`, others: equivalent). Skills then load as context without referencing `framework-skills/` paths. After framework updates, run the `maintenance` skill — Phase B re-syncs the agent directory.
|
|
293
302
|
|
|
294
303
|
Available skills:
|
|
295
304
|
|
|
@@ -301,6 +310,7 @@ Available skills:
|
|
|
301
310
|
| `add-app-tool` | Scaffold an MCP App tool + paired UI resource |
|
|
302
311
|
| `add-resource` | Scaffold a new resource definition |
|
|
303
312
|
| `add-prompt` | Scaffold a new prompt definition |
|
|
313
|
+
| `add-skill` | Author a skill the server publishes in root `skills/` for the agents that use it — when one earns its place, structure, validation, delivery |
|
|
304
314
|
| `add-service` | Scaffold a new service integration |
|
|
305
315
|
| `add-test` | Scaffold test file for a tool, resource, or service |
|
|
306
316
|
| `field-test` | Exercise tools/resources/prompts with real inputs, verify behavior, report issues |
|
|
@@ -351,7 +361,7 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
351
361
|
| `bun run format` | Auto-fix formatting (safe fixes only) |
|
|
352
362
|
| `bun run format:unsafe` | Also apply Biome's unsafe autofixes — review the diff; they can change behavior |
|
|
353
363
|
| `bun run lint:mcp` | Run the MCP definition linter standalone (rule catalog: `api-linter` skill) |
|
|
354
|
-
| `bun run lint:packaging` | Packaging surface checks — `server.json`/`manifest.json` env-var parity (run by devcheck) |
|
|
364
|
+
| `bun run lint:packaging` | Packaging surface checks — `server.json`/`manifest.json` env-var parity, `.mcpbignore` coverage, plugin manifests, the published `skills/` tree (run by devcheck) |
|
|
355
365
|
| `bun run list-skills` | Print an index of available skills from `framework-skills/` |
|
|
356
366
|
| `bun run changelog:build` | Regenerate `CHANGELOG.md` from `changelog/*.md` |
|
|
357
367
|
| `bun run changelog:check` | Verify `CHANGELOG.md` is in sync with `changelog/` (used by devcheck) |
|
|
@@ -370,7 +380,7 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
370
380
|
|
|
371
381
|
## Bundling
|
|
372
382
|
|
|
373
|
-
`bun run bundle` produces a `.mcpb` extension bundle for one-click install in Claude Desktop. The pack step is followed by `scripts/clean-mcpb.ts`, which prunes dev dependencies (`mcpb clean`) and strips
|
|
383
|
+
`bun run bundle` produces a `.mcpb` extension bundle for one-click install in Claude Desktop. `mcpb pack` reads `.mcpbignore`, not `.gitignore`, so every root path the bundle must not ship — source, tests, docs, caches, logs, agent docs, plugin manifests, gitignored local files such as the `/data/` mirror directory — needs a root-anchored line there (`/docs/`, never `docs/`, which also strips `node_modules/*/docs/`); `lint:packaging` names each missing or unanchored line. Root `skills/` ships on purpose. The pack step is followed by `scripts/clean-mcpb.ts`, which prunes dev dependencies (`mcpb clean`) and strips four classes of `node_modules/**` content that root-anchored `.mcpbignore` patterns cannot reach: dependency-shipped agent docs (`framework-skills/`, `skills/`, `.claude/`, `.agents/`, `SKILL.md`); platform-specific native bindings, which would otherwise lock the bundle to the platform it was packed on; node-gyp build output beside a `binding.gyp` (`config.gypi`, Makefiles, `obj.target/`, `.deps/`, object files and debug symbols), which records the packing machine's paths — compiled `*.node` addons stay; and build-only package files — the framework's `changelog/`, `scripts/`, `templates/`, `CLAUDE.md`, `AGENTS.md`, and config bases (`biome.json`, `vitest.config.base.mjs`, `config/tsconfig.base.json`), and `better-sqlite3`'s `deps/` and `src/` C sources. The bundle therefore ships without the DuckDB native, and `manifest.json` sets `CANVAS_PROVIDER_TYPE=none` so the canvas is off instead of failing to load `@duckdb/node-api`: the dataframe tools fail as `canvas_unavailable` and every other tool works normally (the data tools answer inline without registering a dataframe). MCPB is stdio-only — HTTP deployments are unaffected. Delete `manifest.json` and `.mcpbignore` to opt out; `lint:packaging` skips cleanly.
|
|
374
384
|
|
|
375
385
|
**Adding an env var requires both files:** `server.json` (registry discovery, `environmentVariables[]`) and `manifest.json` (bundle install UX, `mcp_config.env` + `user_config`). `lint:packaging` (run by `devcheck`) verifies the env var names match, that every `user_config` option is wired into `mcp_config.env` as `"X": "${user_config.X}"` (the host substitutes nothing else — `"${X}"` reaches the server as that literal string), and that an optional string option carries `"default": ""`.
|
|
376
386
|
|
|
@@ -440,7 +450,10 @@ import { getEdgarApiService } from '@/services/edgar/edgar-api-service.js';
|
|
|
440
450
|
- [ ] EDGAR upstream sparsity: schemas reflect real nullability; `format()` preserves uncertainty (don't fabricate facts from missing XBRL fields); tests cover at least one sparse payload
|
|
441
451
|
- [ ] Registered in `createApp()` arrays (directly or via barrel exports)
|
|
442
452
|
- [ ] Tests use `createMockContext({ errors: tool.errors })` from `@cyanheads/mcp-ts-core/testing` for tools with declared contracts
|
|
453
|
+
- [ ] `plugin.json` + `mcp.json` (Agent Plugins 1.0.0 — read by Codex, Cursor, GitHub Copilot, VS Code, Kiro) populated — `plugin.json` `name` = the unscoped repo name (lowercase `a-z`, `0-9`, `-`, `.` only); `version`, `description`, `author`, `homepage`, `repository`, `license`, `keywords` from `package.json`; no other top-level field, `extensions` included. `mcp.json` holds one `"type": "stdio"` entry keyed by the unscoped name, `npx -y <full package name>`, and the same fixed `env` as `.codex-plugin/mcp.json`. Neither file carries a user-supplied variable (no `""`, no `${…}` besides `${PLUGIN_ROOT}`/`${PLUGIN_DATA}`): keys reach Codex through `.codex-plugin/mcp.json` `env_vars`, which works only while `.codex-plugin/plugin.json` keeps `"mcpServers": "./.codex-plugin/mcp.json"`
|
|
443
454
|
- [ ] `.codex-plugin/plugin.json` populated — `name`, `version`, `description`, `repository`, `license` from `package.json`; `interface.displayName` = the unscoped repo name (never the npm scope — `lint:packaging` enforces this); `interface.shortDescription` from `package.json` description
|
|
444
455
|
- [ ] `.codex-plugin/mcp.json` updated — server name key is the unscoped repo name; every user-supplied variable (API key, contact email, instance URL) is listed in `env_vars` so Codex forwards it from the user's environment. Never write `"KEY": ""` into `env` — an empty value replaces the user's exported key and is read as unset
|
|
445
456
|
- [ ] `.claude-plugin/plugin.json` populated — `name`, `version`, `description`, `author`, `repository`, `license`, `keywords` from `package.json`; inline `mcpServers` entry keyed by the unscoped repo name. Every user-supplied variable is declared under `userConfig` (`type`, `title`, `description`; `sensitive: true` for keys and tokens; `required: true` or `default: ""`) and referenced from `env` as `"KEY": "${user_config.<option>}"` — mirror the `user_config` block in `manifest.json`. Never write `"KEY": ""` into `env`
|
|
457
|
+
- [ ] Any new root dev path has a root-anchored line in `.mcpbignore`; `bun run bundle` and `lint:packaging` pass
|
|
458
|
+
- [ ] If the server publishes a skill in root `skills/`: `lint:packaging` passes, and `field-test` verified every gate and identifier claim in it against live tool behavior
|
|
446
459
|
- [ ] `bun run devcheck` passes
|
package/CLAUDE.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Agent Protocol
|
|
2
2
|
|
|
3
3
|
**Server:** secedgar-mcp-server
|
|
4
|
-
**Version:** 0.
|
|
5
|
-
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.
|
|
4
|
+
**Version:** 0.17.0
|
|
5
|
+
**Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.18`
|
|
6
6
|
**Engines:** Bun ≥1.4.0, Node ≥24.0.0
|
|
7
|
-
**MCP SDK:** `@modelcontextprotocol/server` ^2.
|
|
7
|
+
**MCP SDK:** `@modelcontextprotocol/server` ^2.4.0
|
|
8
8
|
**Zod:** ^4.6.7
|
|
9
9
|
|
|
10
10
|
Query SEC EDGAR filings, XBRL financials, and company data through MCP. Read-only, no API keys required. Full design: `docs/sec-edgar-mcp-design.md`.
|
|
@@ -107,7 +107,7 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
107
107
|
- **Heading detection:** `detectHeadings` runs three line-anchored patterns — the all-caps forms, the mixed-case same-line `Item N. Title` / `Part II` forms (#71), and a bare `Item N` marker whose title lands on a later line. **One NBSP may open a same-line Item line (#176):** both Item arms admit a single U+00A0 ahead of the marker, since an ` ` before a styled span left Horizon Space's 8-K line `<NBSP>Item 1.01. Entry into a Material Definitive Agreement.` (0001929980-26-000542) out of an outline that listed Items 7.01 and 9.01. A lookahead holds the NBSP to an Item marker; the entry's offset is the marker's (1960, not the NBSP's 1959), `isWrappedProse` and `followsMidSentence` still read the line from its real start, and a line that is a bare marker once its page cell is stripped (`<NBSP>ITEM 2.02`, `<NBSP>Item 1. 3`) yields nothing (`itemEntryOffset`). Nothing wider: ASCII-space or tab indentation, two NBSPs, and NBSP-led Part lines stay out — Microsoft's 2004 10-K (0001193125-04-150689) stamps an NBSP-led `Part I` on every page, and admitting those turns its body `PART I` (3832), `PART II` (64076), and `PART IV` (308565) into running-header furniture. Across the 67-filing corpus the change adds that one entry and removes none. Every whitespace match is horizontal-only: `\s` matches `\n`, which fused runs of consecutive all-caps lines into composite entries (`"TABLE OF CONTENTS\n\nPART I"`) (#105). Horizontal-only is not space-or-tab — filings put NBSP inside all-caps headings too (`MICROSOFT<NBSP>CORPORATION`), so the all-caps arm nests the class-set difference `[\s--\n]` in its class (the `v` flag). It is one class, not the class alternated with `[^\S\n]`: the quantified alternation slows sharply on very long lines, while a single class scans in linear time. The all-caps line admits `-`, `–`, `—`, digits, and apostrophes, and `isAllCapsHeadingText` bounds it and narrows the last two. Its heading text runs at most `MAX_TITLE_CHARS` (140): past it the line is a paragraph set in capitals, which the widened class otherwise admitted (Alternative Ballistics' 308-character risk-factor coda took `section: "business"` by substring from `OUR BUSINESS`; EuroSPAC's 860-character tax legend; GT Biopharma's 288- and 326-character voting notices). A digit is admitted only inside a whitespace-delimited `\d{1,2}[A-Z]?\.?` token judged on the whole line, a TOC page cell included (`NOTE 10`, `SCHEDULE 14A`; `USD 2,929`, `FMR LLC(4)`, `WASHINGTON, D.C. 20549`, and `…ACT OF 1934` stay out), and an apostrophe only inside a word or after a word-final `S` (`MANAGEMENT’S`, `STOCKHOLDERS’ EQUITY`), never opening or closing the heading. The all-caps `ITEM …` arm takes the rest of its line whatever it holds, and neither Item arm admits a line that runs a table of contents together: a second Item marker — after whitespace, or abutting the period, digit, or parenthesis that ends the cell before it (`Item 2.Item 3.`, `Item 9.01Item 9.02`, #180), while an opening quote or parenthesis before it is a title citing another Item (`(Item 408 of Regulation S-K)`) — or a page cell mid-line followed by two or more spaces and the next cell (`isRunTogetherToc`). Without it a 238-character 8-K table of contents (`ITEM 8.01. OTHER EVENTS. 3 ITEM 9.01. …`, Amazon's 0001104659-26-107526) was one heading and took `item 8`, `item 9`, and `exhibits`. The test is on shape, not length: a 140-character bound on that arm dropped a proxy's one proposal heading, Modine's 185-character `ITEM 1 – TO APPROVE THE PROPOSAL TO AMEND THE Amended and Restated Articles…` (DEF 14A 0001104659-26-115309), and turned `item 1` and `proposal` into misses. Across 104 cached filings (244 all-caps and 568 mixed-case Item lines) the shape test flags only the Amazon line, and no all-caps Item line runs past 240 characters, so the arm carries no outer length bound. The mixed-case same-line arm keeps its title bound as its prose guard. A colon stays out: every colon line it admitted across 26 sampled documents was a statement caption (`CURRENT ASSETS:`). A Part line may carry a title — whitespace, an optional `-`/`–`/`—`/`:`/`.`, then an uppercase letter, at most 140 characters — and is a cross-reference rather than a heading (`isPartCrossReference`), on the mixed-case and the all-caps arm alike, when it sits under a line ending on a comma or semicolon like an unpunctuated Item, or when its title opens on another marker (`Part II — Item 5. Market for Registrant’s Common Equity`, a bank 10-K's cross-reference index row), which would otherwise also displace the bare body Part it names. After dedup, a bare Part entry gives way to a later titled heading for the same numeral (`dropBarePartsTitledLater`, which applies the same marker-title test): a 10-Q's TOC row `Part II` and its body `PART II — OTHER INFORMATION` have different keys, so without the merge `section: "part ii"` landed on the TOC row 50,045 characters early (Apple Q3 FY2026). Over the 26-document corpus these rules add 72 headings and remove 4 TOC Part rows: 66 section headings (all 18 of Microsoft FY2026's `NOTE N — TITLE` lines, statement titles, titled Parts) and 6 furniture (statement captions, `SCHEDULE 14A`, a `PAGE` stamp, a wrapped legend line), no numeric table row (#170). The Item code spans `16A`–`16K` and the current 8-K's dotted `N.NN` (`ITEM_CODE`, shared by all four Item patterns), and the marker's period is optional — 20-F filers write the lettered markers unpunctuated. A dotted code may also abut its title or carry a paragraph suffix (`Item 9.01Financial Statements…`, `Item 9.01(d). Financial…`, the shapes html-to-text leaves when the marker and title are separate spans), and then the title must start with a letter, so a bare `Item 5.02.` line never matches on its own. **Any other code may abut its title too (#180):** past its period (`Item 2.Management’s Discussion…`, `Item 1A.Risk Factors`, `Item 16A.Audit…`), the title again opening on a letter, while a digit after the period stays part of the code (`Item 2.02` and the dotted forms above read as before; `Item 2.0 something` is no heading). With no period, a code or Part numeral runs into a title only where a capital followed by a lowercase letter opens it (`TITLE_RUN_IN`, a case-sensitive `(?-i:)` lookahead inside the case-insensitive patterns): `Item 1Financial Statements` is Item `1` and `Part IFinancial Information` Part `I`, while a lettered code or a longer numeral reads whole (`Item 16KCybersecurity`, `Part IVExhibits`) and an all-caps run-in (`Item 1ARISK FACTORS`) matches nothing. New Peoples Bankshares' 10-Q (0001903596-26-000307) prints every body marker this way and had no Item in its outline; it now lists all eleven Items and Part I (28 → 40 entries; the period form alone 28 → 38), `item 1` lands on Part I's `Item 1Financial Statements` (4167) instead of Part II's `Item 1.Legal Proceedings` (113660), and `risk factors` and `management's discussion` land at 114156 and 73945 instead of missing. Across the other 25 corpus filings no outline entry or landing moved. Every new alternative is a fixed-width lookahead, so the scans stay linear. A code in `CURRENT_EIGHT_K_ITEMS` takes a 160-character title bound (`EIGHT_K_MAX_TITLE_CHARS`) on the same-line arm and in `usableTitle` — Item 5.02's 146-character title is the longest across the 53 sampled 8-Ks, and the 14-character margin admits a short trailing note while a paragraph opening on the marker line (`Item 5.02(b) On June 2, 2026, the Board…`) runs to hundreds of characters and stays out — and every other marker keeps the 140-character prose guard, a dotted code outside the table (`Item 7.09`) included. Across 53 current 8-Ks the outline holds all 112 item codes EDGAR lists for them (24 before); without the bound exemption it holds 89, without the abutting/suffix form 103 (#165). When an all-caps marker line (`ITEM 2.02`) both matches the all-caps arm alone and joins the title below it, and both entries survive dedup at one offset, the bare entry is dropped — one heading, listed once. Because the period is optional, an unpunctuated same-line Item is dropped as wrapped prose (`isWrappedProse`) when its next word starts lowercase or the line above ends on a comma or semicolon, a title run straight into its code included (`…in Part II,` over `Item 8Financial Statements of this report`, #180). A line ending on a lowercase word is no evidence: a running page header (`Table of Contents`) sits directly above an unpunctuated body heading (`Item 5 — Other information`) at the top of a page. A plain-text 10-K's cross-reference that wraps to start a line with `Item 8 on this Form 10-K…` otherwise takes the `item 8` landing ahead of the real heading (#136). **A bare marker is admitted on the shape of what follows, not just on a title being there:** a blank line between marker and title admits the pair; without one, the pair is admitted only when no bare page-number line sits under the title, since marker/title/page-number on three tight lines is the TOC-cell artifact #71's guard exists to reject. That test is one-way — it rejects a TOC row, it does not certify a heading — so a marker line anywhere else (a page footer, an unwrapped paragraph) is admitted too, taking the next line as its title; the entry carries the marker's own offset, so `section` still lands in the right place. A heading text repeated `RUNNING_HEADER_MIN_OCCURRENCES` (5) times at distinct offsets is page furniture — C3is' FY2025 20-F repeats `TABLE OF CONTENTS` 129 times and a 10-Q stamps `PART I` on all 47 of its pages, while a heading printed once in the TOC and once in the body occurs twice. **Furniture is then handled by shape, two ways:** a bare marker (`Item 1`, `PART I`) carries no navigable text, so no occurrence of it earns an entry and every one is dropped; anything else keeps its FIRST occurrence and drops the rest, inverting the keep-later rule — a filing that runs a section's own heading across every page of that section (`NOTES TO THE CONSOLIDATED FINANCIAL STATEMENTS` through an S-1's financial statements) prints it first at the section start, and keep-later would land it on the section's last page instead. So the outline carries no fused composite and at most one entry per running header, at its first occurrence. **A bare marker is counted on its own line, never on the joined heading:** a page footer's marker takes its title from whatever prose the next page opens with, so every composite it produces is a different string and only the marker line repeats — count composites instead and a 10-Q's per-page `Item 1` footer yields one bogus entry per page, crowding the real Items past `maxEntries`. That count is per rendered form (`Item 1` and `Item 1.` are separate), since a filing stamping the bare form on every page still writes the punctuated one for the real heading. Dedup (keep the later, i.e. body, occurrence) keys on `headingKey` — `foldForHeadingMatch`, a terminal-punctuation strip, and the separator between an Item/Part marker and its title folded to one space — never raw lowercase: the same filing prints one marker with different space characters and a trailing period in its TOC and its body (`Item 3.` with U+0020 vs U+2009; `…Policies` vs `…Policies.`), and a raw key leaves both in the outline. The separator is the third difference: Worthington Steel's 10-Q (0001968487-26-000036) prints `Item 1A. Risk Factors` in its TOC and `Item 1A. – Risk Factors` in its body, so both kept an entry and `risk factors` landed on the TOC row at 4292 instead of 250397. A marker key is rebuilt as the marker, one space, and the title (`item 1a risk factors`), whatever period, dash, or colon stood between them (`MARKER_TITLE_RE`, exported for the section matcher); across the 67-filing corpus that drops the 10-Q's 11 TOC copies (34 → 23 entries) and moves no other outline entry (#174). A title run straight into its marker gets its space where detection ends the marker, before the fold lowercases the capital that marks it (`RUN_IN_MARKER_RE`): `Item 1Financial Statements` keys as `item 1 financial statements`, not `item 1f inancial statements`, so it dedups against a TOC row's `Item 1. Financial Statements`, `dropBarePartsTitledLater` reads `Part IFinancial Information` as a titled Part I, and `section: "part i"` lands on it as Part I's heading (#180). `section` reads the same space in through `spaceRunInMarker`: the Item/Part test (`isStructuralHeading`, which the outline cut shares) and `sectionMatches`' operands (`foldForSectionMatch`, heading and value alike), so the numeral ends there too — `part i` reaches New Peoples' `Part IFinancial Information` (4136) in tier 2 instead of through the line tier with a line notice, and a run-in Part past the 50th heading stays in a cut outline, while `part i` never reaches `Part IIOther Information` or `Part IVExhibits` and `item 1` never `Item 12Security Ownership`. A run-in line keys one character longer than it prints, so the line scan's length short-circuit allows that one character. The furniture count, and the copy furniture keeps, key on the printed form instead — `headingKey` short of the separator fold (`printedKey`) — so a TOC row and a body heading never pool with a running header printed with another separator and lose the body heading to the TOC row. A TOC row's trailing page cell — a page number or a lettered page (`F-3`), optionally after a `Page` word (`Page 1`), set off by a tab or two or more spaces (`TOC_PAGE_TAIL_RE`) — is stripped from every heading for the same reason. Without the `Page` form, First Bancorp's 8-K TOC row `Item 8.01 Other Events Page 1` (0000765207-26-000110) kept its cell, never deduped against the body `Item 8.01 Other Events.`, and took `item 8` at 1998 instead of 2227. A number after a single space is the heading's own (`PROPOSAL NO. 1`, `ARTICLE 5`, `…Securities Exchange Act of 1934`): the single-space strip cut it off, and dedup then merged GT Biopharma's three `PROPOSAL NO. N` headings into one `PROPOSAL NO.` at the last (#172). Across 81 cached filings the change moved nothing else in any outline. Lettered sub-item markers (a bare `A.`/`D.` inside an Item) are deliberately not detected — single letters recur throughout filing prose — and neither is a digit-led sub-item line (`3D. RISK FACTORS`; the all-caps arm needs a leading letter): `section` reaches a labelled line by stripping the label at comparison time instead (#175, see Section matching), so no detected heading is added.
|
|
108
108
|
- **Section matching:** `get_filing`'s `section` compares through `foldForHeadingMatch` (`filing-to-text.ts`) on **both** operands — Unicode whitespace runs collapse to one space, U+2018/U+2019 → `'`, U+201C/U+201D → `"`, then lowercase. Comparison-time only: `FilingHeading.heading`, the `outline` output, `content[]`, and every character offset keep the document's own bytes. EDGAR headings carry NBSP runs and typographic quotes, so a caller re-sending an outline heading through a UI that normalizes either one never byte-matched the heading it came from — which broke the `section_not_found` recovery loop the error's own hint points at (#106). No fuzzy matching — `"item 8"` resolves only to Item 8 — but every tier runs on folded operands, so a single-spaced needle also reaches a heading whose marker carries an NBSP run or a double space (`Item 9.` in a 2004 10-K). Folding only one operand does not fix the round trip. One boundary (`sectionMatches`): a needle ending in a number never continues into a longer one. A trailing digit never runs on into another digit, so `item 1` matches `Item 1.` and `Item 1A` but never `Item 12`, and `item 5` reaches an 8-K's `Item 5.02` while `item 5.0` does not. A trailing roman numeral (a final word of `i`, `v`, `x`) must end at a space, punctuation, or the heading's end, so `part i` never matches `PART II`, `PART III`, `PART IV`, or `PART II, III`: Sono-Tek's 10-Q (0001171520-26-000305), which prints Part I only inside a one-line TOC, landed `part i` on `PART II - OTHER INFORMATION`, and Microsoft's FY2026 10-K on `PART II, III` (348204). `part i` now falls through to the line tier (Microsoft: the body `PART I` at 6900) or misses with the outline (Sono-Tek; an S-1, whose Part I is the unlabeled prospectus). Plain-text outlines keep TOC rows worded differently from their body heading, and a TOC row for Item 10–16 precedes the body Item 1 — a bare substring test landed `item 1` on another Item in 6 of 37 plain 10-K/10-Q outlines and 1 of 270 HTML landings (#136). **Resolution order (#169, #166, #177)**, first hit wins, over every detected heading (`detectHeadings(text)`, which takes no cap, run once per call — the 50-entry cap belongs to the rendered outline alone; with the first 50 only, a heavy 10-K's financial-statement captions put Items 9B–16 out of reach) and the document's own lines: (1) **exact** (`resolveSection`) — a heading whose `headingKey` (the dedup key: the fold, a terminal-punctuation strip, and the marker separator folded to one space) equals the needle's, or whose title after its Item/Part marker does — failing that, the same with a leading sub-item label removed (below); a bare marker (`ITEM 1.`, `Part II`) has no title and is never a candidate, or GSBD's `item 1` jumps to a bare TOC `ITEM 1.`. The tier exists for #170's widened all-caps class: CCC's `SPECIAL NOTE REGARDING FORWARD-LOOKING STATEMENTS AND RISK FACTORS` (7562) otherwise takes `risk factors` from `Item 1A. Risk Factors.` (67351). (2) **Item/Part substring** — the first Item or Part heading (`isStructuralHeading`, bare and run-in markers included) that `sectionMatches`: a value inside an Item title names that Item, while a line equal to it is often a table-row label (CCC's `compensation` → `Item 11. Executive Compensation` at 493087, not the `Compensation` cell at 418658). (3) **an equal line** and (5) **a table-row label, else a prefix line** — a document line (`matchSectionLine`, `filing-to-text.ts`, which reports whether its line ranks in tier 3; `findSectionLine` is its offset alone), for a mixed-case subheading the outline never lists (`Liquidity and Capital Resources`): a line whose `headingKey` equals the needle's and that labels no table row (3), else — only when no such line exists — the first equal line that labels one (5, below), else one that starts with it (5) at a word boundary within `MAX_TITLE_CHARS` (a date-suffixed title: `Consolidated Schedule of Investments as of June 30, 2026`), unless a one-word needle runs on into a longer Title Case title (`continuesIntoLongerTitle`: the next word opens on a capital in a line that is not all caps). One word heads many unrelated titles — `business` landed on `Business Combinations.` (C3is), `Business Seasonality and Product Introductions` (Apple 10-Q), and a cross-reference line `Business Overview—Governmental regulations…` (DRDGOLD); a needle of two or more words keeps its continuation (`Human Capital` → `Human Capital Management`), and a suffix opening lowercase or on punctuation (`(continued)`, `—Parent Company Only`) always does. The cost: `forward-looking` no longer reaches `Forward-Looking Statements`, nor `revenue` `Revenue Recognition`; both miss with the outline. Equality goes first because a prefix match alone lands `Competition` on an earlier line that merely begins with the word. **A table-row label ranks in tier 5, ahead of a prefix line (#184):** an equal line labels a row when its next line with content (spacer lines passed over) is a figure row, or is a label — at most 60 characters, opening uppercase, ending on no `.`, `:`, `;`, or `,`, closing quotes and brackets passed over (a cross-reference `See “Item 18. Financial Statements.”` over its page number is no label) — whose next line with content is one (`labelsTableRow`; Microsoft's segment table `Revenue` / `Client` / `$ 9,350 …`). A figure row is a lone currency or nil cell (`$`, `-`, `–`, `—`), or a line whose text after its last letter holds a digit and, when the line has a letter, opens on a dotted leader, a tab, or two or more spaces behind a label of at most 60 characters (`isFigureRow`: `Product sales....... $4,759`; ``$ 7,746 $ `7,835``, a stray backtick in a 2004 cell); one space never qualifies (`The effective rate was 21%.`), and past the label bound the line is a table set on one line, which a heading may sit over (AMC Robotics' S-1 0001493152-26-047958: MD&A `Revenue` at 177926 over `Three months ended June 30, 2026 2025 $ Change …`). All 13 `Revenue` lines in Microsoft's 2004 10-K (0001193125-04-150689) label rows, and as tier-3 lines took `revenue` from `SEGMENT PRODUCT REVENUE/OPERATING INCOME (LOSS)` (81018) to the selected-data row at 66987; Aware's `Revenue:` over `Product sales....... $4,759` (112947) took it from `CONTRACT REVENUE` (49489). Demoted rather than skipped, a row label still lands when no heading contains the value (`total revenue` keeps its income-statement row), under the plain line notice. The test reads at most two content lines past an equal line. Over 26 corpus filings × 104 section values it labels 124 lines on the resolution path, none a detected heading, and moves 22 landings and no outline entry: 15 from a row label onto a heading or prose subheading for the value (Microsoft FY2026 `revenue` → its MD&A `Revenue` subheading at 217873; `earnings per share` → the EPS note in both Microsoft 10-Ks), 2 onto a table's caption line, 1 onto another row label (BAB's `goodwill`, from its balance-sheet row to a note table's), and 4 onto a heading that only contains the words (Arxis `net income` → `NOTE 16. NET INCOME (LOSS) PER SHARE`, and Nutanix's → `NOTE 11. NET INCOME (LOSS) PER SHARE` past a `Net income` row of `—` cells; CCC `net income` → the all-caps row label `NET INCOME (LOSS) INCLUDING NON-CONTROLLING INTEREST`, a detected heading; CCC `revenues` → `COST OF REVENUES` at 332106, since all four of its `Revenues` lines label rows). In both line tiers a line is skipped when it opens lowercase (a wrapped sentence or bullet); when it sits under a line ending on a comma or semicolon, or opens on an unpunctuated Item marker followed by a lowercase word (a wrapped cross-reference, the `followsMidSentence` / `isWrappedProse` guards heading detection applies: `…in Part II,\nItem 8 on this Form 10-K…`); or when it is a TOC row (`isTocRow`). A TOC row ends in a page cell set off by **two or more** spaces or a tab (`TOC_PAGE_TAIL_RE`; one space would read the year in `as of June 30, 2026` as a page number), or its next line with content is a page cell (`PAGE_CELL_LINE_RE`: a number, `F-3`, or a lowercase roman `ii`, which counts only on its own line — `Part iv` is not a page cell); either form may put a `Page` word before the number (`Page 3`), spacer lines passed over (an HTML TOC table puts NBSP rows between its cells). A line that is not a marker, followed directly by an Item/Part marker, is a TOC title cell with no page column; a marker run straight into its title counts (`MARKER_LINE_RE` takes `TITLE_RUN_IN`, #180: `Item 2Properties`). A marker line — a Part over its first Item — is a TOC row when that Item row is: the Item line ends in a page cell, the line below it holds or ends in one (a title cell, or Aware's wrapped `of Equity Securities....... 12`), or, for a bare marker, the line below its title holds one. Microsoft's FY2026 10-K prints body `PART IV` only as page furniture, so `part iv` reaches the line tier, where `PART IV / Item 15. / Exhibit and Financial Statement Schedules / 93` was taken; it now lands on the body's first `PART IV` (352884). A body Part sits over its Item too, but prose follows, not a page cell. The first remaining line wins, at its first non-blank character; one linear pass serves tiers 3 and 5, run only after tiers 1 and 2 miss, so a value landing on a tier-4 heading now pays that pass (7–11 ms on a 1 MB S-1). A cut page's guidance after a `section` landing says to pass `next_offset` as `offset` without `section`, which outranks `offset` — kept, it returns the same page. A line landing says so in the notice, composed into the truncation guidance when the page is cut (`notice` is last-wins): `No detected heading matches section '<value>', so this page starts at the first document line that does.` — no new output field. An equal line that outranked a heading merely containing the value says `No detected heading equals section '<value>', so this page starts at the first document line that does, ahead of headings that only contain it.` instead, since that heading does match by substring. Every #166 acceptance offset holds. (4) **other substring** — the first other heading that `sectionMatches`, after the equal line: a heading that merely holds the word took Arxis' S-1 `business` to `NOTE 3. BUSINESS COMBINATIONS` (544705) past its `Business` heading line (323578), and CCC's `revenue` to `COST OF REVENUES` (332106) past the MD&A `Revenue` subheading (256790). An exact or Item/Part heading still beats an equal line (Reddit's `use of proceeds` → the `USE OF PROCEEDS` heading); every other heading ranks after one. Over 67 corpus filings × 103 section values the order moved 24 landings, each one the validation predicted: 17 onto the section named (Microsoft FY2026 `general` → `GENERAL` at 8720, not `GENERAL RISKS`; DRDGOLD's `risk`, `directors`, `management`, and `employees` → body Items instead of TOC rows), and 7 onto a different place that is defensible but that a reader may like less, since an Item heading or a line equal to the value now outranks a heading that only holds it — JPM's 10-Q `risk` lands on Item 3, a one-sentence cross-reference, not `FIRMWIDE RISK MANAGEMENT`; `revenue` in Microsoft's 2004 and FY2026 10-Ks and Aware's on a selected-data, MD&A-table, or income-statement row label rather than a heading containing the word — the three landings tier 5's row labels move (#184, above); `summary` on Item 16 `Form 10-K Summary`; Intel's 2000 `consolidated balance sheets` on an Item 14 index row. (6) `section_not_found`. **A proxy statement keeps its headings ahead of the lines:** for a proxy or information statement form (`PROXY_FORM_RE`: DEF 14A, DEFA14A, DEFM14A, PRE 14A, DEF 14C, …), tier 1 is followed by the first heading of any kind that `sectionMatches`, then the line tiers — its ITEM headings are ballot proposals, not the Regulation S-K item taxonomy, and its lines equal to a value are mostly table cells. Under tiers 2–4 Goldman Sachs' DEF 14A (0001193125-26-117433) took `compensation` from the CD&A (123680) to `ITEM 2. … (SAY ON PAY)` (274744) and `summary` from `EXECUTIVE SUMMARY` (16953) to a Pay versus Performance column header (282286). The form is read before `section` resolves, so the `.hdr.sgml` fallback request (#126) now precedes a `section_not_found`. **Inside each heading tier a body heading ranks ahead of a table-of-contents row (#174):** a heading is a TOC row when its title line — its own line, or the next line with content after a bare or joined marker — is one by `isTocRow` (the line tiers' test), or the next line with content below it ends in a page cell (a TOC title wrapped onto a second line, Aware's `of Equity Securities....... 12`), or the title wraps once with its page cell on its own line under the wrap — the title line, one continuation line, and a `PAGE_CELL_LINE_RE` line on three consecutive lines, the continuation in the title line's case: no lowercase letter under an all-caps title line, an opening lowercase one under a mixed-case one (`of Equity Securities`) (#183). DRDGOLD's TOC row `ITEM 14.` / `MATERIAL MODIFICATIONS TO THE RIGHTS OF SECURITY HOLDERS AND USE OF` / `PROCEEDS` / `79` took `item 14` and `material modifications to the rights of security holders` at 8146 from the body heading at 433144. A body heading over one line of body text and a page number has the same three lines: across 26 filings the step without the consecutive-line and case conditions flags 47 body headings (`Item 4. Mine Safety Disclosures` / `Not applicable.` / `31`, spacer lines between or not, and DRDGOLD's `NOTE 22 ASSET HELD FOR SALE` over its page footer and `F-50`); with them it flags only that TOC row (8146) and its title cell (8155), moves those two landings, and changes no outline. An all-caps body heading over a one-word all-caps line and a page number on consecutive lines (`ITEM 4. MINE SAFETY DISCLOSURES` / `NONE` / `31`) still reads as a TOC row and lands as its tier's only match; `isTocHeading` in `filing-to-text.ts` owns the test, finding the line start from the newline before the offset since an NBSP-led Item's offset sits one past it. A TOC row still lands when it is its tier's only match. Three TOC shapes survive dedup: a row worded apart from its body heading (First Bancorp's 8-K `Item 9.01 Financial Statements and Exhibits. Page 1` took `item 9` at 2041 from the body Item at 2402), a bare all-caps marker over its title and page cells (DRDGOLD's FY2026 20-F `ITEM 3.` / `KEY INFORMATION` / `6` took `item 3` at 6174 from `ITEM 3. KEY INFORMATION` at 34346, and nearly every other `item N` the same way), and the title cell itself (`key information`). TOC rows are ranked, never dropped or pushed behind the line tiers: the test also reads a body heading over a page number (CCC's `Item 6. Reserved.` at 234199, Reddit's S-1 `PART II` at 1021631), and behind the line tiers CCC's and Microsoft's `item 6` left the body heading for a bare TOC marker line. With the dedup-key fold, the ranking moved 83 landings over the corpus, each from a TOC row onto a body heading (DRDGOLD 35, Worthington Steel 20, GSBD 9, Amazon's 2003 10-K 8, Aware 6, First Bancorp 4, CCC 1), and no other: 78 onto the body heading the row copies, 5 onto another the value names (Amazon 2003's `part ii`–`part iv` onto its body Parts, Aware's `management` from its TOC Item 12 to body Item 7, Worthington Steel's `other information` from its TOC Item 5 to body `PART II. OTHER INFORMATION`). **A leading 20-F sub-item label is ignored (#175):** 20-F filers label sub-items (`3D. RISK FACTORS`, `B. Business Overview`), so a plain-phrase value never equalled the line. Tier 1 and both line tiers also compare the heading or line key with a leading label removed (`withoutSubItemLabel`: `/^(?:\d{1,2}[a-z]|[a-z])\. /` on the folded key, anchored, so constant-time), and the one-word longer-title rule judges the line after its label — Regencell's `B. Business Overview` keeps `business` a miss while `business overview` lands (151573). The value itself is never stripped, so `3D. RISK FACTORS` still matches its own line; a line opening on an initial (`L. John Doerr`) strips the same way and so matches only a value equal to the rest of the line. The landing is the line's first character, label included (DRDGOLD `risk factors` → 34503). Tier 1 strips only once no heading equals the value as printed, so an earlier `A. RISK FACTORS` never takes `risk factors` from a later `Item 1A. Risk Factors`; a labelled body heading still ranks ahead of a TOC row. The line tiers keep the first line equal either way: across the corpus the labelled line they land on is the 20-F sub-item itself and a later unlabelled one is narrower (DRDGOLD `business overview` → `4B. BUSINESS OVERVIEW` at 176975, not `Business overview` at 269017). Over the corpus the strip moved 23 landings, all in the three 20-Fs: 19 misses now land on the labelled sub-item, and 4 earlier landings move onto one (DRDGOLD `compensation` from the `Compensation of key management` subheading inside Item 6B, 328266, to `6B. COMPENSATION` at 324690); no filing without labelled sub-item lines changed. **The outline cut (#169):** past 50 detected headings the outline is every Item/Part heading (`isStructuralHeading`, bare and run-in markers included) plus the earliest other headings up to 50, in offset order (`selectOutline`) — shared by the first-page `outline` and the `section_not_found` message and data. When the Item/Part headings alone pass 50, each bare marker that a later titled heading for the same Item or Part repeats is left out first (`outlineStructuralHeadings`): DRDGOLD's FY2026 20-F renders all 31 TOC Items as bare markers (`ITEM 3.`), which filled the cut and pushed body Items 16A–19 out; its outline now runs `ITEM 1.` through `ITEM 19. EXHIBITS` from the body. A bare marker no titled heading repeats stays, and `section` still matches every heading. The miss message heads a cut outline `Outline (50 of N detected headings):` (else `Outline:`), and a cut first page appends `The outline lists 50 of N detected headings, every Item and Part heading among them; section matches all of them.` to the truncation guidance — `…among them, bare table-of-contents copies of titled ones aside; …` when bare markers were left out (DRDGOLD: `The outline lists 50 of 142 detected headings, every Item and Part heading among them, bare table-of-contents copies of titled ones aside; section matches all of them.`), and no Item/Part clause when the kept Item/Part headings alone pass 50, which reaches `structuredContent.notice` and `content[]` — no new output field, since the outline is emitted only on a truncated first page. A document of 50 or fewer headings renders as before.
|
|
109
109
|
- **Binary filing documents:** a filing index routinely carries scanned pages, PDF exhibits, `Financial_Report.xlsx`, and `-xbrl.zip` — `filingToExtract` runs `html-to-text` over whatever bytes arrive and never inspects a content type, so requesting one used to return decoded binary as `content` with no error (#96). `get-filing.tool.ts` detects them with one rule, `binaryType`: the filename extension (`binaryTypeFromName`), the one signal every filing carries, plus a header-page `GRAPHIC` type for a scan named oddly — the header page is read before the body (#161). Two halves, both needed — `documents[]` entries carry `binary: true`, and `resolveFilingArchive` throws `binary_document` before `tryGetFilingDocument` runs, since the `document` input is an unconstrained string and a catalog marker alone cannot stop a caller. Extension-driven and bucket-independent on purpose: a PDF exhibit is typed `EX-99.*` and files under `exhibits`, not among the graphics.
|
|
110
|
-
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); a company's submissions feed also lists the Form 4s it filed as a reporting owner of another issuer (JPMorgan's 10% stake in a BlackRock fund), so a parsed document whose `issuerCik` differs from the resolved CIK, compared with zero-padding stripped, adds no row inline or to the dataframe and is counted in `filings_other_issuer` — it still counts toward `filings_scanned`, the scan floor, and `history_scanned_through`, and the no-canvas stop rule is unchanged — though a skipped filing adds no transactions, so a no-canvas call reads on through such filings until `limit` transactions are in hand, within the same scan cap (#157); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). In an annual matrix an instant enters only on one of the filer's fiscal-year ends and takes the key of the annual duration ending that day. The ends come from the companyfacts payload already in hand (`filerFiscalYearEndsWithComparatives` in `concept-series.ts`, shared with `get_financials`' annual view of an instant): each annual report's own year (the #148 rule) plus the comparative years it restates — its year-long facts ending a whole number of years earlier, within a week — so the first XBRL 10-K's prior-year balance keys too, while a year-long figure ending off the report's calendar (a December filer's trailing year to 09-30) stays out. The #148 test of a 10-Q-held annual frame keeps `filerFiscalYearEnds` (each report's own year only): it matches day of year, where a comparative year adds nothing but a wider tolerance. The key is the first `CY####` frame on any concept in the payload ending then, so a balance lands beside its fiscal year's income lines even when none is requested; with no framed duration ending on it, the calendar year in which that fiscal year's second half starts (SEC's own framing: a year ending 06-30 is that year, one ending 06-27 or 05-31 the year before — Seagate's and Western Digital's years ending 2025-06-27 are `CY2024`; SEC departs from it for the year both ended 2012-06-29, framed `CY2012`, which is why a framed key wins). Interim and cover-date instants stay out of the annual matrix, and a company-concept left with none gets a gap plus a caveat pointing at quarterly; when that empties the whole matrix, `no_comparable_data` says the values were all off fiscal-year ends and its hint names `period_type: "quarterly"` (omitting `period_type` keeps annual). Quarterly keys are unchanged. The year ends are read lazily, only when an annual matrix meets an instant (#171). The period-ends caveat checks every inline column — the newest can hold one company alone — spelling out the newest column whose ends differ and naming the others; the inline window is the newest periods of the union across companies, so a company-concept pair whose values all predate it is named in `caveats` — one line per concept, each company with its newest period and the dataframe pointer — and never in `gaps`, which keeps meaning no value in any period (#144). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, `sql_read_only`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints — since mcp-ts-core 0.13.15 the gate also denies engine-setting and SQL-from-string functions (`current_setting`, `query_table`) as `denied_function` and a file named as a table (`FROM 'data.csv'`) as `sql_read_only`, and since 0.13.16 it refuses any unqualified `current_schema`, since the parse tree cannot tell the bare function from a column of that name — a staged or aliased column so named is read qualified (`c.current_schema`). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and
|
|
110
|
+
- **Dataframes:** `secedgar_search_filings`, `secedgar_get_financials`, `secedgar_fetch_frames`, `secedgar_compare_companies`, `secedgar_get_insider_transactions`, `secedgar_get_institutional_holdings`, `secedgar_get_material_events`, `secedgar_find_holders`, `secedgar_get_beneficial_owners`, and `secedgar_get_fund_holdings` materialize their full result set as `df_<id>` on a shared DuckDB-backed canvas (one per tenant). The two ownership tools register the full parsed set (inline response is a preview capped at `limit`); `get_insider_transactions` scans at least `INSIDER_CANVAS_FILING_SCAN` recent filings when canvas is available (so small inline limits don't under-scan the window), capped at 100, and flags `dataset.truncated` when more Form 4 filings exist beyond the scanned window (#63); with a `filed_after` / `filed_before` window it reads the in-window Form 4 / 4-A rows with an XML primary document — `recent`, then the archive pages overlapping the window, newest first, stopping once the scan cap plus a sentinel row are in hand — and with a canvas parses every one up to 100 instead of the 40-filing floor, since the window already bounds the set; `dataset.truncated` is then the sentinel or unread overlapping pages, and `history_scanned_through` the oldest filing parsed (#127); a company's submissions feed also lists the Form 4s it filed as a reporting owner of another issuer (JPMorgan's 10% stake in a BlackRock fund), so a parsed document whose `issuerCik` differs from the resolved CIK, compared with zero-padding stripped, adds no row inline or to the dataframe and is counted in `filings_other_issuer` — it still counts toward `filings_scanned`, the scan floor, and `history_scanned_through`, and the no-canvas stop rule is unchanged — though a skipped filing adds no transactions, so a no-canvas call reads on through such filings until `limit` transactions are in hand, within the same scan cap (#157); its `shares_traded` is an unsigned magnitude paired with a `direction` (acquire/dispose) column, so net activity is `SUM(CASE WHEN direction='dispose' THEN -shares_traded ELSE shares_traded END)` (#46); `get_financials`' dataframe materializes the source-filing fiscal keys as `source_filing_fy`/`source_filing_fp` (renamed from bare `fiscal_year`/`fiscal_period` — every comparative period restated in one filing carries that filing's fy/fp, so `period_end` is the time key for ORDER BY/GROUP BY/windowing) (#72); `compare_companies`' dataframe holds the FULL aligned series across every period while the inline matrix is capped at `periods`, and each row carries both the aligned `period` key and the underlying XBRL `frame` (they differ for point-in-time concepts) (#85). In an annual matrix an instant enters only on one of the filer's fiscal-year ends and takes the key of the annual duration ending that day. The ends come from the companyfacts payload already in hand (`filerFiscalYearEndsWithComparatives` in `concept-series.ts`, shared with `get_financials`' annual view of an instant): each annual report's own year (the #148 rule) plus the comparative years it restates — its year-long facts ending a whole number of years earlier, within a week — so the first XBRL 10-K's prior-year balance keys too, while a year-long figure ending off the report's calendar (a December filer's trailing year to 09-30) stays out. The #148 test of a 10-Q-held annual frame keeps `filerFiscalYearEnds` (each report's own year only): it matches day of year, where a comparative year adds nothing but a wider tolerance. The key is the first `CY####` frame on any concept in the payload ending then, so a balance lands beside its fiscal year's income lines even when none is requested; with no framed duration ending on it, the calendar year in which that fiscal year's second half starts (SEC's own framing: a year ending 06-30 is that year, one ending 06-27 or 05-31 the year before — Seagate's and Western Digital's years ending 2025-06-27 are `CY2024`; SEC departs from it for the year both ended 2012-06-29, framed `CY2012`, which is why a framed key wins). Interim and cover-date instants stay out of the annual matrix, and a company-concept left with none gets a gap plus a caveat pointing at quarterly; when that empties the whole matrix, `no_comparable_data` says the values were all off fiscal-year ends and its hint names `period_type: "quarterly"` (omitting `period_type` keeps annual). Quarterly keys are unchanged. The year ends are read lazily, only when an annual matrix meets an instant (#171). The period-ends caveat checks every inline column — the newest can hold one company alone — spelling out the newest column whose ends differ and naming the others; the inline window is the newest periods of the union across companies, so a company-concept pair whose values all predate it is named in `caveats` — one line per concept, each company with its newest period and the dataframe pointer — and never in `gaps`, which keeps meaning no value in any period (#144). Each row set carries an all-nullable schema (framework's `inferSchemaFromRows` always emits `nullable: true` since mcp-ts-core 0.10.4). Per-table TTL is delegated to the framework via `RegisterTableOptions.ttlMs` (resolved in mcp-ts-core#140); bridge still tracks metadata expiry in `ctx.state` for provenance. Every producer emits the describe-then-query pointer on the branch that registers the dataframe, composed once by `dataframeGuidance(dataset)` in the bridge so the wording cannot drift; where a truncation call already fires there it rides as that call's `guidance`, since `ctx.enrich.notice` is last-wins across `notice`/`truncated` and a second call would clobber the first (#104). `secedgar_dataframe_query` detects a `row_limit`-bound result from `QueryResult.truncated` rather than `rowCount > rows.length` — `row_limit` is pushed into the query as the provider's cap, so a capped result comes back with `rowCount === rows.length` and the arithmetic cannot see it; `rowCount` is then the cap and not a total, so that branch names the cap instead of printing "of N rows", `row_count_capped` carries the same disclosure into `format()`, and the fallback comparison stays for the `registerAs` path, where `truncated` is never set. A SQL `LIMIT` exactly equal to the cap is indistinguishable from a result that holds that many rows and is reported as exact (#109). `secedgar_dataframe_query` runs framework's SQL gate with `denySystemCatalogs: true` to block `information_schema`, `pg_catalog`, `sqlite_master`, and `duckdb_*` catalog access. Raw DuckDB execution errors (missing table, syntax) are caught in `bridge.query` and re-thrown as structured `missing_table`/`invalid_sql` reasons, with DuckDB's "Did you mean …?" hint stripped so internal catalog names don't leak (#47). The framework's own classifications pass through untouched and are declared on the tool's contract (`thrownBy: 'service'`, since the handler never names them): a SELECT that prepares and then fails on the data — a `Conversion` / `Invalid Input` / `Out of Range` error, e.g. `CAST('abc' AS INTEGER)` — is `ValidationError` with `sql_execution_error` (mcp-ts-core 0.13.4; `DatabaseError` with no reason before), and the gate's `multi_statement`, `denied_function`, `sql_read_only`, and `plan_operator_not_allowed` rejections arrive with their own recovery hints — since mcp-ts-core 0.13.15 the gate also denies engine-setting and SQL-from-string functions (`current_setting`, `query_table`) as `denied_function` and a file named as a table (`FROM 'data.csv'`) as `sql_read_only`, and since 0.13.16 it refuses any unqualified `current_schema`, since the parse tree cannot tell the bare function from a column of that name — a staged or aliased column so named is read qualified (`c.current_schema`). `get_beneficial_owners` registers one row per reporting person rather than one per filing, since powers and percent of class are per person; `get_fund_holdings` registers every position in the report while the inline list is one `limit`-sized page from `offset`, and its rows carry the fund keys (`series_id`, `registrant_cik`, `report_period_date`, `accession_number`) so a portfolio joins the 13F and 13D/G rows on `cusip`. Insider rows carry `issuer_cik` and no `cusip`, 13F rows the reverse; 13D/G rows carry both.
|
|
111
111
|
- **Beneficial ownership (13D/13G):** SEC replaced the legacy `SC 13D` / `SC 13G` text filings with structured XML under the `SCHEDULE 13D` / `SCHEDULE 13G` names on **2024-12-18**; the changeover is total, so a legacy-name filter matches nothing recent and a current-name filter matches nothing older. The two schedules are **separate schemas, not one shape with optional fields** — 13D repeats `reportingPersons/reportingPersonInfo` with the power fields flat on each person and the percentage in `percentOfClass`; 13G repeats `coverPageHeaderReportingPersonDetails` with the powers nested inside `reportingPersonBeneficiallyOwnedNumberOfShares`, the percentage in `classPercent`, no reporting-person CIK anywhere, and no purpose-of-transaction field at all (its item 4 is structured ownership data, which is what makes it the passive form). `beneficial-ownership-parser.ts` therefore branches per schema and dispatches on the submission type, falling back to whichever repeated reporting-person element the document carries. **Ownership powers and percent of class are per reporting person, including on joint filings** — a fund, its adviser, and its controlling principal each report the same underlying shares, so the parsed owner list is never collapsed to a group total and the percentages must not be summed. Discovery runs off the issuer's own `getSubmissions()`: SEC cross-lists a schedule under every CIK party to it, so no entity-scoped EFTS step is needed.
|
|
112
112
|
- **NPORT-P series routing:** a fund's public portfolio report covers **exactly one series**, a registrant trust holds many (iShares Trust: 390 listed series, 200+ reports for a single period), and the series identity appears **only inside the document** — neither the submissions feed nor EFTS carries `seriesId` on an NPORT-P row. Routing is therefore series-first: EDGAR's company browse accepts a series ID in place of a CIK (`getFundSeriesFilings`), which is the one SEC surface mapping a series to its accessions and which also names the registrant, so a bare series ID resolves in one request. The browse feed's registrant outranks any registrant the `fund` input implied, since `series_id` can name a series of an entirely different trust and the archive path is keyed on the registrant. The bounded header scan (`tryGetFilingDocumentHead` + `parseNportHeader`, cap `SERIES_SCAN_LIMIT`) is the fallback and the disambiguator for a trust whose series carry no listed ticker; SEC's archive host **ignores `Range`** (a ranged request answers 200 with the whole body), so the saving is client-side stream cancellation only — and the cutoff cannot cut below one read chunk, which Bun sets at 262,144 bytes, so `HEADER_MAX_BYTES` binds only on a document arriving in several chunks. **A registrant's own NPORT-P list is never the candidate set for a series-structured trust**: series fiscal quarters are staggered, so the list interleaves funds and one report at the newest period does not mean one fund — it means the fund whose quarter ends latest. The scan therefore runs on the newest batch even when that batch holds one report, and a series found there re-routes through the browse feed; only a report naming no series at all (a closed-end fund, which also carries no `classId`) is answered from the registrant's list. Candidates are ordered by **reported period, not filing date**: an `NPORT-P/A` restating an old period is filed after every report of the periods that followed it. Reports publish on a ~60-day lag, so `report_period_date` and `publication_lag_days` are first-class output and the `as_of` enrichment states both.
|
|
113
113
|
- **Local mirror (opt-in, `EDGAR_MIRROR_ENABLED`):** routes `resolveCik`, `tryGetCompanyConcept`, `tryGetCompanyFacts`, and `tryGetFrames` to a local SQLite mirror (framework `MirrorService`) of `company_tickers.json` + the `companyfacts.zip` bulk archive; the live API is the fallback on a miss (`EDGAR_MIRROR_FALLBACK_LIVE`). Bootstrap out-of-band with `bun run mirror:init`; refresh nightly via cron (HTTP) or `bun run mirror:refresh`. The three `mirror:*` commands also ship in the production Docker image — run them against a deployed container with `docker exec <container> bun run mirror:<init|refresh|verify>`. Node/Bun only — skipped on Workers. Frames are assembled from the company-facts store, so `loc` (business location) is absent. The ingester stores a tag's own name when SEC serves it with no label (`InterestExpenseNonoperating`), so every read path treats a stored label equal to a camel-case compound tag as no label (`storedLabel`) — callers fall back to the concept's label, and `getFrames` answers an empty `label` as the live frames API does; a one-word element's label is the word itself (`Revenues`) and is kept. Read-side, so stored rows need no migration. No FTS5 — every routed lookup is exact/indexed (cik+taxonomy+tag point, taxonomy+tag scan, cik scan for the whole-company read, ticker/CIK). Mirror ingests MF fund symbols from `company_tickers_mf.json`; on the mirror path a live MF merge supplements them when `FALLBACK_LIVE` is on (superseding the mirror's bare fund rows with series/class, and covering a mirror synced before MF ingestion), so funds resolve there too (#43, #135).
|
|
@@ -122,7 +122,7 @@ Tailor suggestions to what's actually missing or stale — don't recite the full
|
|
|
122
122
|
| `EDGAR_RATE_LIMIT_RPS` | No | `10` | Max requests/second to SEC APIs. Do not exceed 10. |
|
|
123
123
|
| `EDGAR_RATE_LIMIT_COOLDOWN_SECONDS` | No | `600` | Seconds to stop sending to SEC after a 429. Calls are refused locally for this long, then the first one goes out alone as a probe. SEC lifts its block only after ten quiet minutes, so a shorter value just probes into it. |
|
|
124
124
|
| `EDGAR_TICKER_CACHE_TTL` | No | `3600` | Seconds to cache the ticker index (company_tickers.json + company_tickers_mf.json). A failed fund-file load is retried after 60 s (or the rate-limit cool-down) instead of standing for the TTL. |
|
|
125
|
-
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes
|
|
125
|
+
| `EDGAR_DATASET_TTL_SECONDS` | No | `86400` | Per-table TTL for canvas-registered dataframes, counted from when the table is staged; queries do not extend it. |
|
|
126
126
|
| `EDGAR_DATAFRAME_DROP_ENABLED` | No | `false` | Set to `true` to expose `secedgar_dataframe_drop`. TTL handles cleanup otherwise. Off, the tool is registered through `disabledTool()`: absent from `tools/list` and uncallable, but rendered on the HTTP landing page in a `disabled` group carrying the reason and `EDGAR_DATAFRAME_DROP_ENABLED=true`, so an operator can see the capability exists. The list `createApp()` receives — and `buildServerManifest()`'s `definitionCounts.tools` — is 17 either way; `/.well-known/mcp.json` carries no per-tool definitions at all, only capability flags, so nothing changes there (#103). |
|
|
127
127
|
| `EDGAR_CLAMP_OVERSIZED_LIMITS` | No | `true` | Serve a count or size input above the cap its tool applies at that cap (up to each field's rejection bound) instead of rejecting it, so the call returns what the cap returns. Set `false` to advertise each cap as the field's maximum and reject anything larger. Read before `createApp()` from its own variable, like `EDGAR_DATAFRAME_DROP_ENABLED`; every other server variable is validated in `setup()` behind the configuration banner (#197). |
|
|
128
128
|
| `EDGAR_MIRROR_ENABLED` | No | `false` | Enable the local SQLite mirror of company_tickers + XBRL company-facts. Node/Bun only (skipped on Workers). Bootstrap once with `bun run mirror:init`. |
|
|
@@ -287,9 +287,18 @@ src/
|
|
|
287
287
|
|
|
288
288
|
## Skills
|
|
289
289
|
|
|
290
|
-
|
|
290
|
+
Two skill trees live at the project root, written for different readers:
|
|
291
291
|
|
|
292
|
-
|
|
292
|
+
| Tree | For | Written by |
|
|
293
|
+
|:-----|:----|:-----------|
|
|
294
|
+
| `framework-skills/` | The agent developing this server — framework skills (`metadata.audience: external`) plus any development-time skills of the server's own | `init` and the `maintenance` sync (framework skills); you (your own) |
|
|
295
|
+
| `skills/` | The agents that install the server and call its tools — tool chains, identifier sources, gates | You, deliberately, with the `add-skill` skill. Absent by default; usually none is needed |
|
|
296
|
+
|
|
297
|
+
Development skills are modular instructions. Read them directly when a task matches — e.g., `framework-skills/add-tool/SKILL.md` when adding a tool. `bun run list-skills` prints the full registry. Their directory is deliberately not `skills/`: Claude Code and Codex auto-load a plugin's root `skills/`, so a server that ships `.claude-plugin/` or `.codex-plugin/` would hand them to every agent that installs it.
|
|
298
|
+
|
|
299
|
+
A published skill in root `skills/` reaches plugin installs and the `.mcpb` bundle as-is, ships in the npm package and the Docker image, and reaches MCP clients when the server opts in with `createApp({ skills: 'skills' })`. `lint:packaging` validates the tree.
|
|
300
|
+
|
|
301
|
+
**Agent skill directory:** Copy `framework-skills/` into the directory your agent discovers (Claude Code: `.claude/skills/`, others: equivalent). Skills then load as context without referencing `framework-skills/` paths. After framework updates, run the `maintenance` skill — Phase B re-syncs the agent directory.
|
|
293
302
|
|
|
294
303
|
Available skills:
|
|
295
304
|
|
|
@@ -301,6 +310,7 @@ Available skills:
|
|
|
301
310
|
| `add-app-tool` | Scaffold an MCP App tool + paired UI resource |
|
|
302
311
|
| `add-resource` | Scaffold a new resource definition |
|
|
303
312
|
| `add-prompt` | Scaffold a new prompt definition |
|
|
313
|
+
| `add-skill` | Author a skill the server publishes in root `skills/` for the agents that use it — when one earns its place, structure, validation, delivery |
|
|
304
314
|
| `add-service` | Scaffold a new service integration |
|
|
305
315
|
| `add-test` | Scaffold test file for a tool, resource, or service |
|
|
306
316
|
| `field-test` | Exercise tools/resources/prompts with real inputs, verify behavior, report issues |
|
|
@@ -351,7 +361,7 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
351
361
|
| `bun run format` | Auto-fix formatting (safe fixes only) |
|
|
352
362
|
| `bun run format:unsafe` | Also apply Biome's unsafe autofixes — review the diff; they can change behavior |
|
|
353
363
|
| `bun run lint:mcp` | Run the MCP definition linter standalone (rule catalog: `api-linter` skill) |
|
|
354
|
-
| `bun run lint:packaging` | Packaging surface checks — `server.json`/`manifest.json` env-var parity (run by devcheck) |
|
|
364
|
+
| `bun run lint:packaging` | Packaging surface checks — `server.json`/`manifest.json` env-var parity, `.mcpbignore` coverage, plugin manifests, the published `skills/` tree (run by devcheck) |
|
|
355
365
|
| `bun run list-skills` | Print an index of available skills from `framework-skills/` |
|
|
356
366
|
| `bun run changelog:build` | Regenerate `CHANGELOG.md` from `changelog/*.md` |
|
|
357
367
|
| `bun run changelog:check` | Verify `CHANGELOG.md` is in sync with `changelog/` (used by devcheck) |
|
|
@@ -370,7 +380,7 @@ When you complete a skill's checklist, check the boxes and add a completion time
|
|
|
370
380
|
|
|
371
381
|
## Bundling
|
|
372
382
|
|
|
373
|
-
`bun run bundle` produces a `.mcpb` extension bundle for one-click install in Claude Desktop. The pack step is followed by `scripts/clean-mcpb.ts`, which prunes dev dependencies (`mcpb clean`) and strips
|
|
383
|
+
`bun run bundle` produces a `.mcpb` extension bundle for one-click install in Claude Desktop. `mcpb pack` reads `.mcpbignore`, not `.gitignore`, so every root path the bundle must not ship — source, tests, docs, caches, logs, agent docs, plugin manifests, gitignored local files such as the `/data/` mirror directory — needs a root-anchored line there (`/docs/`, never `docs/`, which also strips `node_modules/*/docs/`); `lint:packaging` names each missing or unanchored line. Root `skills/` ships on purpose. The pack step is followed by `scripts/clean-mcpb.ts`, which prunes dev dependencies (`mcpb clean`) and strips four classes of `node_modules/**` content that root-anchored `.mcpbignore` patterns cannot reach: dependency-shipped agent docs (`framework-skills/`, `skills/`, `.claude/`, `.agents/`, `SKILL.md`); platform-specific native bindings, which would otherwise lock the bundle to the platform it was packed on; node-gyp build output beside a `binding.gyp` (`config.gypi`, Makefiles, `obj.target/`, `.deps/`, object files and debug symbols), which records the packing machine's paths — compiled `*.node` addons stay; and build-only package files — the framework's `changelog/`, `scripts/`, `templates/`, `CLAUDE.md`, `AGENTS.md`, and config bases (`biome.json`, `vitest.config.base.mjs`, `config/tsconfig.base.json`), and `better-sqlite3`'s `deps/` and `src/` C sources. The bundle therefore ships without the DuckDB native, and `manifest.json` sets `CANVAS_PROVIDER_TYPE=none` so the canvas is off instead of failing to load `@duckdb/node-api`: the dataframe tools fail as `canvas_unavailable` and every other tool works normally (the data tools answer inline without registering a dataframe). MCPB is stdio-only — HTTP deployments are unaffected. Delete `manifest.json` and `.mcpbignore` to opt out; `lint:packaging` skips cleanly.
|
|
374
384
|
|
|
375
385
|
**Adding an env var requires both files:** `server.json` (registry discovery, `environmentVariables[]`) and `manifest.json` (bundle install UX, `mcp_config.env` + `user_config`). `lint:packaging` (run by `devcheck`) verifies the env var names match, that every `user_config` option is wired into `mcp_config.env` as `"X": "${user_config.X}"` (the host substitutes nothing else — `"${X}"` reaches the server as that literal string), and that an optional string option carries `"default": ""`.
|
|
376
386
|
|
|
@@ -440,7 +450,10 @@ import { getEdgarApiService } from '@/services/edgar/edgar-api-service.js';
|
|
|
440
450
|
- [ ] EDGAR upstream sparsity: schemas reflect real nullability; `format()` preserves uncertainty (don't fabricate facts from missing XBRL fields); tests cover at least one sparse payload
|
|
441
451
|
- [ ] Registered in `createApp()` arrays (directly or via barrel exports)
|
|
442
452
|
- [ ] Tests use `createMockContext({ errors: tool.errors })` from `@cyanheads/mcp-ts-core/testing` for tools with declared contracts
|
|
453
|
+
- [ ] `plugin.json` + `mcp.json` (Agent Plugins 1.0.0 — read by Codex, Cursor, GitHub Copilot, VS Code, Kiro) populated — `plugin.json` `name` = the unscoped repo name (lowercase `a-z`, `0-9`, `-`, `.` only); `version`, `description`, `author`, `homepage`, `repository`, `license`, `keywords` from `package.json`; no other top-level field, `extensions` included. `mcp.json` holds one `"type": "stdio"` entry keyed by the unscoped name, `npx -y <full package name>`, and the same fixed `env` as `.codex-plugin/mcp.json`. Neither file carries a user-supplied variable (no `""`, no `${…}` besides `${PLUGIN_ROOT}`/`${PLUGIN_DATA}`): keys reach Codex through `.codex-plugin/mcp.json` `env_vars`, which works only while `.codex-plugin/plugin.json` keeps `"mcpServers": "./.codex-plugin/mcp.json"`
|
|
443
454
|
- [ ] `.codex-plugin/plugin.json` populated — `name`, `version`, `description`, `repository`, `license` from `package.json`; `interface.displayName` = the unscoped repo name (never the npm scope — `lint:packaging` enforces this); `interface.shortDescription` from `package.json` description
|
|
444
455
|
- [ ] `.codex-plugin/mcp.json` updated — server name key is the unscoped repo name; every user-supplied variable (API key, contact email, instance URL) is listed in `env_vars` so Codex forwards it from the user's environment. Never write `"KEY": ""` into `env` — an empty value replaces the user's exported key and is read as unset
|
|
445
456
|
- [ ] `.claude-plugin/plugin.json` populated — `name`, `version`, `description`, `author`, `repository`, `license`, `keywords` from `package.json`; inline `mcpServers` entry keyed by the unscoped repo name. Every user-supplied variable is declared under `userConfig` (`type`, `title`, `description`; `sensitive: true` for keys and tokens; `required: true` or `default: ""`) and referenced from `env` as `"KEY": "${user_config.<option>}"` — mirror the `user_config` block in `manifest.json`. Never write `"KEY": ""` into `env`
|
|
457
|
+
- [ ] Any new root dev path has a root-anchored line in `.mcpbignore`; `bun run bundle` and `lint:packaging` pass
|
|
458
|
+
- [ ] If the server publishes a skill in root `skills/`: `lint:packaging` passes, and `field-test` verified every gate and identifier claim in it against live tool behavior
|
|
446
459
|
- [ ] `bun run devcheck` passes
|
package/Dockerfile
CHANGED
|
@@ -33,6 +33,11 @@ RUN --mount=type=cache,target=/root/.bun/install/cache \
|
|
|
33
33
|
# Copy the rest of the source code
|
|
34
34
|
COPY . .
|
|
35
35
|
|
|
36
|
+
# The skills a server publishes (`createApp({ skills: 'skills' })`) are read at
|
|
37
|
+
# runtime from the application root. Created when the project has none, so the
|
|
38
|
+
# production stage's copy below holds with or without one.
|
|
39
|
+
RUN mkdir -p skills
|
|
40
|
+
|
|
36
41
|
# Build the application
|
|
37
42
|
RUN bun run build
|
|
38
43
|
|
|
@@ -101,7 +106,8 @@ RUN --mount=type=cache,target=/root/.bun/install/cache \
|
|
|
101
106
|
# glibc and musl variants (DuckDB's bindings, for one) installs both. The
|
|
102
107
|
# runtime image is Debian (glibc) and never loads the musl copy; the script
|
|
103
108
|
# deletes every package whose own `libc` admits only musl. It follows every
|
|
104
|
-
# install, since a later `bun install` restores what it removes.
|
|
109
|
+
# install, since a later `bun install` restores what it removes. A server on a
|
|
110
|
+
# musl (Alpine) runtime image drops these two lines.
|
|
105
111
|
COPY scripts/prune-musl-packages.ts ./scripts/
|
|
106
112
|
RUN bun scripts/prune-musl-packages.ts
|
|
107
113
|
|
|
@@ -126,7 +132,7 @@ ENV NODE_ENV=production
|
|
|
126
132
|
|
|
127
133
|
# OCI image metadata (https://github.com/opencontainers/image-spec/blob/main/annotations.md)
|
|
128
134
|
ARG APP_VERSION
|
|
129
|
-
LABEL org.opencontainers.image.title="
|
|
135
|
+
LABEL org.opencontainers.image.title="secedgar-mcp-server"
|
|
130
136
|
LABEL org.opencontainers.image.description="Query SEC EDGAR filings, XBRL financials, and company data via MCP. STDIO or Streamable HTTP."
|
|
131
137
|
LABEL org.opencontainers.image.source="https://github.com/cyanheads/secedgar-mcp-server"
|
|
132
138
|
LABEL org.opencontainers.image.licenses="Apache-2.0"
|
|
@@ -139,6 +145,7 @@ COPY --from=deps /usr/src/app/node_modules ./node_modules
|
|
|
139
145
|
|
|
140
146
|
# Copy the compiled application code from the build stage
|
|
141
147
|
COPY --from=build /usr/src/app/dist ./dist
|
|
148
|
+
COPY --from=build /usr/src/app/skills ./skills
|
|
142
149
|
|
|
143
150
|
# Copy the mirror lifecycle scripts. `bun run mirror:init` / `mirror:refresh` /
|
|
144
151
|
# `mirror:verify` invoke these directly (Bun runs `.ts` natively) — they must
|
|
@@ -162,6 +169,10 @@ RUN echo '{"compilerOptions":{"baseUrl":".","paths":{"@/*":["./dist/*"]}}}' > ts
|
|
|
162
169
|
# Create and set permissions for the log directory, assigning ownership to the 'bun' user.
|
|
163
170
|
RUN mkdir -p /var/log/secedgar-mcp-server && chown -R bun:bun /var/log/secedgar-mcp-server
|
|
164
171
|
|
|
172
|
+
# Writable data dir for the on-disk SQLite mirror (EDGAR_MIRROR_PATH defaults to
|
|
173
|
+
# ./data/edgar-mirror), owned by the runtime user. Mount a volume over it in production.
|
|
174
|
+
RUN mkdir -p /usr/src/app/data && chown -R bun:bun /usr/src/app/data
|
|
175
|
+
|
|
165
176
|
# Switch to the non-root user
|
|
166
177
|
USER bun
|
|
167
178
|
|