@yottagraph-app/data-model-skill 0.0.33 → 0.0.35

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yottagraph-app/data-model-skill",
-  "version": "0.0.33",
+  "version": "0.0.35",
   "description": "Data model skill documentation for AI agents - entity types, properties, and schemas from Lovelace fetch sources",
   "repository": {
     "type": "git",

package/skill/fjcidb/DATA_DICTIONARY.md

@@ -0,0 +1,178 @@
+# Data Dictionary: FJC IDB (Federal Judicial Center — Civil Cases)
+
+Last updated: 2026-05-20
+
+## Source Overview
+
+The **Federal Judicial Center Integrated Database (IDB)** provides administrative statistics on civil cases filed in U.S. district courts. Lovelace ingests the **civil SAS extract** published on the FJC site (annual FY files such as `cv26.sas7bdat`), not PACER dockets. Each SAS row is one civil case with coded fields for court location, docket number, party labels, nature of suit, and disposition.
+
+The streamer uses a **diffing** pipeline: it materializes the SAS file (HTTP download or local path), parses rows in chunks, stores a normalized JSON snapshot per case under the raw store (`fjcidb/download/{case-id}.json`), and republishes atom batches only when a row’s JSON changes.
+
+| Item | Value |
+|------|--------|
+| Stream source constant | `fjcidb-source` |
+| `Record.Source` | `fjcidb` |
+| Default dataset URL | FJC FY civil SAS (see FJC civil cases landing page) |
+| Poll cadence | Configurable (`pollTimeMin`); typical dev runs use a large interval or one-shot via fetcheval |
+
+**Data quality notes**
+
+- Party fields **PLT** and **DEF** are short text labels (often truncated). They are not full party rosters; numeric values mean “count of plaintiffs/defendants” rather than a name.
+- Person vs organization for textual parties uses **regex heuristics** on the label (see `party_regex.go`), not LLM classification in the default ingest path.
+- Code values (district, disposition, etc.) follow the FJC codebook; this dictionary documents KG mapping, not codebook semantics.
+
+---
+
+## Entity Types
+
+### `legal_case`
+
+One civil case in the IDB extract, identified by district, office, docket, and filing year.
+
+- **Subject name:** Human-readable label including `fjcidb_case_id`.
+- **Strong id:** `fjcidb_case_id` on the case subject.
+- **Resolver:** `NOT_MERGEABLE` — passive administrative case node.
+- **Timestamp:** Atomization time (microseconds) for the ingest pass.
+
+### `person`
+
+An individual named on the plaintiff or defendant side when the IDB field is **textual** (not digits-only) and classified as a person by regex rules.
+
+- **Subject name:** Normalized party display (trailing `, ET AL` removed from the raw field).
+- **Property:** `name` (normalized party label from PLT/DEF).
+- **Resolver:** `MERGEABLE` named entity; lawsuit-context snippet on the record (who sues whom; NOS phrase included).
+- **Examples:** `SHANKS`, `BECERRA`, `BIDEN` (with or without `, ET AL`).
+
+### `organization`
+
+An institution or collective named on the plaintiff or defendant side when the field is textual and classified as organization (regex), or when classification is ambiguous and defaults to organization.
+
+- **Subject name:** Same normalization as person parties.
+- **Property:** `name` (normalized party label from PLT/DEF).
+- **Resolver:** `MERGEABLE` named entity; same lawsuit snippet pattern as person parties.
+- **Examples:** `DEPARTMENT OF DEFENSE`, `CUMMINS INC.`, `INTERNATIONAL UNION OF , ET AL`.
+
+### `nature_of_suit`
+
+A federal civil **nature of suit (NOS)** code from the U.S. Courts classification, linked from the case’s **NOS** field.
+
+- **Subject name:** Short title when known, e.g. `Employment`; otherwise `Nature of suit {code} (federal civil)`.
+- **Strong id:** `nos_code`.
+- **Resolver:** `NOT_MERGEABLE`.
+- **Reference:** U.S. Courts civil NOS code descriptions PDF (titles/descriptions embedded in ingest).
+
+---
+
+## Properties
+
+### Legal case
+
+* `fjcidb_case_id`
+  * Definition: Stable case identifier for this IDB row.
+  * Examples: `3-1-12345-2025`, `90-1-2303817-2023`
+  * Derivation: `{DISTRICT}-{OFFICE}-{DOCKET}-{year}` where year comes from `FILEDATE`, or from `FDATEUSE` when `FILEDATE` is absent. Row omitted if year cannot be determined.
+
+* `district_code`
+  * Definition: FJC district court code (`DISTRICT`).
+  * Examples: `3`, `90`
+  * Derivation: SAS `DISTRICT`.
+
+* `office_code`
+  * Definition: FJC office within the district (`OFFICE`).
+  * Examples: `1`
+  * Derivation: SAS `OFFICE`.
+
+* `case_docket_number`
+  * Definition: Court docket number (`DOCKET`).
+  * Examples: `12345`, `2303817`
+  * Derivation: SAS `DOCKET`.
+
+* `case_filing_date`
+  * Definition: Filing date as stored in the extract.
+  * Derivation: SAS `FILEDATE`.
+
+* `termination_date`
+  * Definition: Termination date when present.
+  * Derivation: SAS `TERMDATE`.
+
+* `origin_code`, `jurisdiction_code`, `disposition_code`, `class_action_code`, `procedural_progress_code`
+  * Definition: FJC codebook fields for procedural status.
+  * Derivation: SAS `ORIGIN`, `JURIS`, `DISP`, `CLASSACT`, `PROCPROG` when non-empty.
+
+### Person and organization (shared)
+
+* `name`
+  * Definition: Normalized party label from PLT or DEF.
+  * Examples: `SHANKS`, `INTERNATIONAL UNION OF`
+  * Derivation: Trim and remove trailing `, ET AL` from the raw IDB field; matches the record subject name.
+
+### Nature of suit
+
+* `nos_code`
+  * Definition: Numeric NOS code from the case row.
+  * Examples: `110`, `442`
+  * Derivation: SAS `NOS`.
+
+* `nos_title`
+  * Definition: Short title from the U.S. Courts NOS codebook.
+  * Examples: `Insurance`, `Employment`
+  * Derivation: Lookup table when code is known; omitted if unknown.
+
+* `nos_description`
+  * Definition: Long description from the same codebook.
+  * Derivation: Lookup table when code is known; omitted if unknown.
+
+---
+
+## Entity Relationships
+
+```
+legal_case  ──[has_nature_of_suit]──→  nature_of_suit
+
+person       ──[is_plaintiff_in]────→  legal_case
+organization ──[is_plaintiff_in]────→  legal_case
+
+person       ──[is_defendant_in]────→  legal_case
+organization ──[is_defendant_in]────→  legal_case
+```
+
+- **`is_plaintiff_in` / `is_defendant_in`:** Emitted on the **party** record; target atom points at the case entity (with case strong id on the target for graph linkage). Only for non-empty, non-numeric, non-`SEALED` PLT/DEF values.
+- **`has_nature_of_suit`:** Emitted on the **case** record when `NOS` is present; target is the `nature_of_suit` entity for that code.
+
+---
+
+## Records Per Case
+
+Typical atomization for one row with textual PLT and DEF, and a NOS code:
+
+1. One `legal_case` record (case properties + `has_nature_of_suit` target).
+2. One `nature_of_suit` record (when NOS present).
+3. Up to two party records (`person` or `organization` per side).
+
+Rows with only numeric PLT/DEF produce a case record with count properties only (no party entities). `SEALED` parties produce no party entities.
+
+---
+
+## Party labeling (person vs organization)
+
+Textual PLT/DEF labels are classified before flavor assignment:
+
+1. **Skip** — empty, all digits, or withheld (`SEALED`).
+2. **Organization** — label matches org indicators (legal suffixes, government words, `OF`/`AND`/`THE`, commas, truncated org stems) or has **three or more** name tokens after normalization.
+3. **Person** — one or two tokens that look like personal name parts (letters, hyphen, apostrophe).
+4. **Default** — organization.
+
+Consumers should treat regex labels as heuristic, especially on truncated government and corporate strings.
+
+---
+
+## Citations
+
+Citation text on atoms and entities: `Federal Judicial Center Integrated Database, civil SAS extract` with link to the FJC IDB research page. NOS entities additionally cite the U.S. Courts NOS descriptions document.
+
+---
+
+## Validation
+
+- **Unit fixture:** `testdata/sample_fjcidb_case.pb.txt` (SHANKS vs International Union, NOS 442) — regenerate via `./run_legal_test.sh` at repo root.
+- **recordeval:** Same script runs schema validation on `testdata/*.pb.txt`.

package/skill/fjcidb/schema.yaml

@@ -0,0 +1,206 @@
+# Dataset schema for FJC Integrated Database — U.S. district court civil cases (SAS extracts).
+#
+# Source: https://www.fjc.gov/research/idb/civil-cases-filed-terminated-and-pending-sy-1988-present
+# Codebook: Civil Codebook 1988 Forward (FJC PDF).
+#
+# Default ingest target is the FY civil SAS file (e.g. cv26.sas7bdat), refreshed by FJC on a schedule.
+name: "fjcidb"
+description: "Federal Judicial Center Integrated Database civil case records for U.S. district courts, including office and docket identity, party labels (paintif/defendant), filing and termination timing, jurisdiction, nature-of-suit entities linked from each case, and disposition codes"
+
+extraction:
+  flavors: closed
+  properties: closed
+  relationships: closed
+  attributes: closed
+  events: closed
+
+flavors:
+  - name: "legal_case"
+    description: "A civil case docketed in a U.S. federal district court, identified by FJC administrative codes rather than PACER"
+    display_name: "Legal case"
+    mergeability: not_mergeable
+    strong_id_properties: ["fjcidb_case_id"]
+    passive: true
+
+  - name: "organization"
+    description: "A particular business, institution, or organization such as a corporation, university, government agency, or non-profit"
+    display_name: "Organization"
+    mergeability: mergeable
+    passive: true
+
+  - name: "person"
+    description: "A real person as opposed to a fictional character, such as a CEO, politician, or public figure"
+    display_name: "Person"
+    mergeability: mergeable
+    passive: true
+
+  - name: "nature_of_suit"
+    description: "Federal civil nature-of-suit (NOS) code from the U.S. Courts classification"
+    display_name: "Nature of suit"
+    mergeability: not_mergeable
+    strong_id_properties: ["nos_code"]
+    passive: true
+
+properties:
+  - name: "fjcidb_case_id"
+    namespace: "fjcidb"
+    type: string
+    description: "Stable FJC civil case identifier formed from district, office, docket, and filing year"
+    display_name: "FJC civil case ID"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "name"
+    namespace: "fjcidb"
+    type: string
+    description: "Display name of the entity"
+    display_name: "Name"
+    mergeability: not_mergeable
+    domain_flavors: ["organization", "person"]
+    passive: true
+
+  - name: "nos_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Nature of suit numeric code (matches IDB NOS and the U.S. Courts civil NOS codebook)"
+    display_name: "NOS code"
+    mergeability: not_mergeable
+    domain_flavors: ["nature_of_suit"]
+    passive: true
+
+  - name: "nos_title"
+    namespace: "fjcidb"
+    type: string
+    description: "Short title for this nature of suit code from the U.S. Courts civil NOS descriptions"
+    display_name: "Nature of suit title"
+    mergeability: not_mergeable
+    domain_flavors: ["nature_of_suit"]
+    passive: true
+
+  - name: "nos_description"
+    namespace: "fjcidb"
+    type: string
+    description: "Official description for this nature of suit code from the U.S. Courts civil NOS descriptions"
+    display_name: "Nature of suit description"
+    mergeability: not_mergeable
+    domain_flavors: ["nature_of_suit"]
+    passive: true
+
+  - name: "district_code"
+    namespace: "fjcidb"
+    type: string
+    description: "FJC district court code for where the civil action was filed (IDB DISTRICT)"
+    display_name: "FJC district code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "office_code"
+    namespace: "fjcidb"
+    type: string
+    description: "FJC office code within the district where the case was filed (IDB OFFICE)"
+    display_name: "FJC office code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "case_docket_number"
+    namespace: "fjcidb"
+    type: string
+    description: "Docket number assigned by the filing office (IDB DOCKET)"
+    display_name: "Docket number"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "case_filing_date"
+    namespace: "fjcidb"
+    type: string
+    description: "Case filing date"
+    display_name: "Filing date"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "termination_date"
+    namespace: "fjcidb"
+    type: string
+    description: "Termination date if the case terminated in the reporting window"
+    display_name: "Termination date"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "origin_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Procedural origin code for the filing"
+    display_name: "Origin code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "jurisdiction_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Jurisdiction code"
+    display_name: "Jurisdiction code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "disposition_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Disposition code when the case terminated"
+    display_name: "Disposition code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "class_action_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Class action designation code"
+    display_name: "Class action code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+  - name: "procedural_progress_code"
+    namespace: "fjcidb"
+    type: string
+    description: "Procedural progress code"
+    display_name: "Procedural progress code"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    passive: true
+
+relationships:
+  - name: "is_plaintiff_in"
+    namespace: "fjcidb"
+    description: "The party is named as the plaintiff side in this FJC civil case"
+    display_name: "Is plaintiff in"
+    mergeability: not_mergeable
+    domain_flavors: ["organization", "person"]
+    target_flavors: ["legal_case"]
+    passive: true
+
+  - name: "is_defendant_in"
+    namespace: "fjcidb"
+    description: "The party is named as the defendant side in this FJC civil case"
+    display_name: "Is defendant in"
+    mergeability: not_mergeable
+    domain_flavors: ["organization", "person"]
+    target_flavors: ["legal_case"]
+    passive: true
+
+  - name: "has_nature_of_suit"
+    namespace: "fjcidb"
+    description: "The civil case is classified under this federal nature-of-suit code (IDB NOS field)"
+    display_name: "Has nature of suit"
+    mergeability: not_mergeable
+    domain_flavors: ["legal_case"]
+    target_flavors: ["nature_of_suit"]
+    passive: true

package/skill/sbaloans/DATA_DICTIONARY.md

@@ -0,0 +1,240 @@
+# Data Dictionary: SBA Loans (FOIA)
+
+## Source Overview
+
+The SBA 7(a) & 504 FOIA dataset is published by the U.S. Small Business Administration, Office of Capital Access. It contains loan-level records for the two main SBA loan guarantee programs:
+
+- **7(a) Loan Program** -- the SBA's primary and most flexible loan program, covering working capital, equipment, real estate, and debt refinancing. Maximum loan amount is $5 million.
+- **504 Loan Program** -- provides long-term, fixed-rate financing for major fixed assets (real estate, heavy equipment) through Certified Development Companies (CDCs). Structured as a three-party arrangement: borrower (10%), CDC-backed SBA portion (40%), and a conventional third-party lender (50%).
+
+The data is distributed as CSV files via the CKAN open data platform at `https://data.sba.gov/dataset/7-a-504-foia`. File URLs change with each quarterly update (contain an "as of" date); the CKAN package API is used to discover current resource URLs dynamically.
+
+The dataset is updated quarterly, typically available one month after the quarter ends.
+
+| Program | CSV Files | Approx Size |
+|---------|-----------|-------------|
+| 7(a) | 4 files by decade: FY1991-1999, FY2000-2009, FY2010-2019, FY2020-Present | ~780MB |
+| 504 | 2 files: FY1991-2009, FY2010-Present | ~104MB |
+
+Both programs share a common column subset (borrower info, approval details, NAICS codes, loan status). 7(a) adds bank/lender fields and interest rate details; 504 adds CDC and third-party lender fields.
+
+| Pipeline | `Record.Source` |
+|----------|----------------|
+| All files | `sbaloans` |
+
+---
+
+## Entity Types
+
+### `financial_instrument`
+
+An SBA-backed loan under the 7(a) or 504 program, identified by its location ID (a unique loan identifier assigned by the SBA).
+
+- Primary key: Composite of program and location ID (e.g., `"7a_317954"`, `"504_188194"`)
+- Entity resolver: named entity. Strong ID = `sba_loan_id`. Not mergeable across sources.
+
+### `organization`
+
+A business, bank, CDC, or third-party lender involved in an SBA loan. Three roles:
+- **Borrower** -- the small business receiving the loan
+- **Bank/Primary Lender** (7(a)) -- the financial institution originating the loan, identified by FDIC certificate number when available
+- **CDC** (504) -- the Certified Development Company administering the 504 portion
+- **Third-Party Lender** (504) -- the conventional lender providing the remaining portion
+
+- Primary key: Organization name (borrowers), or FDIC certificate number (banks)
+- Entity resolver: named entity, mergeable. Banks with an FDIC certificate number use it as a strong ID (property `fdic_certificate_number`), enabling cross-source entity resolution with existing FDIC-sourced bank entities.
+
+### `location`
+
+A geographic location representing the project county and state where the loan-funded activity takes place.
+
+- Primary key: County + state combination (e.g., `"EAST BATON ROUGE, LA"`)
+- Entity resolver: named entity, mergeable. Disambiguation via county name + state code.
+
+---
+
+## Properties
+
+### Loan Properties (`financial_instrument`)
+
+#### Identity
+
+* `sba_loan_id`
+  * Definition: Composite SBA loan identifier combining program and location ID.
+  * Examples: `"7a_317954"`, `"504_188194"`
+  * Derivation: Constructed from `program` (normalized) + `"_"` + `locationid` CSV column.
+
+* `program`
+  * Definition: SBA loan program name.
+  * Examples: `"7a"`, `"504"`
+  * Derivation: `program` CSV column, normalized (leading/trailing whitespace trimmed, " 7A" becomes "7a").
+
+#### Financial Terms
+
+* `gross_approval`
+  * Definition: Total approved loan amount in dollars.
+  * Examples: `"450000"`, `"810000"`
+  * Derivation: `grossapproval` CSV column.
+
+* `sba_guaranteed_approval`
+  * Definition: SBA-guaranteed portion of the loan amount in dollars. 7(a) only.
+  * Examples: `"337500"`
+  * Derivation: `sbaguaranteedapproval` CSV column. Empty for 504 loans.
+
+* `initial_interest_rate`
+  * Definition: Initial interest rate of the loan as a percentage. 7(a) only.
+  * Examples: `"6"`, `"5.5"`
+  * Derivation: `initialinterestrate` CSV column.
+
+* `interest_type`
+  * Definition: Whether the interest rate is fixed or variable. 7(a) only.
+  * Examples: `"Fixed"`, `"Variable"`
+  * Derivation: `fixedorvariableinterestind` CSV column, decoded: F=Fixed, V=Variable.
+
+* `term_in_months`
+  * Definition: Loan term length in months.
+  * Examples: `"120"`, `"240"`, `"300"`
+  * Derivation: `terminmonths` CSV column.
+
+* `third_party_dollars`
+  * Definition: Third-party lender contribution in dollars. 504 only.
+  * Examples: `"1334500"`
+  * Derivation: `thirdpartydollars` CSV column.
+
+#### Dates
+
+* `approval_date`
+  * Definition: Date the loan was approved.
+  * Examples: `"12/5/2020"`, `"10/2/2009"`
+  * Derivation: `approvaldate` CSV column, format M/D/YYYY.
+
+* `approval_fiscal_year`
+  * Definition: Federal fiscal year of approval (Oct 1 - Sep 30).
+  * Examples: `"2021"`, `"2010"`
+  * Derivation: `approvalfy` CSV column.
+
+* `first_disbursement_date`
+  * Definition: Date of first loan disbursement.
+  * Examples: `"1/1/2021"`, `"9/15/2010"`
+  * Derivation: `firstdisbursementdate` CSV column.
+
+* `paid_in_full_date`
+  * Definition: Date the loan was paid in full.
+  * Derivation: `paidinfulldate` CSV column. Empty if loan is not paid off.
+
+* `chargeoff_date`
+  * Definition: Date the loan was charged off.
+  * Derivation: `chargeoffdate` CSV column. Empty if loan has not been charged off.
+
+#### Processing
+
+* `processing_method`
+  * Definition: SBA processing method used for the loan.
+  * Examples: `"Preferred Lenders Program"`, `"504 Basic"`
+  * Derivation: `processingmethod` CSV column.
+
+* `sub_program`
+  * Definition: SBA sub-program designation.
+  * Examples: `"Guaranty"`, `"Sec. 504 - Loan Guarantees - Private Sector Financed"`
+  * Derivation: `subprogram` CSV column.
+
+#### Industry and Business
+
+* `naics_code`
+  * Definition: North American Industry Classification System code for the borrower's business.
+  * Examples: `"531390"`, `"721110"`
+  * Derivation: `naicscode` CSV column.
+
+* `naics_description`
+  * Definition: Description of the NAICS industry classification.
+  * Examples: `"Other Activities Related to Real Estate"`, `"Hotels (except Casino Hotels) and Motels"`
+  * Derivation: `naicsdescription` CSV column.
+
+* `franchise_name`
+  * Definition: Name of the franchise if the borrower is a franchisee.
+  * Examples: `"CHOICE HOTELS INTERNATIONAL INC."`
+  * Derivation: `franchisename` CSV column. Empty if not a franchise.
+
+* `business_type`
+  * Definition: Legal structure of the borrowing business.
+  * Examples: `"CORPORATION"`, `"LLC"`, `"PARTNERSHIP"`, `"INDIVIDUAL"`
+  * Derivation: `businesstype` CSV column.
+
+* `business_age`
+  * Definition: Age category of the borrowing business at time of loan.
+  * Examples: `"Existing or more than 2 years old"`, `"New Business or 2 years or less"`, `"Less than 4 years old but at least 3"`
+  * Derivation: `businessage` CSV column.
+
+#### Status
+
+* `loan_status`
+  * Definition: Current status of the loan.
+  * Examples: `"LIQUID"` (active), `"P I F"` (paid in full), `"CHGOFF"` (charged off), `"EXEMPT"`, `"PREPAID IN FULL"`
+  * Derivation: `loanstatus` CSV column.
+
+* `gross_chargeoff_amount`
+  * Definition: Gross charge-off amount in dollars.
+  * Examples: `"0"`, `"125000"`
+  * Derivation: `grosschargeoffamount` CSV column.
+
+* `revolver_status`
+  * Definition: Whether the loan is a revolving line of credit. 7(a) only.
+  * Examples: `"TRUE"`, `"FALSE"`
+  * Derivation: `revolverstatus` CSV column.
+
+* `jobs_supported`
+  * Definition: Number of jobs supported by the loan.
+  * Examples: `"9"`, `"250"`, `"0"`
+  * Derivation: `jobssupported` CSV column.
+
+#### Geography
+
+* `sba_district_office`
+  * Definition: SBA district office that processed the loan.
+  * Examples: `"LOUISIANA DISTRICT OFFICE"`, `"ILLINOIS DISTRICT OFFICE"`
+  * Derivation: `sbadistrictoffice` CSV column.
+
+* `congressional_district`
+  * Definition: Congressional district of the project.
+  * Examples: `"6"`, `"13"`
+  * Derivation: `congressionaldistrict` CSV column.
+
+### Organization Properties
+
+* `address`
+  * Definition: Physical address of the organization formatted as "street, city, state zip".
+  * Examples: `"11851 Wentling Ave, BATON ROUGE, LA 70816"`, `"1981 Marcus Avenue, LAKE SUCCESS, NY 11042"`
+  * Derivation: Formatted from borrower address fields (`borrstreet`/`borrcity`/`borrstate`/`borrzip`) for borrowers, bank fields (`bankstreet`/`bankcity`/`bankstate`/`bankzip`) for 7(a) banks, or CDC fields (`cdc_street`/`cdc_city`/`cdc_state`/`cdc_zip`) for 504 CDCs.
+
+* `fdic_certificate_number`
+  * Definition: FDIC certificate number identifying a bank. Used as a strong ID for entity resolution with FDIC-sourced bank entities.
+  * Examples: `"59345"`
+  * Derivation: `bankfdicnumber` CSV column. Only present for 7(a) bank entities.
+  * Note: Enables cross-source entity resolution with the FDIC dataset (`fdic` source), which also uses `fdic_certificate_number` as a strong ID property on bank organizations.
+
+* `ncua_number`
+  * Definition: NCUA charter number identifying a credit union.
+  * Derivation: `bankncuanumber` CSV column. Only present for 7(a) entities where the lender is a credit union.
+  * Note: Sparse -- most lenders are banks, not credit unions.
+
+---
+
+## Entity Relationships
+
+```
+organization (borrower)       ──[borrower_of]──→ financial_instrument (loan)
+organization (bank/CDC)       ──[lender_of]────→ financial_instrument (loan)    attr: lender_role=primary|cdc
+organization (third-party)    ──[lender_of]────→ financial_instrument (loan)    attr: lender_role=third_party
+financial_instrument (loan)   ──[is_located_at]─→ location (project county/state)
+```
+
+---
+
+## Attributes
+
+* `lender_role` (on `lender_of` relationship)
+  * Definition: Role of the lending organization in the loan.
+  * Values: `"primary"` (7(a) bank), `"cdc"` (504 Certified Development Company), `"third_party"` (504 conventional third-party lender)
+  * Derivation: Set based on which organization record is being emitted and the loan program type.
+
+---

package/skill/sbaloans/schema.yaml

@@ -0,0 +1,380 @@
+# Dataset schema for SBA 7(a) & 504 FOIA loan data.
+#
+# Source: https://data.sba.gov/dataset/7-a-504-foia
+# Updated quarterly (typically one month after quarter end).
+#
+# Each CSV row represents a single SBA-backed loan. The atomizer produces:
+#   - A financial_instrument entity for the loan itself
+#   - An organization entity for the borrower
+#   - An organization entity for the bank/CDC/lender
+#   - Optionally, a third-party lender organization (504 loans)
+#   - A location entity for the project county/state
+name: "sbaloans"
+description: "SBA 7(a) and 504 FOIA loan data including loan terms, borrower and lender organizations, and project locations"
+
+extraction:
+  flavors: closed
+  properties: closed
+  relationships: closed
+  attributes: closed
+  events: closed
+
+flavors:
+  - name: "sba_loan"
+    description: "An SBA-backed loan under the 7(a) or 504 program"
+    display_name: "SBA Loan"
+    mergeability: not_mergeable
+    strong_id_properties: ["sba_loan_id"]
+    passive: true
+
+  - name: "organization"
+    description: "A particular business, institution, or organization such as a corporation, university, government agency, or non-profit"
+    display_name: "Organization"
+    mergeability: not_mergeable
+    strong_id_properties: ["fdic_certificate_number"]
+    passive: true
+
+  - name: "sba_program"
+    description: "An SBA loan program or sub-program (e.g., SBA 7a: Guaranty, SBA 504: Sec. 504 - Loan Guarantees)"
+    display_name: "SBA Program"
+    mergeability: not_mergeable
+    strong_id_properties: ["sba_program_id"]
+    passive: true
+
+  - name: "industry"
+    description: "An industry classification of economic activity (e.g. NAICS or SIC) identifying the line of business associated with an organization or award"
+    display_name: "Industry"
+    mergeability: not_mergeable
+    strong_id_properties: ["naics_code"]
+    passive: true
+
+  - name: "location"
+    description: "A specific named geographic location such as a city, country, region, or landmark"
+    display_name: "Location"
+    mergeability: not_mergeable
+    examples: ["EAST BATON ROUGE, LA", "LINN, IA"]
+    passive: true
+
+properties:
+  # Loan identity
+  - name: "sba_loan_id"
+    type: string
+    description: "Composite SBA loan identifier derived from program, borrower name, lender name, and approval date"
+    display_name: "SBA Loan ID"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["7a_Title2Land,_L.L.C._Newtek_Small_Business_Finance,_Inc._12_5_2020"]
+    passive: true
+
+
+  - name: "sba_program_id"
+    type: string
+    description: "Composite identifier for an SBA program/sub-program (lowercased program name)"
+    display_name: "SBA Program ID"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_program"]
+    examples: ["sba 7a: guaranty", "sba 504: sec. 504 - loan guarantees - private sector financed"]
+    passive: true
+
+  - name: "program"
+    type: string
+    description: "SBA loan program name"
+    display_name: "Program"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_program"]
+    examples: ["7a", "504"]
+    passive: true
+
+  - name: "gross_approval"
+    type: float
+    description: "Total approved loan amount in dollars"
+    display_name: "Gross Approval"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["450000", "810000"]
+    passive: true
+
+  - name: "sba_guaranteed_approval"
+    type: float
+    description: "SBA-guaranteed portion of the loan amount in dollars (7(a) only)"
+    display_name: "SBA Guaranteed Approval"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["337500"]
+    passive: true
+
+  - name: "approval_date"
+    type: string
+    description: "Date the loan was approved"
+    display_name: "Approval Date"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["12/5/2020"]
+    passive: true
+
+  - name: "approval_fiscal_year"
+    type: string
+    description: "Federal fiscal year of approval"
+    display_name: "Approval Fiscal Year"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["2021", "2010"]
+    passive: true
+
+  - name: "first_disbursement_date"
+    type: string
+    description: "Date of first loan disbursement"
+    display_name: "First Disbursement Date"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "term_in_months"
+    type: float
+    description: "Loan term length in months"
+    display_name: "Term in Months"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["120", "240", "300"]
+    passive: true
+
+  - name: "initial_interest_rate"
+    type: float
+    description: "Initial interest rate of the loan as a percentage (7(a) only)"
+    display_name: "Initial Interest Rate"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["6", "5.5"]
+    passive: true
+
+  - name: "interest_type"
+    type: string
+    description: "Whether the interest rate is fixed or variable (7(a) only)"
+    display_name: "Interest Type"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["Fixed", "Variable"]
+    passive: true
+
+  - name: "processing_method"
+    type: string
+    description: "SBA processing method used for the loan"
+    display_name: "Processing Method"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["Preferred Lenders Program", "504 Basic"]
+    passive: true
+
+  - name: "sub_program"
+    type: string
+    description: "SBA sub-program designation"
+    display_name: "Sub-Program"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_program"]
+    examples: ["Guaranty", "Sec. 504 - Loan Guarantees - Private Sector Financed"]
+    passive: true
+
+  - name: "naics_code"
+    type: string
+    description: "North American Industry Classification System code (typically 6 digits) identifying the industry of work performed under a contract (e.g., \"524114\" for Direct Health and Medical Insurance Carriers)"
+    display_name: "NAICS code"
+    mergeability: not_mergeable
+    domain_flavors: ["industry"]
+    examples: ["531390", "721110"]
+    passive: true
+
+  - name: "naics_description"
+    type: string
+    description: "Human-readable name of the NAICS industry (e.g., \"DIRECT HEALTH AND MEDICAL INSURANCE CARRIERS\")"
+    display_name: "NAICS description"
+    mergeability: not_mergeable
+    domain_flavors: ["industry"]
+    examples: ["Other Activities Related to Real Estate", "Hotels (except Casino Hotels) and Motels"]
+    passive: true
+
+  - name: "franchise_name"
+    type: string
+    description: "Name of the franchise if the borrower is a franchisee"
+    display_name: "Franchise Name"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "business_type"
+    type: string
+    description: "Legal structure of the borrowing business"
+    display_name: "Business Type"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["CORPORATION", "LLC", "PARTNERSHIP"]
+    passive: true
+
+  - name: "business_age"
+    type: string
+    description: "Age category of the borrowing business at time of loan"
+    display_name: "Business Age"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["Existing or more than 2 years old", "New Business or 2 years or less"]
+    passive: true
+
+  - name: "loan_status"
+    type: string
+    description: "Current status of the loan"
+    display_name: "Loan Status"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["LIQUID", "P I F", "CHGOFF", "EXEMPT"]
+    passive: true
+
+  - name: "paid_in_full_date"
+    type: string
+    description: "Date the loan was paid in full"
+    display_name: "Paid in Full Date"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "chargeoff_date"
+    type: string
+    description: "Date the loan was charged off"
+    display_name: "Charge-Off Date"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "gross_chargeoff_amount"
+    type: float
+    description: "Gross charge-off amount in dollars"
+    display_name: "Gross Charge-Off Amount"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "revolver_status"
+    type: string
+    description: "Whether the loan is a revolving line of credit (7(a) only)"
+    display_name: "Revolver Status"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "jobs_supported"
+    type: float
+    description: "Number of jobs supported by the loan"
+    display_name: "Jobs Supported"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    examples: ["9", "250"]
+    passive: true
+
+  - name: "third_party_dollars"
+    type: float
+    description: "Third-party lender contribution in dollars (504 only)"
+    display_name: "Third-Party Dollars"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "sba_district_office"
+    type: string
+    description: "SBA district office that processed the loan"
+    display_name: "SBA District Office"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  - name: "congressional_district"
+    type: string
+    description: "Congressional district of the project"
+    display_name: "Congressional District"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    passive: true
+
+  # Organization properties
+  - name: "address"
+    type: string
+    description: "Physical street address of the entity"
+    display_name: "Address"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["11851 Wentling Ave, BATON ROUGE, LA 70816"]
+    passive: true
+
+  - name: "fdic_certificate_number"
+    type: string
+    description: "FDIC Certificate Number uniquely identifying an insured institution, derived from the CERT field in the Institutions API"
+    display_name: "FDIC Certificate Number"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "ncua_number"
+    type: string
+    description: "NCUA charter number identifying a credit union"
+    display_name: "NCUA Number"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+relationships:
+  - name: "borrower_of"
+    description: "An organization is the borrower, applicant, or account party on a financial instrument such as a letter of credit, loan, or bond issue."
+    display_name: "Borrower Of"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    target_flavors: ["sba_loan"]
+    examples: ["Title2Land, L.L.C. is borrower of SBA 7a Loan 317954"]
+    passive: true
+
+  - name: "lender_of"
+    description: "An organization (bank, CDC, or third-party lender) is the lender on an SBA loan"
+    display_name: "Lender Of"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    target_flavors: ["sba_loan"]
+    examples: ["Newtek Small Business Finance, Inc. is lender of SBA 7a Loan 317954"]
+    passive: true
+
+  - name: "in_program"
+    description: "The SBA program/sub-program under which this loan was made"
+    display_name: "In Program"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    target_flavors: ["sba_program"]
+    passive: true
+
+  - name: "in_industry"
+    description: "The industry (NAICS) classification of the work performed under this contract or IDV"
+    display_name: "In industry (NAICS)"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    target_flavors: ["industry"]
+    passive: true
+
+  - name: "project_is_located_at"
+    description: "Geographic location of the SBA loan's project"
+    display_name: "Project Located At"
+    mergeability: not_mergeable
+    domain_flavors: ["sba_loan"]
+    target_flavors: ["location"]
+    examples: ["SBA 7a Loan 317954 is located at EAST BATON ROUGE, LA"]
+    passive: true
+
+attributes:
+  - properties: ["lender_of"]
+    name: "lender_type"
+    type: string
+    description: "Type of lender in the loan (primary, cdc, third_party)"
+    display_name: "Lender Type"
+    mergeability: not_mergeable
+    examples: ["primary", "cdc", "third_party"]
+
+  - properties: ["lender_of"]
+    name: "lender_amount"
+    type: float
+    description: "Dollar amount lent by this lender"
+    display_name: "Lender Amount"
+    mergeability: not_mergeable
+    examples: ["450000", "810000", "1334500"]