@yottagraph-app/data-model-skill 0.0.18 → 0.0.20

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yottagraph-app/data-model-skill",
-  "version": "0.0.18",
+  "version": "0.0.20",
   "description": "Data model skill documentation for AI agents - entity types, properties, and schemas from Lovelace fetch sources",
   "repository": {
     "type": "git",

package/skill/dotcensus/DATA_DICTIONARY.md

@@ -0,0 +1,157 @@
+# Data Dictionary: DOT Census
+
+## Source Overview
+
+The FMCSA Company Census File is a daily snapshot of all motor carriers, brokers, shippers, and hazmat carriers registered with the Federal Motor Carrier Safety Administration (FMCSA), part of the U.S. Department of Transportation.
+
+- **Publisher:** FMCSA / U.S. Department of Transportation
+- **URL:** https://data.transportation.gov/Trucking-and-Motorcoaches/Company-Census-File/az4n-8mr2
+- **Format:** CSV (~500K+ rows)
+- **Cadence:** Updated daily from a 24-hour-old FMCSA database snapshot
+- **Source name:** `dotcensus`
+
+Each row represents one registered entity identified by a unique USDOT number. The file contains identity, physical location, fleet composition, carrier operation type, and safety ratings.
+
+**Limitations:** The census reflects registration data, not real-time operational status. Fleet sizes and driver counts are self-reported by carriers during MCS-150 filings and may be stale. Safety ratings are assigned only after compliance reviews and are absent for most carriers.
+
+---
+
+## Entity Types
+
+### `organization`
+
+A motor carrier, broker, shipper, or other entity registered with FMCSA.
+
+- Primary key: `usdot_number` (USDOT Number, unique per registrant)
+- Entity resolver: named entity, MERGEABLE. Strong ID = `usdot_number`. Disambiguation snippet includes the physical address when available.
+- Name: uses `DBA_NAME` when it differs from `LEGAL_NAME`; otherwise uses `LEGAL_NAME`.
+
+### `legal_entity`
+
+The legal owner of a carrier when `LEGAL_NAME` differs from `DBA_NAME`. Represents the person or organization that legally owns the business operating under the DBA name.
+
+- Primary key: none (resolved by name)
+- Entity resolver: named entity, MERGEABLE. Disambiguation snippet includes the entity name.
+- Only produced when `DBA_NAME` is present and differs from `LEGAL_NAME`.
+
+### `location`
+
+The physical location (city + state) of a registered carrier.
+
+- Primary key: none (resolved by name)
+- Entity resolver: named entity, MERGEABLE. Disambiguation snippet includes the formatted location name.
+- Name format: `"{city}, {state}"` or `"{city}, {state}, {country}"` when country is present.
+
+---
+
+## Properties
+
+### Organization Properties
+
+#### Identity and Registration
+
+* `usdot_number`
+  * Definition: USDOT number assigned by FMCSA, unique per registered entity.
+  * Examples: `"12345"`, `"99999"`
+  * Derivation: `DOT_NUMBER` column. Emitted as both a strong ID for entity resolution and as a property atom on every record.
+
+* `address`
+  * Definition: Physical street address of the carrier.
+  * Examples: `"21154 HWY EAST, SILOAM SPRINGS, AR 72761"`, `"100 Main St, Dallas, TX 75201"`
+  * Derivation: Concatenation of `PHY_STREET`, `PHY_CITY`, `PHY_STATE`, and `PHY_ZIP` columns.
+
+* `dot_phone_number`
+  * Definition: Primary phone number on file with FMCSA.
+  * Examples: `"5551234567"`
+  * Derivation: `PHONE` column.
+
+* `dot_business_org_type`
+  * Definition: Business organization type of the registrant.
+  * Examples: `"Individual"`, `"Partnership"`, `"Corporation"`
+  * Derivation: `BUSINESS_ORG_ID` column decoded from numeric codes (1=Individual, 2=Partnership, 3=Corporation).
+
+* `dot_entity_type`
+  * Definition: FMCSA entity type indicating the registrant's role in freight transportation.
+  * Examples: `"Carrier"`, `"Carrier; Broker"`, `"Carrier; Shipper"`
+  * Derivation: `CARSHIP` column. Semicolon-delimited code list decoded from single-letter codes (C=Carrier, B=Broker, S=Shipper, T=Cargo Tank, R=Registrant).
+
+* `dot_operating_authority`
+  * Definition: Operating authority classification describing the carrier's authorization type.
+  * Examples: `"AUTHORIZED FOR HIRE"`, `"PRIVATE PROPERTY"`, `"EXEMPT FOR HIRE"`
+  * Derivation: `CLASSDEF` column, passed through as-is.
+
+#### Carrier Status and Operations
+
+* `dot_carrier_status`
+  * Definition: FMCSA carrier registration status.
+  * Examples: `"A (Active)"`, `"I (Inactive)"`, `"N (Not Authorized)"`
+  * Derivation: `STATUS_CODE` column decoded from single-letter codes.
+
+* `dot_carrier_operation_type`
+  * Definition: Type of carrier operation based on registration.
+  * Examples: `"A (Interstate)"`, `"B (Intrastate Hazmat)"`, `"C (Intrastate Non-Hazmat)"`
+  * Derivation: `CARRIER_OPERATION` column decoded from single-letter codes.
+  * Note: The B/C distinction reflects the carrier's registration type, not actual hazmat activity. A carrier registered as C (non-hazmat) can still transport hazmat (see `dot_hazmat_indicator`).
+
+* `dot_hazmat_indicator`
+  * Definition: Whether the carrier transports hazardous materials.
+  * Examples: `1.0` (yes), `0.0` (no)
+  * Derivation: `HM_Ind` column. `"Y"` → 1.0, `"N"` → 0.0. Omitted when blank.
+
+#### Fleet Composition
+
+* `dot_total_drivers`
+  * Definition: Total number of drivers reported by the carrier.
+  * Examples: `10.0`, `250.0`
+  * Derivation: `TOTAL_DRIVERS` column, parsed as float. Omitted when zero or blank.
+
+* `dot_total_power_units`
+  * Definition: Total number of power units (trucks, tractors) operated by the carrier.
+  * Examples: `5.0`, `1200.0`
+  * Derivation: `POWER_UNITS` column, parsed as float. Omitted when zero or blank.
+
+* `dot_total_bus_units`
+  * Definition: Total number of bus units operated by the carrier.
+  * Examples: `2.0`, `50.0`
+  * Derivation: `BUS_UNITS` column, parsed as float. Omitted when zero or blank.
+
+* `dot_fleet_size_category`
+  * Definition: Fleet size category assigned by FMCSA based on total power units.
+  * Examples: `"A (1-6 power units)"`, `"D (20-100 power units)"`, `"F (1000+ power units)"`
+  * Derivation: `FLEETSIZE` column decoded from single-letter codes. Omitted when `"0"` (none reported).
+
+#### Safety
+
+* `dot_safety_rating`
+  * Definition: FMCSA safety rating assigned during a compliance review.
+  * Examples: `"Satisfactory"`, `"Conditional"`, `"Unsatisfactory"`
+  * Derivation: `SAFETY_RATING` column decoded from single-letter codes (S, C, U).
+  * Note: Most carriers have no safety rating — it is only assigned after a compliance review.
+
+* `dot_safety_rating_date`
+  * Definition: Date the safety rating was assigned.
+  * Examples: `"2024-01-01"`
+  * Derivation: `SAFETY_RATING_DATE` column parsed from `YYYYMMDD` format, output as `YYYY-MM-DD`.
+
+#### Filing Dates
+
+* `dot_last_mcs150_filing_date`
+  * Definition: Date the carrier last filed or updated their MCS-150 Motor Carrier Identification Report.
+  * Examples: `"2024-01-01"`
+  * Derivation: `MCS150_DATE` column parsed from `YYYYMMDD HHMM` format, output as `YYYY-MM-DD`.
+
+### Timestamp Derivation
+
+Record timestamps are derived in priority order: `ADD_DATE` (carrier registration date), then `MCS150_DATE` (last MCS-150 filing). Rows where neither date parses successfully are dropped.
+
+---
+
+## Entity Relationships Summary
+
+```
+legal_entity  ──[doing_business_as]──→ organization
+organization  ──[is_located_at]──────→ location
+```
+
+- `doing_business_as`: Links the legal owner (`LEGAL_NAME`) to the business operating name (`DBA_NAME`). Only produced when the two names differ.
+- `is_located_at`: Links the carrier to its physical location. Only produced when both `PHY_CITY` and `PHY_STATE` are present.

package/skill/dotcensus/schema.yaml

@@ -0,0 +1,196 @@
+# Dataset schema for FMCSA Company Census File (DOT Motor Carrier Census).
+#
+# Source: https://data.transportation.gov/Trucking-and-Motorcoaches/Company-Census-File/az4n-8mr2
+# Updated daily by FMCSA from a 24-hour-old database snapshot.
+#
+# This schema describes motor carrier entities registered with the Federal
+# Motor Carrier Safety Administration, including identity, physical location,
+# fleet composition, and safety ratings.
+name: "dotcensus"
+description: "FMCSA motor carrier company census data including registration, fleet size, carrier operations, and safety ratings from the DOT Company Census File"
+
+extraction:
+  flavors: closed
+  properties: closed
+  relationships: closed
+  attributes: closed
+  events: closed
+
+flavors:
+  - name: "organization"
+    description: "A particular business, institution, or organization such as a corporation, university, government agency, or non-profit"
+    display_name: "Organization"
+    mergeability: not_mergeable
+    strong_id_properties: ["usdot_number"]
+    passive: true
+
+  - name: "person"
+    description: "A named individual such as a business owner, executive, or public figure"
+    display_name: "Person"
+    mergeability: not_mergeable
+    passive: true
+
+  - name: "location"
+    description: "A specific named geographic location such as a city, country, region, or landmark"
+    display_name: "Location"
+    mergeability: not_mergeable
+    examples: ["New York City", "San Francisco", "North America", "Bakery Square"]
+    passive: true
+
+properties:
+  - name: "usdot_number"
+    type: string
+    description: "USDOT number assigned by FMCSA, uniquely identifying a registered motor carrier, broker, or shipper"
+    display_name: "USDOT Number"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["12345", "99999"]
+    passive: true
+
+  - name: "address"
+    type: string
+    description: "Physical street address of the entity"
+    display_name: "Address"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["21154 HWY EAST, SILOAM SPRINGS, AR 72761"]
+    passive: true
+
+  - name: "dot_carrier_status"
+    type: string
+    description: "FMCSA carrier registration status"
+    display_name: "Carrier Status"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["A (Active)", "I (Inactive)", "N (Not Authorized)"]
+    passive: true
+
+  - name: "dot_phone_number"
+    type: string
+    description: "Primary phone number of the carrier"
+    display_name: "Phone Number"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["5551234567"]
+    passive: true
+
+  - name: "dot_carrier_operation_type"
+    type: string
+    description: "Type of carrier operation"
+    display_name: "Carrier Operation Type"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["A (Interstate)", "B (Intrastate Hazmat)", "C (Intrastate Non-Hazmat)"]
+    passive: true
+
+  - name: "dot_safety_rating"
+    type: string
+    description: "FMCSA safety rating assigned during compliance review"
+    display_name: "Safety Rating"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["Satisfactory", "Conditional", "Unsatisfactory"]
+    passive: true
+
+  - name: "dot_safety_rating_date"
+    type: string
+    description: "Date the safety rating was assigned, derived from the SAFETY_RATING_DATE field"
+    display_name: "Safety Rating Date"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_last_mcs150_filing_date"
+    type: string
+    description: "Date the carrier last filed or updated their MCS-150 Motor Carrier Identification Report with FMCSA"
+    display_name: "Last MCS-150 Filing Date"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_hazmat_indicator"
+    type: float
+    description: "Whether the carrier transports hazardous materials (1.0 = yes, 0.0 = no), derived from the HM_Ind field"
+    display_name: "Hazmat Indicator"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_total_drivers"
+    type: float
+    description: "Total number of drivers reported by the carrier"
+    display_name: "Total Drivers"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_total_power_units"
+    type: float
+    description: "Total number of power units (trucks, tractors) operated by the carrier"
+    display_name: "Total Power Units"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_total_bus_units"
+    type: float
+    description: "Total number of bus units operated by the carrier"
+    display_name: "Total Bus Units"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    passive: true
+
+  - name: "dot_fleet_size_category"
+    type: string
+    description: "Fleet size category assigned by FMCSA based on the number of power units"
+    display_name: "Fleet Size Category"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["A (1-6 power units)", "D (20-100 power units)", "F (1000+ power units)"]
+    passive: true
+
+  - name: "dot_business_org_type"
+    type: string
+    description: "Business organization type of the carrier"
+    display_name: "Business Organization Type"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["Individual", "Partnership", "Corporation"]
+    passive: true
+
+  - name: "dot_entity_type"
+    type: string
+    description: "FMCSA entity type indicating whether the entity operates as a carrier, broker, shipper, or combination"
+    display_name: "Entity Type"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["Carrier", "Carrier; Broker", "Carrier; Shipper"]
+    passive: true
+
+  - name: "dot_operating_authority"
+    type: string
+    description: "Operating authority classification describing the carrier's authorization type"
+    display_name: "Operating Authority"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    examples: ["AUTHORIZED FOR HIRE", "PRIVATE PROPERTY", "EXEMPT FOR HIRE"]
+    passive: true
+
+relationships:
+  - name: "doing_business_as"
+    description: "A legal entity is doing business as (DBA) an organization"
+    display_name: "Doing Business As"
+    mergeability: not_mergeable
+    domain_flavors: ["person", "organization"]
+    target_flavors: ["organization"]
+    examples: ["LUIS GALVAN doing business as G & G TRANSPORTATION CO"]
+    passive: true
+
+  - name: "is_located_at"
+    description: "An entity is located at, operates in, resides in, is headquartered in, was born in, visits, or died in a location"
+    display_name: "Located At"
+    mergeability: not_mergeable
+    domain_flavors: ["organization"]
+    target_flavors: ["location"]
+    examples: ["DAYSPRING CARDS INC is located at SILOAM SPRINGS, AR"]
+    passive: true

package/skill/edgar/schema.yaml

@@ -2763,6 +2763,33 @@ relationships:
     target_flavors: ["organization"]
     passive: true
 
+  - name: "has_administrator"
+    description: "Link from a fund series to its fund administrator (N-CEN). Sub-admins are excluded (mirrors the custodied_by sub-custodian filter); the is_admin_affiliated attribute distinguishes in-family from third-party admin relationships."
+    display_name: "Has Administrator"
+    mergeability: not_mergeable
+    domain_flavors: ["financial_instrument"]
+    target_flavors: ["organization"]
+    passive: true
+
+  # has_broker (N-CEN Item E.1, agency-execution brokers) is intentionally
+  # NOT modelled. A day of evals showed it produced ~6.5k edges — 5.8x the
+  # volume of advised_by — and the source XML reports broker names in ALL
+  # CAPS while every other section uses mixed case, so the same firm gets
+  # fragmented into two organization entities. High volume + bad merge
+  # quality + low query value (top-10 brokers are the same big banks for
+  # every active equity fund) → the data isn't worth surfacing in the graph.
+  # has_broker_dealer (Item E.2, principal trades) is kept because
+  # principal-trade counterparties are economically distinct, the lists are
+  # short, the names dedup cleanly, and the captive-distributor pattern is
+  # genuinely informative.
+  - name: "has_broker_dealer"
+    description: "Link from a fund series to a broker-dealer that the fund traded with on a principal basis during the reporting period (N-CEN Item E.2). Broker-dealers act as principals in the trade — they take the other side of it — so this captures market-making and captive-distributor relationships rather than mere agency execution."
+    display_name: "Has Broker-Dealer"
+    mergeability: not_mergeable
+    domain_flavors: ["financial_instrument"]
+    target_flavors: ["organization"]
+    passive: true
+
   - name: "has_master_fund"
     description: "Link from a feeder fund to its master fund in a master-feeder structure (N-CEN Item C.3). The feeder invests substantially all assets in the master, so resolving this edge is required to see the feeder's true portfolio (which lives on the master's N-PORT)."
     display_name: "Has Master Fund"
@@ -3064,6 +3091,24 @@ attributes:
     display_name: "Sub-Adviser Affiliated"
     mergeability: not_mergeable
 
+  # ── has_administrator relationship attributes (N-CEN) ──
+
+  - property: "has_administrator"
+    name: "is_admin_affiliated"
+    type: string
+    description: "Whether the fund administrator is affiliated with the registrant (Y or N) per N-CEN"
+    display_name: "Admin Affiliated"
+    mergeability: not_mergeable
+
+  # ── has_broker_dealer relationship attributes (N-CEN Item E.2) ──
+
+  - property: "has_broker_dealer"
+    name: "broker_dealer_commission"
+    type: string
+    description: "Commission paid to this broker-dealer for principal trades during the reporting period (USD) per N-CEN"
+    display_name: "Broker-Dealer Commission"
+    mergeability: not_mergeable
+
   # ── is_director relationship attributes (N-CEN) ──
 
   - property: "is_director"