sheeply 0.0.0-stage → 0.1.0

package/.env.example

@@ -0,0 +1,15 @@
+# Dealroom API key: create one at https://app.dealroom.co (Settings > API). A key is a client id + secret pair.
+DEALROOM_CLIENT_ID=
+DEALROOM_CLIENT_SECRET=
+
+# Optional. Defaults shown. Keys issued for the open beta host need
+# DEALROOM_API_BASE_URL=https://api.beta.dealroom.app
+DEALROOM_API_BASE_URL=
+DEALROOM_AUDIENCE=
+DEALROOM_TOKEN_URL=
+DEALROOM_API_VERSION=
+DEALROOM_REQUESTS_PER_SECOND=
+DEALROOM_TIMEOUT_MS=
+
+# Optional. Where the local SQLite cache lives (default ./sheeply.sqlite).
+SHEEPLY_DB=

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 40 for 40
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,156 @@
-# Temporary Holding Version
+# Sheeply
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Is your VC a black sheep or one of the flock?
+
+Every investor says they back founders early and lead with conviction. Sheeply checks. Give it a firm's name and it pulls the firm's portfolio and every funding round from [Dealroom](https://dealroom.co), works out how often the firm leads or goes in first rather than following someone else, and draws you a sheep.
+
+```text
+$ sheeply "Example Ventures"
+Investor: Example Ventures
+          .-~~-.-~~-.-~~-.
+  ,--.  .( @ @ @@ @ @ @@ ).
+ ( o  \(@@ @ @@ @ @ @@ @ @ )
+  \   /(@ @ @ @@ @ @ @@ @ @)
+   `-'  `(@ @@ @ @ @@ @ @)'
+          `-~~-'-~~-'-~~-'
+            ||        ||
+            ^^        ^^
+42% black sheep: Grey area. Shear indecision.
+```
+
+The fuller the fleece, the more the firm follows the flock. An empty outline is a true black sheep.
+
+Built as dry run #1 for [40 for 40](https://build40for40.com). It runs on your machine: no server, no accounts, no AI guessing.
+
+## What the score means
+
+The black sheep score runs from 0 to 100 and asks two questions about each firm's first investment in a company since 2020:
+
+1. **Did it lead?** Was it the round's lead investor, or the only investor, rather than joining a round someone else led?
+2. **Did it go in first?** Was this the company's first VC round, rather than joining a company that had already raised behind someone else's lead?
+
+Each answer becomes a rank against a peer set of 31 UK and European early-stage investors, and the score is the average of the two ranks. So 50 means "about as independent as a typical early-stage peer", and 100 means "more independent than every one of them".
+
+| Score | Label |
+| --- | --- |
+| 0 to 15 | Pure white sheep. Baaa-sically the whole flock. |
+| 16 to 35 | Follows the flock |
+| 36 to 55 | Grey area. Shear indecision. |
+| 56 to 75 | Ewe-nique taste |
+| 76 to 90 | Ram-pant contrarian |
+| 91 to 100 | True Blaaaaack Sheep |
+
+**Take it with a pinch of salt.** It only sees announced rounds in Dealroom. It can't see who committed first, how fast a firm moved or whether it asked "who's leading?" before saying yes. Dealroom also leaves gaps: some rounds name no lead at all, and some list only one investor because nobody recorded the others. The JSON report gives a range for every score (firms whose ranges overlap shouldn't be called different), plus the counts behind it. A firm with too little recent data, or one that mostly invests after seed, gets an explanation instead of a score.
+
+## Getting started
+
+You need Node.js 22.13 or later and a Dealroom API key.
+
+```bash
+git clone https://github.com/build40for40/sheeply.git
+cd sheeply
+npm install
+npm run build
+cp .env.example .env    # then add your Dealroom key, see below
+node dist/cli/index.js "Seedcamp"
+```
+
+Once it's published to npm, this becomes `npx sheeply "Seedcamp"`.
+
+### Your Dealroom key
+
+A Dealroom API key is a client id and a secret, created under API settings in Dealroom. Put both in `.env` in the folder you run Sheeply from:
+
+```bash
+DEALROOM_CLIENT_ID=your-client-id
+DEALROOM_CLIENT_SECRET=your-client-secret
+```
+
+Keys issued during Dealroom's open beta also need `DEALROOM_API_BASE_URL=https://api.beta.dealroom.app`.
+
+**It costs credits.** Dealroom charges about one credit per record. A firm with about 500 portfolio companies costs roughly 2,300 credits the first time. After that the data is cached, and rerunning is free.
+
+## Using it
+
+```bash
+sheeply "Seedcamp"                  # draw the sheep (fetches the firm the first time)
+sheeply "Seedcamp" --format json    # the full report, with every number behind the score
+sheeply "Seedcamp" --offline        # use only what's cached, never call Dealroom
+sheeply "Seedcamp" --refresh        # fetch fresh data from Dealroom
+```
+
+**If a name matches more than one firm,** Sheeply asks which one you mean. In a script, or with `--no-interactive`, it stops and lists the candidates with their ids instead. Pass one back with `--investor-id <id>`.
+
+All the options:
+
+| Option | What it does |
+| --- | --- |
+| `--format sheep` or `json` | The sheep (default), or the full JSON report |
+| `--verbose` | Adds every company and round behind the numbers to the JSON |
+| `--output <file>` | Writes the output to a file instead of the screen |
+| `--investor-id <id>` | Skips name matching and uses an exact Dealroom id |
+| `--offline` | Uses only cached data, never calls Dealroom |
+| `--refresh` | Fetches the firm again even if it's cached |
+| `--db <file>` | Uses a different cache file (default `./sheeply.sqlite`) |
+| `--no-interactive` | Fails with the candidates instead of asking which firm you meant |
+
+Two housekeeping commands: `sheeply db status` lists what's cached, and `sheeply db rebuild` rebuilds the cache's tables from the stored Dealroom responses without calling the API.
+
+Messages go to stderr and the result to stdout, so piping works: `sheeply "Seedcamp" --format json | jq .black_sheep_score`. Exit code `2` means a usage problem, an ambiguous name or not enough data; `3` means Dealroom returned an error.
+
+## The full report
+
+`--format json` gives you everything the score is built from, and more:
+
+- **Black sheep score:** the score, its range, both ranks and the raw counts behind them.
+- **Entry maturity:** the stage of each first investment (pre-seed, seed, Series A, Series B+ or unknown), plus company age and how much the company had raised before.
+- **Investment leadership:** how often the firm is recorded as lead, and how complete Dealroom's lead data is for that firm.
+- **Sector exposure:** which industries the firm's companies are in.
+- **Syndicate relationships:** who led the rounds the firm joined, and who it keeps investing alongside.
+- **Data quality:** what was missing, what was left out and why.
+- **Interpretation:** plain-English statements, each with the rule and threshold behind it.
+
+Every percentage comes with its count and denominator. The same cached data always gives exactly the same report, so results can be checked and compared.
+
+How each number is calculated is in [docs/methodology.md](docs/methodology.md). Which Dealroom field feeds what is in [docs/dealroom-field-mapping.md](docs/dealroom-field-mapping.md), and known gaps in Dealroom's API are in [docs/api-limitations.md](docs/api-limitations.md).
+
+## Your data stays yours
+
+Everything Sheeply fetches lives in a SQLite file on your machine. Dealroom's data is licensed, so none of it is committed to this repo or shipped in the package. The only Dealroom-derived numbers in the package are the anonymised peer figures the score ranks against: two sorted lists of shares with no names attached.
+
+## How it works
+
+```text
+Dealroom API ──► raw responses, stored verbatim ──► tables ──► analysis ──► sheep or JSON
+                 (sheeply.sqlite)                   rebuilt     pure functions,
+                                                    any time    no network
+```
+
+The cached file is a convenience, not the source of truth: delete it and Sheeply fetches everything again. The peer figures come from an internal script, `npm run calibrate`, which fetches the peer list in `scripts/peers.ts` and regenerates `src/analysis/black-sheep-calibration.ts`. Users never need to run it.
+
+```text
+src/
+  cli/        the commands
+  dealroom/   API client: login, retries, rate limits, schema checks, paging
+  db/         the SQLite cache
+  ingest/     finding the firm, fetching it and turning responses into tables
+  analysis/   the metrics, the black sheep score and its peer figures
+  reports/    the JSON report and the sheep
+scripts/      peer list and calibration (internal)
+tests/        fictional Dealroom data, an in-memory fake API, CLI tests and an optional live check
+```
+
+## Development
+
+```bash
+npm run dev -- "Seedcamp"   # run from source
+npm test                    # all tests, no Dealroom key needed
+npm run typecheck
+npm run test:smoke          # a quick live check, about a dozen credits, skipped without a key
+```
+
+Unit tests sit next to the file they cover (`leadership.ts` and `leadership.test.ts`). Test data is made up: never commit real Dealroom responses. All settings, including rate limits and timeouts, are read from environment variables in `src/config.ts`.
+
+## Licence
+
+MIT

package/dist/analysis/analyse.js

@@ -0,0 +1,28 @@
+import { dataQuality } from "./data-quality.js";
+import { entryMaturity } from "./entry-maturity.js";
+import { interpret } from "./interpretation.js";
+import { buildInvestmentHistory } from "./investments.js";
+import { investmentLeadership } from "./leadership.js";
+import { sectorExposure } from "./sector-exposure.js";
+import { syndicates } from "./syndicates.js";
+/**
+ * Two metrics and two descriptive outputs, each computed independently from the same investment
+ * history. The black sheep score is not one of them: it ranks an investor against stored peers, so
+ * it lives in `black-sheep.ts` and is computed by the caller that has the peer set.
+ */
+export function analyse(dataset) {
+    const history = buildInvestmentHistory(dataset);
+    const entry = entryMaturity(history);
+    const leadership = investmentLeadership(history);
+    const sectors = sectorExposure(history, dataset.companies);
+    const quality = dataQuality(dataset, history, { entry, leadership, sectors });
+    return {
+        history,
+        entryMaturity: entry,
+        investmentLeadership: leadership,
+        sectorExposure: sectors,
+        syndicateRelationships: syndicates(history),
+        dataQuality: quality,
+        interpretation: interpret(entry, leadership, quality),
+    };
+}

package/dist/analysis/black-sheep-calibration.js

@@ -0,0 +1,7 @@
+/** Generated by `npm run calibrate` from the peer set in scripts/peers.ts. Shares only, sorted, with no peer names or ids. */
+export const CALIBRATION = {
+    peer_set_version: "2026-10-28",
+    algorithm_version: "0.3.0",
+    leads: [0.031151, 0.050179, 0.08, 0.115702, 0.128205, 0.158416, 0.177778, 0.191489, 0.193548, 0.25, 0.265306, 0.291667, 0.303571, 0.304348, 0.310345, 0.313725, 0.315068, 0.333333, 0.393258, 0.416667, 0.435714, 0.459459, 0.47619, 0.47619, 0.5, 0.533333, 0.544, 0.560976, 0.642857, 0.75, 0.827381],
+    independent_entry: [0.010277, 0.010384, 0.03277, 0.04977, 0.08284, 0.086124, 0.092593, 0.102041, 0.12987, 0.131494, 0.133333, 0.137456, 0.147959, 0.15625, 0.157396, 0.16124, 0.162955, 0.171429, 0.181409, 0.197802, 0.223529, 0.242424, 0.254464, 0.258906, 0.258912, 0.265187, 0.267492, 0.5],
+};

package/dist/analysis/black-sheep.js

@@ -0,0 +1,167 @@
+import { canonicalJson, sha256 } from "../util.js";
+import { identified } from "./investments.js";
+import { leadStatus } from "./leadership.js";
+import { rate } from "./ratio.js";
+export const BLACK_SHEEP = {
+    sinceYear: 2020,
+    minimumSample: 10,
+    minimumPeers: 10,
+    minimumEarlyStageShare: 0.5,
+    weights: { leads: 0.5, independent_entry: 0.5 },
+    resamples: 1000,
+    interval: 0.9,
+    seed: 1,
+};
+export function sheepProfile(history, entry, name) {
+    const d = entry.distribution;
+    const observations = identified(history)
+        .filter((company) => Number(company.initial.announcedDate?.slice(0, 4)) >= BLACK_SHEEP.sinceYear)
+        .map((company) => {
+        const round = company.initial;
+        const status = leadStatus(round, history.investorId);
+        const soleInvestor = round.participants.length === 1;
+        const context = company.entry;
+        return {
+            late: context.priorVcRounds > 0 ? true : context.uncertainVcRounds > 0 ? null : false,
+            led: soleInvestor || status === "sole_lead" || status === "co_lead" ? true : status === "not_led" ? false : null,
+            soleInvestor,
+        };
+    });
+    return {
+        investorId: history.investorId,
+        name,
+        earlyStageShare: entry.known_stage === 0 ? null : (d.pre_seed.count + d.seed.count) / entry.known_stage,
+        observations,
+    };
+}
+const share = (r) => (r.denominator === 0 ? null : r.numerator / r.denominator);
+function behaviour(observations) {
+    const usable = observations.filter((o) => o.led !== null);
+    const ordered = observations.filter((o) => o.late !== null);
+    const late = ordered.filter((o) => o.late);
+    const lateUsable = late.filter((o) => o.led !== null);
+    const follow = rate(usable.filter((o) => !o.led).length, usable.length);
+    const lateRate = rate(late.length, ordered.length);
+    const followWhenLate = rate(lateUsable.filter((o) => !o.led).length, lateUsable.length);
+    const lateShare = share(lateRate);
+    const followLateShare = share(followWhenLate);
+    return {
+        follow,
+        late: lateRate,
+        followWhenLate,
+        followShare: share(follow),
+        lateFollowShare: lateShare === null || followLateShare === null ? null : lateShare * followLateShare,
+    };
+}
+const SHARE_DECIMALS = 1_000_000;
+const roundShare = (value) => Math.round(value * SHARE_DECIMALS) / SHARE_DECIMALS;
+const isEarlyStage = (profile) => profile.earlyStageShare !== null && profile.earlyStageShare >= BLACK_SHEEP.minimumEarlyStageShare;
+/** Builds the peer distributions from stored peers, keeping only peers with enough early-stage data for each component. */
+export function calibrate(peers, peerSetVersion, algorithmVersion) {
+    const points = (pick, minimum) => peers
+        .filter(isEarlyStage)
+        .map((peer) => behaviour(peer.observations))
+        .filter(minimum)
+        .map((b) => roundShare(pick(b)))
+        .sort((a, b) => a - b);
+    const enoughLeads = (b) => b.follow.denominator >= BLACK_SHEEP.minimumSample;
+    return {
+        peer_set_version: peerSetVersion,
+        algorithm_version: algorithmVersion,
+        leads: points((b) => b.followShare, enoughLeads),
+        independent_entry: points((b) => b.lateFollowShare, (b) => enoughLeads(b) && b.followWhenLate.denominator >= BLACK_SHEEP.minimumSample),
+    };
+}
+/** Drops one share equal to the investor's own, so a peer is not ranked against itself. */
+function withoutSelf(shares, own) {
+    const index = own === null ? -1 : shares.indexOf(roundShare(own));
+    return index === -1 ? shares : [...shares.slice(0, index), ...shares.slice(index + 1)];
+}
+/** Share of peers below `value`, ties counted half, so 100 means above every peer. */
+const percentile = (value, peers) => (100 * (peers.filter((p) => p < value).length + peers.filter((p) => p === value).length / 2)) / peers.length;
+function mulberry32(seed) {
+    let state = seed;
+    return () => {
+        state = (state + 0x6d2b79f5) | 0;
+        let t = Math.imul(state ^ (state >>> 15), 1 | state);
+        t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
+        return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+    };
+}
+const quantile = (sorted, q) => sorted[Math.floor(q * (sorted.length - 1))];
+/**
+ * Ranks one investor against the calibrated early-stage peers on two behaviours: how rarely it follows another
+ * lead, and how rarely it joins an already funded company behind someone else's lead. Each is a
+ * percentile among peers, inverted so that higher is more contrarian, and the score averages them.
+ */
+export function blackSheepScore(target, calibration) {
+    const p = BLACK_SHEEP;
+    const own = behaviour(target.observations);
+    const followPeers = withoutSelf(calibration.leads, own.followShare);
+    const latePeers = withoutSelf(calibration.independent_entry, own.lateFollowShare);
+    const reasons = [];
+    if (!isEarlyStage(target)) {
+        reasons.push("Most first investments with a known stage are after seed, so the early-stage peer set does not apply.");
+    }
+    if (own.follow.denominator < p.minimumSample) {
+        reasons.push(`Only ${own.follow.denominator} first investments since ${p.sinceYear} have a usable lead status (minimum ${p.minimumSample}).`);
+    }
+    if (own.followWhenLate.denominator < p.minimumSample) {
+        reasons.push(`Only ${own.followWhenLate.denominator} first investments since ${p.sinceYear} joined an already funded company with a usable lead status (minimum ${p.minimumSample}).`);
+    }
+    if (followPeers.length < p.minimumPeers || latePeers.length < p.minimumPeers) {
+        reasons.push(`The calibration has too few peers (minimum ${p.minimumPeers}).`);
+    }
+    const scoreOf = (b) => {
+        const leads = 100 - percentile(roundShare(b.followShare), followPeers);
+        const independent = 100 - percentile(roundShare(b.lateFollowShare), latePeers);
+        return { leads, independent, value: p.weights.leads * leads + p.weights.independent_entry * independent };
+    };
+    let scored = null;
+    let range = null;
+    if (reasons.length === 0) {
+        scored = scoreOf(own);
+        const random = mulberry32(p.seed);
+        const n = target.observations.length;
+        const values = [];
+        for (let i = 0; i < p.resamples; i++) {
+            const sample = Array.from({ length: n }, () => target.observations[Math.floor(random() * n)]);
+            const b = behaviour(sample);
+            if (b.followShare !== null && b.lateFollowShare !== null)
+                values.push(scoreOf(b).value);
+        }
+        values.sort((a, b) => a - b);
+        const tail = (1 - p.interval) / 2;
+        range = { low: Math.round(quantile(values, tail)), high: Math.round(quantile(values, 1 - tail)), interval: p.interval, resamples: values.length };
+    }
+    return {
+        role: "score",
+        question: "How often does this investor lead or go in first, rather than follow others, compared with early-stage peers?",
+        scale: "0 follows the flock the most among peers, 100 is the most contrarian",
+        status: scored ? "scored" : "not_scored",
+        reasons,
+        value: scored && Math.round(scored.value),
+        range,
+        components: {
+            leads: {
+                score: scored && Math.round(scored.leads),
+                follows_others: own.follow,
+                sole_investor: target.observations.filter((o) => o.soleInvestor).length,
+                peers: followPeers.length,
+            },
+            independent_entry: {
+                score: scored && Math.round(scored.independent),
+                late_entry: own.late,
+                follows_others_when_late: own.followWhenLate,
+                late_follow_share: own.lateFollowShare === null ? null : Math.round(own.lateFollowShare * 10_000) / 10_000,
+                peers: latePeers.length,
+            },
+        },
+        calibration: {
+            peer_set_version: calibration.peer_set_version,
+            algorithm_version: calibration.algorithm_version,
+            fingerprint: `sha256:${sha256(canonicalJson(calibration))}`,
+        },
+        parameters: BLACK_SHEEP,
+    };
+}

package/dist/analysis/data-quality.js

@@ -0,0 +1,90 @@
+import { identified } from "./investments.js";
+import { rate } from "./ratio.js";
+import { stageBucket } from "./stages.js";
+const STATUSES = [
+    "initial_identified",
+    "no_round_attribution",
+    "rounds_not_matched",
+    "participation_undisclosed",
+    "not_a_company",
+];
+const FLAGS = [
+    "initial_ordering_ambiguous",
+    "undated_participation",
+    "initial_undated",
+    "earlier_round_investors_undisclosed",
+];
+export function dataQuality(dataset, history, outputs) {
+    const byStatus = Object.fromEntries(STATUSES.map((s) => [s, 0]));
+    for (const company of history.companies)
+        byStatus[company.status]++;
+    const flags = Object.fromEntries(FLAGS.map((f) => [f, 0]));
+    for (const company of identified(history))
+        for (const flag of company.flags)
+            flags[flag]++;
+    const companies = history.companies.filter((c) => c.status !== "not_a_company");
+    const reported = dataset.investor.reportedPortfolioTotal;
+    const { entry, leadership, sectors } = outputs;
+    const initial = leadership.initial_investments;
+    const roundsById = new Map(dataset.rounds.map((round) => [round.id, round]));
+    const lateStart = identified(history).filter((company) => {
+        const firstVc = company.entry.companyFirstVcRoundId;
+        const bucket = firstVc === null ? null : stageBucket(roundsById.get(firstVc)?.roundLabel ?? null);
+        return bucket === "series_a" || bucket === "series_b_plus";
+    }).length;
+    const withParticipants = dataset.rounds.filter((r) => r.participants.length > 0);
+    const participants = dataset.rounds.flatMap((r) => r.participants);
+    const exclusions = [
+        { output: "all", reason: "portfolio row is not a company", count: byStatus.not_a_company },
+        { output: "all", reason: "no round attribution in Dealroom (rounds_count 0)", count: byStatus.no_round_attribution },
+        { output: "all", reason: "portfolio claims rounds but none of the returned rounds lists the investor", count: byStatus.rounds_not_matched },
+        { output: "all", reason: "investor not found and some company rounds have no investor list", count: byStatus.participation_undisclosed },
+        { output: "entry_maturity", reason: "first investment has no Dealroom round_label (counted as unknown)", count: sum(entry.unknown_breakdown.no_round_label_by_standardized_round) },
+        { output: "entry_maturity", reason: "first investment has a round_label outside the mapped ladder (counted as unknown)", count: sum(entry.unknown_breakdown.unmapped_round_labels) },
+        { output: "investment_leadership", reason: "is_lead false but nobody on the first-investment round is flagged as lead", count: initial.false_with_no_lead_on_round },
+        { output: "investment_leadership", reason: "is_lead null on the first-investment round", count: initial.null_lead_flag },
+        { output: "sector_exposure", reason: "taxonomy not returned for company", count: sectors.taxonomy_not_returned },
+        { output: "sector_exposure", reason: "company has no industry label", count: sectors.without_industry },
+    ];
+    return {
+        portfolio: {
+            rows_returned: dataset.portfolio.length,
+            reported_by_dealroom: reported,
+            complete: reported === null ? null : dataset.portfolio.length >= reported,
+            by_status: byStatus,
+            rounds_count_mismatch: identified(history).filter((c) => c.participations.length !== c.roundsCountReported).length,
+        },
+        rounds: {
+            returned: dataset.rounds.length,
+            reported_by_dealroom: dataset.provenance.transactionsReportedTotal,
+            undated: dataset.rounds.filter((r) => r.announcedDate === null).length,
+            year_precision_only: dataset.rounds.filter((r) => r.announcedDate?.length === 4).length,
+            investors_not_disclosed: dataset.rounds.filter((r) => !r.investorsDisclosed).length,
+            without_round_label: dataset.rounds.filter((r) => r.roundLabel === null).length,
+            outside_portfolio_listing: history.roundsOutsidePortfolio.length,
+        },
+        funding_histories: {
+            companies_without_any_round: companies.filter((c) => c.companyRoundsObserved === 0).length,
+            history_starts_at_series_a_or_later: lateStart,
+            first_investment_flags: flags,
+        },
+        lead_flags: {
+            participant_flags: {
+                true: participants.filter((p) => p.isLead === true).length,
+                false: participants.filter((p) => p.isLead === false).length,
+                null: participants.filter((p) => p.isLead === null).length,
+            },
+            rounds_with_participants: withParticipants.length,
+            rounds_with_a_flagged_lead: rate(withParticipants.filter((r) => r.participants.some((p) => p.isLead === true)).length, withParticipants.length),
+            single_participant_rounds_flagged_false: withParticipants.filter((r) => r.participants.length === 1 && r.participants[0].isLead === false).length,
+        },
+        coverage: {
+            first_investment_identification: rate(byStatus.initial_identified, companies.length),
+            entry_stage: rate(entry.known_stage, entry.first_investments),
+            lead_status: initial.lead_status_completeness,
+            sector: rate(sectors.labelled, sectors.first_investments),
+        },
+        exclusions: exclusions.filter((e) => e.count > 0),
+    };
+}
+const sum = (record) => Object.values(record).reduce((a, b) => a + b, 0);

package/dist/analysis/dataset.js

@@ -0,0 +1,126 @@
+import { compareStrings } from "../util.js";
+export class NoStoredDataError extends Error {
+    name = "NoStoredDataError";
+}
+const str = (value) => (value == null ? null : String(value));
+const flag = (value) => (value == null ? null : Number(value) === 1);
+export function latestCompleteRun(db, investorId) {
+    const row = db.prepare("SELECT run_id FROM extraction_state WHERE investor_id = ?").get(investorId);
+    return row ? Number(row.run_id) : null;
+}
+/** Reads one investor's analysable dataset from the derived tables, in a fixed order. */
+export function loadDataset(db, investorId) {
+    const runId = latestCompleteRun(db, investorId);
+    const investorRow = db.prepare("SELECT * FROM investors WHERE uuid = ?").get(investorId);
+    if (runId === null || !investorRow) {
+        throw new NoStoredDataError(`No complete ingestion is stored for investor ${investorId}.`);
+    }
+    const run = db.prepare("SELECT * FROM ingestion_runs WHERE id = ?").get(runId);
+    const responses = db
+        .prepare(`SELECT count(*) AS n, min(retrieved_at) AS first, max(retrieved_at) AS last FROM raw_responses WHERE run_id = ?`)
+        .get(runId);
+    const firstTransactions = db
+        .prepare("SELECT body_json FROM raw_responses WHERE run_id = ? AND kind = 'transactions' ORDER BY page_index LIMIT 1")
+        .get(runId);
+    const transactionsTotal = firstTransactions
+        ? JSON.parse(String(firstTransactions.body_json)).page?.total ?? null
+        : null;
+    const portfolio = db
+        .prepare(`SELECT p.company_id, c.name, p.company_subtype, p.rounds_count, p.is_lead_any, p.last_round_date
+       FROM portfolio_relationships p JOIN companies c ON c.uuid = p.company_id
+       WHERE p.investor_id = ? ORDER BY p.company_id`)
+        .all(investorId)
+        .map((row) => ({
+        companyId: String(row.company_id),
+        companyName: String(row.name),
+        companySubtype: str(row.company_subtype),
+        roundsCount: Number(row.rounds_count),
+        isLeadAny: Number(row.is_lead_any) === 1,
+        lastRoundDate: str(row.last_round_date),
+    }));
+    const roundRows = db
+        .prepare(`SELECT r.* FROM investor_round_scope s JOIN funding_rounds r ON r.id = s.round_id
+       WHERE s.investor_id = ? ORDER BY r.id`)
+        .all(investorId);
+    const participantRows = db
+        .prepare(`SELECT rp.round_id, rp.entity_id, rp.is_lead, e.name, e.type, e.subtype
+       FROM investor_round_scope s
+       JOIN round_participants rp ON rp.round_id = s.round_id
+       JOIN entities e ON e.uuid = rp.entity_id
+       WHERE s.investor_id = ? ORDER BY rp.round_id, rp.position, rp.entity_id`)
+        .all(investorId);
+    const participantsByRound = new Map();
+    for (const row of participantRows) {
+        const list = participantsByRound.get(Number(row.round_id)) ?? [];
+        list.push({
+            id: String(row.entity_id),
+            name: String(row.name),
+            type: String(row.type),
+            subtype: str(row.subtype),
+            isLead: flag(row.is_lead),
+        });
+        participantsByRound.set(Number(row.round_id), list);
+    }
+    const rounds = roundRows.map((row) => ({
+        id: Number(row.id),
+        companyId: String(row.company_id),
+        announcedDate: str(row.announced_date),
+        roundLabel: str(row.round_label),
+        standardizedRound: str(row.standardized_round),
+        roundType: str(row.round_type),
+        amount: row.amount == null ? null : Number(row.amount),
+        currency: str(row.currency),
+        isVcRound: flag(row.is_vc_round),
+        investorsDisclosed: Number(row.investors_disclosed) === 1,
+        participants: participantsByRound.get(Number(row.id)) ?? [],
+    }));
+    const companyIds = [...new Set([...portfolio.map((p) => p.companyId), ...rounds.map((r) => r.companyId)])].sort(compareStrings);
+    const companies = {};
+    const companyStatement = db.prepare("SELECT * FROM companies WHERE uuid = ?");
+    const taxonomyStatement = db.prepare(`SELECT t.id, t.name, t.type FROM company_taxonomy ct JOIN taxonomy_terms t ON t.id = ct.term_id
+     WHERE ct.company_id = ? ORDER BY t.type, t.id`);
+    for (const id of companyIds) {
+        const row = companyStatement.get(id);
+        if (!row)
+            continue;
+        companies[id] = {
+            id,
+            name: String(row.name),
+            launchDate: str(row.launch_date),
+            taxonomyKnown: Number(row.taxonomy_known) === 1,
+            taxonomy: taxonomyStatement.all(id).map((term) => ({ id: Number(term.id), name: String(term.name), type: String(term.type) })),
+        };
+    }
+    return {
+        investor: {
+            id: investorId,
+            name: String(investorRow.name),
+            type: String(investorRow.type),
+            subtype: str(investorRow.subtype),
+            firmTypes: JSON.parse(String(investorRow.firm_types_json)),
+            launchDate: str(investorRow.launch_date),
+            hqCountry: str(investorRow.hq_country),
+            dealroomUrl: str(investorRow.dealroom_url),
+            reportedPortfolioTotal: investorRow.reported_portfolio_total == null ? null : Number(investorRow.reported_portfolio_total),
+            reportedRoundsCount: investorRow.reported_rounds_count == null ? null : Number(investorRow.reported_rounds_count),
+        },
+        portfolio,
+        companies,
+        rounds,
+        provenance: {
+            runId,
+            startedAt: String(run.started_at),
+            finishedAt: String(run.finished_at),
+            apiBaseUrl: String(run.api_base_url),
+            apiVersion: String(run.api_version),
+            roundsStrategy: String(run.rounds_strategy),
+            extractionVersion: String(db.prepare("SELECT extraction_version FROM extraction_state WHERE investor_id = ?").get(investorId).extraction_version),
+            responsesStored: Number(responses.n),
+            firstRetrievedAt: String(responses.first),
+            lastRetrievedAt: String(responses.last),
+            requests: run.requests == null ? null : Number(run.requests),
+            creditsUsed: run.credits_used == null ? null : Number(run.credits_used),
+            transactionsReportedTotal: transactionsTotal,
+        },
+    };
+}