gasp-standard-mcp 1.16.0 → 1.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/README.md +25 -5
  2. package/build/cel.d.ts +138 -0
  3. package/build/cel.js +376 -0
  4. package/build/checks.d.ts +173 -0
  5. package/build/checks.js +465 -0
  6. package/build/compile.d.ts +4 -0
  7. package/build/compile.js +6 -1
  8. package/build/derive.d.ts +8 -0
  9. package/build/derive.js +11 -2
  10. package/build/dialect.d.ts +5 -2
  11. package/build/dialect.js +6 -1
  12. package/build/formula.d.ts +3 -0
  13. package/build/formula.js +3 -0
  14. package/build/index.js +115 -29
  15. package/build/loader.js +17 -1
  16. package/build/ossie/adapt.d.ts +96 -0
  17. package/build/ossie/adapt.js +363 -0
  18. package/build/ossie/check.d.ts +18 -4
  19. package/build/ossie/check.js +88 -18
  20. package/build/ossie/company.d.ts +55 -0
  21. package/build/ossie/company.js +215 -0
  22. package/build/ossie/conformance.d.ts +100 -0
  23. package/build/ossie/conformance.js +186 -0
  24. package/build/ossie/decide.d.ts +81 -0
  25. package/build/ossie/decide.js +532 -0
  26. package/build/ossie/demo.d.ts +62 -0
  27. package/build/ossie/demo.js +100 -0
  28. package/build/ossie/drift.d.ts +159 -0
  29. package/build/ossie/drift.js +650 -0
  30. package/build/ossie/example.d.ts +16 -0
  31. package/build/ossie/example.js +203 -0
  32. package/build/ossie/index.js +1 -1
  33. package/build/ossie/mapping.d.ts +26 -1
  34. package/build/ossie/mapping.js +45 -15
  35. package/build/ossie/metrics.d.ts +2 -0
  36. package/build/ossie/metrics.js +13 -10
  37. package/build/ossie/model.d.ts +39 -4
  38. package/build/ossie/model.js +56 -26
  39. package/build/ossie/names.d.ts +4 -0
  40. package/build/ossie/names.js +4 -0
  41. package/build/ossie/portability.d.ts +56 -0
  42. package/build/ossie/portability.js +109 -0
  43. package/build/ossie/profile.d.ts +157 -0
  44. package/build/ossie/profile.js +758 -0
  45. package/build/ossie/propose.d.ts +107 -0
  46. package/build/ossie/propose.js +454 -0
  47. package/build/ossie/types.d.ts +36 -7
  48. package/build/ossie/types.js +39 -1
  49. package/build/ranges.d.ts +20 -0
  50. package/build/ranges.js +51 -0
  51. package/build/readiness.d.ts +139 -0
  52. package/build/readiness.js +232 -0
  53. package/build/standard-data.json +1 -1
  54. package/build/standard-version.json +1 -1
  55. package/build/synthetic.d.ts +230 -0
  56. package/build/synthetic.js +1206 -0
  57. package/build/tools.d.ts +27 -1
  58. package/build/tools.js +10 -8
  59. package/build/types.d.ts +12 -0
  60. package/package.json +7 -3
package/README.md CHANGED
@@ -40,8 +40,17 @@ Restart your MCP client after adding the config.
40
40
  | `get_relationships` | One node of the relationship graph (metric, entity or concept) with its direct edges: upstream, measured by, downstream and measures. 21 relationship types across signal, lifecycle, ownership, interaction and financial flow categories. Metric names resolve as in `lookup_metric`, ambiguity included; a metric outside the graph says so. An entity also lists every metric whose formula reads its datasets (from `entityMappings`), with department and URL |
41
41
  | `get_data_requirements` | Data requirements for one metric, a department or every metric with a formula: per metric, each field and dataset pair with its type, the staged flag, source, grain, frequency, page URL and example SQL. The metric resolves exactly as in `lookup_metric`; `department` lists a department or picks between metrics sharing a name. Optional `breakdown` names a breakdown the metric declares (`by_plan` on MRR) and returns that breakdown's requirement: the dimension among the fields and example SQL with one row per value; an unknown name lists the declared ones. Optional `dialect` (`postgres` by default and for Redshift, `snowflake`, `bigquery`, `databricks`) writes the example SQL for that warehouse |
42
42
  | `search` | Whole-word phrase search across metrics (names, aliases, definitions, formulas, fields, categories, departments) and glossary terms (terms, definitions, sections); the last word may take a plural or verb ending. Up to 10 results, one per metric or term, ranked name, alias and term first, then words within them, then definitions, then formulas and fields, each with what matched and its gaspwiki.com URL. A query that matches nothing returns up to 6 close names as "Did you mean" |
43
- | `generate_query` | Compile warehouse SQL from the metric's canonical formula and an adapter. One scalar subquery per aggregate, each reading the table mapped for its dataset, with `:period_start` and `:period_end` date parameters (both inclusive). Unmapped pairs stay as `{{field@dataset}}` placeholders and are listed with any dataset lacking a table and any mapping conflict. Optional `breakdown` names a breakdown the metric declares (`by_plan` on MRR) and returns one row per value; an unknown name lists the declared ones. Optional `dialect` (`postgres`, `snowflake`, `bigquery`, `databricks`; `postgres` also covers Redshift) picks the SQL syntax for date arithmetic, casts and percentiles; without it the adapter's `warehouse` decides when it names one of them, else postgres. The output says which dialect it used |
44
- | `validate_adapter` | Validate a GASP adapter. Applies the bundled adapter JSON Schema (draft 2020-12, versions 1.0 and 1.1) first and lists each violation with its JSON path, then reports unknown fields and datasets, fields mapped to a dataset that does not carry them, bare column names on fields that live on several datasets, dataset table conflicts, enum fields mapped without a transform, covered metrics without an owner and owners naming unknown metrics. Scores every metric with a formula against the exact field and dataset pairs its compiled SQL reads (full, partial with the missing pairs or none) and ranks up to 15 unmapped pairs by how many partially covered metrics need them |
43
+ | `generate_query` | Compile warehouse SQL from the metric's canonical formula and a mapping (a GASP adapter, or an Ossie mapping with its `semantic_model`). One scalar subquery per aggregate, each reading the table mapped for its dataset, with `:period_start` and `:period_end` date parameters (both inclusive). Unmapped pairs stay as `{{field@dataset}}` placeholders and are listed with any dataset lacking a table and any mapping conflict. Optional `breakdown` names a breakdown the metric declares (`by_plan` on MRR) and returns one row per value; an unknown name lists the declared ones. Optional `dialect` (`postgres`, `snowflake`, `bigquery`, `databricks`; `postgres` also covers Redshift) picks the SQL syntax for date arithmetic, casts and percentiles; without it the adapter's `warehouse` decides when it names one of them, else postgres. The output says which dialect it used |
44
+ | `validate_mapping` | Validate a warehouse mapping in either format: a GASP adapter (JSON) or an Apache Ossie mapping document (YAML or JSON) over your semantic model, passed as `semantic_model`. A GASP adapter is converted on load into an Ossie mapping over a model synthesised from its tables and columns. Reports in layers: the schema (the adapter JSON Schema, and the bundled Ossie mapping schema for the Ossie form), GASP's cross-document checks against the GASP ontology and the model, then coverage: unknown fields and datasets, fields mapped to a dataset that does not carry them, table conflicts, every metric with a formula scored against the field and dataset pairs its compiled SQL reads (full, partial with the missing pairs or none), enum fields mapped without a transform, covered metrics without an owner, owners naming unknown metrics and up to 15 unmapped pairs ranked by how many partially covered metrics need them. `validate_adapter` is the same tool under its original name |
45
+ | `score_drift` | Score how far your own metrics drift from GASP's canonical definitions, by result. Takes your mapping (either format) and your `semantic_model` holding your metrics; each is matched to a GASP metric by exact name, by the synonyms in its `ai_context` or by a pair in `matches`, a name ending in -O or -M matching that declared form, with near names listed for review and never scored on a guess. The canonical metric is compiled into your names and both run on the GASP synthetic SaaS dataset for six monthly periods in an in-process Postgres (PGlite, an optional dependency). Differences come with both values per period and the likely cause read from the canonical formula: a null on one side (a sum over no rows is NULL in your SQL and 0 in GASP's), a cohort or opening value the expression cannot state, a filter one side applies, a dataset one side reads, an aggregate with no equal on the other side. Your values are checked against the standard's range for the metric per period, and where both forms of a dual-lens metric are matched the O/M bridge (the Durability Gap for ARR, the Activation Delta for CAC and the others the Core metrics name) is scored beside the canonical gap. `output: json` returns the data |
46
+ | `profile_schema` | Profile your warehouse schema into a minimal Ossie semantic model, the first step towards a mapping. Pass the rows of `information_schema.columns` (JSON or CSV), a tables list or `CREATE TABLE` statements, read through your own warehouse MCP or from a dump, and optionally statistics per column (`output: statistics` returns the query that gathers them). Every table gets a key, a time field and a kind (snapshot, event, fact or reference) with the evidence and a confidence, plus the relationships the columns imply; what the names cannot settle is a question for you rather than a guess. The model is returned as YAML to pass on as `semantic_model` |
47
+ | `propose_mapping` | Propose a GASP mapping from the model `profile_schema` returned (or your own Ossie model, or a schema profiled on the way in). Every table is scored against the GASP datasets on its columns, its profiled kind, key and time field and its name; every column of a proposed table against the dataset's fields by name, structural role, a short synonym list, word order and near names, checked against the type and the listed values. A column in cents gets a divide-by-100 transform and enum values the standard does not use a `CASE` recode when one is known. High-confidence pairs become an Ossie mapping over the model (a transform a calculated field) for `validate_mapping`; medium ones are candidates with their evidence and what would settle them; `output: adapter` gives the pairs as a GASP adapter |
48
+ | `derive_cel` | SQL that derives CEL `Revenue_Event` rows from your billing data by the rules in `cel.md`. `source: subscription` (default): the subscription dataset your mapping covers (snapshots at any cadence or a version history) becomes one event per movement, with the customer's `prior_mrr` and `new_mrr` chained so the row identity holds, the event type read from the customer's MRR around the step, the ATL lifecycle tag and an empty `expansion_classification` the ledger cannot fill. `source: invoice`: the invoice dataset read as one row per service period (`recurring_amount` over the months between `service_period_start` and `service_period_end`), for a billing system that keeps no subscription history. When the mapping covers the `usage_record` dataset (or `usage: true`), each period whose `usage_amount` exceeds its `committed_amount` adds an `expansion` tagged `usage_realisation` with `change_nature` `temporary` on the period's end and its `scheduled_reversal` the next day, chained with the ledger's steps. Same mapping input as `generate_query`, plus the ledger's currency; dialect from the request or the warehouse; unmapped fields listed |
49
+ | `generate_checks` | SQL for the readiness checks the standard states, over the datasets your mapping covers: CEL row identity, MRR continuity, unique event ids, the ATL tags (lifecycle, expansion classification, cost function), customer identity both ways, freshness per dataset and, for every fully mapped metric with a hard range rule in the ontology (NRR at least 0, GRR at most 100%), its value in every reporting period of your period spine (`period_spine`, the table `compile_model` reads), one row per check. Paste the result rows back as `results` for pass or fail per check and per metric, each check naming the metrics it protects. Same mapping input as `generate_query`; dialect from the request or the warehouse |
50
+ | `plan_fixes` | The ranked fix list: every unmapped field and every failing check, ordered by Core metrics unlocked, then all metrics unlocked, with an effort class shown and used only as a tiebreak. A fix unlocks a metric when it is the last thing between that metric and covered-and-passing. Takes the mapping and the `results` from `generate_checks` |
51
+ | `readiness_report` | Which readiness level a company has reached and what stands between it and the next, read from the documents the other tools wrote: L1 Profiled (a semantic model exists), L2 Mapped (a metric fully covered, no open question or candidate), L3 Clean core (the CEL revenue events mapped, the checks run and every covered Core metric passing every check that protects it), L4 LLM-ready (the canonical metrics compiled into the model and none of your own metrics differing from its canonical counterpart). Takes the mapping, the `semantic_model` (with an adapter, the one `compile_model` returned) and the `results` from `generate_checks`; each blocker names the tool that clears it and the Next line is the one action to take
52
+ | `record_decision` | Record the decisions people make on the proposed mapping, each named by content with its owner and date: a profiler question answered (key, time field, kind, datatype) is written into the model; a candidate accepted or rejected into the mapping (an accepted table matched column by column, an accepted column with an optional transform as a calculated field); a metric owner into `metricOwners`; a judgement call GASP cannot apply (reporting currency, churn date) recorded. All land in the mapping's GASP extension; a later decision on a subject replaces the earlier one. With no decisions it lists the open questions, candidates and unowned metrics and whether readiness level 2 is reached |
53
+ | `compile_model` | Compile the GASP metrics your mapping covers into your Apache Ossie semantic model. Same input as `validate_mapping`; returns your model with a `gasp_` metric per covered formula entry (main formula, O and M forms, variants, named breakdowns): a direct `OSSIE_SQL_2026` expression over your datasets and fields where the language can state it, else staged through a `gasp_staged_` per-period dataset whose `source` is the compiled SQL over your tables in your warehouse `dialect`, reading the `period_spine` table you name. Entries with unmapped pairs are listed with the pairs they need; recompiling replaces the `gasp_` additions and leaves the rest. Optional `metrics` and `department` compile a subset; `output: report` returns the coverage without the document |
45
54
  | `infer_drivers` | Diagnose what could explain an observed metric movement: up to 25 upstream candidates ranked by attenuation score, each with its implied direction and path. Direction flips across `reduces` edges and negative-polarity inputs; a baseline input (MRR read at period start by NRR) is listed but not walked through. Metric nodes only; `maxDepth` 1 to 8 (default 6) |
46
55
  | `explain_path` | Up to 5 forward edge chains from one graph node to another, shortest first, with edge type, classification (metric_signal / structural / lifecycle / behavioral / assignment / generative), attenuation and direction flips per step and the target's end direction per path. A baseline edge can only start a path |
47
56
  | `coverage_gap` | Audit a metric set: metrics with a direct edge into or out of it that it leaves out, plus graph metrics with no path to or from it. Inputs that are ambiguous, outside the graph or not metrics are listed as unresolved with the reason |
@@ -50,7 +59,7 @@ The traversal tools (`infer_drivers`, `explain_path`, `coverage_gap`) return pat
50
59
 
51
60
  ## Data
52
61
 
53
- Version 1.16.0.
62
+ Version 1.18.0.
54
63
 
55
64
  - 306 metrics across 13 departments
56
65
  - 94 glossary definitions (81 SaaS domain + 13 GASP framework)
@@ -58,18 +67,29 @@ Version 1.16.0.
58
67
  - 21 relationship types: metric signal (drives, reduces, predicts, component, measures), entity lifecycle (converts_to, parent_of, belongs_to), entity ownership (owns, grants, priced_by), entity interactions (manages, serves, engages_via, submits, represented_by, sources), entity financial flows (generates, produces, contributes_to, incurs)
59
68
  - 279 data requirements, one per metric with a formula, compiled from the formula at load time: fields with types, datasets, grain and a compiled example query
60
69
  - 279 query templates, one per metric with a formula, compiled by the same step. None of the SQL is hand-written. Every field placeholder names the dataset it is read from (`{{customer_status@customer}}`), so `generate_query` resolves per-dataset adapter mappings by tag, inside cohort subqueries too. The grammar's `window(expr, n unit)` gives Daily, Weekly and Monthly Active Users (DAU, WAU, MAU) their 1, 7 and 30 day windows. `all_time(expr)` lets retention numerators read past the period end. A multi-argument `count_distinct` counts distinct tuples (Breadth of Adoption counts account and feature pairs). A formula with a `breakdown` field compiles to one row per value of that field, with `total(expr)` reading the whole population for shares of total (Ticket Distribution by Type, CAC by Channel)
61
- - The test suite executes every compiled template against a seeded fixture in an in-process Postgres (PGlite) and checks 25 metrics against independently computed values
70
+ - The test suite executes every compiled template against a seeded fixture in an in-process Postgres (PGlite) and checks selected metrics against independently computed values
62
71
  - Apache Ossie (incubating) export, written by `npm run derive` to `standard/exports/ossie/` and served at https://gaspwiki.com/downloads/ossie/: `gasp.ontology.yaml` (every dataset and GASP concept as an entity type, every field as a value type or relationship, enum values as constraints), `gasp.semantic_model.yaml` (every dataset with `OSSIE_SQL_2026` field expressions, the foreign-key relationships between them and every canonical formula as a metric, either a direct expression or staged through a per-period dataset that carries the compiled SQL; O/M forms and variants are separate metrics with the form in `custom_extensions`), `gasp.mapping.yaml` (the ontology mapped onto the model) and `coverage.json` (how each formula was exported and why). All three documents pass the upstream validator (vendored at `ossie-validator/`) and GASP's own cross-reference checks in `src/ossie/check.ts`, which cover references the upstream validator leaves alone
63
72
  - 28 business entities (Customer, Subscription, License, Lead, Opportunity, User, Ticket, Employee, Invoice, Partner, Project, Plan, Product, Campaign, Contact, Incident, Deployment, Release, Issue, Vulnerability, Feature, Requisition, Project Milestone, Deliverable, Quote, Order, Knowledge Article and Deal Registration). Every dataset with its own identity maps to an entity; ledger and period-level facts (cost events, financials, forecasts, web traffic) stay entity-free
64
73
  - 7 business concepts (Revenue, Bookings, COGS, Cohort, Pipeline, Segment and TCV)
65
- - 268 canonical fields across 52 datasets (event, snapshot and fact tables with date and key fields, entities, CEL class and activation rules). Enum fields declare their allowed values
74
+ - 274 canonical fields across 53 datasets (event, snapshot and fact tables with date and key fields, entities, CEL class and activation rules). Enum fields declare their allowed values
66
75
  - 65 structured formulas with decomposed inputs (metric refs + field refs, with baseline inputs marked)
67
76
  - Canonical formulas for every metric in the GASP formula grammar (`standard/formulas.json`): 294 entries covering all 306 metrics (279 with formulas, 15 marked not computable with a reason, 2 department-qualified ids for shared headings), with units, O/M forms for ARR, NRR, CAC, LTV, LTV:CAC, CAC Payback and Gross Margin (a form can reference another metric's form, `metric.gross_margin.O`) and variants such as Adjusted NRR (served by `get_formula` and compiled by `generate_query` and `get_data_requirements`)
68
77
  - Access control classifications for every graph metric (sensitivity tier + access scope)
69
78
  - CEL `change_nature` field for revenue quality analysis (Adjusted NRR/GRR) and `segment` on revenue, contract and amendment events, so MRR, retention and churn metrics break down by customer segment
70
79
  - Per-department cross-references (NRR under Customer Success, CAC under Marketing) resolve to the full canonical record, with its access classification, structured formula and entity mappings
71
80
  - Every metric and definition result carries its gaspwiki.com page URL; definitions keep their full body and reference links
81
+ - One mapping format: a company's warehouse mapping is an Apache Ossie mapping document over its semantic model, and the v1 adapter is accepted everywhere and converted on load (`src/ossie/adapt.ts`). The upstream Ossie schemas travel in the bundled data as `ossieSchemas`, so `validate_mapping` checks an Ossie mapping offline
82
+ - The schema profiler (`src/ossie/profile.ts`, behind `profile_schema`) turns a schema dump into a minimal Ossie semantic model with its evidence; the Acme DDL in `test/fixture/acme-schema.sql` is its fixture.
83
+ - The mapping proposer (`src/ossie/propose.ts`, behind `propose_mapping`) turns that model into a draft GASP mapping with the evidence per pair, transforms where the name or the values call for one and candidates for the pairs the evidence does not settle; on the Acme DDL it reproduces the Acme adapter.
84
+ - The CEL derivations (`src/cel.ts`, behind `derive_cel`) are the standard's rules for turning a subscription ledger, an invoice ledger or usage records into `Revenue_Event` rows, compiled over a company's mapping in its dialect. The test suite runs them over the synthetic dataset: every generated movement comes back from the subscription snapshots with monthly sums equal to the compiled MRR movement metrics, the monthly invoices give the month-start movements with a lapsed subscription churning the day after its last period, and the usage records give a tagged pair per month over the commitment, chained with the ledger's steps.
85
+ - Two vendor worked examples beside Acme in `test/fixture/` (`stripe/`, Redshift; `salesforce/`, BigQuery): a DDL, a GASP mapping and a README stating the source documentation and every choice, written from public API documentation and never standard content. The Stripe example is taken from a validated mapping to readiness level 4 in the test suite, derives from its item snapshots with a metered item's usage records and from its invoices, and its drift report names a planted difference; the Salesforce example compiles in BigQuery and stops at level 2 because a CRM maps no revenue event
86
+ - The readiness checks and the fix plan (`src/checks.ts`, behind `generate_checks` and `plan_fixes`) compile the rules in `cel.md` and `atl.md`, and the ontology's value ranges per reporting period (`src/ranges.ts`), into SQL over a mapping and rank what to fix by the metrics it unlocks; the synthetic dataset passes every check and a table of planted violations fails the right ones.
87
+ - The readiness report (`src/readiness.ts`, behind `readiness_report`) reads the level a company has reached (L1 to L4 of the interop plan) from the mapping, the model and the pasted check rows, and names what blocks the next level; the test suite takes the synthetic dataset to L4.
88
+ - The Decide step (`src/ossie/decide.ts`, behind `record_decision`) applies the answers people give to the model and the mapping and keeps every decision, with its owner, in the mapping's GASP extension; it closes readiness level 2 when no question and no candidate is open.
89
+ - The drift scorer (`src/ossie/drift.ts`, behind `score_drift`) runs a company's metrics and the canonical ones side by side on the synthetic SaaS dataset (`src/synthetic.ts`, also the test fixture) in PGlite, loaded on first use from the optional dependency `@electric-sql/pglite`; without it the tool says so and runs nothing
90
+ - The metric compiler (`src/ossie/company.ts`, behind `compile_model`) writes every covered GASP metric into a company's own Ossie model, direct or staged, over their datasets, sources and field expressions. The test suite compiles the fixture adapter and checks every direct and staged value against the reference export on the fixture rows
72
91
  - Adapter schema 1.1 (`standard/gasp-adapter-schema.json`, JSON Schema draft 2020-12, served at https://gaspwiki.com/schemas/gasp-adapter-v1.json): a `datasets` map, per-dataset mappings (a mapping names its `dataset`), informational fields and `$schema`. The schema travels in the bundled data as `adapterSchema` and `validate_adapter` applies it with Ajv, so validation works offline. `npm run derive` writes a ready template to `standard/exports/gasp-adapter-template.json` (served at https://gaspwiki.com/downloads/gasp-adapter-template.json) and the tests validate it against the schema
92
+ - GASP Ossie extension schema 1 (`standard/gasp-ossie-extension-v1.json`, JSON Schema draft 2020-12, served at https://gaspwiki.com/schemas/gasp-ossie-extension-v1.json): the parsed `data` of every `custom_extensions` entry GASP writes under `vendor_name: GASP`, one definition per placement (model, dataset, metric, mapping), strict so each key GASP adds outside Ossie is listed. It travels in the bundled data as `extensionSchema` and the cross-document checks (`checkExport` in `src/ossie/check.ts`) validate every GASP payload against it in the export, `validate_mapping`, `compile_model`, `propose_mapping` and `record_decision`
73
93
  - Sources: the canonical [GASP Standard](https://gaspwiki.com) files
74
94
 
75
95
  ## Examples
package/build/cel.d.ts ADDED
@@ -0,0 +1,138 @@
1
+ /**
2
+ * Deriving CEL Revenue_Events from a subscription ledger (standard/cel.md,
3
+ * "Deriving Revenue Events from a Subscription Ledger").
4
+ *
5
+ * Most billing systems hold subscriptions, not events: a row per subscription
6
+ * whose MRR and status change over time, kept as snapshots or as a version
7
+ * history. The CEL needs the movements between those states. The rules here
8
+ * turn the GASP `subscription` dataset into `revenue_event` rows and are the
9
+ * same for every source system; a vendor appears only in a mapping.
10
+ *
11
+ * 1. A subscription's MRR on a snapshot is its `mrr` when `subscription_status`
12
+ * is active, else 0. A customer's MRR is the sum over its subscriptions.
13
+ * 2. A step is a snapshot whose MRR differs from the subscription's previous
14
+ * snapshot (by snapshot_date). A subscription's first snapshot steps from 0.
15
+ * 3. Rows on the ledger's first snapshot date are opening balances: they seed
16
+ * the customer's MRR and produce no event (`firstSnapshot: "steps"` turns
17
+ * that off for a ledger that starts with the business). The ledger holds
18
+ * one row per subscription per snapshot date and keeps a cancelled
19
+ * subscription's row, since a subscription that stops appearing has no step.
20
+ * 4. One Revenue_Event per step, dated the snapshot that first shows the change.
21
+ * mrr_change is the subscription's change; prior_mrr and new_mrr are the
22
+ * customer's MRR before and after it, chained through the customer's steps
23
+ * in (snapshot_date, increases before decreases, subscription_id) order, so
24
+ * prior_mrr + mrr_change = new_mrr holds on every row and the rows of one
25
+ * date add up.
26
+ * 5. event_type from the customer's MRR at the start and end of the date: an
27
+ * increase from 0 is `new`, or `reactivation` when the customer was active
28
+ * on an earlier date; a decrease to 0 is `churn`; any other increase is
29
+ * `expansion` and any other decrease `contraction`. So a plan swap onto a
30
+ * new subscription id is an expansion and a contraction, two subscriptions
31
+ * cancelled on one date are two churns. `renewal` is not a movement and
32
+ * comes from Contract_Event or the renewal fields.
33
+ * 6. lifecycle_attribution follows the ATL rules for each type; change_nature
34
+ * is `permanent`; segment is the snapshot's; expansion_classification is
35
+ * NULL, because the ledger does not say what kind of expansion it was (the
36
+ * amendment does), and the readiness check for the ATL tag names the gap.
37
+ * 7. event_id is the subscription id and the snapshot date.
38
+ *
39
+ * The cadence of the ledger is the grain of the events: with daily snapshots
40
+ * every movement is seen on its day; with month-end snapshots the movements
41
+ * inside a month net to one event dated the month end, and a churn followed by
42
+ * a reactivation inside one interval leaves no event. Where amendment events
43
+ * exist they are the better source and the pairing rule in cel.md applies.
44
+ *
45
+ * Two more sources feed the same rules (cel.md, "Deriving Revenue Events from
46
+ * an Invoice Ledger" and "Deriving Usage Realisation"):
47
+ *
48
+ * - An invoice ledger (`source: "invoice"`) is a subscription ledger with one
49
+ * row per service period: an invoice's MRR is its `recurring_amount` over
50
+ * the months its service period covers, a subscription's MRR on a date is
51
+ * the sum over the invoices covering it, and its ledger dates are the
52
+ * starts of its service periods and the day after each end, up to the
53
+ * ledger's horizon (the latest service period start). Rules 2 to 7 then
54
+ * apply unchanged. The invoice dataset carries no segment.
55
+ * - Usage records (`usage: true`) add one pair of events per period in which
56
+ * `usage_amount` exceeds `committed_amount`: an `expansion` dated the
57
+ * period's end for the overage over the months the period covers, tagged
58
+ * `usage_realisation` with `change_nature` `temporary`, and its
59
+ * `scheduled_reversal`, a `contraction` dated the next day. The commitment
60
+ * is already in the subscription's MRR, so usage within it is no event. The
61
+ * pairs chain with the ledger's steps, so prior_mrr and new_mrr hold.
62
+ *
63
+ * The template uses the compiler's placeholders ({{field@dataset}},
64
+ * {{_table:dataset}}), so `fillTemplate` compiles it over any adapter or Ossie
65
+ * mapping that covers the fields it reads.
66
+ */
67
+ import { type DialectName } from "./dialect.js";
68
+ import type { QueryTemplate, StandardData } from "./types.js";
69
+ export declare const DERIVATION_DATASET = "subscription";
70
+ /** The subscription fields the derivation reads, in the order the ledger CTE selects them. */
71
+ export declare const DERIVATION_FIELDS: readonly ["subscription_id", "customer_id", "snapshot_date", "mrr", "subscription_status", "segment"];
72
+ /** The invoice fields the invoice source reads. */
73
+ export declare const INVOICE_FIELDS: readonly ["invoice_id", "subscription_id", "customer_id", "snapshot_date", "service_period_start", "service_period_end", "recurring_amount"];
74
+ /** The usage_record fields the usage branch reads. */
75
+ export declare const USAGE_FIELDS: readonly ["subscription_id", "customer_id", "period_start", "period_end", "committed_amount", "usage_amount"];
76
+ /** The revenue_event fields the derivation writes, in column order. */
77
+ export declare const DERIVED_FIELDS: readonly ["event_id", "event_date", "customer_id", "subscription_id", "event_type", "mrr_change", "currency", "prior_mrr", "new_mrr", "expansion_classification", "change_nature", "lifecycle_attribution", "segment"];
78
+ export type DerivationSource = "subscription" | "invoice";
79
+ export declare const DERIVATION_SOURCES: readonly DerivationSource[];
80
+ export interface DerivationOptions {
81
+ dialect?: DialectName;
82
+ /** ISO 4217 code written on every event: the ledger's currency (the subscription dataset carries none). */
83
+ currency: string;
84
+ /**
85
+ * What the rows on the ledger's first snapshot date are: `opening` (the default) seeds each customer's
86
+ * MRR and produces no event, for a ledger that starts after the oldest subscription; `steps` treats them
87
+ * as steps from 0, for a ledger that starts with the business.
88
+ */
89
+ firstSnapshot?: "opening" | "steps";
90
+ /** The ledger: the `subscription` dataset (the default) or the `invoice` dataset read as one row per service period. */
91
+ source?: DerivationSource;
92
+ /** Read the `usage_record` dataset too and add the usage realisation pairs. */
93
+ usage?: boolean;
94
+ }
95
+ /** The `ledger` CTE of the subscription source: one row per subscription per snapshot date, MRR when active. */
96
+ export declare function subscriptionLedgerSql(): string;
97
+ /**
98
+ * The `ledger` CTE of the invoice source: each invoice's latest snapshot, its MRR
99
+ * as the recurring amount over the months of its service period, the boundaries
100
+ * of each subscription's periods as its ledger dates and the invoices covering
101
+ * each date summed into its MRR, up to the ledger's horizon.
102
+ */
103
+ export declare function invoiceLedgerSql(dialect?: DialectName): string;
104
+ /**
105
+ * The usage CTEs: each period whose usage exceeds its commitment, the overage
106
+ * over the months the period covers, and the pair of changes it adds to the
107
+ * customer's steps, a `usage` increase on the period's end and a
108
+ * `usage_reversal` decrease the day after.
109
+ */
110
+ export declare function usageChangesSql(dialect?: DialectName): string;
111
+ /** The SQL that derives revenue events from the ledger (and the usage records), with compiler placeholders. */
112
+ export declare function revenueEventDerivationSql(opts: DerivationOptions): string;
113
+ /** The (field, dataset) pairs the derivation reads for the given source and usage setting. */
114
+ export declare function derivationPairs(opts: Pick<DerivationOptions, "source" | "usage">): Array<[field: string, dataset: string]>;
115
+ /** The derivation as a query template, so `fillTemplate` resolves it over an adapter or Ossie mapping like a metric's. */
116
+ export declare function revenueEventDerivation(opts: DerivationOptions): QueryTemplate;
117
+ /** The reference SQL over the GASP field names, as cel.md prints it. */
118
+ export declare function referenceDerivationSql(currency?: string): string;
119
+ /** The invoice source's ledger CTEs over the GASP field names, as cel.md prints them. */
120
+ export declare function referenceInvoiceLedgerSql(): string;
121
+ /** The usage CTEs over the GASP field names, as cel.md prints them. */
122
+ export declare function referenceUsageSql(): string;
123
+ export interface DeriveCelOptions {
124
+ dialect?: string;
125
+ currency?: string;
126
+ firstSnapshot?: "opening" | "steps";
127
+ source?: string;
128
+ usage?: boolean;
129
+ output?: "report" | "sql";
130
+ }
131
+ /**
132
+ * The derive_cel tool: the derivation compiled over a company's mapping (a GASP
133
+ * adapter or an Ossie mapping with its model), from the subscription or the
134
+ * invoice dataset, with the usage records when the mapping covers them, in the
135
+ * dialect asked for or the one the adapter's warehouse names, as a report or
136
+ * the SQL alone.
137
+ */
138
+ export declare function deriveCelFromText(documentText: string | undefined, modelText: string | undefined, data: StandardData, opts?: DeriveCelOptions): string;
package/build/cel.js ADDED
@@ -0,0 +1,376 @@
1
+ /**
2
+ * Deriving CEL Revenue_Events from a subscription ledger (standard/cel.md,
3
+ * "Deriving Revenue Events from a Subscription Ledger").
4
+ *
5
+ * Most billing systems hold subscriptions, not events: a row per subscription
6
+ * whose MRR and status change over time, kept as snapshots or as a version
7
+ * history. The CEL needs the movements between those states. The rules here
8
+ * turn the GASP `subscription` dataset into `revenue_event` rows and are the
9
+ * same for every source system; a vendor appears only in a mapping.
10
+ *
11
+ * 1. A subscription's MRR on a snapshot is its `mrr` when `subscription_status`
12
+ * is active, else 0. A customer's MRR is the sum over its subscriptions.
13
+ * 2. A step is a snapshot whose MRR differs from the subscription's previous
14
+ * snapshot (by snapshot_date). A subscription's first snapshot steps from 0.
15
+ * 3. Rows on the ledger's first snapshot date are opening balances: they seed
16
+ * the customer's MRR and produce no event (`firstSnapshot: "steps"` turns
17
+ * that off for a ledger that starts with the business). The ledger holds
18
+ * one row per subscription per snapshot date and keeps a cancelled
19
+ * subscription's row, since a subscription that stops appearing has no step.
20
+ * 4. One Revenue_Event per step, dated the snapshot that first shows the change.
21
+ * mrr_change is the subscription's change; prior_mrr and new_mrr are the
22
+ * customer's MRR before and after it, chained through the customer's steps
23
+ * in (snapshot_date, increases before decreases, subscription_id) order, so
24
+ * prior_mrr + mrr_change = new_mrr holds on every row and the rows of one
25
+ * date add up.
26
+ * 5. event_type from the customer's MRR at the start and end of the date: an
27
+ * increase from 0 is `new`, or `reactivation` when the customer was active
28
+ * on an earlier date; a decrease to 0 is `churn`; any other increase is
29
+ * `expansion` and any other decrease `contraction`. So a plan swap onto a
30
+ * new subscription id is an expansion and a contraction, two subscriptions
31
+ * cancelled on one date are two churns. `renewal` is not a movement and
32
+ * comes from Contract_Event or the renewal fields.
33
+ * 6. lifecycle_attribution follows the ATL rules for each type; change_nature
34
+ * is `permanent`; segment is the snapshot's; expansion_classification is
35
+ * NULL, because the ledger does not say what kind of expansion it was (the
36
+ * amendment does), and the readiness check for the ATL tag names the gap.
37
+ * 7. event_id is the subscription id and the snapshot date.
38
+ *
39
+ * The cadence of the ledger is the grain of the events: with daily snapshots
40
+ * every movement is seen on its day; with month-end snapshots the movements
41
+ * inside a month net to one event dated the month end, and a churn followed by
42
+ * a reactivation inside one interval leaves no event. Where amendment events
43
+ * exist they are the better source and the pairing rule in cel.md applies.
44
+ *
45
+ * Two more sources feed the same rules (cel.md, "Deriving Revenue Events from
46
+ * an Invoice Ledger" and "Deriving Usage Realisation"):
47
+ *
48
+ * - An invoice ledger (`source: "invoice"`) is a subscription ledger with one
49
+ * row per service period: an invoice's MRR is its `recurring_amount` over
50
+ * the months its service period covers, a subscription's MRR on a date is
51
+ * the sum over the invoices covering it, and its ledger dates are the
52
+ * starts of its service periods and the day after each end, up to the
53
+ * ledger's horizon (the latest service period start). Rules 2 to 7 then
54
+ * apply unchanged. The invoice dataset carries no segment.
55
+ * - Usage records (`usage: true`) add one pair of events per period in which
56
+ * `usage_amount` exceeds `committed_amount`: an `expansion` dated the
57
+ * period's end for the overage over the months the period covers, tagged
58
+ * `usage_realisation` with `change_nature` `temporary`, and its
59
+ * `scheduled_reversal`, a `contraction` dated the next day. The commitment
60
+ * is already in the subscription's MRR, so usage within it is no event. The
61
+ * pairs chain with the ledger's steps, so prior_mrr and new_mrr hold.
62
+ *
63
+ * The template uses the compiler's placeholders ({{field@dataset}},
64
+ * {{_table:dataset}}), so `fillTemplate` compiles it over any adapter or Ossie
65
+ * mapping that covers the fields it reads.
66
+ */
67
+ import { DIALECT } from "./dialect.js";
68
+ import { exampleSql } from "./compile.js";
69
+ import { chooseDialect, fillTemplate, resolveAdapter } from "./tools.js";
70
+ import { loadAdapter } from "./ossie/adapt.js";
71
+ import { resolveDocLink } from "./urls.js";
72
+ export const DERIVATION_DATASET = "subscription";
73
+ /** The subscription fields the derivation reads, in the order the ledger CTE selects them. */
74
+ export const DERIVATION_FIELDS = ["subscription_id", "customer_id", "snapshot_date", "mrr", "subscription_status", "segment"];
75
+ /** The invoice fields the invoice source reads. */
76
+ export const INVOICE_FIELDS = ["invoice_id", "subscription_id", "customer_id", "snapshot_date", "service_period_start", "service_period_end", "recurring_amount"];
77
+ /** The usage_record fields the usage branch reads. */
78
+ export const USAGE_FIELDS = ["subscription_id", "customer_id", "period_start", "period_end", "committed_amount", "usage_amount"];
79
+ /** The revenue_event fields the derivation writes, in column order. */
80
+ export const DERIVED_FIELDS = ["event_id", "event_date", "customer_id", "subscription_id", "event_type", "mrr_change", "currency", "prior_mrr", "new_mrr", "expansion_classification", "change_nature", "lifecycle_attribution", "segment"];
81
+ export const DERIVATION_SOURCES = ["subscription", "invoice"];
82
+ const ph = (field, dataset = DERIVATION_DATASET) => `{{${field}@${dataset}}}`;
83
+ /** MRR is kept to the cent, so a chained total within half a cent of zero is zero (the sums are floats on some warehouses). */
84
+ const ZERO = "0.005";
85
+ /** The length of a period of whole days in months (365.25 / 12 days each), as a fraction. */
86
+ const monthsExpr = (d, start, end) => `${d.periodDays(end, start)} * 12.0 / 365.25`;
87
+ /**
88
+ * The months a period covers, from its length in months: the nearest whole
89
+ * number (so January and February are one month each and a year is twelve), or
90
+ * the fraction itself for a period shorter than about a fortnight.
91
+ */
92
+ const MONTHS_COVERED = "CASE WHEN ROUND(months) < 1 THEN months ELSE ROUND(months) END";
93
+ /** The `ledger` CTE of the subscription source: one row per subscription per snapshot date, MRR when active. */
94
+ export function subscriptionLedgerSql() {
95
+ return `ledger AS (
96
+ SELECT ${ph("subscription_id")} AS subscription_id,
97
+ ${ph("customer_id")} AS customer_id,
98
+ ${ph("snapshot_date")} AS snapshot_date,
99
+ ${ph("segment")} AS segment,
100
+ CASE WHEN ${ph("subscription_status")} = 'active' THEN ROUND(COALESCE(${ph("mrr")}, 0), 2) ELSE 0 END AS mrr
101
+ FROM {{_table:${DERIVATION_DATASET}}}
102
+ WHERE ${ph("snapshot_date")} IS NOT NULL
103
+ )`;
104
+ }
105
+ /**
106
+ * The `ledger` CTE of the invoice source: each invoice's latest snapshot, its MRR
107
+ * as the recurring amount over the months of its service period, the boundaries
108
+ * of each subscription's periods as its ledger dates and the invoices covering
109
+ * each date summed into its MRR, up to the ledger's horizon.
110
+ */
111
+ export function invoiceLedgerSql(dialect = "postgres") {
112
+ const d = DIALECT[dialect];
113
+ const i = (f) => ph(f, "invoice");
114
+ return `invoices AS (
115
+ SELECT invoice_id, subscription_id, customer_id, service_period_start, service_period_end, recurring_amount
116
+ FROM (
117
+ SELECT ${i("invoice_id")} AS invoice_id,
118
+ ${i("subscription_id")} AS subscription_id,
119
+ ${i("customer_id")} AS customer_id,
120
+ ${i("service_period_start")} AS service_period_start,
121
+ ${i("service_period_end")} AS service_period_end,
122
+ ${i("recurring_amount")} AS recurring_amount,
123
+ ROW_NUMBER() OVER (PARTITION BY ${i("invoice_id")} ORDER BY ${i("snapshot_date")} DESC) AS latest
124
+ FROM {{_table:invoice}}
125
+ WHERE ${i("subscription_id")} IS NOT NULL AND ${i("service_period_start")} IS NOT NULL AND ${i("service_period_end")} IS NOT NULL
126
+ ) i
127
+ WHERE latest = 1
128
+ ),
129
+ periods AS (
130
+ SELECT invoice_id, subscription_id, customer_id, service_period_start, service_period_end,
131
+ ROUND(COALESCE(recurring_amount, 0) / (${MONTHS_COVERED}), 2) AS mrr
132
+ FROM (
133
+ SELECT i.*, ${monthsExpr(d, "service_period_start", "service_period_end")} AS months FROM invoices i
134
+ ) m
135
+ ),
136
+ boundaries AS (
137
+ SELECT subscription_id, customer_id, service_period_start AS snapshot_date FROM periods
138
+ UNION
139
+ SELECT subscription_id, customer_id, CAST(${d.plus("service_period_end", 1, "day")} AS DATE) FROM periods
140
+ ),
141
+ horizon AS (
142
+ SELECT MAX(service_period_start) AS latest FROM periods
143
+ ),
144
+ ledger AS (
145
+ SELECT b.subscription_id, b.customer_id, b.snapshot_date,
146
+ ${d.toText("NULL")} AS segment,
147
+ ROUND(COALESCE(SUM(p.mrr), 0), 2) AS mrr
148
+ FROM boundaries b
149
+ LEFT JOIN periods p ON p.subscription_id = b.subscription_id AND p.service_period_start <= b.snapshot_date AND b.snapshot_date <= p.service_period_end
150
+ WHERE b.snapshot_date <= (SELECT latest FROM horizon)
151
+ GROUP BY b.subscription_id, b.customer_id, b.snapshot_date
152
+ )`;
153
+ }
154
+ /**
155
+ * The usage CTEs: each period whose usage exceeds its commitment, the overage
156
+ * over the months the period covers, and the pair of changes it adds to the
157
+ * customer's steps, a `usage` increase on the period's end and a
158
+ * `usage_reversal` decrease the day after.
159
+ */
160
+ export function usageChangesSql(dialect = "postgres") {
161
+ const d = DIALECT[dialect];
162
+ const u = (f) => ph(f, "usage_record");
163
+ return `usage_periods AS (
164
+ SELECT subscription_id, customer_id, period_end,
165
+ ROUND(overage / (${MONTHS_COVERED}), 2) AS overage
166
+ FROM (
167
+ SELECT ${u("subscription_id")} AS subscription_id,
168
+ ${u("customer_id")} AS customer_id,
169
+ ${u("period_end")} AS period_end,
170
+ ${u("usage_amount")} - ${u("committed_amount")} AS overage,
171
+ ${monthsExpr(d, u("period_start"), u("period_end"))} AS months
172
+ FROM {{_table:usage_record}}
173
+ WHERE ${u("period_start")} IS NOT NULL AND ${u("period_end")} IS NOT NULL
174
+ AND ${u("usage_amount")} > ${u("committed_amount")}
175
+ ) u
176
+ ),
177
+ usage_changes AS (
178
+ SELECT subscription_id, customer_id, period_end AS snapshot_date, ${d.toText("NULL")} AS segment, 0 AS opening, overage AS mrr_change, 'usage' AS kind
179
+ FROM usage_periods WHERE overage >= 0.01
180
+ UNION ALL
181
+ SELECT subscription_id, customer_id, CAST(${d.plus("period_end", 1, "day")} AS DATE), ${d.toText("NULL")}, 0, -overage, 'usage_reversal'
182
+ FROM usage_periods WHERE overage >= 0.01
183
+ )`;
184
+ }
185
+ /** The SQL that derives revenue events from the ledger (and the usage records), with compiler placeholders. */
186
+ export function revenueEventDerivationSql(opts) {
187
+ const d = DIALECT[opts.dialect ?? "postgres"];
188
+ if (!/^[A-Z]{3}$/.test(opts.currency))
189
+ throw new Error(`currency must be an ISO 4217 code, got ${JSON.stringify(opts.currency)}`);
190
+ const source = opts.source ?? "subscription";
191
+ if (!DERIVATION_SOURCES.includes(source))
192
+ throw new Error(`source must be one of ${DERIVATION_SOURCES.join(", ")}, got ${JSON.stringify(source)}`);
193
+ const opening = opts.firstSnapshot === "steps" ? "0" : "CASE WHEN snapshot_date = MIN(snapshot_date) OVER () THEN 1 ELSE 0 END";
194
+ // Within a date a customer's increases come before its decreases, then subscription id and kind, so the chained MRR is deterministic.
195
+ const chain = "PARTITION BY c.customer_id ORDER BY c.snapshot_date, CASE WHEN c.mrr_change > 0 THEN 0 ELSE 1 END, c.subscription_id, c.kind";
196
+ const ledger = source === "invoice" ? invoiceLedgerSql(d.name) : subscriptionLedgerSql();
197
+ const usage = opts.usage ? `,
198
+ ${usageChangesSql(d.name)}` : "";
199
+ const union = opts.usage ? `
200
+ UNION ALL
201
+ SELECT subscription_id, customer_id, snapshot_date, segment, opening, mrr_change, kind FROM usage_changes` : "";
202
+ return `WITH ${ledger},
203
+ steps AS (
204
+ SELECT subscription_id, customer_id, snapshot_date, segment, mrr,
205
+ COALESCE(LAG(mrr) OVER (PARTITION BY subscription_id ORDER BY snapshot_date), 0) AS prior_sub_mrr,
206
+ ${opening} AS opening
207
+ FROM ledger
208
+ )${usage},
209
+ changes AS (
210
+ SELECT subscription_id, customer_id, snapshot_date, segment, opening,
211
+ mrr - prior_sub_mrr AS mrr_change, 'ledger' AS kind
212
+ FROM steps
213
+ WHERE mrr <> prior_sub_mrr${union}
214
+ ),
215
+ day_changes AS (
216
+ SELECT customer_id, snapshot_date, SUM(mrr_change) AS day_change
217
+ FROM changes
218
+ GROUP BY customer_id, snapshot_date
219
+ ),
220
+ days AS (
221
+ SELECT customer_id, snapshot_date, day_change,
222
+ SUM(day_change) OVER (PARTITION BY customer_id ORDER BY snapshot_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS day_end
223
+ FROM day_changes
224
+ ),
225
+ day_state AS (
226
+ SELECT customer_id, snapshot_date, day_end - day_change AS day_start, day_end,
227
+ COALESCE(MAX(CASE WHEN day_end > ${ZERO} THEN 1 ELSE 0 END) OVER (PARTITION BY customer_id ORDER BY snapshot_date ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), 0) AS active_before
228
+ FROM days
229
+ ),
230
+ chained AS (
231
+ SELECT c.subscription_id, c.customer_id, c.snapshot_date, c.segment, c.opening, c.mrr_change, c.kind,
232
+ COALESCE(SUM(c.mrr_change) OVER (${chain} ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), 0) AS prior_mrr,
233
+ SUM(c.mrr_change) OVER (${chain} ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS new_mrr,
234
+ s.day_start, s.day_end, s.active_before
235
+ FROM changes c
236
+ JOIN day_state s ON s.customer_id = c.customer_id AND s.snapshot_date = c.snapshot_date
237
+ ),
238
+ typed AS (
239
+ SELECT subscription_id, customer_id, snapshot_date, segment, opening, mrr_change, prior_mrr, new_mrr, kind,
240
+ CASE
241
+ WHEN kind = 'usage' THEN 'expansion'
242
+ WHEN kind = 'usage_reversal' THEN 'contraction'
243
+ WHEN mrr_change > 0 AND ABS(day_start) < ${ZERO} AND active_before = 1 THEN 'reactivation'
244
+ WHEN mrr_change > 0 AND ABS(day_start) < ${ZERO} THEN 'new'
245
+ WHEN mrr_change < 0 AND ABS(day_end) < ${ZERO} THEN 'churn'
246
+ WHEN mrr_change > 0 THEN 'expansion'
247
+ ELSE 'contraction'
248
+ END AS event_type
249
+ FROM chained
250
+ )
251
+ SELECT ${d.toText("subscription_id")} || CASE WHEN kind = 'ledger' THEN ':' ELSE ':usage:' END || ${d.toText("snapshot_date")} AS event_id,
252
+ snapshot_date AS event_date,
253
+ customer_id,
254
+ subscription_id,
255
+ event_type,
256
+ mrr_change,
257
+ '${opts.currency}' AS currency,
258
+ prior_mrr,
259
+ new_mrr,
260
+ CASE WHEN kind = 'ledger' THEN ${d.toText("NULL")} ELSE 'usage_realisation' END AS expansion_classification,
261
+ CASE kind WHEN 'usage' THEN 'temporary' WHEN 'usage_reversal' THEN 'scheduled_reversal' ELSE 'permanent' END AS change_nature,
262
+ CASE event_type
263
+ WHEN 'new' THEN 'acquisition'
264
+ WHEN 'expansion' THEN 'expansion'
265
+ WHEN 'reactivation' THEN 'reactivation'
266
+ ELSE 'retention'
267
+ END AS lifecycle_attribution,
268
+ segment
269
+ FROM typed
270
+ WHERE opening = 0
271
+ ORDER BY snapshot_date, customer_id, CASE WHEN mrr_change > 0 THEN 0 ELSE 1 END, subscription_id, kind`;
272
+ }
273
+ /** The (field, dataset) pairs the derivation reads for the given source and usage setting. */
274
+ export function derivationPairs(opts) {
275
+ const source = opts.source ?? "subscription";
276
+ const ledger = source === "invoice" ? INVOICE_FIELDS.map((f) => [f, "invoice"]) : DERIVATION_FIELDS.map((f) => [f, DERIVATION_DATASET]);
277
+ return opts.usage ? [...ledger, ...USAGE_FIELDS.map((f) => [f, "usage_record"])] : ledger;
278
+ }
279
+ /** The derivation as a query template, so `fillTemplate` resolves it over an adapter or Ossie mapping like a metric's. */
280
+ export function revenueEventDerivation(opts) {
281
+ const source = opts.source ?? "subscription";
282
+ const pairs = derivationPairs(opts);
283
+ const datasets = [...new Set(pairs.map(([, ds]) => ds))];
284
+ return {
285
+ metric: "cel_revenue_events",
286
+ fields: [...new Set(pairs.map(([f]) => f))],
287
+ datasets,
288
+ sourceTable: source,
289
+ grain: "event",
290
+ frequency: source === "invoice" ? "as the service periods start and end" : "as the ledger is snapshotted",
291
+ source: source === "invoice" ? "invoice ledger" : "subscription ledger",
292
+ parameterizedQuery: revenueEventDerivationSql(opts),
293
+ staged: false,
294
+ placeholders: pairs.map(([f, ds]) => `${f}@${ds}`),
295
+ };
296
+ }
297
+ /** The reference SQL over the GASP field names, as cel.md prints it. */
298
+ export function referenceDerivationSql(currency = "USD") {
299
+ return exampleSql(revenueEventDerivationSql({ dialect: "postgres", currency }));
300
+ }
301
+ /** The invoice source's ledger CTEs over the GASP field names, as cel.md prints them. */
302
+ export function referenceInvoiceLedgerSql() {
303
+ return exampleSql(invoiceLedgerSql("postgres"));
304
+ }
305
+ /** The usage CTEs over the GASP field names, as cel.md prints them. */
306
+ export function referenceUsageSql() {
307
+ return exampleSql(usageChangesSql("postgres"));
308
+ }
309
+ /**
310
+ * The derive_cel tool: the derivation compiled over a company's mapping (a GASP
311
+ * adapter or an Ossie mapping with its model), from the subscription or the
312
+ * invoice dataset, with the usage records when the mapping covers them, in the
313
+ * dialect asked for or the one the adapter's warehouse names, as a report or
314
+ * the SQL alone.
315
+ */
316
+ export function deriveCelFromText(documentText, modelText, data, opts = {}) {
317
+ if (!documentText?.trim())
318
+ return "Pass adapter: your mapping as text, a GASP adapter (JSON) or an Ossie mapping document (YAML or JSON) with its semantic_model. Check it with validate_mapping first.";
319
+ const currency = opts.currency?.trim().toUpperCase();
320
+ if (!currency)
321
+ return "Pass currency: the ISO 4217 code of the ledger's currency (USD, EUR, GBP ...). Every derived event carries it; the subscription dataset has no currency field to read it from.";
322
+ if (!/^[A-Z]{3}$/.test(currency))
323
+ return `currency must be a three-letter ISO 4217 code, got ${JSON.stringify(opts.currency)}.`;
324
+ const source = (opts.source?.trim().toLowerCase() || "subscription");
325
+ if (!DERIVATION_SOURCES.includes(source))
326
+ return `source must be one of ${DERIVATION_SOURCES.join(", ")}, got ${JSON.stringify(opts.source)}.`;
327
+ const loaded = loadAdapter(documentText, modelText, data);
328
+ if (!loaded.adapter)
329
+ return loaded.error;
330
+ const chosen = chooseDialect(opts.dialect, loaded.adapter.warehouse);
331
+ if ("error" in chosen)
332
+ return chosen.error;
333
+ const resolved = resolveAdapter(loaded.adapter, data);
334
+ const usageMapped = Boolean(resolved.tables.usage_record) || Object.keys(resolved.columns.usage_record ?? {}).length > 0;
335
+ const usage = opts.usage ?? usageMapped;
336
+ const template = revenueEventDerivation({ dialect: chosen.dialect, currency, firstSnapshot: opts.firstSnapshot, source, usage });
337
+ const r = fillTemplate(template, loaded.adapter, data);
338
+ if (opts.output === "sql")
339
+ return r.sql;
340
+ const ledgerName = source === "invoice" ? "invoice ledger" : "subscription ledger";
341
+ const rulesAnchor = source === "invoice" ? "cel.md#deriving-revenue-events-from-an-invoice-ledger" : "cel.md#deriving-revenue-events-from-a-subscription-ledger";
342
+ const ledgerPairs = template.placeholders.filter((p) => !p.endsWith("@usage_record"));
343
+ const usagePairs = template.placeholders.filter((p) => p.endsWith("@usage_record"));
344
+ const mappedOf = (pairs) => pairs.filter((p) => r.mapped.some((m) => `${m.field}@${m.dataset}` === p)).length;
345
+ const p = [
346
+ `# CEL Revenue_Events derived from your ${ledgerName}${usage ? " and usage records" : ""}`,
347
+ `**Rules:** ${resolveDocLink(rulesAnchor)}${usage ? ` and ${resolveDocLink("cel.md#deriving-usage-realisation")}` : ""}`,
348
+ `**Mapping:** ${loaded.format === "v1" ? "GASP adapter" : "Ossie mapping"}${loaded.adapter.organization ? ` (${loaded.adapter.organization})` : ""}`,
349
+ `**Warehouse:** ${loaded.adapter.warehouse || "not specified"}`,
350
+ `**Dialect:** ${chosen.note}`,
351
+ `**Reads:** the \`${source}\` dataset, ${mappedOf(ledgerPairs)} / ${ledgerPairs.length} fields mapped${usage ? `; the \`usage_record\` dataset, ${mappedOf(usagePairs)} / ${usagePairs.length} fields mapped${opts.usage === undefined ? " (read because the mapping covers it; pass usage: false to leave it out)" : ""}` : usageMapped ? "" : "; no usage records (map the `usage_record` dataset, or pass usage: true, for the usage realisation events)"}`,
352
+ `**Writes:** one \`revenue_event\` row per movement (${DERIVED_FIELDS.join(", ")}), currency \`${currency}\``,
353
+ ];
354
+ if (r.missingFields.length)
355
+ p.push(`**Unmapped fields (left as placeholders):** \`${r.missingFields.join("\`, \`")}\`. Map each on its dataset and run again.`);
356
+ if (r.missingTables.length)
357
+ p.push(`**Dataset with no table:** ${r.missingTables.join(", ")}. Name its table under "datasets" in the adapter, or map one of its fields with a schema.table column reference.`);
358
+ for (const c of r.conflicts)
359
+ p.push(`**Conflict:** ${c}`);
360
+ const firstRows = opts.firstSnapshot === "steps"
361
+ ? "Every first row is a step from 0 (first_snapshot: steps), so the ledger must start with the business."
362
+ : "Rows on the ledger's first date are opening balances and produce no event; a customer that churned before the ledger starts comes back as new, not reactivation. Pass first_snapshot: steps if the ledger starts with the business.";
363
+ const limits = source === "invoice"
364
+ ? `**What the invoices can and cannot say.** Each invoice is read at its latest snapshot as one service period: its MRR is \`recurring_amount\` over the months the period covers (whole months for monthly, quarterly and annual billing; a prorated invoice for part of a period is approximate). A subscription's MRR on a date is the sum over the invoices covering it, its ledger dates are the starts of its service periods and the day after each end, and a date after the ledger's horizon (the latest service period start) is not read, so a subscription whose last invoice is still current does not churn. Invoices with no subscription or no service period (one-off charges) produce no event. ${firstRows} Each movement is dated the service period that first bills it, so a change agreed mid-period shows on the next period's start unless a prorated invoice was issued. The invoice dataset carries no segment, so \`segment\` is empty; fill it from the customer snapshot. \`expansion_classification\` is empty: the invoices show that MRR rose, not what was bought, and the Operating forms of ARR and NRR read that tag; fill it from your amendment records or record the classification as a decision. Where a subscription ledger exists it is the better source (source: subscription).`
365
+ : `**What the ledger can and cannot say.** Each movement is dated the snapshot that first shows it, so the ledger's cadence is the grain of the events: daily snapshots show every movement on its day, month-end snapshots net a month's movements into one event dated the month end. ${firstRows} The ledger needs one row per subscription per snapshot date and must keep a cancelled subscription's row; a subscription that stops appearing produces no churn. A null MRR on an active row counts as 0. \`expansion_classification\` is empty: the ledger shows that MRR rose, not what was bought, and the Operating forms of ARR and NRR read that tag; fill it from your amendment records or record the classification as a decision. Renewals change no MRR and are not derived here.`;
366
+ p.push("", limits);
367
+ if (usage)
368
+ p.push("", "**Usage realisation.** Each usage period whose `usage_amount` exceeds its `committed_amount` adds a pair of events: an `expansion` dated the period's end for the overage over the months the period covers, tagged `usage_realisation` with `change_nature` `temporary`, and a `contraction` the next day with `change_nature` `scheduled_reversal`. The commitment is already in the subscription's MRR, so usage within it is no event, and a commitment raised to cover the usage shows on the ledger as a permanent expansion. The pairs chain with the ledger's steps, so `prior_mrr` and `new_mrr` hold across both. Their ids are `<subscription_id>:usage:<date>`.");
369
+ p.push("", `Run the query to build (or rebuild) the \`revenue_event\` table; it reads the whole ledger and carries no period parameters. Map the result as the \`revenue_event\` dataset and the MRR movement metrics, NRR and GRR compile over it with generate_query or compile_model.`, "", "\`\`\`sql\n" + r.sql + "\n\`\`\`");
370
+ if (r.mapped.length) {
371
+ p.push("", "**Field mapping used:**");
372
+ for (const m of r.mapped)
373
+ p.push(`- \`${m.field}\` (${m.dataset}) → \`${m.ref}\`${m.transform ? ` (transform: ${m.transform})` : ""}`);
374
+ }
375
+ return p.join("\n");
376
+ }