gasp-standard-mcp 1.17.0 → 1.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -5
- package/build/cel.d.ts +58 -4
- package/build/cel.js +200 -35
- package/build/index.js +7 -5
- package/build/ossie/profile.js +5 -1
- package/build/standard-data.json +1 -1
- package/build/standard-version.json +1 -1
- package/build/synthetic.d.ts +7 -0
- package/build/synthetic.js +42 -11
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -45,7 +45,7 @@ Restart your MCP client after adding the config.
|
|
|
45
45
|
| `score_drift` | Score how far your own metrics drift from GASP's canonical definitions, by result. Takes your mapping (either format) and your `semantic_model` holding your metrics; each is matched to a GASP metric by exact name, by the synonyms in its `ai_context` or by a pair in `matches`, a name ending in -O or -M matching that declared form, with near names listed for review and never scored on a guess. The canonical metric is compiled into your names and both run on the GASP synthetic SaaS dataset for six monthly periods in an in-process Postgres (PGlite, an optional dependency). Differences come with both values per period and the likely cause read from the canonical formula: a null on one side (a sum over no rows is NULL in your SQL and 0 in GASP's), a cohort or opening value the expression cannot state, a filter one side applies, a dataset one side reads, an aggregate with no equal on the other side. Your values are checked against the standard's range for the metric per period, and where both forms of a dual-lens metric are matched the O/M bridge (the Durability Gap for ARR, the Activation Delta for CAC and the others the Core metrics name) is scored beside the canonical gap. `output: json` returns the data |
|
|
46
46
|
| `profile_schema` | Profile your warehouse schema into a minimal Ossie semantic model, the first step towards a mapping. Pass the rows of `information_schema.columns` (JSON or CSV), a tables list or `CREATE TABLE` statements, read through your own warehouse MCP or from a dump, and optionally statistics per column (`output: statistics` returns the query that gathers them). Every table gets a key, a time field and a kind (snapshot, event, fact or reference) with the evidence and a confidence, plus the relationships the columns imply; what the names cannot settle is a question for you rather than a guess. The model is returned as YAML to pass on as `semantic_model` |
|
|
47
47
|
| `propose_mapping` | Propose a GASP mapping from the model `profile_schema` returned (or your own Ossie model, or a schema profiled on the way in). Every table is scored against the GASP datasets on its columns, its profiled kind, key and time field and its name; every column of a proposed table against the dataset's fields by name, structural role, a short synonym list, word order and near names, checked against the type and the listed values. A column in cents gets a divide-by-100 transform and enum values the standard does not use a `CASE` recode when one is known. High-confidence pairs become an Ossie mapping over the model (a transform a calculated field) for `validate_mapping`; medium ones are candidates with their evidence and what would settle them; `output: adapter` gives the pairs as a GASP adapter |
|
|
48
|
-
| `derive_cel` | SQL that derives CEL `Revenue_Event` rows from your
|
|
48
|
+
| `derive_cel` | SQL that derives CEL `Revenue_Event` rows from your billing data by the rules in `cel.md`. `source: subscription` (default): the subscription dataset your mapping covers (snapshots at any cadence or a version history) becomes one event per movement, with the customer's `prior_mrr` and `new_mrr` chained so the row identity holds, the event type read from the customer's MRR around the step, the ATL lifecycle tag and an empty `expansion_classification` the ledger cannot fill. `source: invoice`: the invoice dataset read as one row per service period (`recurring_amount` over the months between `service_period_start` and `service_period_end`), for a billing system that keeps no subscription history. When the mapping covers the `usage_record` dataset (or `usage: true`), each period whose `usage_amount` exceeds its `committed_amount` adds an `expansion` tagged `usage_realisation` with `change_nature` `temporary` on the period's end and its `scheduled_reversal` the next day, chained with the ledger's steps. Same mapping input as `generate_query`, plus the ledger's currency; dialect from the request or the warehouse; unmapped fields listed |
|
|
49
49
|
| `generate_checks` | SQL for the readiness checks the standard states, over the datasets your mapping covers: CEL row identity, MRR continuity, unique event ids, the ATL tags (lifecycle, expansion classification, cost function), customer identity both ways, freshness per dataset and, for every fully mapped metric with a hard range rule in the ontology (NRR at least 0, GRR at most 100%), its value in every reporting period of your period spine (`period_spine`, the table `compile_model` reads), one row per check. Paste the result rows back as `results` for pass or fail per check and per metric, each check naming the metrics it protects. Same mapping input as `generate_query`; dialect from the request or the warehouse |
|
|
50
50
|
| `plan_fixes` | The ranked fix list: every unmapped field and every failing check, ordered by Core metrics unlocked, then all metrics unlocked, with an effort class shown and used only as a tiebreak. A fix unlocks a metric when it is the last thing between that metric and covered-and-passing. Takes the mapping and the `results` from `generate_checks` |
|
|
51
51
|
| `readiness_report` | Which readiness level a company has reached and what stands between it and the next, read from the documents the other tools wrote: L1 Profiled (a semantic model exists), L2 Mapped (a metric fully covered, no open question or candidate), L3 Clean core (the CEL revenue events mapped, the checks run and every covered Core metric passing every check that protects it), L4 LLM-ready (the canonical metrics compiled into the model and none of your own metrics differing from its canonical counterpart). Takes the mapping, the `semantic_model` (with an adapter, the one `compile_model` returned) and the `results` from `generate_checks`; each blocker names the tool that clears it and the Next line is the one action to take
|
|
@@ -59,7 +59,7 @@ The traversal tools (`infer_drivers`, `explain_path`, `coverage_gap`) return pat
|
|
|
59
59
|
|
|
60
60
|
## Data
|
|
61
61
|
|
|
62
|
-
Version 1.
|
|
62
|
+
Version 1.18.0.
|
|
63
63
|
|
|
64
64
|
- 306 metrics across 13 departments
|
|
65
65
|
- 94 glossary definitions (81 SaaS domain + 13 GASP framework)
|
|
@@ -71,7 +71,7 @@ Version 1.17.0.
|
|
|
71
71
|
- Apache Ossie (incubating) export, written by `npm run derive` to `standard/exports/ossie/` and served at https://gaspwiki.com/downloads/ossie/: `gasp.ontology.yaml` (every dataset and GASP concept as an entity type, every field as a value type or relationship, enum values as constraints), `gasp.semantic_model.yaml` (every dataset with `OSSIE_SQL_2026` field expressions, the foreign-key relationships between them and every canonical formula as a metric, either a direct expression or staged through a per-period dataset that carries the compiled SQL; O/M forms and variants are separate metrics with the form in `custom_extensions`), `gasp.mapping.yaml` (the ontology mapped onto the model) and `coverage.json` (how each formula was exported and why). All three documents pass the upstream validator (vendored at `ossie-validator/`) and GASP's own cross-reference checks in `src/ossie/check.ts`, which cover references the upstream validator leaves alone
|
|
72
72
|
- 28 business entities (Customer, Subscription, License, Lead, Opportunity, User, Ticket, Employee, Invoice, Partner, Project, Plan, Product, Campaign, Contact, Incident, Deployment, Release, Issue, Vulnerability, Feature, Requisition, Project Milestone, Deliverable, Quote, Order, Knowledge Article and Deal Registration). Every dataset with its own identity maps to an entity; ledger and period-level facts (cost events, financials, forecasts, web traffic) stay entity-free
|
|
73
73
|
- 7 business concepts (Revenue, Bookings, COGS, Cohort, Pipeline, Segment and TCV)
|
|
74
|
-
-
|
|
74
|
+
- 274 canonical fields across 53 datasets (event, snapshot and fact tables with date and key fields, entities, CEL class and activation rules). Enum fields declare their allowed values
|
|
75
75
|
- 65 structured formulas with decomposed inputs (metric refs + field refs, with baseline inputs marked)
|
|
76
76
|
- Canonical formulas for every metric in the GASP formula grammar (`standard/formulas.json`): 294 entries covering all 306 metrics (279 with formulas, 15 marked not computable with a reason, 2 department-qualified ids for shared headings), with units, O/M forms for ARR, NRR, CAC, LTV, LTV:CAC, CAC Payback and Gross Margin (a form can reference another metric's form, `metric.gross_margin.O`) and variants such as Adjusted NRR (served by `get_formula` and compiled by `generate_query` and `get_data_requirements`)
|
|
77
77
|
- Access control classifications for every graph metric (sensitivity tier + access scope)
|
|
@@ -81,8 +81,8 @@ Version 1.17.0.
|
|
|
81
81
|
- One mapping format: a company's warehouse mapping is an Apache Ossie mapping document over its semantic model, and the v1 adapter is accepted everywhere and converted on load (`src/ossie/adapt.ts`). The upstream Ossie schemas travel in the bundled data as `ossieSchemas`, so `validate_mapping` checks an Ossie mapping offline
|
|
82
82
|
- The schema profiler (`src/ossie/profile.ts`, behind `profile_schema`) turns a schema dump into a minimal Ossie semantic model with its evidence; the Acme DDL in `test/fixture/acme-schema.sql` is its fixture.
|
|
83
83
|
- The mapping proposer (`src/ossie/propose.ts`, behind `propose_mapping`) turns that model into a draft GASP mapping with the evidence per pair, transforms where the name or the values call for one and candidates for the pairs the evidence does not settle; on the Acme DDL it reproduces the Acme adapter.
|
|
84
|
-
- The CEL
|
|
85
|
-
- Two vendor worked examples beside Acme in `test/fixture/` (`stripe/`, Redshift; `salesforce/`, BigQuery): a DDL, a GASP mapping and a README stating the source documentation and every choice, written from public API documentation and never standard content. The Stripe example is taken from a validated mapping to readiness level 4 in the test suite and its drift report names a planted difference; the Salesforce example compiles in BigQuery and stops at level 2 because a CRM maps no revenue event
|
|
84
|
+
- The CEL derivations (`src/cel.ts`, behind `derive_cel`) are the standard's rules for turning a subscription ledger, an invoice ledger or usage records into `Revenue_Event` rows, compiled over a company's mapping in its dialect. The test suite runs them over the synthetic dataset: every generated movement comes back from the subscription snapshots with monthly sums equal to the compiled MRR movement metrics, the monthly invoices give the month-start movements with a lapsed subscription churning the day after its last period, and the usage records give a tagged pair per month over the commitment, chained with the ledger's steps.
|
|
85
|
+
- Two vendor worked examples beside Acme in `test/fixture/` (`stripe/`, Redshift; `salesforce/`, BigQuery): a DDL, a GASP mapping and a README stating the source documentation and every choice, written from public API documentation and never standard content. The Stripe example is taken from a validated mapping to readiness level 4 in the test suite, derives from its item snapshots with a metered item's usage records and from its invoices, and its drift report names a planted difference; the Salesforce example compiles in BigQuery and stops at level 2 because a CRM maps no revenue event
|
|
86
86
|
- The readiness checks and the fix plan (`src/checks.ts`, behind `generate_checks` and `plan_fixes`) compile the rules in `cel.md` and `atl.md`, and the ontology's value ranges per reporting period (`src/ranges.ts`), into SQL over a mapping and rank what to fix by the metrics it unlocks; the synthetic dataset passes every check and a table of planted violations fails the right ones.
|
|
87
87
|
- The readiness report (`src/readiness.ts`, behind `readiness_report`) reads the level a company has reached (L1 to L4 of the interop plan) from the mapping, the model and the pasted check rows, and names what blocks the next level; the test suite takes the synthetic dataset to L4.
|
|
88
88
|
- The Decide step (`src/ossie/decide.ts`, behind `record_decision`) applies the answers people give to the model and the mapping and keeps every decision, with its owner, in the mapping's GASP extension; it closes readiness level 2 when no question and no candidate is open.
|
package/build/cel.d.ts
CHANGED
|
@@ -42,17 +42,41 @@
|
|
|
42
42
|
* a reactivation inside one interval leaves no event. Where amendment events
|
|
43
43
|
* exist they are the better source and the pairing rule in cel.md applies.
|
|
44
44
|
*
|
|
45
|
+
* Two more sources feed the same rules (cel.md, "Deriving Revenue Events from
|
|
46
|
+
* an Invoice Ledger" and "Deriving Usage Realisation"):
|
|
47
|
+
*
|
|
48
|
+
* - An invoice ledger (`source: "invoice"`) is a subscription ledger with one
|
|
49
|
+
* row per service period: an invoice's MRR is its `recurring_amount` over
|
|
50
|
+
* the months its service period covers, a subscription's MRR on a date is
|
|
51
|
+
* the sum over the invoices covering it, and its ledger dates are the
|
|
52
|
+
* starts of its service periods and the day after each end, up to the
|
|
53
|
+
* ledger's horizon (the latest service period start). Rules 2 to 7 then
|
|
54
|
+
* apply unchanged. The invoice dataset carries no segment.
|
|
55
|
+
* - Usage records (`usage: true`) add one pair of events per period in which
|
|
56
|
+
* `usage_amount` exceeds `committed_amount`: an `expansion` dated the
|
|
57
|
+
* period's end for the overage over the months the period covers, tagged
|
|
58
|
+
* `usage_realisation` with `change_nature` `temporary`, and its
|
|
59
|
+
* `scheduled_reversal`, a `contraction` dated the next day. The commitment
|
|
60
|
+
* is already in the subscription's MRR, so usage within it is no event. The
|
|
61
|
+
* pairs chain with the ledger's steps, so prior_mrr and new_mrr hold.
|
|
62
|
+
*
|
|
45
63
|
* The template uses the compiler's placeholders ({{field@dataset}},
|
|
46
64
|
* {{_table:dataset}}), so `fillTemplate` compiles it over any adapter or Ossie
|
|
47
|
-
* mapping that covers the
|
|
65
|
+
* mapping that covers the fields it reads.
|
|
48
66
|
*/
|
|
49
67
|
import { type DialectName } from "./dialect.js";
|
|
50
68
|
import type { QueryTemplate, StandardData } from "./types.js";
|
|
51
69
|
export declare const DERIVATION_DATASET = "subscription";
|
|
52
70
|
/** The subscription fields the derivation reads, in the order the ledger CTE selects them. */
|
|
53
71
|
export declare const DERIVATION_FIELDS: readonly ["subscription_id", "customer_id", "snapshot_date", "mrr", "subscription_status", "segment"];
|
|
72
|
+
/** The invoice fields the invoice source reads. */
|
|
73
|
+
export declare const INVOICE_FIELDS: readonly ["invoice_id", "subscription_id", "customer_id", "snapshot_date", "service_period_start", "service_period_end", "recurring_amount"];
|
|
74
|
+
/** The usage_record fields the usage branch reads. */
|
|
75
|
+
export declare const USAGE_FIELDS: readonly ["subscription_id", "customer_id", "period_start", "period_end", "committed_amount", "usage_amount"];
|
|
54
76
|
/** The revenue_event fields the derivation writes, in column order. */
|
|
55
77
|
export declare const DERIVED_FIELDS: readonly ["event_id", "event_date", "customer_id", "subscription_id", "event_type", "mrr_change", "currency", "prior_mrr", "new_mrr", "expansion_classification", "change_nature", "lifecycle_attribution", "segment"];
|
|
78
|
+
export type DerivationSource = "subscription" | "invoice";
|
|
79
|
+
export declare const DERIVATION_SOURCES: readonly DerivationSource[];
|
|
56
80
|
export interface DerivationOptions {
|
|
57
81
|
dialect?: DialectName;
|
|
58
82
|
/** ISO 4217 code written on every event: the ledger's currency (the subscription dataset carries none). */
|
|
@@ -63,22 +87,52 @@ export interface DerivationOptions {
|
|
|
63
87
|
* as steps from 0, for a ledger that starts with the business.
|
|
64
88
|
*/
|
|
65
89
|
firstSnapshot?: "opening" | "steps";
|
|
90
|
+
/** The ledger: the `subscription` dataset (the default) or the `invoice` dataset read as one row per service period. */
|
|
91
|
+
source?: DerivationSource;
|
|
92
|
+
/** Read the `usage_record` dataset too and add the usage realisation pairs. */
|
|
93
|
+
usage?: boolean;
|
|
66
94
|
}
|
|
67
|
-
/** The
|
|
95
|
+
/** The `ledger` CTE of the subscription source: one row per subscription per snapshot date, MRR when active. */
|
|
96
|
+
export declare function subscriptionLedgerSql(): string;
|
|
97
|
+
/**
|
|
98
|
+
* The `ledger` CTE of the invoice source: each invoice's latest snapshot, its MRR
|
|
99
|
+
* as the recurring amount over the months of its service period, the boundaries
|
|
100
|
+
* of each subscription's periods as its ledger dates and the invoices covering
|
|
101
|
+
* each date summed into its MRR, up to the ledger's horizon.
|
|
102
|
+
*/
|
|
103
|
+
export declare function invoiceLedgerSql(dialect?: DialectName): string;
|
|
104
|
+
/**
|
|
105
|
+
* The usage CTEs: each period whose usage exceeds its commitment, the overage
|
|
106
|
+
* over the months the period covers, and the pair of changes it adds to the
|
|
107
|
+
* customer's steps, a `usage` increase on the period's end and a
|
|
108
|
+
* `usage_reversal` decrease the day after.
|
|
109
|
+
*/
|
|
110
|
+
export declare function usageChangesSql(dialect?: DialectName): string;
|
|
111
|
+
/** The SQL that derives revenue events from the ledger (and the usage records), with compiler placeholders. */
|
|
68
112
|
export declare function revenueEventDerivationSql(opts: DerivationOptions): string;
|
|
113
|
+
/** The (field, dataset) pairs the derivation reads for the given source and usage setting. */
|
|
114
|
+
export declare function derivationPairs(opts: Pick<DerivationOptions, "source" | "usage">): Array<[field: string, dataset: string]>;
|
|
69
115
|
/** The derivation as a query template, so `fillTemplate` resolves it over an adapter or Ossie mapping like a metric's. */
|
|
70
116
|
export declare function revenueEventDerivation(opts: DerivationOptions): QueryTemplate;
|
|
71
117
|
/** The reference SQL over the GASP field names, as cel.md prints it. */
|
|
72
118
|
export declare function referenceDerivationSql(currency?: string): string;
|
|
119
|
+
/** The invoice source's ledger CTEs over the GASP field names, as cel.md prints them. */
|
|
120
|
+
export declare function referenceInvoiceLedgerSql(): string;
|
|
121
|
+
/** The usage CTEs over the GASP field names, as cel.md prints them. */
|
|
122
|
+
export declare function referenceUsageSql(): string;
|
|
73
123
|
export interface DeriveCelOptions {
|
|
74
124
|
dialect?: string;
|
|
75
125
|
currency?: string;
|
|
76
126
|
firstSnapshot?: "opening" | "steps";
|
|
127
|
+
source?: string;
|
|
128
|
+
usage?: boolean;
|
|
77
129
|
output?: "report" | "sql";
|
|
78
130
|
}
|
|
79
131
|
/**
|
|
80
132
|
* The derive_cel tool: the derivation compiled over a company's mapping (a GASP
|
|
81
|
-
* adapter or an Ossie mapping with its model),
|
|
82
|
-
*
|
|
133
|
+
* adapter or an Ossie mapping with its model), from the subscription or the
|
|
134
|
+
* invoice dataset, with the usage records when the mapping covers them, in the
|
|
135
|
+
* dialect asked for or the one the adapter's warehouse names, as a report or
|
|
136
|
+
* the SQL alone.
|
|
83
137
|
*/
|
|
84
138
|
export declare function deriveCelFromText(documentText: string | undefined, modelText: string | undefined, data: StandardData, opts?: DeriveCelOptions): string;
|
package/build/cel.js
CHANGED
|
@@ -42,32 +42,57 @@
|
|
|
42
42
|
* a reactivation inside one interval leaves no event. Where amendment events
|
|
43
43
|
* exist they are the better source and the pairing rule in cel.md applies.
|
|
44
44
|
*
|
|
45
|
+
* Two more sources feed the same rules (cel.md, "Deriving Revenue Events from
|
|
46
|
+
* an Invoice Ledger" and "Deriving Usage Realisation"):
|
|
47
|
+
*
|
|
48
|
+
* - An invoice ledger (`source: "invoice"`) is a subscription ledger with one
|
|
49
|
+
* row per service period: an invoice's MRR is its `recurring_amount` over
|
|
50
|
+
* the months its service period covers, a subscription's MRR on a date is
|
|
51
|
+
* the sum over the invoices covering it, and its ledger dates are the
|
|
52
|
+
* starts of its service periods and the day after each end, up to the
|
|
53
|
+
* ledger's horizon (the latest service period start). Rules 2 to 7 then
|
|
54
|
+
* apply unchanged. The invoice dataset carries no segment.
|
|
55
|
+
* - Usage records (`usage: true`) add one pair of events per period in which
|
|
56
|
+
* `usage_amount` exceeds `committed_amount`: an `expansion` dated the
|
|
57
|
+
* period's end for the overage over the months the period covers, tagged
|
|
58
|
+
* `usage_realisation` with `change_nature` `temporary`, and its
|
|
59
|
+
* `scheduled_reversal`, a `contraction` dated the next day. The commitment
|
|
60
|
+
* is already in the subscription's MRR, so usage within it is no event. The
|
|
61
|
+
* pairs chain with the ledger's steps, so prior_mrr and new_mrr hold.
|
|
62
|
+
*
|
|
45
63
|
* The template uses the compiler's placeholders ({{field@dataset}},
|
|
46
64
|
* {{_table:dataset}}), so `fillTemplate` compiles it over any adapter or Ossie
|
|
47
|
-
* mapping that covers the
|
|
65
|
+
* mapping that covers the fields it reads.
|
|
48
66
|
*/
|
|
49
67
|
import { DIALECT } from "./dialect.js";
|
|
50
68
|
import { exampleSql } from "./compile.js";
|
|
51
|
-
import { chooseDialect, fillTemplate } from "./tools.js";
|
|
69
|
+
import { chooseDialect, fillTemplate, resolveAdapter } from "./tools.js";
|
|
52
70
|
import { loadAdapter } from "./ossie/adapt.js";
|
|
53
71
|
import { resolveDocLink } from "./urls.js";
|
|
54
72
|
export const DERIVATION_DATASET = "subscription";
|
|
55
73
|
/** The subscription fields the derivation reads, in the order the ledger CTE selects them. */
|
|
56
74
|
export const DERIVATION_FIELDS = ["subscription_id", "customer_id", "snapshot_date", "mrr", "subscription_status", "segment"];
|
|
75
|
+
/** The invoice fields the invoice source reads. */
|
|
76
|
+
export const INVOICE_FIELDS = ["invoice_id", "subscription_id", "customer_id", "snapshot_date", "service_period_start", "service_period_end", "recurring_amount"];
|
|
77
|
+
/** The usage_record fields the usage branch reads. */
|
|
78
|
+
export const USAGE_FIELDS = ["subscription_id", "customer_id", "period_start", "period_end", "committed_amount", "usage_amount"];
|
|
57
79
|
/** The revenue_event fields the derivation writes, in column order. */
|
|
58
80
|
export const DERIVED_FIELDS = ["event_id", "event_date", "customer_id", "subscription_id", "event_type", "mrr_change", "currency", "prior_mrr", "new_mrr", "expansion_classification", "change_nature", "lifecycle_attribution", "segment"];
|
|
59
|
-
const
|
|
81
|
+
export const DERIVATION_SOURCES = ["subscription", "invoice"];
|
|
82
|
+
const ph = (field, dataset = DERIVATION_DATASET) => `{{${field}@${dataset}}}`;
|
|
60
83
|
/** MRR is kept to the cent, so a chained total within half a cent of zero is zero (the sums are floats on some warehouses). */
|
|
61
84
|
const ZERO = "0.005";
|
|
62
|
-
/** The
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
85
|
+
/** The length of a period of whole days in months (365.25 / 12 days each), as a fraction. */
|
|
86
|
+
const monthsExpr = (d, start, end) => `${d.periodDays(end, start)} * 12.0 / 365.25`;
|
|
87
|
+
/**
|
|
88
|
+
* The months a period covers, from its length in months: the nearest whole
|
|
89
|
+
* number (so January and February are one month each and a year is twelve), or
|
|
90
|
+
* the fraction itself for a period shorter than about a fortnight.
|
|
91
|
+
*/
|
|
92
|
+
const MONTHS_COVERED = "CASE WHEN ROUND(months) < 1 THEN months ELSE ROUND(months) END";
|
|
93
|
+
/** The `ledger` CTE of the subscription source: one row per subscription per snapshot date, MRR when active. */
|
|
94
|
+
export function subscriptionLedgerSql() {
|
|
95
|
+
return `ledger AS (
|
|
71
96
|
SELECT ${ph("subscription_id")} AS subscription_id,
|
|
72
97
|
${ph("customer_id")} AS customer_id,
|
|
73
98
|
${ph("snapshot_date")} AS snapshot_date,
|
|
@@ -75,18 +100,117 @@ export function revenueEventDerivationSql(opts) {
|
|
|
75
100
|
CASE WHEN ${ph("subscription_status")} = 'active' THEN ROUND(COALESCE(${ph("mrr")}, 0), 2) ELSE 0 END AS mrr
|
|
76
101
|
FROM {{_table:${DERIVATION_DATASET}}}
|
|
77
102
|
WHERE ${ph("snapshot_date")} IS NOT NULL
|
|
103
|
+
)`;
|
|
104
|
+
}
|
|
105
|
+
/**
|
|
106
|
+
* The `ledger` CTE of the invoice source: each invoice's latest snapshot, its MRR
|
|
107
|
+
* as the recurring amount over the months of its service period, the boundaries
|
|
108
|
+
* of each subscription's periods as its ledger dates and the invoices covering
|
|
109
|
+
* each date summed into its MRR, up to the ledger's horizon.
|
|
110
|
+
*/
|
|
111
|
+
export function invoiceLedgerSql(dialect = "postgres") {
|
|
112
|
+
const d = DIALECT[dialect];
|
|
113
|
+
const i = (f) => ph(f, "invoice");
|
|
114
|
+
return `invoices AS (
|
|
115
|
+
SELECT invoice_id, subscription_id, customer_id, service_period_start, service_period_end, recurring_amount
|
|
116
|
+
FROM (
|
|
117
|
+
SELECT ${i("invoice_id")} AS invoice_id,
|
|
118
|
+
${i("subscription_id")} AS subscription_id,
|
|
119
|
+
${i("customer_id")} AS customer_id,
|
|
120
|
+
${i("service_period_start")} AS service_period_start,
|
|
121
|
+
${i("service_period_end")} AS service_period_end,
|
|
122
|
+
${i("recurring_amount")} AS recurring_amount,
|
|
123
|
+
ROW_NUMBER() OVER (PARTITION BY ${i("invoice_id")} ORDER BY ${i("snapshot_date")} DESC) AS latest
|
|
124
|
+
FROM {{_table:invoice}}
|
|
125
|
+
WHERE ${i("subscription_id")} IS NOT NULL AND ${i("service_period_start")} IS NOT NULL AND ${i("service_period_end")} IS NOT NULL
|
|
126
|
+
) i
|
|
127
|
+
WHERE latest = 1
|
|
128
|
+
),
|
|
129
|
+
periods AS (
|
|
130
|
+
SELECT invoice_id, subscription_id, customer_id, service_period_start, service_period_end,
|
|
131
|
+
ROUND(COALESCE(recurring_amount, 0) / (${MONTHS_COVERED}), 2) AS mrr
|
|
132
|
+
FROM (
|
|
133
|
+
SELECT i.*, ${monthsExpr(d, "service_period_start", "service_period_end")} AS months FROM invoices i
|
|
134
|
+
) m
|
|
135
|
+
),
|
|
136
|
+
boundaries AS (
|
|
137
|
+
SELECT subscription_id, customer_id, service_period_start AS snapshot_date FROM periods
|
|
138
|
+
UNION
|
|
139
|
+
SELECT subscription_id, customer_id, CAST(${d.plus("service_period_end", 1, "day")} AS DATE) FROM periods
|
|
140
|
+
),
|
|
141
|
+
horizon AS (
|
|
142
|
+
SELECT MAX(service_period_start) AS latest FROM periods
|
|
78
143
|
),
|
|
144
|
+
ledger AS (
|
|
145
|
+
SELECT b.subscription_id, b.customer_id, b.snapshot_date,
|
|
146
|
+
${d.toText("NULL")} AS segment,
|
|
147
|
+
ROUND(COALESCE(SUM(p.mrr), 0), 2) AS mrr
|
|
148
|
+
FROM boundaries b
|
|
149
|
+
LEFT JOIN periods p ON p.subscription_id = b.subscription_id AND p.service_period_start <= b.snapshot_date AND b.snapshot_date <= p.service_period_end
|
|
150
|
+
WHERE b.snapshot_date <= (SELECT latest FROM horizon)
|
|
151
|
+
GROUP BY b.subscription_id, b.customer_id, b.snapshot_date
|
|
152
|
+
)`;
|
|
153
|
+
}
|
|
154
|
+
/**
|
|
155
|
+
* The usage CTEs: each period whose usage exceeds its commitment, the overage
|
|
156
|
+
* over the months the period covers, and the pair of changes it adds to the
|
|
157
|
+
* customer's steps, a `usage` increase on the period's end and a
|
|
158
|
+
* `usage_reversal` decrease the day after.
|
|
159
|
+
*/
|
|
160
|
+
export function usageChangesSql(dialect = "postgres") {
|
|
161
|
+
const d = DIALECT[dialect];
|
|
162
|
+
const u = (f) => ph(f, "usage_record");
|
|
163
|
+
return `usage_periods AS (
|
|
164
|
+
SELECT subscription_id, customer_id, period_end,
|
|
165
|
+
ROUND(overage / (${MONTHS_COVERED}), 2) AS overage
|
|
166
|
+
FROM (
|
|
167
|
+
SELECT ${u("subscription_id")} AS subscription_id,
|
|
168
|
+
${u("customer_id")} AS customer_id,
|
|
169
|
+
${u("period_end")} AS period_end,
|
|
170
|
+
${u("usage_amount")} - ${u("committed_amount")} AS overage,
|
|
171
|
+
${monthsExpr(d, u("period_start"), u("period_end"))} AS months
|
|
172
|
+
FROM {{_table:usage_record}}
|
|
173
|
+
WHERE ${u("period_start")} IS NOT NULL AND ${u("period_end")} IS NOT NULL
|
|
174
|
+
AND ${u("usage_amount")} > ${u("committed_amount")}
|
|
175
|
+
) u
|
|
176
|
+
),
|
|
177
|
+
usage_changes AS (
|
|
178
|
+
SELECT subscription_id, customer_id, period_end AS snapshot_date, ${d.toText("NULL")} AS segment, 0 AS opening, overage AS mrr_change, 'usage' AS kind
|
|
179
|
+
FROM usage_periods WHERE overage >= 0.01
|
|
180
|
+
UNION ALL
|
|
181
|
+
SELECT subscription_id, customer_id, CAST(${d.plus("period_end", 1, "day")} AS DATE), ${d.toText("NULL")}, 0, -overage, 'usage_reversal'
|
|
182
|
+
FROM usage_periods WHERE overage >= 0.01
|
|
183
|
+
)`;
|
|
184
|
+
}
|
|
185
|
+
/** The SQL that derives revenue events from the ledger (and the usage records), with compiler placeholders. */
|
|
186
|
+
export function revenueEventDerivationSql(opts) {
|
|
187
|
+
const d = DIALECT[opts.dialect ?? "postgres"];
|
|
188
|
+
if (!/^[A-Z]{3}$/.test(opts.currency))
|
|
189
|
+
throw new Error(`currency must be an ISO 4217 code, got ${JSON.stringify(opts.currency)}`);
|
|
190
|
+
const source = opts.source ?? "subscription";
|
|
191
|
+
if (!DERIVATION_SOURCES.includes(source))
|
|
192
|
+
throw new Error(`source must be one of ${DERIVATION_SOURCES.join(", ")}, got ${JSON.stringify(source)}`);
|
|
193
|
+
const opening = opts.firstSnapshot === "steps" ? "0" : "CASE WHEN snapshot_date = MIN(snapshot_date) OVER () THEN 1 ELSE 0 END";
|
|
194
|
+
// Within a date a customer's increases come before its decreases, then subscription id and kind, so the chained MRR is deterministic.
|
|
195
|
+
const chain = "PARTITION BY c.customer_id ORDER BY c.snapshot_date, CASE WHEN c.mrr_change > 0 THEN 0 ELSE 1 END, c.subscription_id, c.kind";
|
|
196
|
+
const ledger = source === "invoice" ? invoiceLedgerSql(d.name) : subscriptionLedgerSql();
|
|
197
|
+
const usage = opts.usage ? `,
|
|
198
|
+
${usageChangesSql(d.name)}` : "";
|
|
199
|
+
const union = opts.usage ? `
|
|
200
|
+
UNION ALL
|
|
201
|
+
SELECT subscription_id, customer_id, snapshot_date, segment, opening, mrr_change, kind FROM usage_changes` : "";
|
|
202
|
+
return `WITH ${ledger},
|
|
79
203
|
steps AS (
|
|
80
204
|
SELECT subscription_id, customer_id, snapshot_date, segment, mrr,
|
|
81
205
|
COALESCE(LAG(mrr) OVER (PARTITION BY subscription_id ORDER BY snapshot_date), 0) AS prior_sub_mrr,
|
|
82
206
|
${opening} AS opening
|
|
83
207
|
FROM ledger
|
|
84
|
-
),
|
|
208
|
+
)${usage},
|
|
85
209
|
changes AS (
|
|
86
210
|
SELECT subscription_id, customer_id, snapshot_date, segment, opening,
|
|
87
|
-
mrr - prior_sub_mrr AS mrr_change
|
|
211
|
+
mrr - prior_sub_mrr AS mrr_change, 'ledger' AS kind
|
|
88
212
|
FROM steps
|
|
89
|
-
WHERE mrr <> prior_sub_mrr
|
|
213
|
+
WHERE mrr <> prior_sub_mrr${union}
|
|
90
214
|
),
|
|
91
215
|
day_changes AS (
|
|
92
216
|
SELECT customer_id, snapshot_date, SUM(mrr_change) AS day_change
|
|
@@ -104,7 +228,7 @@ day_state AS (
|
|
|
104
228
|
FROM days
|
|
105
229
|
),
|
|
106
230
|
chained AS (
|
|
107
|
-
SELECT c.subscription_id, c.customer_id, c.snapshot_date, c.segment, c.opening, c.mrr_change,
|
|
231
|
+
SELECT c.subscription_id, c.customer_id, c.snapshot_date, c.segment, c.opening, c.mrr_change, c.kind,
|
|
108
232
|
COALESCE(SUM(c.mrr_change) OVER (${chain} ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING), 0) AS prior_mrr,
|
|
109
233
|
SUM(c.mrr_change) OVER (${chain} ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS new_mrr,
|
|
110
234
|
s.day_start, s.day_end, s.active_before
|
|
@@ -112,8 +236,10 @@ chained AS (
|
|
|
112
236
|
JOIN day_state s ON s.customer_id = c.customer_id AND s.snapshot_date = c.snapshot_date
|
|
113
237
|
),
|
|
114
238
|
typed AS (
|
|
115
|
-
SELECT subscription_id, customer_id, snapshot_date, segment, opening, mrr_change, prior_mrr, new_mrr,
|
|
239
|
+
SELECT subscription_id, customer_id, snapshot_date, segment, opening, mrr_change, prior_mrr, new_mrr, kind,
|
|
116
240
|
CASE
|
|
241
|
+
WHEN kind = 'usage' THEN 'expansion'
|
|
242
|
+
WHEN kind = 'usage_reversal' THEN 'contraction'
|
|
117
243
|
WHEN mrr_change > 0 AND ABS(day_start) < ${ZERO} AND active_before = 1 THEN 'reactivation'
|
|
118
244
|
WHEN mrr_change > 0 AND ABS(day_start) < ${ZERO} THEN 'new'
|
|
119
245
|
WHEN mrr_change < 0 AND ABS(day_end) < ${ZERO} THEN 'churn'
|
|
@@ -122,7 +248,7 @@ typed AS (
|
|
|
122
248
|
END AS event_type
|
|
123
249
|
FROM chained
|
|
124
250
|
)
|
|
125
|
-
SELECT ${d.toText("subscription_id")} || ':' || ${d.toText("snapshot_date")} AS event_id,
|
|
251
|
+
SELECT ${d.toText("subscription_id")} || CASE WHEN kind = 'ledger' THEN ':' ELSE ':usage:' END || ${d.toText("snapshot_date")} AS event_id,
|
|
126
252
|
snapshot_date AS event_date,
|
|
127
253
|
customer_id,
|
|
128
254
|
subscription_id,
|
|
@@ -131,8 +257,8 @@ SELECT ${d.toText("subscription_id")} || ':' || ${d.toText("snapshot_date")} AS
|
|
|
131
257
|
'${opts.currency}' AS currency,
|
|
132
258
|
prior_mrr,
|
|
133
259
|
new_mrr,
|
|
134
|
-
${d.toText("NULL")} AS expansion_classification,
|
|
135
|
-
'permanent' AS change_nature,
|
|
260
|
+
CASE WHEN kind = 'ledger' THEN ${d.toText("NULL")} ELSE 'usage_realisation' END AS expansion_classification,
|
|
261
|
+
CASE kind WHEN 'usage' THEN 'temporary' WHEN 'usage_reversal' THEN 'scheduled_reversal' ELSE 'permanent' END AS change_nature,
|
|
136
262
|
CASE event_type
|
|
137
263
|
WHEN 'new' THEN 'acquisition'
|
|
138
264
|
WHEN 'expansion' THEN 'expansion'
|
|
@@ -142,31 +268,50 @@ SELECT ${d.toText("subscription_id")} || ':' || ${d.toText("snapshot_date")} AS
|
|
|
142
268
|
segment
|
|
143
269
|
FROM typed
|
|
144
270
|
WHERE opening = 0
|
|
145
|
-
ORDER BY snapshot_date, customer_id, CASE WHEN mrr_change > 0 THEN 0 ELSE 1 END, subscription_id`;
|
|
271
|
+
ORDER BY snapshot_date, customer_id, CASE WHEN mrr_change > 0 THEN 0 ELSE 1 END, subscription_id, kind`;
|
|
272
|
+
}
|
|
273
|
+
/** The (field, dataset) pairs the derivation reads for the given source and usage setting. */
|
|
274
|
+
export function derivationPairs(opts) {
|
|
275
|
+
const source = opts.source ?? "subscription";
|
|
276
|
+
const ledger = source === "invoice" ? INVOICE_FIELDS.map((f) => [f, "invoice"]) : DERIVATION_FIELDS.map((f) => [f, DERIVATION_DATASET]);
|
|
277
|
+
return opts.usage ? [...ledger, ...USAGE_FIELDS.map((f) => [f, "usage_record"])] : ledger;
|
|
146
278
|
}
|
|
147
279
|
/** The derivation as a query template, so `fillTemplate` resolves it over an adapter or Ossie mapping like a metric's. */
|
|
148
280
|
export function revenueEventDerivation(opts) {
|
|
281
|
+
const source = opts.source ?? "subscription";
|
|
282
|
+
const pairs = derivationPairs(opts);
|
|
283
|
+
const datasets = [...new Set(pairs.map(([, ds]) => ds))];
|
|
149
284
|
return {
|
|
150
285
|
metric: "cel_revenue_events",
|
|
151
|
-
fields: [...
|
|
152
|
-
datasets
|
|
153
|
-
sourceTable:
|
|
286
|
+
fields: [...new Set(pairs.map(([f]) => f))],
|
|
287
|
+
datasets,
|
|
288
|
+
sourceTable: source,
|
|
154
289
|
grain: "event",
|
|
155
|
-
frequency: "as the ledger is snapshotted",
|
|
156
|
-
source: "subscription ledger",
|
|
290
|
+
frequency: source === "invoice" ? "as the service periods start and end" : "as the ledger is snapshotted",
|
|
291
|
+
source: source === "invoice" ? "invoice ledger" : "subscription ledger",
|
|
157
292
|
parameterizedQuery: revenueEventDerivationSql(opts),
|
|
158
293
|
staged: false,
|
|
159
|
-
placeholders:
|
|
294
|
+
placeholders: pairs.map(([f, ds]) => `${f}@${ds}`),
|
|
160
295
|
};
|
|
161
296
|
}
|
|
162
297
|
/** The reference SQL over the GASP field names, as cel.md prints it. */
|
|
163
298
|
export function referenceDerivationSql(currency = "USD") {
|
|
164
299
|
return exampleSql(revenueEventDerivationSql({ dialect: "postgres", currency }));
|
|
165
300
|
}
|
|
301
|
+
/** The invoice source's ledger CTEs over the GASP field names, as cel.md prints them. */
|
|
302
|
+
export function referenceInvoiceLedgerSql() {
|
|
303
|
+
return exampleSql(invoiceLedgerSql("postgres"));
|
|
304
|
+
}
|
|
305
|
+
/** The usage CTEs over the GASP field names, as cel.md prints them. */
|
|
306
|
+
export function referenceUsageSql() {
|
|
307
|
+
return exampleSql(usageChangesSql("postgres"));
|
|
308
|
+
}
|
|
166
309
|
/**
|
|
167
310
|
* The derive_cel tool: the derivation compiled over a company's mapping (a GASP
|
|
168
|
-
* adapter or an Ossie mapping with its model),
|
|
169
|
-
*
|
|
311
|
+
* adapter or an Ossie mapping with its model), from the subscription or the
|
|
312
|
+
* invoice dataset, with the usage records when the mapping covers them, in the
|
|
313
|
+
* dialect asked for or the one the adapter's warehouse names, as a report or
|
|
314
|
+
* the SQL alone.
|
|
170
315
|
*/
|
|
171
316
|
export function deriveCelFromText(documentText, modelText, data, opts = {}) {
|
|
172
317
|
if (!documentText?.trim())
|
|
@@ -176,32 +321,52 @@ export function deriveCelFromText(documentText, modelText, data, opts = {}) {
|
|
|
176
321
|
return "Pass currency: the ISO 4217 code of the ledger's currency (USD, EUR, GBP ...). Every derived event carries it; the subscription dataset has no currency field to read it from.";
|
|
177
322
|
if (!/^[A-Z]{3}$/.test(currency))
|
|
178
323
|
return `currency must be a three-letter ISO 4217 code, got ${JSON.stringify(opts.currency)}.`;
|
|
324
|
+
const source = (opts.source?.trim().toLowerCase() || "subscription");
|
|
325
|
+
if (!DERIVATION_SOURCES.includes(source))
|
|
326
|
+
return `source must be one of ${DERIVATION_SOURCES.join(", ")}, got ${JSON.stringify(opts.source)}.`;
|
|
179
327
|
const loaded = loadAdapter(documentText, modelText, data);
|
|
180
328
|
if (!loaded.adapter)
|
|
181
329
|
return loaded.error;
|
|
182
330
|
const chosen = chooseDialect(opts.dialect, loaded.adapter.warehouse);
|
|
183
331
|
if ("error" in chosen)
|
|
184
332
|
return chosen.error;
|
|
185
|
-
const
|
|
333
|
+
const resolved = resolveAdapter(loaded.adapter, data);
|
|
334
|
+
const usageMapped = Boolean(resolved.tables.usage_record) || Object.keys(resolved.columns.usage_record ?? {}).length > 0;
|
|
335
|
+
const usage = opts.usage ?? usageMapped;
|
|
336
|
+
const template = revenueEventDerivation({ dialect: chosen.dialect, currency, firstSnapshot: opts.firstSnapshot, source, usage });
|
|
186
337
|
const r = fillTemplate(template, loaded.adapter, data);
|
|
187
338
|
if (opts.output === "sql")
|
|
188
339
|
return r.sql;
|
|
340
|
+
const ledgerName = source === "invoice" ? "invoice ledger" : "subscription ledger";
|
|
341
|
+
const rulesAnchor = source === "invoice" ? "cel.md#deriving-revenue-events-from-an-invoice-ledger" : "cel.md#deriving-revenue-events-from-a-subscription-ledger";
|
|
342
|
+
const ledgerPairs = template.placeholders.filter((p) => !p.endsWith("@usage_record"));
|
|
343
|
+
const usagePairs = template.placeholders.filter((p) => p.endsWith("@usage_record"));
|
|
344
|
+
const mappedOf = (pairs) => pairs.filter((p) => r.mapped.some((m) => `${m.field}@${m.dataset}` === p)).length;
|
|
189
345
|
const p = [
|
|
190
|
-
|
|
191
|
-
`**Rules:** ${resolveDocLink("cel.md#deriving-
|
|
346
|
+
`# CEL Revenue_Events derived from your ${ledgerName}${usage ? " and usage records" : ""}`,
|
|
347
|
+
`**Rules:** ${resolveDocLink(rulesAnchor)}${usage ? ` and ${resolveDocLink("cel.md#deriving-usage-realisation")}` : ""}`,
|
|
192
348
|
`**Mapping:** ${loaded.format === "v1" ? "GASP adapter" : "Ossie mapping"}${loaded.adapter.organization ? ` (${loaded.adapter.organization})` : ""}`,
|
|
193
349
|
`**Warehouse:** ${loaded.adapter.warehouse || "not specified"}`,
|
|
194
350
|
`**Dialect:** ${chosen.note}`,
|
|
195
|
-
`**Reads:** the \`${
|
|
351
|
+
`**Reads:** the \`${source}\` dataset, ${mappedOf(ledgerPairs)} / ${ledgerPairs.length} fields mapped${usage ? `; the \`usage_record\` dataset, ${mappedOf(usagePairs)} / ${usagePairs.length} fields mapped${opts.usage === undefined ? " (read because the mapping covers it; pass usage: false to leave it out)" : ""}` : usageMapped ? "" : "; no usage records (map the `usage_record` dataset, or pass usage: true, for the usage realisation events)"}`,
|
|
196
352
|
`**Writes:** one \`revenue_event\` row per movement (${DERIVED_FIELDS.join(", ")}), currency \`${currency}\``,
|
|
197
353
|
];
|
|
198
354
|
if (r.missingFields.length)
|
|
199
|
-
p.push(`**Unmapped fields (left as placeholders):** \`${r.missingFields.join("\`, \`")}\`. Map each on
|
|
355
|
+
p.push(`**Unmapped fields (left as placeholders):** \`${r.missingFields.join("\`, \`")}\`. Map each on its dataset and run again.`);
|
|
200
356
|
if (r.missingTables.length)
|
|
201
357
|
p.push(`**Dataset with no table:** ${r.missingTables.join(", ")}. Name its table under "datasets" in the adapter, or map one of its fields with a schema.table column reference.`);
|
|
202
358
|
for (const c of r.conflicts)
|
|
203
359
|
p.push(`**Conflict:** ${c}`);
|
|
204
|
-
|
|
360
|
+
const firstRows = opts.firstSnapshot === "steps"
|
|
361
|
+
? "Every first row is a step from 0 (first_snapshot: steps), so the ledger must start with the business."
|
|
362
|
+
: "Rows on the ledger's first date are opening balances and produce no event; a customer that churned before the ledger starts comes back as new, not reactivation. Pass first_snapshot: steps if the ledger starts with the business.";
|
|
363
|
+
const limits = source === "invoice"
|
|
364
|
+
? `**What the invoices can and cannot say.** Each invoice is read at its latest snapshot as one service period: its MRR is \`recurring_amount\` over the months the period covers (whole months for monthly, quarterly and annual billing; a prorated invoice for part of a period is approximate). A subscription's MRR on a date is the sum over the invoices covering it, its ledger dates are the starts of its service periods and the day after each end, and a date after the ledger's horizon (the latest service period start) is not read, so a subscription whose last invoice is still current does not churn. Invoices with no subscription or no service period (one-off charges) produce no event. ${firstRows} Each movement is dated the service period that first bills it, so a change agreed mid-period shows on the next period's start unless a prorated invoice was issued. The invoice dataset carries no segment, so \`segment\` is empty; fill it from the customer snapshot. \`expansion_classification\` is empty: the invoices show that MRR rose, not what was bought, and the Operating forms of ARR and NRR read that tag; fill it from your amendment records or record the classification as a decision. Where a subscription ledger exists it is the better source (source: subscription).`
|
|
365
|
+
: `**What the ledger can and cannot say.** Each movement is dated the snapshot that first shows it, so the ledger's cadence is the grain of the events: daily snapshots show every movement on its day, month-end snapshots net a month's movements into one event dated the month end. ${firstRows} The ledger needs one row per subscription per snapshot date and must keep a cancelled subscription's row; a subscription that stops appearing produces no churn. A null MRR on an active row counts as 0. \`expansion_classification\` is empty: the ledger shows that MRR rose, not what was bought, and the Operating forms of ARR and NRR read that tag; fill it from your amendment records or record the classification as a decision. Renewals change no MRR and are not derived here.`;
|
|
366
|
+
p.push("", limits);
|
|
367
|
+
if (usage)
|
|
368
|
+
p.push("", "**Usage realisation.** Each usage period whose `usage_amount` exceeds its `committed_amount` adds a pair of events: an `expansion` dated the period's end for the overage over the months the period covers, tagged `usage_realisation` with `change_nature` `temporary`, and a `contraction` the next day with `change_nature` `scheduled_reversal`. The commitment is already in the subscription's MRR, so usage within it is no event, and a commitment raised to cover the usage shows on the ledger as a permanent expansion. The pairs chain with the ledger's steps, so `prior_mrr` and `new_mrr` hold across both. Their ids are `<subscription_id>:usage:<date>`.");
|
|
369
|
+
p.push("", `Run the query to build (or rebuild) the \`revenue_event\` table; it reads the whole ledger and carries no period parameters. Map the result as the \`revenue_event\` dataset and the MRR movement metrics, NRR and GRR compile over it with generate_query or compile_model.`, "", "\`\`\`sql\n" + r.sql + "\n\`\`\`");
|
|
205
370
|
if (r.mapped.length) {
|
|
206
371
|
p.push("", "**Field mapping used:**");
|
|
207
372
|
for (const m of r.mapped)
|
package/build/index.js
CHANGED
|
@@ -664,14 +664,16 @@ server.tool("record_decision", "Record the decisions people make on a profiled a
|
|
|
664
664
|
output: z.enum(["report", "mapping", "model", "json"]).optional().describe("'report' (default): what was applied, what is open and the updated documents as YAML; 'mapping': the mapping alone; 'model': the model alone; 'json': everything as data."),
|
|
665
665
|
}, async ({ semantic_model, mapping, decisions, output }) => ({ content: [{ type: "text", text: recordDecisionFromText(semantic_model, mapping, decisions, data, { output }) }] }));
|
|
666
666
|
// --- Tool 14: derive_cel ---
|
|
667
|
-
server.tool("derive_cel", "SQL that derives CEL Revenue_Events from your
|
|
668
|
-
adapter: z.string().describe("Your mapping as text: a GASP adapter (JSON; schema: https://gaspwiki.com/schemas/gasp-adapter-v1.json) or an Ossie mapping document (YAML or JSON). It must cover the subscription dataset. Check it with validate_mapping first."),
|
|
667
|
+
server.tool("derive_cel", "SQL that derives CEL Revenue_Events from your billing data, by the rules in the standard (cel.md, 'Deriving Revenue Events from a Subscription Ledger', 'Deriving Revenue Events from an Invoice Ledger' and 'Deriving Usage Realisation'). Most billing systems hold subscriptions or invoices, not events. source: subscription (default) turns the subscription dataset your mapping covers (subscription_id, customer_id, snapshot_date, mrr, subscription_status, segment, as snapshots at any cadence or a version history) into one revenue_event row per movement: a step is a change in a subscription's active MRR, the ledger's first date is the opening balance, prior_mrr and new_mrr are the customer's MRR chained so prior_mrr + mrr_change = new_mrr holds on every row, and the event type (new, expansion, contraction, churn, reactivation) is read from the customer's MRR around the step, with the ATL lifecycle tag, change_nature permanent and an empty expansion_classification (the ledger cannot know it). source: invoice reads the invoice dataset instead (invoice_id, subscription_id, customer_id, snapshot_date, service_period_start, service_period_end, recurring_amount): each invoice is one service period whose MRR is the recurring amount over the months it covers, and the same rules apply. When the mapping covers the usage_record dataset (subscription_id, customer_id, period_start, period_end, committed_amount, usage_amount), or usage is true, each period whose usage exceeds its commitment adds an expansion tagged usage_realisation with change_nature temporary dated the period's end and its scheduled_reversal the next day, chained with the ledger's steps. Takes the same mapping as generate_query (a GASP adapter or an Ossie mapping with its semantic_model) and the ledger's currency; the dialect is the one asked for, else the adapter's warehouse, else postgres. Unmapped fields stay as {{field@dataset}} placeholders and are listed. Run the result to build the revenue_event table, map it, and the MRR movement metrics compile over it.", {
|
|
668
|
+
adapter: z.string().describe("Your mapping as text: a GASP adapter (JSON; schema: https://gaspwiki.com/schemas/gasp-adapter-v1.json) or an Ossie mapping document (YAML or JSON). It must cover the subscription dataset (or the invoice dataset with source: invoice). Check it with validate_mapping first."),
|
|
669
669
|
currency: z.string().describe("ISO 4217 code of the ledger's currency (USD, EUR, GBP ...), written on every derived event."),
|
|
670
670
|
semantic_model: z.string().optional().describe("With an Ossie mapping: your semantic model document (YAML or JSON), so table names and field expressions resolve. Ignored for a GASP adapter."),
|
|
671
|
+
source: z.enum(["subscription", "invoice"]).optional().describe("The ledger to read: 'subscription' (default), the subscription dataset as snapshots or a version history; 'invoice', the invoice dataset read as one row per service period (recurring_amount over the months between service_period_start and service_period_end). Use invoice when the billing system keeps no subscription history."),
|
|
672
|
+
usage: z.boolean().optional().describe("Read the usage_record dataset too and add the usage realisation pairs. Default: true when the mapping covers usage_record, else false."),
|
|
671
673
|
dialect: z.enum(DIALECTS).optional().describe("SQL dialect: postgres (also Redshift), snowflake, bigquery or databricks. Omit to use the adapter's warehouse when it names one of them, else postgres."),
|
|
672
|
-
first_snapshot: z.enum(["opening", "steps"]).optional().describe("What the rows on the ledger's first
|
|
673
|
-
output: z.enum(["report", "sql"]).optional().describe("'report' (default): the SQL with the mapping it used, the unmapped fields and what the
|
|
674
|
-
}, async ({ adapter, currency, semantic_model, dialect, first_snapshot, output }) => ({ content: [{ type: "text", text: deriveCelFromText(adapter, semantic_model, data, { dialect, currency, firstSnapshot: first_snapshot, output }) }] }));
|
|
674
|
+
first_snapshot: z.enum(["opening", "steps"]).optional().describe("What the rows on the ledger's first date are. 'opening' (default): opening balances that seed each customer's MRR and produce no event, for a ledger that starts after the oldest subscription. 'steps': steps from 0, for a ledger that starts with the business."),
|
|
675
|
+
output: z.enum(["report", "sql"]).optional().describe("'report' (default): the SQL with the mapping it used, the unmapped fields and what the source can and cannot say; 'sql': the query alone."),
|
|
676
|
+
}, async ({ adapter, currency, semantic_model, source, usage, dialect, first_snapshot, output }) => ({ content: [{ type: "text", text: deriveCelFromText(adapter, semantic_model, data, { dialect, currency, firstSnapshot: first_snapshot, source, usage, output }) }] }));
|
|
675
677
|
// --- Tool 15: generate_checks ---
|
|
676
678
|
server.tool("generate_checks", "The readiness checks the GASP Standard states, as SQL over the datasets your mapping covers (readiness level 3, the Check step). From cel.md and atl.md: CEL row identity (prior_mrr + mrr_change = new_mrr on every revenue event), MRR continuity (a subscription's mrr_start equals its mrr on the previous snapshot), unique event_id per event dataset, the ATL tags (one lifecycle_attribution per event, an expansion_classification on every expansion, contraction and reactivation revenue event and every amendment, a cost_function on every cost event), customer identity both ways (every revenue event's customer is a known customer, every customer active on the latest snapshot has a revenue event) freshness (the latest date per dataset within a window) and, for every fully mapped metric with a hard range rule in the ontology (NRR at least 0, GRR at most 100%, Gross Margin, ARR and others), its value in every reporting period of your period spine (the table compile_model reads; period_spine names another). One statement, one row per check with its violation count. GASP has no warehouse access: run it, then pass the rows back as results to get pass or fail per check and per metric; each check names the metrics whose formulas read what it covers. Checks whose fields the mapping lacks are listed with what they need. Takes the mapping generate_query takes; the dialect is the one asked for, else the warehouse's, else postgres. Follow with plan_fixes for the ranked list.", {
|
|
677
679
|
adapter: z.string().describe("Your mapping as text: a GASP adapter (JSON; schema: https://gaspwiki.com/schemas/gasp-adapter-v1.json) or an Ossie mapping document (YAML or JSON). Check it with validate_mapping first."),
|
package/build/ossie/profile.js
CHANGED
|
@@ -407,12 +407,16 @@ function inferTime(t) {
|
|
|
407
407
|
return { value: top.c.column, confidence: "high", evidence: "the only date column" };
|
|
408
408
|
const runnersUp = scored.slice(1).filter((x) => x.s[0] >= top.s[0] - 5);
|
|
409
409
|
// A start date beside its end date is a span: which end places the row is the business's call (a trial counts when it ends).
|
|
410
|
+
// period_start beside period_end is not: the pair bounds the period the row describes, and the row is dated its start.
|
|
410
411
|
const topStem = name(top.c).replace(/_(date|at|on|ts|time|day)$/, "");
|
|
411
|
-
const
|
|
412
|
+
const periodPair = name(top.c) === "period_start" && cands.some((c) => name(c) === "period_end");
|
|
413
|
+
const endOf = !periodPair && (/_(start|begin|from|open|opened)$/.test(topStem) || /^(start|begin)(_|$)/.test(topStem))
|
|
412
414
|
? cands.find((c) => c !== top.c && name(c).replace(/_(date|at|on|ts|time|day)$/, "") === topStem.replace(/_(start|begin|from|open|opened)$/, "_end").replace(/^(start|begin)(_|$)/, "end$2"))
|
|
413
415
|
: undefined;
|
|
414
416
|
if (endOf && !runnersUp.some((x) => x.c === endOf))
|
|
415
417
|
runnersUp.unshift(scored.find((x) => x.c === endOf));
|
|
418
|
+
if (periodPair)
|
|
419
|
+
return { value: top.c.column, confidence: "high", evidence: `${top.s[1]}; period_end is the end of the same period${scored.length > 2 ? `, ${scored.slice(1).filter((x) => name(x.c) !== "period_end").map((x) => x.c.column).join(", ")} ${scored.length === 3 ? "is a secondary date" : "are secondary dates"}` : ""}` };
|
|
416
420
|
if (!runnersUp.length)
|
|
417
421
|
return { value: top.c.column, confidence: top.s[0] >= 60 ? "high" : "medium", evidence: `${top.s[1]}; ${scored.slice(1).map((x) => x.c.column).join(", ")} ${scored.length === 2 ? "is a later or secondary date" : "are later or secondary dates"}` };
|
|
418
422
|
return {
|