semanticdb-v2 2.0.0-alpha.4 → 2.0.0-alpha.6

package/README.md

@@ -1,9 +1,23 @@
 # SemanticDB
 
-SemanticDB translates its own LogicForm into SQL and executes it on a single core compute layer.
+SemanticDB translates its own LogicForm into SQL and executes it on one or more core compute layers.
 Callers provide schemas, schema mappings, operators, and database connection information.
 SemanticDB creates the matching provider and owns its connection lifecycle.
 
+Install SemanticDB together with only the database driver used by the application. Database
+drivers are optional peer dependencies and are loaded on the first execution for that connection;
+Redis cache support is included by the main package.
+
+```bash
+npm install semanticdb-v2 mysql2
+```
+
+Use `@clickhouse/client`, `@databricks/sql`, `dmdb`, `mysql2` (MySQL, Doris, StarRocks, and GBase),
+`mssql`, `pg`, `snowflake-sdk`, or `oracledb` for the corresponding connection kind. MaxCompute
+uses a caller-supplied parameterized execution client. If a configured driver is missing, execution
+fails with an installation command for that driver. SemanticDB caches its internally created
+providers across executions; call `await Logicform.close()` during application shutdown.
+
 ## Public API
 
 ```ts
@@ -34,8 +48,9 @@ const response = await Logicform.execute(
     connections: [{
       id: 'analytics',
       kind: 'clickhouse',
-      url: 'http://localhost:8123',
-      username: 'default',
+      host: 'localhost',
+      port: 8123,
+      user: 'default',
       password: process.env.CLICKHOUSE_PASSWORD,
       database: 'analytics',
     }],
@@ -49,6 +64,11 @@ const response = await Logicform.execute(
 );
 ```
 
+A successful response always includes `total`, the number of result rows before global
+`limit`/`page` pagination. For grouped LogicForms this is the number of groups after `having` and
+`limitBy`. Paginated physical queries include a separate `sqls` entry with `type: 'total'` for the
+count query; unpaginated and executor-merged results derive the value without another SQL query.
+
 Every executable LogicForm, including LogicForm objects used as subqueries, must declare `version`. This
 SDK accepts semantic versions greater than or equal to `2.0`; missing, malformed, or older versions
 fail with `UNSUPPORTED_LOGICFORM_VERSION` before Schema resolution or database access. A relation
@@ -56,19 +76,28 @@ query shaped as `{ schema, query }` is part of its parent LogicForm and inherits
 
 PredItems without an explicit `name` receive a human-readable name during normalization. Generated
 names default to Chinese; pass `locale: 'en-US'` in `ExecuteOptions` for the built-in English
-names. Every operator owns its localized default names and receives the requested locale directly:
+names. SemanticDB ships `zh-CN` and `en-US` locale files backed by i18next. Applications can add or
+override locale files through `ExecuteOptions.locales`:
 
 ```ts
-operators.register({
-  name: '$weightedAverage',
-  defaultName: ({ operandName, locale }) => locale.toLowerCase().startsWith('en')
-    ? `Weighted average of ${operandName}`
-    : `${operandName}加权平均值`,
-  kind: 'aggregate',
-  toSQL: ({ column }) => `weighted_avg(${column})`,
+await Logicform.execute(logicform, {
+  locale: 'ja-JP',
+  locales: {
+    locale: 'ja-JP',
+    messages: {
+      'operator.$sum': '{operandName}の合計',
+      'component.numerator': '{outputName} 分子',
+      'component.denominator': '{outputName} 分母',
+    },
+  },
+  // schemas, schemaMappings, connections, ...
 });
 ```
 
+Locale lookup uses the exact locale first, then the matching language, and finally `zh-CN`.
+Multiple locale files can be supplied as an array. A custom operator still owns its fallback name
+through `defaultName()` and can call the context's `t()` helper when it defines translation keys.
+
 The same locale applies to `preds`, `groupby`, nested expressions, custom-function subqueries,
 and entity-population queries. A PredItem's normalized `query` is included in its
 generated name, for example `门店为上海的销售额合计`. Operator-specific `args` also participate
@@ -90,12 +119,18 @@ Plugins may own Schema metadata, legacy conversion, `entity_id` selection, and O
 remaining scoped to one execution. See [Register an external plugin](docs/register-plugin.md).
 
 Parent-child Schemas may declare `parentProperty`. `$recursive` filters a property to a root node and
-all descendants; object properties infer the tree Schema from `ref`. MySQL, Doris, StarRocks, and
-PostgreSQL compile this as a parameterized recursive CTE.
+all descendants; object properties infer the tree Schema from `ref`. MySQL, Doris, StarRocks,
+PostgreSQL, ClickHouse 24.4+, and Databricks compile this as a parameterized recursive CTE.
 
 Top-level projections may set `schema` to read metrics from another registered Schema. SemanticDB
 splits those projections into independent provider queries, aligns shared query/groupby dimensions,
 and merges rows by the group keys before applying `having`, `sort`, `limitBy`, and pagination.
+When the selected Schema mappings use separate physical connections, SemanticDB executes every
+single-connection fragment with its native provider and registers the aggregated fragment results
+in an in-process DuckDB coordinator. DuckDB applies the final full-outer group merge, cross-fragment
+conditions, ordering, totals, and pagination. It is embedded in the Node.js process and does not
+require a service or listening port. `federation.maxMaterializedRows` limits the total number of
+fragment rows admitted to the coordinator and defaults to 100,000.
 
 The progress stream emits `started`, `validated`, `normalized`, `planned`, `lineage`,
 `compiled`, `executing`, `hydrating`, `result`, and `completed`. Every event includes a cumulative
@@ -108,7 +143,9 @@ SQL string for diagnostics:
 
 ```ts
 interface ExecutedSql {
-  type: 'semantic-resolution' | 'computation';
+  type: 'semantic-resolution' | 'computation' | 'total';
+  connectionId?: string;
+  provider?: CoreComputeProvider['kind'];
   explain: string;
   logicform: LogicformType;
   sql: string;
@@ -121,6 +158,10 @@ interface ExecutedSql {
 time-watermark resolution and object entity population. `computation` identifies SQL that calculates
 the requested LogicForm result. `sqls` includes every physical SQL involved in the execution; an
 entry remains visible when its result is served by the query cache.
+Coordinator SQL uses the reserved connection id `__semanticdb_coordinator__` and provider
+`duckdb`; other entries identify the physical connection that executed them. Successful results
+also expose `providers`, the physical connection/provider pairs that contributed rows. The legacy
+singular `provider` field is `duckdb` for a federated result.
 
 Providers execute only parameterized `sql + parameters`; SemanticDB never executes `displaySql`.
 Because `displaySql` contains literal values and may expose sensitive data, use it only in trusted
@@ -181,13 +222,27 @@ Composite metrics are regular `OperatorDefinition` values with `kind: 'component
 them in the same `OperatorRegistry` as scalar and aggregate operators. Operator names are free-form;
 the `$` prefix is a convention used by built-in operators, not a requirement.
 
+Every composite component is exposed in result rows by default. Its `name` is its final result-column
+name and may contain Chinese characters or spaces. Built-ins derive child names from the parent output,
+for example `总销售额占比分子` and `总销售额占比分母`. Operators do not return `outputName` or an
+exposure flag; the executor owns the parent output binding. Components are joined on groupby output
+names shared by both sides unless an exceptional custom operator explicitly supplies `joinKeys`.
+
 See [Register an operator](docs/register-operator.md) for the component contract.
 
 The root entry point accepts a `connections` array of `DatabaseConnection` items. Every item has a unique `id`; set `kind` to `mysql`,
-`doris`, `starrocks`, `clickhouse`, `snowflake`, `oracle`, or `postgresql`, then add that driver's
+`doris`, `starrocks`, `gbase`, `maxcompute`, `clickhouse`, `databricks`, `dameng`, `snowflake`,
+`sqlserver`, `oracle`, or `postgresql`, then add that driver's
 native connection fields directly on the item. Each Schema Mapping selects an item through
 `connectionId`. SemanticDB lazily creates and reuses the matching internal provider. Call
 `await Logicform.close()` during application shutdown to close every connection pool and session.
+One LogicForm may reference mappings on multiple connections. This is one logical execution, not
+one cross-database SQL statement: physical fragments run independently and their aggregate results
+are combined by the embedded coordinator. A relation join whose two sides themselves live on
+different connections is not yet materialized automatically. When the related Schema has replicas
+on multiple connections, the Planner selects the mapping colocated with the main mapping and emits
+the `LEFT JOIN` on that connection. It fails explicitly only when no target mapping exists on the
+main connection.
 
 MySQL-compatible providers default to 30 connections, at most 10 idle connections, a 60-second
 idle timeout, and TCP keepalive. Native `mysql2` pool options on the connection item override these
@@ -223,6 +278,66 @@ pagination, date, aggregate, string, regex, parameter, and display-SQL behavior
 MySQL-compatible SQL helper. Database-specific differences should be implemented by overriding the
 corresponding internal dialect capability.
 
+### GBase connection
+
+GBase 8a uses its MySQL-compatible protocol through `mysql2`. It defaults to port `5258`; all
+ordinary `mysql2` connection and pool fields remain available:
+
+```ts
+const gbase = {
+  id: 'gbase-main',
+  kind: 'gbase',
+  host: process.env.GBASE_HOST,
+  user: process.env.GBASE_USER,
+  password: process.env.GBASE_PASSWORD,
+  database: process.env.GBASE_DATABASE,
+} as const;
+```
+
+### MaxCompute connection
+
+MaxCompute requires a caller-owned client that preserves the Provider's parameterized execution
+contract. The client receives SQL and parameters separately and owns the actual MaxCompute/JDBC
+transport:
+
+```ts
+const maxcompute = {
+  id: 'maxcompute-main',
+  kind: 'maxcompute',
+  client: {
+    execute: ({ sql, parameters, signal }) => maxComputeGateway.execute({
+      sql, parameters, signal,
+    }),
+    close: () => maxComputeGateway.close(),
+  },
+} as const;
+```
+
+The legacy UDBC `Statement` endpoint accepts only a complete SQL string, so SemanticDB does not
+use it or interpolate values locally. A gateway implementation must bind `parameters` before this
+connection can execute queries.
+
+### SQL Server connection
+
+SQL Server uses `mssql`, bracket-quoted identifiers, and named `@p0`, `@p1` parameters:
+
+```ts
+const sqlserver = {
+  id: 'sqlserver-main',
+  kind: 'sqlserver',
+  host: process.env.SQLSERVER_HOST,
+  port: 1433,
+  user: process.env.SQLSERVER_USER,
+  password: process.env.SQLSERVER_PASSWORD,
+  database: process.env.SQLSERVER_DATABASE,
+  options: { encrypt: true },
+} as const;
+```
+
+Pagination uses `OFFSET ... FETCH`; SemanticDB supplies a neutral top-level ordering when the
+LogicForm has no sort. JSON array relations use `OPENJSON`. SQL Server has no native regular
+expression predicate, so `$regex` is rejected explicitly.
+
 ### Snowflake connection
 
 Snowflake connection fields are passed to the official `snowflake-sdk`:
@@ -245,6 +360,52 @@ await Logicform.execute(logicform, {
 });
 ```
 
+### Databricks connection
+
+Databricks uses `@databricks/sql` and supports a personal access token or OAuth M2M. `catalog` and
+`schema` select the initial SQL Warehouse session namespace (`database` is accepted as a legacy
+alias for `schema`):
+
+```ts
+const databricks = {
+  id: 'databricks-main',
+  kind: 'databricks',
+  host: process.env.DATABRICKS_SERVER_HOSTNAME,
+  path: process.env.DATABRICKS_HTTP_PATH,
+  authType: 'databricks-oauth',
+  oauthClientId: process.env.DATABRICKS_CLIENT_ID,
+  oauthClientSecret: process.env.DATABRICKS_CLIENT_SECRET,
+  catalog: 'main',
+  schema: 'analytics',
+} as const;
+```
+
+Use `authType: 'access-token'` with `token` for PAT authentication. Values are sent through the
+driver's ordinal parameter binding and are not interpolated into executable SQL. `authType` is
+required: PAT and OAuth fields are mutually exclusive, and authentication is never inferred from
+the presence of credentials.
+
+### Dameng connection
+
+Dameng uses the official `dmdb` pool and the `dameng` connection kind:
+
+```ts
+const dameng = {
+  id: 'dameng-main',
+  kind: 'dameng',
+  host: process.env.DAMENG_HOST,
+  port: 5236,
+  user: process.env.DAMENG_USER,
+  password: process.env.DAMENG_PASSWORD,
+  schema: process.env.DAMENG_SCHEMA,
+} as const;
+```
+
+The provider uses positional binds and object-format result rows. Recursive hierarchy queries and
+JSON-array relations are rejected until their behavior can be verified against supported DM server
+versions; scalar queries, grouping, date operations, regex, windows, joins, and pagination compile
+with the DM/Oracle-style dialect.
+
 ### Oracle connection
 
 Oracle uses the official `oracledb` driver in Thin mode and does not require Oracle Client
@@ -315,9 +476,20 @@ comments, or parameter placeholders. Property UDFs do not support the legacy `fu
 
 All compiled provider queries, including the internal time-watermark query, can share one
 caller-owned query cache. Redis is published from `semanticdb-v2/cache/redis`; cache identity
-includes namespace, provider, SQL, and parameters. Physical expiration belongs to each Schema
+includes namespace, connection id, provider, SQL, and parameters. Physical expiration belongs to each Schema
 mapping's `freshness` policy. See [Query cache and Redis](docs/query-cache.md).
 
+Schema mappings can also expose same-connection entity feature tables as lazy physical
+enrichments. A mapped feature property adds a `LEFT JOIN` only when referenced, and a point-in-time
+feature can be assigned to the containing valid-time entity row. See
+[Schema Mapping feature enrichments](docs/schema-mapping-enrichments.md).
+
+For aggregate sources, set `granularity` on mapped date or hierarchy Properties.
+Other available dimensions are inferred from their Property mappings, so there is no
+separate Schema Mapping `dimensions` field. The planner selects a source that covers
+the requested properties and physical time or hierarchy grain. See
+[Schema Mapping physical granularity](docs/schema-mapping-granularity.md).
+
 When Redis is shared with other applications, pass an application-specific key prefix:
 
 ```ts
@@ -439,7 +611,8 @@ npm run debug
 
 ## Current scope
 
-- ClickHouse, MySQL, Oracle, and Snowflake core compute providers
+- ClickHouse, Databricks, Dameng, Doris, GBase, MaxCompute, MySQL, Oracle, PostgreSQL, Snowflake,
+  SQL Server, and StarRocks core compute providers
 - Projections, conditional aggregates, and `$sum`, `$count`, `$countDistinct`, `$avg`, `$min`, `$max`
 - Structured arithmetic, share-of-total, period comparison, period-boundary, date-bucket,
   cumulative period, trusted SQL expression, string, case, and window operators

package/dist/cache/redis-query-cache.esm.js

@@ -1 +1 @@
-import{createClient as b}from"redis";import{createHash as g}from"node:crypto";const a=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?e.map(a):e&&typeof e=="object"?Object.fromEntries(Object.entries(e).sort(([t],[r])=>t.localeCompare(r)).map(([t,r])=>[t,a(r)])):e,d=e=>g("sha256").update(JSON.stringify(a(e))).digest("hex"),A=e=>d({namespace:e.namespace,provider:e.provider,sql:e.sql,parameters:e.parameters}),S=(e,t)=>d({namespace:e,source:t}),o=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?{type:"array",value:e.map(o)}:e&&typeof e=="object"?{type:"object",value:Object.entries(e).map(([t,r])=>[t,o(r)])}:e,y=e=>{if(e===null||typeof e!="object")return e;switch(e.type){case"undefined":return;case"bigint":return BigInt(e.value);case"number":return Number(e.value);case"date":return new Date(e.value);case"array":return e.value.map(y);case"object":return Object.fromEntries(e.value.map(([t,r])=>[t,y(r)]))}},w=e=>JSON.stringify(o(e)),x=e=>{const t=y(JSON.parse(e));if(!(!Array.isArray(t)||t.some(r=>!r||typeof r!="object"||Array.isArray(r))))return t},l=Number.MAX_SAFE_INTEGER;class p{#e;#t;#r;constructor(t,r={}){this.#e=t,this.#t=r.keyPrefix??"semanticdb:cache",this.#r=r.ownsClient??!1}#n(t){return`${this.#t}:query:${A(t)}`}#i(t,r){return`${this.#t}:source:${S(t,r)}`}async get(t){const r=await this.#e.get(this.#n(t));if(r!==null)return x(r)}async set(t,r,i){const n=this.#n(t),f=w(r),c=i.expiresAt?.getTime(),m=c??l,s=this.#e.multi();c===void 0?s.set(n,f):s.set(n,f,{PXAT:c});for(const h of new Set(i.sources)){const u=this.#i(t.namespace,h);s.zRemRangeByScore(u,0,Date.now()),s.zAdd(u,{score:m,value:n})}await s.exec()}async invalidateSource(t,r){const i=this.#i(t,r),n=await this.#e.zRange(i,0,-1);n.length>0&&await this.#e.del(n),await this.#e.del(i)}async close(){this.#r&&await this.#e.close()}}const j=async(e={})=>{const t=b(e.url?{url:e.url}:{});return t.on("error",()=>{}),await t.connect(),new p(t,{...e.keyPrefix?{keyPrefix:e.keyPrefix}:{},ownsClient:!0})};export{p as RedisQueryCache,j as createRedisQueryCache};
+import{createClient as b}from"redis";import{createHash as g}from"node:crypto";const o=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?e.map(o):e&&typeof e=="object"?Object.fromEntries(Object.entries(e).sort(([t],[r])=>t.localeCompare(r)).map(([t,r])=>[t,o(r)])):e,d=e=>g("sha256").update(JSON.stringify(o(e))).digest("hex"),A=e=>d({namespace:e.namespace,provider:e.provider,connectionId:e.connectionId,sql:e.sql,parameters:e.parameters}),S=(e,t)=>d({namespace:e,source:t}),a=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?{type:"array",value:e.map(a)}:e&&typeof e=="object"?{type:"object",value:Object.entries(e).map(([t,r])=>[t,a(r)])}:e,f=e=>{if(e===null||typeof e!="object")return e;switch(e.type){case"undefined":return;case"bigint":return BigInt(e.value);case"number":return Number(e.value);case"date":return new Date(e.value);case"array":return e.value.map(f);case"object":return Object.fromEntries(e.value.map(([t,r])=>[t,f(r)]))}},w=e=>JSON.stringify(a(e)),x=e=>{const t=f(JSON.parse(e));if(!(!Array.isArray(t)||t.some(r=>!r||typeof r!="object"||Array.isArray(r))))return t},l=Number.MAX_SAFE_INTEGER;class p{#e;#t;#r;constructor(t,r={}){this.#e=t,this.#t=r.keyPrefix??"semanticdb:cache",this.#r=r.ownsClient??!1}#n(t){return`${this.#t}:query:${A(t)}`}#i(t,r){return`${this.#t}:source:${S(t,r)}`}async get(t){const r=await this.#e.get(this.#n(t));if(r!==null)return x(r)}async set(t,r,i){const n=this.#n(t),y=w(r),c=i.expiresAt?.getTime(),m=c??l,s=this.#e.multi();c===void 0?s.set(n,y):s.set(n,y,{PXAT:c});for(const h of new Set(i.sources)){const u=this.#i(t.namespace,h);s.zRemRangeByScore(u,0,Date.now()),s.zAdd(u,{score:m,value:n})}await s.exec()}async invalidateSource(t,r){const i=this.#i(t,r),n=await this.#e.zRange(i,0,-1);n.length>0&&await this.#e.del(n),await this.#e.del(i)}async close(){this.#r&&await this.#e.close()}}const j=async(e={})=>{const t=b(e.url?{url:e.url}:{});return t.on("error",()=>{}),await t.connect(),new p(t,{...e.keyPrefix?{keyPrefix:e.keyPrefix}:{},ownsClient:!0})};export{p as RedisQueryCache,j as createRedisQueryCache};

package/dist/cache/redis-query-cache.js

@@ -1 +1 @@
-"use strict";var b=require("redis"),g=require("node:crypto");const a=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?e.map(a):e&&typeof e=="object"?Object.fromEntries(Object.entries(e).sort(([t],[r])=>t.localeCompare(r)).map(([t,r])=>[t,a(r)])):e,d=e=>g.createHash("sha256").update(JSON.stringify(a(e))).digest("hex"),A=e=>d({namespace:e.namespace,provider:e.provider,sql:e.sql,parameters:e.parameters}),S=(e,t)=>d({namespace:e,source:t}),o=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?{type:"array",value:e.map(o)}:e&&typeof e=="object"?{type:"object",value:Object.entries(e).map(([t,r])=>[t,o(r)])}:e,y=e=>{if(e===null||typeof e!="object")return e;switch(e.type){case"undefined":return;case"bigint":return BigInt(e.value);case"number":return Number(e.value);case"date":return new Date(e.value);case"array":return e.value.map(y);case"object":return Object.fromEntries(e.value.map(([t,r])=>[t,y(r)]))}},w=e=>JSON.stringify(o(e)),x=e=>{const t=y(JSON.parse(e));if(!(!Array.isArray(t)||t.some(r=>!r||typeof r!="object"||Array.isArray(r))))return t},l=Number.MAX_SAFE_INTEGER;class p{#e;#t;#r;constructor(t,r={}){this.#e=t,this.#t=r.keyPrefix??"semanticdb:cache",this.#r=r.ownsClient??!1}#n(t){return`${this.#t}:query:${A(t)}`}#i(t,r){return`${this.#t}:source:${S(t,r)}`}async get(t){const r=await this.#e.get(this.#n(t));if(r!==null)return x(r)}async set(t,r,i){const n=this.#n(t),u=w(r),c=i.expiresAt?.getTime(),h=c??l,s=this.#e.multi();c===void 0?s.set(n,u):s.set(n,u,{PXAT:c});for(const m of new Set(i.sources)){const f=this.#i(t.namespace,m);s.zRemRangeByScore(f,0,Date.now()),s.zAdd(f,{score:h,value:n})}await s.exec()}async invalidateSource(t,r){const i=this.#i(t,r),n=await this.#e.zRange(i,0,-1);n.length>0&&await this.#e.del(n),await this.#e.del(i)}async close(){this.#r&&await this.#e.close()}}const C=async(e={})=>{const t=b.createClient(e.url?{url:e.url}:{});return t.on("error",()=>{}),await t.connect(),new p(t,{...e.keyPrefix?{keyPrefix:e.keyPrefix}:{},ownsClient:!0})};exports.RedisQueryCache=p,exports.createRedisQueryCache=C;
+"use strict";var b=require("redis"),g=require("node:crypto");const a=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?e.map(a):e&&typeof e=="object"?Object.fromEntries(Object.entries(e).sort(([t],[r])=>t.localeCompare(r)).map(([t,r])=>[t,a(r)])):e,d=e=>g.createHash("sha256").update(JSON.stringify(a(e))).digest("hex"),A=e=>d({namespace:e.namespace,provider:e.provider,connectionId:e.connectionId,sql:e.sql,parameters:e.parameters}),S=(e,t)=>d({namespace:e,source:t}),o=e=>e===void 0?{type:"undefined"}:typeof e=="bigint"?{type:"bigint",value:e.toString()}:typeof e=="number"&&!Number.isFinite(e)?{type:"number",value:String(e)}:e instanceof Date?{type:"date",value:e.toISOString()}:Array.isArray(e)?{type:"array",value:e.map(o)}:e&&typeof e=="object"?{type:"object",value:Object.entries(e).map(([t,r])=>[t,o(r)])}:e,u=e=>{if(e===null||typeof e!="object")return e;switch(e.type){case"undefined":return;case"bigint":return BigInt(e.value);case"number":return Number(e.value);case"date":return new Date(e.value);case"array":return e.value.map(u);case"object":return Object.fromEntries(e.value.map(([t,r])=>[t,u(r)]))}},w=e=>JSON.stringify(o(e)),x=e=>{const t=u(JSON.parse(e));if(!(!Array.isArray(t)||t.some(r=>!r||typeof r!="object"||Array.isArray(r))))return t},l=Number.MAX_SAFE_INTEGER;class p{#e;#t;#r;constructor(t,r={}){this.#e=t,this.#t=r.keyPrefix??"semanticdb:cache",this.#r=r.ownsClient??!1}#n(t){return`${this.#t}:query:${A(t)}`}#i(t,r){return`${this.#t}:source:${S(t,r)}`}async get(t){const r=await this.#e.get(this.#n(t));if(r!==null)return x(r)}async set(t,r,i){const n=this.#n(t),y=w(r),c=i.expiresAt?.getTime(),h=c??l,s=this.#e.multi();c===void 0?s.set(n,y):s.set(n,y,{PXAT:c});for(const m of new Set(i.sources)){const f=this.#i(t.namespace,m);s.zRemRangeByScore(f,0,Date.now()),s.zAdd(f,{score:h,value:n})}await s.exec()}async invalidateSource(t,r){const i=this.#i(t,r),n=await this.#e.zRange(i,0,-1);n.length>0&&await this.#e.del(n),await this.#e.del(i)}async close(){this.#r&&await this.#e.close()}}const C=async(e={})=>{const t=b.createClient(e.url?{url:e.url}:{});return t.on("error",()=>{}),await t.connect(),new p(t,{...e.keyPrefix?{keyPrefix:e.keyPrefix}:{},ownsClient:!0})};exports.RedisQueryCache=p,exports.createRedisQueryCache=C;

package/dist/cache/types.d.ts

@@ -2,6 +2,8 @@ import type { CoreComputeProvider, DataRow, SqlParameters } from '../physical/in
 export interface QueryCacheIdentity {
     readonly namespace: string;
     readonly provider: CoreComputeProvider['kind'];
+    /** Distinguishes separate databases using the same provider dialect. */
+    readonly connectionId?: string;
     readonly sql: string;
     readonly parameters: SqlParameters;
 }

package/dist/compiler/compile-cte.d.ts

@@ -24,4 +24,4 @@ export interface PreparedCteQuery {
  * CTE preserves those ranges while allowing the provider to execute one query.
  */
 export declare const compileCteQueries: (units: readonly PreparedCteQuery[], outputPlan: LogicalPlan, provider: CoreComputeProvider, operators?: OperatorRegistry) => CompiledQuery;
-export declare const compileCteCalculation: (calculation: OperatorComponentCalculation, components: readonly PreparedCteComponent[], outputPlan: LogicalPlan, provider: CoreComputeProvider, operators: OperatorRegistry) => CompiledQuery;
+export declare const compileCteCalculation: (calculation: OperatorComponentCalculation, components: readonly PreparedCteComponent[], outputPlan: LogicalPlan, outputName: string, provider: CoreComputeProvider, operators: OperatorRegistry) => CompiledQuery;

package/dist/errors.d.ts

@@ -1,7 +1,8 @@
+import { type LocaleRegistry } from './i18n/index';
 export declare class SemanticDBError extends Error {
     readonly code: string;
     readonly details?: Readonly<Record<string, unknown>> | undefined;
     constructor(code: string, message: string, details?: Readonly<Record<string, unknown>> | undefined);
     /** Return the user-facing message for an execution locale. */
-    localizedMessage(locale?: string): string;
+    localizedMessage(locale?: string, locales?: LocaleRegistry): string;
 }

package/dist/executor/components.d.ts

@@ -1,9 +1,14 @@
 import type { LogicformType } from '../logicform-types';
-import type { OperatorExecutionComponent, OperatorRegistry } from '../operators/index';
+import type { OperatorComponentCalculation, OperatorJavaScriptCalculation, OperatorQueryComponent, OperatorRegistry } from '../operators/index';
 import type { DataRow } from '../physical/index';
 import type { LogicalPlan } from '../planner/index';
 import type { ResolvedSemanticSchema } from '../semantic-types/index';
-export type ExecutionUnit = OperatorExecutionComponent;
-export declare const createExecutionUnits: (logicform: LogicformType, plan: LogicalPlan, schema: ResolvedSemanticSchema, operators: OperatorRegistry, resolveSchema: (schemaId: string) => ResolvedSemanticSchema, sourceLogicform?: LogicformType) => readonly ExecutionUnit[] | undefined;
+import { type LocaleTranslate } from '../i18n/index';
+export type ExecutionUnit = OperatorQueryComponent | (OperatorComponentCalculation & {
+    readonly outputName: string;
+}) | (OperatorJavaScriptCalculation & {
+    readonly outputName: string;
+});
+export declare const createExecutionUnits: (logicform: LogicformType, plan: LogicalPlan, schema: ResolvedSemanticSchema, operators: OperatorRegistry, resolveSchema: (schemaId: string) => ResolvedSemanticSchema, sourceLogicform?: LogicformType, locale?: string, t?: LocaleTranslate) => readonly ExecutionUnit[] | undefined;
 export declare const mergeComponentRows: (results: readonly (readonly DataRow[])[], groupNames: readonly string[]) => DataRow[];
 export declare const mergeExecutionResults: (results: readonly (readonly DataRow[])[], plan: LogicalPlan) => DataRow[];

package/dist/executor/connection-registry.d.ts

@@ -1,4 +1,4 @@
-import type { CoreComputeProvider, DatabaseConnection } from '../physical/index';
+import type { CoreComputeProvider, DatabaseConnection, ProviderRegistry } from '../physical/index';
 type ProviderFactory = (connection: DatabaseConnection) => CoreComputeProvider;
 export declare class ConnectionRegistry {
     #private;
@@ -7,6 +7,6 @@ export declare class ConnectionRegistry {
     get(connection: DatabaseConnection): CoreComputeProvider;
     close(): Promise<void>;
 }
-export declare const getConnectionProvider: (connection: DatabaseConnection) => CoreComputeProvider;
+export declare const getConnectionProvider: (connection: DatabaseConnection, providers?: ProviderRegistry) => CoreComputeProvider;
 export declare const closeConnections: () => Promise<void>;
 export {};

package/dist/executor/execute-query.d.ts

@@ -1,13 +1,18 @@
 import type { CompiledQuery } from '../compiler/index';
 import type { DataRow } from '../physical/index';
 import type { LogicalPlan } from '../planner/index';
+import type { SchemaMappingDefinition, SchemaMappingRegistry } from '../schema-mappings/index';
 import type { InternalExecuteOptions } from './types';
 export interface PhysicalQuerySource {
     readonly schemaId: string;
     readonly source: string;
+    readonly connectionId?: string;
+    /** Base mapping source that owns an enrichment source. */
+    readonly mappingSource?: string;
 }
+export declare const resolvePhysicalSourceMapping: (source: PhysicalQuerySource, mappings: SchemaMappingRegistry) => SchemaMappingDefinition | undefined;
 export declare const collectPlanSources: (plan: LogicalPlan) => readonly PhysicalQuerySource[];
 export declare const mergePhysicalSources: (groups: readonly (readonly PhysicalQuerySource[])[]) => readonly PhysicalQuerySource[];
-export declare const executeCompiledQuery: (compiled: CompiledQuery, sources: readonly PhysicalQuerySource[], options: Pick<InternalExecuteOptions, "cache" | "provider" | "schemaMappings" | "signal">) => Promise<DataRow[]>;
+export declare const executeCompiledQuery: (compiled: CompiledQuery, sources: readonly PhysicalQuerySource[], options: Pick<InternalExecuteOptions, "cache" | "provider" | "schemaMappings" | "signal" | "connectionId">) => Promise<DataRow[]>;
 /** Apply final SQL shaping after all plans, views, and derived sources are compiled. */
 export declare const shortenCompiledQuery: (compiled: CompiledQuery) => CompiledQuery;

package/dist/executor/federated-execution.d.ts

@@ -0,0 +1,37 @@
+import type { CompiledQuery } from '../compiler/index';
+import type { LogicformType } from '../logicform-types';
+import type { OperatorRegistry } from '../operators/index';
+import type { CoreComputeProvider, DataRow } from '../physical/index';
+import type { LogicalPlan } from '../planner/index';
+import type { SchemaMappingRegistry } from '../schema-mappings/index';
+import type { InternalExecuteOptions } from './types';
+export declare const coordinatorConnectionId = "__semanticdb_coordinator__";
+export interface FederatedCompiledQuery {
+    readonly compiled: CompiledQuery;
+    readonly connectionId: string;
+    readonly provider: CoreComputeProvider;
+    readonly type: 'computation' | 'total';
+    readonly logicform: LogicformType;
+}
+interface FederatedExecutionOptions {
+    readonly logicform: LogicformType;
+    readonly operators: OperatorRegistry;
+    readonly schemaMappings: SchemaMappingRegistry;
+    readonly providerForConnection: (connectionId: string) => CoreComputeProvider;
+    readonly cache?: InternalExecuteOptions['cache'];
+    readonly signal?: AbortSignal;
+    readonly maxMaterializedRows: number;
+    readonly onCompiled: (query: FederatedCompiledQuery) => void;
+}
+export interface FederatedExecutionResult {
+    readonly rows: DataRow[];
+    readonly total: number;
+    readonly connections: readonly {
+        readonly connectionId: string;
+        readonly provider: CoreComputeProvider['kind'];
+    }[];
+}
+export declare const planConnectionIds: (plan: LogicalPlan, schemaMappings: SchemaMappingRegistry) => readonly string[];
+export declare const isFederatedComposition: (plan: LogicalPlan, schemaMappings: SchemaMappingRegistry) => boolean;
+export declare const executeFederatedComposition: (plan: LogicalPlan, options: FederatedExecutionOptions) => Promise<FederatedExecutionResult>;
+export {};

package/dist/executor/fetch-entities.d.ts

@@ -9,19 +9,23 @@ import type { Permissions } from '../permissions/index';
 import type { ResolvedSemanticSchema, SemanticTypeRegistry } from '../semantic-types/index';
 import type { EntityFetchRequest } from './result-hydrator/index';
 import type { SemanticDBPlugin } from '../plugins/index';
+import type { LocaleRegistry } from '../i18n/index';
 interface FetchEntityOptions {
     readonly version: string;
     readonly operators: OperatorRegistry;
     readonly locale?: string;
+    readonly localeRegistry?: LocaleRegistry;
     readonly schemaMappings: SchemaMappingRegistry;
     readonly semanticTypes: SemanticTypeRegistry;
     readonly plugins?: readonly SemanticDBPlugin[];
     readonly resolveSchema: (schemaId: string) => ResolvedSemanticSchema;
     readonly permissions?: Permissions;
     readonly provider: CoreComputeProvider;
+    readonly providerForConnection?: (connectionId: string) => CoreComputeProvider;
+    readonly preferredConnectionId?: string;
     readonly cache?: QueryCacheExecutionOptions;
     readonly signal?: AbortSignal;
-    readonly onCompiled?: (compiled: CompiledQuery, logicform: LogicformType) => void;
+    readonly onCompiled?: (compiled: CompiledQuery, logicform: LogicformType, provider: CoreComputeProvider, connectionId?: string) => void;
 }
 export declare const fetchEntities: (request: EntityFetchRequest, options: FetchEntityOptions) => Promise<DataRow[]>;
 export {};

package/dist/executor/result-hydrator/types.d.ts

@@ -1,6 +1,7 @@
 import type { ResultProperty } from '../../operators/index';
 import type { DataRow } from '../../physical/index';
 import type { ResolvedSemanticProperty, ResolvedSemanticSchema, SemanticTypeRegistry } from '../../semantic-types/index';
+import type { SemanticDBPlugin } from '../../plugins/index';
 export type HydrationProperty = Pick<ResultProperty, 'isArray' | 'name' | 'primal_type' | 'ref' | 'type'> | ResolvedSemanticProperty;
 export interface EntityFetchRequest {
     readonly schema: ResolvedSemanticSchema;
@@ -11,4 +12,5 @@ export interface ResultHydratorOptions {
     readonly semanticTypes: SemanticTypeRegistry;
     readonly resolveSchema: (schemaId: string) => ResolvedSemanticSchema;
     readonly fetchEntities: (request: EntityFetchRequest) => Promise<readonly DataRow[]>;
+    readonly plugins?: readonly SemanticDBPlugin[];
 }

package/dist/executor/total.d.ts

@@ -0,0 +1,11 @@
+import type { CompiledQuery } from '../compiler/index';
+import type { CoreComputeProvider, DataRow } from '../physical/index';
+import type { LogicalPlan } from '../planner/index';
+import type { LogicformType } from '../logicform-types';
+/** Remove global result paging while retaining HAVING and per-group limitBy semantics. */
+export declare const withoutGlobalPagination: (plan: LogicalPlan) => LogicalPlan;
+/** Describe the total SQL with the same unpaged global result as its plan. */
+export declare const withoutGlobalPaginationLogicform: (logicform: LogicformType) => LogicformType;
+/** Count the rows produced by a complete unpaginated logical result. */
+export declare const compileTotalQuery: (unpaged: CompiledQuery, provider: CoreComputeProvider) => CompiledQuery;
+export declare const readTotal: (rows: readonly DataRow[]) => number;

package/dist/executor/types.d.ts

@@ -1,13 +1,14 @@
-import type { OperatorRegistry } from '../operators/index';
-import type { CoreComputeProvider, DataRow, SqlParameters } from '../physical/index';
+import type { OperatorDefinition, OperatorRegistry } from '../operators/index';
+import type { CoreComputeProvider, DataRow, ProviderRegistry, SqlParameters } from '../physical/index';
 import type { DatabaseConnection } from '../physical/index';
 import type { LogicalPlan, QueryLineage } from '../planner/index';
-import type { SchemaMappingDefinition, SchemaMappingRegistry } from '../schema-mappings/index';
+import type { SchemaMappingDefinition, SchemaMappingRegistry, SchemaMappingSelection } from '../schema-mappings/index';
 import type { ResolvedSemanticSchema, SemanticSchemaDraft, SemanticTypeRegistry, TimeWatermarkPolicy } from '../semantic-types/index';
 import type { LogicformType, PublicLogicformType } from '../logicform-types';
 import type { QueryCacheExecutionOptions } from '../cache/index';
 import type { Permissions } from '../permissions/index';
 import type { SemanticDBPlugin } from '../plugins/index';
+import type { LocaleInput, LocaleRegistry } from '../i18n/index';
 export type ExecuteProgressEvent = {
     readonly stage: 'started';
     readonly logicform: LogicformType;
@@ -35,6 +36,7 @@ export type ExecuteProgressEvent = {
 } | {
     readonly stage: 'executing';
     readonly provider: CoreComputeProvider['kind'];
+    readonly connectionId?: string;
     readonly partialResult: PartialExecuteResult;
 } | {
     readonly stage: 'hydrating';
@@ -55,12 +57,16 @@ export type ExecuteProgressEvent = {
 export type ProgressCallback = (event: ExecuteProgressEvent) => void | Promise<void>;
 interface ExecuteBaseOptions {
     readonly schemas: Readonly<Record<string, SemanticSchemaDraft>>;
+    /** Caller-owned Registry; plugin SemanticTypes are copied into the execution Registry. */
     readonly semanticTypes?: SemanticTypeRegistry;
-    readonly operators?: OperatorRegistry;
+    /** Caller-defined Operators; explicit definitions override same-name built-ins or plugin Operators. */
+    readonly operators?: OperatorDefinition | readonly OperatorDefinition[];
     /** Caller-owned capability plugins appended to the built-in plugin set for this execution. */
     readonly plugins?: readonly SemanticDBPlugin[];
     /** Locale for generated human-readable PredItem names. Defaults to zh-CN. */
     readonly locale?: string;
+    /** Additional or overriding locale files used by generated names and messages. */
+    readonly locales?: LocaleInput;
     /** Table, row, and column permissions for this execution. */
     readonly permissions?: Permissions;
     /** Optional caller-owned cache shared across executions. */
@@ -69,28 +75,54 @@ interface ExecuteBaseOptions {
     readonly signal?: AbortSignal;
     /** Default time watermark policy; Schema and Property policies override it. */
     readonly timeWatermark?: TimeWatermarkPolicy;
+    /** Resource limits for in-process combination of separate physical connections. */
+    readonly federation?: {
+        /** Defaults to 100,000 rows across all materialized aggregate branches. */
+        readonly maxMaterializedRows?: number;
+    };
 }
 export interface ExecuteOptions extends ExecuteBaseOptions {
     /** Physical mappings supplied as JSON definitions. */
     readonly schemaMappings: readonly SchemaMappingDefinition[];
     /** Database connections addressable by SchemaMappingDefinition.connectionId. */
     readonly connections: readonly DatabaseConnection[];
+    /** Provider factories used for built-in or application-defined connection kinds. */
+    readonly providerRegistry?: ProviderRegistry;
 }
 /** Internal execution seam used by executor tests after a provider has been resolved. */
-export interface InternalExecuteOptions extends ExecuteBaseOptions {
+export interface InternalExecuteOptions extends Omit<ExecuteBaseOptions, 'operators'> {
+    readonly operators?: OperatorRegistry;
+    /** The public execution boundary has already merged built-in, plugin, and explicit Operators. */
+    readonly operatorsResolved?: boolean;
+    /** The public execution boundary has already merged plugin SemanticTypes. */
+    readonly semanticTypesResolved?: boolean;
+    readonly localeRegistry?: LocaleRegistry;
     readonly provider: CoreComputeProvider;
+    /** Resolve providers for additional physical connections in a federated execution. */
+    readonly providerForConnection?: (connectionId: string) => CoreComputeProvider;
     readonly schemaMappings: SchemaMappingRegistry;
     /** Selected connection for public execution; omitted only by internal provider tests. */
     readonly connectionId?: string;
 }
 export interface ExecuteSuccessResult {
     readonly result: DataRow[];
+    /** Result rows or grouped rows before global limit/page pagination. */
+    readonly total: number;
     /** Schema describing the computed result rows. */
     readonly schema: ResolvedSemanticSchema;
     readonly normedLogicform: PublicLogicformType;
     readonly sqls: readonly ExecutedSql[];
     readonly lineage: QueryLineage;
     readonly provider: CoreComputeProvider['kind'];
+    /** Physical connections that contributed rows to this result. */
+    readonly providers: readonly {
+        readonly connectionId: string;
+        readonly provider: CoreComputeProvider['kind'];
+    }[];
+    /** Physical mapping decisions, including why every non-selected candidate lost. */
+    readonly schemaMappingSelections: readonly SchemaMappingSelection[];
+    /** Plugin-owned materialization for a custom LogicForm output type. */
+    readonly output?: unknown;
 }
 export interface ExecuteFailureResult {
     readonly errorCode: '100';
@@ -103,10 +135,13 @@ export interface ExecuteFailureResult {
     readonly lineage?: QueryLineage;
 }
 export type ExecuteResult = ExecuteSuccessResult | ExecuteFailureResult;
-export type ExecutedSqlType = 'computation' | 'semantic-resolution';
+export type ExecutedSqlType = 'computation' | 'semantic-resolution' | 'total';
 export interface ExecutedSql {
     /** Why this SQL participates in the LogicForm execution. */
     readonly type: ExecutedSqlType;
+    /** Physical connection that executed this SQL, or the in-process coordinator id. */
+    readonly connectionId?: string;
+    readonly provider?: CoreComputeProvider['kind'];
     /** Human-readable, localized explanation of why this SQL is executed. */
     readonly explain: string;
     /** Normalized LogicForm represented by this SQL statement. */

package/dist/i18n/index.d.ts

@@ -0,0 +1,4 @@
+export { createLocaleRegistry, defaultLocaleRegistry, LocaleRegistry } from './registry';
+export type { LocaleDefinition, LocaleInput, LocaleTranslate } from './types';
+export { enUS } from './locales/en-US';
+export { zhCN } from './locales/zh-CN';

package/dist/i18n/locales/en-US.d.ts

@@ -0,0 +1,2 @@
+import type { LocaleDefinition } from '../types';
+export declare const enUS: LocaleDefinition;

package/dist/i18n/locales/zh-CN.d.ts

@@ -0,0 +1,2 @@
+import type { LocaleDefinition } from '../types';
+export declare const zhCN: LocaleDefinition;