@powersync/service-sync-rules 0.17.10 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,129 @@
1
1
  # powersync-sync-rules
2
2
 
3
3
  A library containing logic for PowerSync sync rules.
4
+
5
+ This is not intended to be used directly by users of PowerSync. If you are interested in the internals, read on.
6
+
7
+ # Overview
8
+
9
+ A core design constraint is that sync rules define two operations:
10
+
11
+ 1. Given a data row, compute a list of buckets that it belongs to.
12
+ 2. Given an authenticated user, return a list of buckets for the user.
13
+
14
+ This implementation of sync rules use SQL queries to declaratively define those operations using familiar SQL operations.
15
+
16
+ We define (1) using data queries, and (2) using parameter queries.
17
+
18
+ Example:
19
+
20
+ ```yaml
21
+ bucket_definitions:
22
+ by_org:
23
+ # parameter query
24
+ # This defines bucket parameters are `bucket.org_id`
25
+ parameters: select org_id from users where id = token_parameters.user_id
26
+ # data query
27
+ data:
28
+ - select * from documents where org_id = bucket.org_id
29
+ ```
30
+
31
+ For the above example, a document with `org_id: 'org1'` will belong to a single bucket `by_org["org1"]`. Similarly, a user with `org_id: 'org1'` will sync the bucket `by_org["org1"]`.
32
+
33
+ An important aspect is that none of these SQL queries are actually executed against any SQL database. Instead, it is used to pre-process data before storing the data in a format for efficient sync operations.
34
+
35
+ When data is replicated from the source database to PowerSync, we do two things for each row:
36
+
37
+ 1. Evaluate data queries on the row: `syncRules.evaluateRow(row)`.
38
+ 2. Evaluate parameter queries on the row: `syncRules.evaluateParameterRow(row)`.
39
+
40
+ Data queries also have the option to transform the row instead of just using `select *`. We store the transformed data for each of the buckets it belongs to.
41
+
42
+ # Query Structure
43
+
44
+ ## Data queries
45
+
46
+ A data query is turned into a function `(row) => Array<{bucket, data}>`. The main implementation is in the `SqlDataQuery` class.
47
+
48
+ The main clauses in a data query are the ones comparing bucket parameters, for example `WHERE documents.document_org_id = bucket.bucket_org_id`. In this case, a document with `document_org_id: 'org1'` will have a bucket parameter of `bucket_org_id: 'org1'`.
49
+
50
+ A data query must match each bucket parameter. To be able to always compute the bucket ids, there are major limitations on the operators supported with bucket parameters, as well as how expressions can be combined using AND and OR.
51
+
52
+ The WHERE clause of a data query is compiled into a `ParameterMatchClause`.
53
+
54
+ Query clauses are structured as follows:
55
+
56
+ ```SQL
57
+ 'literal' -- StaticValueClause
58
+ mytable.column -- RowValueClause
59
+ fn(mytable.column) -- RowValueClause. This includes most operators.
60
+ bucket.param -- ParameterValueClause
61
+ fn(bucket.param) -- Error: not allowed
62
+
63
+ mytable.column = mytable.other_column -- RowValueClause
64
+ mytable.column = bucket.param -- ParameterMatchClause
65
+ bucket.param IN mytable.some_array -- ParameterMatchClause
66
+ (mytable.column1 = bucket.param1) AND (mytable.column2 = bucket.param2) -- ParameterMatchClause
67
+ (mytable.column1 = bucket.param) OR (mytable.column2 = bucket.param) -- ParameterMatchClause
68
+ ```
69
+
70
+ ## Parameter Queries
71
+
72
+ There are two types of parameter queries:
73
+
74
+ 1. Queries without tables. These just operate on request parameters. Example: `select token_parameters.user_id`. Thes are implemented in the `StaticSqlParameterQuery` class.
75
+ 2. Queries with tables. Example: `select org_id from users where id = token_parameters.user_id`. These use parameter tables, and are implemented in `SqlParameterQuery`. These are used to pre-process rows in the parameter tables for efficient lookup later.
76
+
77
+ ### StaticSqlParameterQuery
78
+
79
+ These are effecitively just a function of `(request) => Array[{bucket}]`. These queries can select values from request parameters, and apply filters from request parameters.
80
+
81
+ The WHERE filter is a ParameterValueClause that operates on the request parameters.
82
+ The bucket parameters are each a ParameterValueClause that operates on the request parameters.
83
+
84
+ Compiled expression clauses are structured as follows:
85
+
86
+ ```SQL
87
+ 'literal' -- StaticValueClause
88
+ token_parameters.param -- ParameterValueClause
89
+ request.parameters() -- ParameterValueClause
90
+ fn(token_parameters.param) -- ParameterValueClause. This includes most operators.
91
+ ```
92
+
93
+ ### SqlParameterQuery
94
+
95
+ These queries pre-process parameter tables to effectively create an "index" for efficient queries when syncing.
96
+
97
+ For a parameter query `select org_id from users where users.org_id = token_parameters.org_id and lower(users.email) = token_parameters.email`, this would effectively create an index on `users.org_id, lower(users.email)`. These indexes are referred to as "lookup" values. Only direct equality lookups are supported on these indexes currently (including the IN operator). Support for more general queries such as "greater than" operators may be added later.
98
+
99
+ A SqlParameterQuery defines the following operations:
100
+
101
+ 1. `evaluateParameterRow(row)`: Given a parameter row, compute the lookup index entries.
102
+ 2. `getLookups(request)`: Given request parameters, compute the lookup index entries we need to find.
103
+ 3. `queryBucketIds(request)`: Uses `getLookups(request)`, combined with a database lookup, to compute bucket ids from request parameters.
104
+
105
+ The compiled query is based on the following:
106
+
107
+ 1. WHERE clause compiled into a `ParameterMatchClause`. This computes the lookup index.
108
+ 2. `lookup_extractors`: Set of `RowValueClause`. Each of these represent a SELECT clause based on a row value, e.g. `SELECT users.org_id`. These are evaluated during the `evaluateParameterRow` call.
109
+ 3. `parameter_extractors`. Set of `ParameterValueClause`. Each of these represent a SELECT clause based on a request parameter, e.g. `SELECT token_parameters.user_id`. These are evaluated during the `queryBucketIds` call.
110
+
111
+ Compiled expression clauses are structured as follows:
112
+
113
+ ```SQL
114
+ 'literal' -- StaticValueClause
115
+ mytable.column -- RowValueClause
116
+ fn(mytable.column) -- RowValueClause. This includes most operators.
117
+ token_parameters.param -- ParameterValueClause
118
+ request.parameters() -- ParameterValueClause
119
+ fn(token_parameters.param) -- ParameterValueClause
120
+ fn(mytable.column, token_parameters.param) -- Error: not allowed
121
+
122
+ mytable.column = mytable.other_column -- RowValueClause
123
+ mytable.column = token_parameters.param -- ParameterMatchClause
124
+ token_parameters.param IN mytable.some_array -- ParameterMatchClause
125
+ mytable.some_value IN token_parameters.some_array -- ParameterMatchClause
126
+
127
+ (mytable.column1 = token_parameters.param1) AND (mytable.column2 = token_parameters.param2) -- ParameterMatchClause
128
+ (mytable.column1 = token_parameters.param) OR (mytable.column2 = token_parameters.param) -- ParameterMatchClause
129
+ ```
@@ -5,7 +5,7 @@ import { SqlParameterQuery } from './SqlParameterQuery.js';
5
5
  import { StaticSqlParameterQuery } from './StaticSqlParameterQuery.js';
6
6
  import { TablePattern } from './TablePattern.js';
7
7
  import { SqlRuleError } from './errors.js';
8
- import { EvaluatedParametersResult, EvaluateRowOptions, EvaluationResult, QueryBucketIdOptions, SourceSchema, SqliteRow, SyncParameters } from './types.js';
8
+ import { EvaluateRowOptions, EvaluatedParametersResult, EvaluationResult, QueryBucketIdOptions, QueryParseOptions, RequestParameters, SourceSchema, SqliteRow } from './types.js';
9
9
  export interface QueryParseResult {
10
10
  /**
11
11
  * True if parsed in some form, even if there are errors.
@@ -26,10 +26,10 @@ export declare class SqlBucketDescriptor {
26
26
  global_parameter_queries: StaticSqlParameterQuery[];
27
27
  parameterIdSequence: IdSequence;
28
28
  addDataQuery(sql: string, schema?: SourceSchema): QueryParseResult;
29
- addParameterQuery(sql: string, schema?: SourceSchema): QueryParseResult;
29
+ addParameterQuery(sql: string, schema: SourceSchema | undefined, options: QueryParseOptions): QueryParseResult;
30
30
  evaluateRow(options: EvaluateRowOptions): EvaluationResult[];
31
31
  evaluateParameterRow(sourceTable: SourceTableInterface, row: SqliteRow): EvaluatedParametersResult[];
32
- getStaticBucketIds(parameters: SyncParameters): string[];
32
+ getStaticBucketIds(parameters: RequestParameters): string[];
33
33
  queryBucketIds(options: QueryBucketIdOptions): Promise<string[]>;
34
34
  getSourceTables(): Set<TablePattern>;
35
35
  tableSyncsData(table: SourceTableInterface): boolean;
@@ -25,8 +25,8 @@ export class SqlBucketDescriptor {
25
25
  errors: dataRows.errors
26
26
  };
27
27
  }
28
- addParameterQuery(sql, schema) {
29
- const parameterQuery = SqlParameterQuery.fromSql(this.name, sql, schema);
28
+ addParameterQuery(sql, schema, options) {
29
+ const parameterQuery = SqlParameterQuery.fromSql(this.name, sql, schema, options);
30
30
  if (this.bucket_parameters == null) {
31
31
  this.bucket_parameters = parameterQuery.bucket_parameters;
32
32
  }
@@ -1 +1 @@
1
- {"version":3,"file":"SqlBucketDescriptor.js","sourceRoot":"","sources":["../src/SqlBucketDescriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AAE7C,OAAO,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AACjD,OAAO,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAC;AAuB3D,MAAM,OAAO,mBAAmB;IAI9B,YAAY,IAAY,EAAS,UAAsB;QAAtB,eAAU,GAAV,UAAU,CAAY;QAIvD;;WAEG;QACH,iBAAY,GAAmB,EAAE,CAAC;QAClC,sBAAiB,GAAwB,EAAE,CAAC;QAC5C,6BAAwB,GAA8B,EAAE,CAAC;QAEzD,wBAAmB,GAAG,IAAI,UAAU,EAAE,CAAC;QAVrC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;IAWD,YAAY,CAAC,GAAW,EAAE,MAAqB;QAC7C,IAAI,IAAI,CAAC,iBAAiB,IAAI,IAAI,EAAE;YAClC,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;SACtD;QACD,MAAM,QAAQ,GAAG,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,iBAAiB,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC;QAEtF,QAAQ,CAAC,MAAM,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,EAAE,CAAC;QAE3C,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QAEjC,OAAO;YACL,MAAM,EAAE,IAAI;YACZ,MAAM,EAAE,QAAQ,CAAC,MAAM;SACxB,CAAC;IACJ,CAAC;IAED,iBAAiB,CAAC,GAAW,EAAE,MAAqB;QAClD,MAAM,cAAc,GAAG,iBAAiB,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC;QACzE,IAAI,IAAI,CAAC,iBAAiB,IAAI,IAAI,EAAE;YAClC,IAAI,CAAC,iBAAiB,GAAG,cAAc,CAAC,iBAAiB,CAAC;SAC3D;aAAM;YACL,IACE,IAAI,GAAG,CAAC,CAAC,GAAG,cAAc,CAAC,iBAAkB,EAAE,GAAG,IAAI,CAAC,iBAAiB,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,iBAAiB,CAAC,MAAM,EAChH;gBACA,MAAM,IAAI,KAAK,CAAC,uEAAuE,CAAC,CAAC;aAC1F;SACF;QACD,cAAc,CAAC,EAAE,GAAG,IAAI,CAAC,mBAAmB,CAAC,MAAM,EAAE,CAAC;QACtD,IAAI,cAAc,YAAY,iBAAiB,EAAE;YAC/C,IAAI,CAAC,iBAAiB,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;SAC7C;aAAM;YACL,IAAI,CAAC,wBAAwB,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;SACpD;QAED,OAAO;YACL,MAAM,EAAE,IAAI;YACZ,MAAM,EAAE,cAAc,CAAC,MAAM;SAC9B,CAAC;IACJ,CAAC;IAED,WAAW,CAAC,OAA2B;QACrC,IAAI,OAAO,GAAuB,EAAE,CAAC;QACrC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE;gBACvC,SAAS;aACV;YAED,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,WAAW,CAAC,OAAO,CAAC,WAAW,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;SACzE;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,oBAAoB,CAAC,WAAiC,EAAE,GAAc;QACpE,IAAI,OAAO,GAAgC,EAAE,CAAC;QAC9C,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,IAAI,KAAK,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE;gBAC9B,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,oBAAoB,CAAC,GAAG,CAAC,CAAC,CAAC;aAClD;SACF;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,kBAAkB,CAAC,UAA0B;QAC3C,IAAI,OAAO,GAAa,EAAE,CAAC;QAC3B,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,wBAAwB,EAAE;YAC/C,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,kBAAkB,CAAC,UAAU,CAAC,CAAC,CAAC;SACvD;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,KAAK,CAAC,cAAc,CAAC,OAA6B;QAChD,IAAI,MAAM,GAAa,IAAI,CAAC,kBAAkB,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;QACnE,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;SACvD;QACD,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,eAAe;QACb,IAAI,MAAM,GAAG,IAAI,GAAG,EAAgB,CAAC;QACrC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,WAAY,CAAC,CAAC;SAChC;QACD,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,WAAY,CAAC,CAAC;SAChC;QAED,wDAAwD;QAExD,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,cAAc,CAAC,KAA2B;QACxC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE;gBACxB,OAAO,IAAI,CAAC;aACb;SACF;QACD,OAAO,KAAK,CAAC;IACf,CAAC;IAED,oBAAoB,CAAC,KAA2B;QAC9C,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE;gBACxB,OAAO,IAAI,CAAC;aACb;SACF;QACD,OAAO,KAAK,CAAC;IACf,CAAC;CACF"}
1
+ {"version":3,"file":"SqlBucketDescriptor.js","sourceRoot":"","sources":["../src/SqlBucketDescriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AAE7C,OAAO,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AACjD,OAAO,EAAE,iBAAiB,EAAE,MAAM,wBAAwB,CAAC;AAwB3D,MAAM,OAAO,mBAAmB;IAI9B,YAAY,IAAY,EAAS,UAAsB;QAAtB,eAAU,GAAV,UAAU,CAAY;QAIvD;;WAEG;QACH,iBAAY,GAAmB,EAAE,CAAC;QAClC,sBAAiB,GAAwB,EAAE,CAAC;QAC5C,6BAAwB,GAA8B,EAAE,CAAC;QAEzD,wBAAmB,GAAG,IAAI,UAAU,EAAE,CAAC;QAVrC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;IAWD,YAAY,CAAC,GAAW,EAAE,MAAqB;QAC7C,IAAI,IAAI,CAAC,iBAAiB,IAAI,IAAI,EAAE;YAClC,MAAM,IAAI,KAAK,CAAC,mCAAmC,CAAC,CAAC;SACtD;QACD,MAAM,QAAQ,GAAG,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,EAAE,IAAI,CAAC,iBAAiB,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC;QAEtF,QAAQ,CAAC,MAAM,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,EAAE,CAAC;QAE3C,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;QAEjC,OAAO;YACL,MAAM,EAAE,IAAI;YACZ,MAAM,EAAE,QAAQ,CAAC,MAAM;SACxB,CAAC;IACJ,CAAC;IAED,iBAAiB,CAAC,GAAW,EAAE,MAAgC,EAAE,OAA0B;QACzF,MAAM,cAAc,GAAG,iBAAiB,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,OAAO,CAAC,CAAC;QAClF,IAAI,IAAI,CAAC,iBAAiB,IAAI,IAAI,EAAE;YAClC,IAAI,CAAC,iBAAiB,GAAG,cAAc,CAAC,iBAAiB,CAAC;SAC3D;aAAM;YACL,IACE,IAAI,GAAG,CAAC,CAAC,GAAG,cAAc,CAAC,iBAAkB,EAAE,GAAG,IAAI,CAAC,iBAAiB,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,iBAAiB,CAAC,MAAM,EAChH;gBACA,MAAM,IAAI,KAAK,CAAC,uEAAuE,CAAC,CAAC;aAC1F;SACF;QACD,cAAc,CAAC,EAAE,GAAG,IAAI,CAAC,mBAAmB,CAAC,MAAM,EAAE,CAAC;QACtD,IAAI,cAAc,YAAY,iBAAiB,EAAE;YAC/C,IAAI,CAAC,iBAAiB,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;SAC7C;aAAM;YACL,IAAI,CAAC,wBAAwB,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;SACpD;QAED,OAAO;YACL,MAAM,EAAE,IAAI;YACZ,MAAM,EAAE,cAAc,CAAC,MAAM;SAC9B,CAAC;IACJ,CAAC;IAED,WAAW,CAAC,OAA2B;QACrC,IAAI,OAAO,GAAuB,EAAE,CAAC;QACrC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE;gBACvC,SAAS;aACV;YAED,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,WAAW,CAAC,OAAO,CAAC,WAAW,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;SACzE;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,oBAAoB,CAAC,WAAiC,EAAE,GAAc;QACpE,IAAI,OAAO,GAAgC,EAAE,CAAC;QAC9C,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,IAAI,KAAK,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE;gBAC9B,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,oBAAoB,CAAC,GAAG,CAAC,CAAC,CAAC;aAClD;SACF;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,kBAAkB,CAAC,UAA6B;QAC9C,IAAI,OAAO,GAAa,EAAE,CAAC;QAC3B,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,wBAAwB,EAAE;YAC/C,OAAO,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,kBAAkB,CAAC,UAAU,CAAC,CAAC,CAAC;SACvD;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,KAAK,CAAC,cAAc,CAAC,OAA6B;QAChD,IAAI,MAAM,GAAa,IAAI,CAAC,kBAAkB,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;QACnE,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,CAAC,cAAc,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;SACvD;QACD,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,eAAe;QACb,IAAI,MAAM,GAAG,IAAI,GAAG,EAAgB,CAAC;QACrC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,WAAY,CAAC,CAAC;SAChC;QACD,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,WAAY,CAAC,CAAC;SAChC;QAED,wDAAwD;QAExD,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,cAAc,CAAC,KAA2B;QACxC,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,YAAY,EAAE;YACnC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE;gBACxB,OAAO,IAAI,CAAC;aACb;SACF;QACD,OAAO,KAAK,CAAC;IACf,CAAC;IAED,oBAAoB,CAAC,KAA2B;QAC9C,KAAK,IAAI,KAAK,IAAI,IAAI,CAAC,iBAAiB,EAAE;YACxC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE;gBACxB,OAAO,IAAI,CAAC;aACb;SACF;QACD,OAAO,KAAK,CAAC;IACf,CAAC;CACF"}
@@ -50,10 +50,12 @@ export class SqlDataQuery {
50
50
  schema: querySchema
51
51
  });
52
52
  const filter = tools.compileWhereClause(where);
53
- const allParams = new Set([...filter.bucketParameters, ...bucket_parameters.map((p) => `bucket.${p}`)]);
54
- if ((!filter.error && allParams.size != filter.bucketParameters.length) ||
53
+ const inputParameterNames = filter.inputParameters.map((p) => p.key);
54
+ const bucketParameterNames = bucket_parameters.map((p) => `bucket.${p}`);
55
+ const allParams = new Set([...inputParameterNames, ...bucketParameterNames]);
56
+ if ((!filter.error && allParams.size != filter.inputParameters.length) ||
55
57
  allParams.size != bucket_parameters.length) {
56
- rows.errors.push(new SqlRuleError(`Query must cover all bucket parameters: ${JSONBig.stringify(bucket_parameters)} != ${JSONBig.stringify(filter.bucketParameters)}`, sql, q._location));
58
+ rows.errors.push(new SqlRuleError(`Query must cover all bucket parameters. Expected: ${JSONBig.stringify(bucketParameterNames)} Got: ${JSONBig.stringify(inputParameterNames)}`, sql, q._location));
57
59
  }
58
60
  rows.sourceTable = sourceTable;
59
61
  rows.table = alias;
@@ -67,7 +69,7 @@ export class SqlDataQuery {
67
69
  for (let column of q.columns ?? []) {
68
70
  const name = tools.getOutputName(column);
69
71
  if (name != '*') {
70
- const clause = tools.compileStaticExtractor(column.expr);
72
+ const clause = tools.compileRowValueExtractor(column.expr);
71
73
  if (isClauseError(clause)) {
72
74
  // Error logged already
73
75
  continue;
@@ -143,7 +145,7 @@ export class SqlDataQuery {
143
145
  evaluateRow(table, row) {
144
146
  try {
145
147
  const tables = { [this.table]: this.addSpecialParameters(table, row) };
146
- const bucketParameters = this.filter.filter(tables);
148
+ const bucketParameters = this.filter.filterRow(tables);
147
149
  const bucketIds = bucketParameters.map((params) => getBucketId(this.descriptor_name, this.bucket_parameters, params));
148
150
  const data = this.transformRow(tables);
149
151
  let id = data.id;
@@ -1 +1 @@
1
- {"version":3,"file":"SqlDataQuery.js","sourceRoot":"","sources":["../src/SqlDataQuery.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,4BAA4B,CAAC;AACrD,OAAO,EAAE,KAAK,EAAkB,MAAM,kBAAkB,CAAC;AACzD,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAoB,cAAc,EAAE,MAAM,qBAAqB,CAAC;AAEvE,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAChD,OAAO,EAAE,wBAAwB,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AAC3E,OAAO,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AAWjD,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC;AAC3E,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAOzD,MAAM,OAAO,YAAY;IACvB,MAAM,CAAC,OAAO,CAAC,eAAuB,EAAE,iBAA2B,EAAE,GAAW,EAAE,MAAqB;QACrG,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,EAAE,EAAE,gBAAgB,EAAE,IAAI,EAAE,CAAC,CAAC;QACtD,MAAM,IAAI,GAAG,IAAI,YAAY,EAAE,CAAC;QAEhC,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE;YACrB,MAAM,IAAI,YAAY,CAAC,6CAA6C,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC;SAClG;QACD,MAAM,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,iBAAiB,CAAC,CAAC,CAAC,EAAE;YACzB,MAAM,IAAI,YAAY,CAAC,sCAAsC,EAAE,GAAG,EAAE,CAAC,CAAC,SAAS,CAAC,CAAC;SAClF;QAED,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,wBAAwB,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC;QAEtD,IAAI,CAAC,CAAC,IAAI,IAAI,IAAI,IAAI,CAAC,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,OAAO,EAAE;YACrE,MAAM,IAAI,YAAY,CAAC,iCAAiC,EAAE,GAAG,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;SACvF;QAED,MAAM,QAAQ,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAClC,IAAI,QAAQ,EAAE,IAAI,IAAI,IAAI,EAAE;YAC1B,MAAM,IAAI,YAAY,CAAC,iCAAiC,EAAE,GAAG,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;SACvF;QACD,MAAM,KAAK,GAAW,QAAQ,CAAC,KAAK,IAAI,QAAQ,CAAC,IAAI,CAAC;QAEtD,MAAM,WAAW,GAAG,IAAI,YAAY,CAAC,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrE,IAAI,WAAW,GAA4B,SAAS,CAAC;QACrD,IAAI,MAAM,EAAE;YACV,MAAM,MAAM,GAAG,MAAM,CAAC,SAAS,CAAC,WAAW,CAAC,CAAC;YAC7C,IAAI,MAAM,CAAC,MAAM,IAAI,CAAC,EAAE;gBACtB,MAAM,CAAC,GAAG,IAAI,YAAY,CACxB,SAAS,WAAW,CAAC,MAAM,IAAI,WAAW,CAAC,YAAY,YAAY,EACnE,GAAG,EACH,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,SAAS,CACvB,CAAC;gBACF,CAAC,CAAC,IAAI,GAAG,SAAS,CAAC;gBAEnB,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;aACrB;iBAAM;gBACL,WAAW,GAAG,IAAI,gBAAgB,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;aACnD;SACF;QAED,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC;QACtB,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC;YACzB,KAAK,EAAE,KAAK;YACZ,gBAAgB,EAAE,CAAC,QAAQ,CAAC;YAC5B,YAAY,EAAE,CAAC,KAAK,CAAC;YACrB,GAAG;YACH,MAAM,EAAE,WAAW;SACpB,CAAC,CAAC;QACH,MAAM,MAAM,GAAG,KAAK,CAAC,kBAAkB,CAAC,KAAK,CAAC,CAAC;QAE/C,MAAM,SAAS,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,gBAAiB,EAAE,GAAG,iBAAiB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QACzG,IACE,CAAC,CAAC,MAAM,CAAC,KAAK,IAAI,SAAS,CAAC,IAAI,IAAI,MAAM,CAAC,gBAAiB,CAAC,MAAM,CAAC;YACpE,SAAS,CAAC,IAAI,IAAI,iBAAiB,CAAC,MAAM,EAC1C;YACA,IAAI,CAAC,MAAM,CAAC,IAAI,CACd,IAAI,YAAY,CACd,2CAA2C,OAAO,CAAC,SAAS,CAAC,iBAAiB,CAAC,OAAO,OAAO,CAAC,SAAS,CACrG,MAAM,CAAC,gBAAgB,CACxB,EAAE,EACH,GAAG,EACH,CAAC,CAAC,SAAS,CACZ,CACF,CAAC;SACH;QAED,IAAI,CAAC,WAAW,GAAG,WAAW,CAAC;QAC/B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;QACf,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,eAAe,GAAG,eAAe,CAAC;QACvC,IAAI,CAAC,iBAAiB,GAAG,iBAAiB,CAAC;QAC3C,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC;QAC/B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QAEnB,IAAI,KAAK,GAAG,KAAK,CAAC;QAClB,KAAK,IAAI,MAAM,IAAI,CAAC,CAAC,OAAO,IAAI,EAAE,EAAE;YAClC,MAAM,IAAI,GAAG,KAAK,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;YACzC,IAAI,IAAI,IAAI,GAAG,EAAE;gBACf,MAAM,MAAM,GAAG,KAAK,CAAC,sBAAsB,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;gBACzD,IAAI,aAAa,CAAC,MAAM,CAAC,EAAE;oBACzB,uBAAuB;oBACvB,SAAS;iBACV;gBACD,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;oBACnB,OAAO,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE;wBAC1B,MAAM,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;oBACzC,CAAC;oBACD,QAAQ,CAAC,MAAM,EAAE,IAAI;wBACnB,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;oBACtD,CAAC;iBACF,CAAC,CAAC;aACJ;iBAAM;gBACL,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;oBACnB,OAAO,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE;wBAC1B,MAAM,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;wBAC1B,KAAK,IAAI,GAAG,IAAI,GAAG,EAAE;4BACnB,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE;gCACvB,SAAS;6BACV;4BACD,MAAM,CAAC,GAAG,MAAV,MAAM,CAAC,GAAG,IAAM,GAAG,CAAC,GAAG,CAAC,EAAC;yBAC1B;oBACH,CAAC;oBACD,QAAQ,CAAC,MAAM,EAAE,IAAI;;wBACnB,KAAK,IAAI,MAAM,IAAI,MAAM,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE;4BAC3C,IAAI,MAAC,MAAM,CAAC,IAAI,MAAhB,IAAI,OAAkB,MAAM,EAAC;yBAC9B;oBACH,CAAC;iBACF,CAAC,CAAC;aACJ;YACD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,GAAG,EAAE;gBAC/B,KAAK,GAAG,IAAI,CAAC;aACd;SACF;QACD,IAAI,CAAC,KAAK,EAAE;YACV,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,YAAY,CAAC,kCAAkC,EAAE,GAAG,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC;SACvG;QACD,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;QAClC,OAAO,IAAI,CAAC;IACd,CAAC;IAgBD;QAVA,eAAU,GAAwB,EAAE,CAAC;QAQrC,WAAM,GAAmB,EAAE,CAAC;IAEb,CAAC;IAEhB,OAAO,CAAC,KAA2B;QACjC,OAAO,IAAI,CAAC,WAAW,EAAE,OAAO,CAAC,KAAK,CAAC,CAAC;IAC1C,CAAC;IAED,oBAAoB,CAAC,KAA2B,EAAE,GAAc;QAC9D,IAAI,IAAI,CAAC,WAAY,CAAC,UAAU,EAAE;YAChC,OAAO;gBACL,GAAG,GAAG;gBACN,aAAa,EAAE,IAAI,CAAC,WAAY,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC;aACrD,CAAC;SACH;aAAM;YACL,OAAO,GAAG,CAAC;SACZ;IACH,CAAC;IAED,aAAa,CAAC,WAAmB;QAC/B,IAAI,IAAI,CAAC,mBAAmB,EAAE,EAAE;YAC9B,sCAAsC;YACtC,OAAO,WAAW,CAAC;SACpB;aAAM;YACL,OAAO,IAAI,CAAC,KAAM,CAAC;SACpB;IACH,CAAC;IAED,mBAAmB;QACjB,OAAO,IAAI,CAAC,WAAY,CAAC,UAAU,IAAI,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,WAAY,CAAC,YAAY,CAAC;IACtF,CAAC;IAED,WAAW,CAAC,KAA2B,EAAE,GAAc;QACrD,IAAI;YACF,MAAM,MAAM,GAAG,EAAE,CAAC,IAAI,CAAC,KAAM,CAAC,EAAE,IAAI,CAAC,oBAAoB,CAAC,KAAK,EAAE,GAAG,CAAC,EAAE,CAAC;YACxE,MAAM,gBAAgB,GAAG,IAAI,CAAC,MAAO,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;YACrD,MAAM,SAAS,GAAG,gBAAgB,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAChD,WAAW,CAAC,IAAI,CAAC,eAAgB,EAAE,IAAI,CAAC,iBAAkB,EAAE,MAAM,CAAC,CACpE,CAAC;YAEF,MAAM,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,CAAC;YACvC,IAAI,EAAE,GAAG,IAAI,CAAC,EAAE,CAAC;YACjB,IAAI,OAAO,EAAE,IAAI,QAAQ,EAAE;gBACzB,0EAA0E;gBAC1E,+EAA+E;gBAC/E,6BAA6B;gBAC7B,oFAAoF;gBACpF,kFAAkF;gBAClF,YAAY;gBACZ,EAAE,GAAG,UAAU,CAAC,EAAE,CAAC,IAAI,EAAE,CAAC;aAC3B;YACD,MAAM,WAAW,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;YAEpD,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;gBAChC,OAAO;oBACL,MAAM,EAAE,QAAQ;oBAChB,KAAK,EAAE,WAAW;oBAClB,EAAE,EAAE,EAAE;oBACN,IAAI;oBACJ,MAAM,EAAE,IAAI,CAAC,MAAM;iBACA,CAAC;YACxB,CAAC,CAAC,CAAC;SACJ;QAAC,OAAO,CAAC,EAAE;YACV,OAAO,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,OAAO,IAAI,8BAA8B,EAAE,CAAC,CAAC;SACjE;IACH,CAAC;IAEO,YAAY,CAAC,MAAuB;QAC1C,IAAI,MAAM,GAAc,EAAE,CAAC;QAC3B,KAAK,IAAI,SAAS,IAAI,IAAI,CAAC,UAAU,EAAE;YACrC,SAAS,CAAC,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;SACnC;QACD,OAAO,aAAa,CAAC,MAAM,CAAC,CAAC;IAC/B,CAAC;IAED,iBAAiB;QACf,OAAO,IAAI,CAAC,OAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YAC7B,OAAO,IAAI,CAAC,KAAM,CAAC,aAAa,CAAC,CAAC,CAAC,CAAC;QACtC,CAAC,CAAC,CAAC;IACL,CAAC;IAED,gBAAgB,CAAC,MAAoB;QACnC,IAAI,MAAM,GAAoD,EAAE,CAAC;QAEjE,IAAI,IAAI,CAAC,mBAAmB,EAAE,EAAE;YAC9B,mBAAmB;YACnB,KAAK,IAAI,WAAW,IAAI,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,WAAY,CAAC,EAAE;gBAC3D,IAAI,MAAM,GAAqC,EAAE,CAAC;gBAElD,IAAI,CAAC,mBAAmB,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;gBAE9C,MAAM,CAAC,IAAI,CAAC;oBACV,IAAI,EAAE,IAAI,CAAC,aAAa,CAAC,WAAW,CAAC,KAAK,CAAC;oBAC3C,OAAO,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC;iBAC/B,CAAC,CAAC;aACJ;SACF;aAAM;YACL,iBAAiB;YACjB,IAAI,MAAM,GAAqC,EAAE,CAAC;YAClD,KAAK,IAAI,WAAW,IAAI,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,WAAY,CAAC,EAAE;gBAC3D,IAAI,CAAC,mBAAmB,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;aAC/C;YACD,MAAM,CAAC,IAAI,CAAC;gBACV,IAAI,EAAE,IAAI,CAAC,KAAM;gBACjB,OAAO,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC;aAC/B,CAAC,CAAC;SACJ;QAED,OAAO,MAAM,CAAC;IAChB,CAAC;IAEO,mBAAmB,CAAC,WAA8B,EAAE,MAAwC;QAClG,MAAM,WAAW,GAAgB;YAC/B,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE;gBACzB,IAAI,KAAK,IAAI,IAAI,CAAC,KAAM,EAAE;oBACxB,OAAO,WAAW,CAAC,OAAO,CAAC,MAAM,CAAC,IAAI,cAAc,CAAC,IAAI,CAAC;iBAC3D;qBAAM;oBACL,2BAA2B;oBAC3B,OAAO,cAAc,CAAC,IAAI,CAAC;iBAC5B;YACH,CAAC;YACD,UAAU,EAAE,CAAC,KAAK,EAAE,EAAE;gBACpB,IAAI,KAAK,IAAI,IAAI,CAAC,KAAM,EAAE;oBACxB,OAAO,WAAW,CAAC,UAAU,EAAE,CAAC;iBACjC;qBAAM;oBACL,OAAO,EAAE,CAAC;iBACX;YACH,CAAC;SACF,CAAC;QACF,KAAK,IAAI,SAAS,IAAI,IAAI,CAAC,UAAU,EAAE;YACrC,SAAS,CAAC,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;SACzC;IACH,CAAC;CACF"}
1
+ {"version":3,"file":"SqlDataQuery.js","sourceRoot":"","sources":["../src/SqlDataQuery.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,4BAA4B,CAAC;AACrD,OAAO,EAAE,KAAK,EAAkB,MAAM,kBAAkB,CAAC;AACzD,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAoB,cAAc,EAAE,MAAM,qBAAqB,CAAC;AAEvE,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAC;AAChD,OAAO,EAAE,wBAAwB,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AAC3E,OAAO,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AAWjD,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC;AAC3E,OAAO,EAAE,gBAAgB,EAAE,MAAM,uBAAuB,CAAC;AAOzD,MAAM,OAAO,YAAY;IACvB,MAAM,CAAC,OAAO,CAAC,eAAuB,EAAE,iBAA2B,EAAE,GAAW,EAAE,MAAqB;QACrG,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,EAAE,EAAE,gBAAgB,EAAE,IAAI,EAAE,CAAC,CAAC;QACtD,MAAM,IAAI,GAAG,IAAI,YAAY,EAAE,CAAC;QAEhC,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE;YACrB,MAAM,IAAI,YAAY,CAAC,6CAA6C,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC;SAClG;QACD,MAAM,CAAC,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,iBAAiB,CAAC,CAAC,CAAC,EAAE;YACzB,MAAM,IAAI,YAAY,CAAC,sCAAsC,EAAE,GAAG,EAAE,CAAC,CAAC,SAAS,CAAC,CAAC;SAClF;QAED,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,wBAAwB,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC;QAEtD,IAAI,CAAC,CAAC,IAAI,IAAI,IAAI,IAAI,CAAC,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,OAAO,EAAE;YACrE,MAAM,IAAI,YAAY,CAAC,iCAAiC,EAAE,GAAG,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;SACvF;QAED,MAAM,QAAQ,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QAClC,IAAI,QAAQ,EAAE,IAAI,IAAI,IAAI,EAAE;YAC1B,MAAM,IAAI,YAAY,CAAC,iCAAiC,EAAE,GAAG,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;SACvF;QACD,MAAM,KAAK,GAAW,QAAQ,CAAC,KAAK,IAAI,QAAQ,CAAC,IAAI,CAAC;QAEtD,MAAM,WAAW,GAAG,IAAI,YAAY,CAAC,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,IAAI,CAAC,CAAC;QACrE,IAAI,WAAW,GAA4B,SAAS,CAAC;QACrD,IAAI,MAAM,EAAE;YACV,MAAM,MAAM,GAAG,MAAM,CAAC,SAAS,CAAC,WAAW,CAAC,CAAC;YAC7C,IAAI,MAAM,CAAC,MAAM,IAAI,CAAC,EAAE;gBACtB,MAAM,CAAC,GAAG,IAAI,YAAY,CACxB,SAAS,WAAW,CAAC,MAAM,IAAI,WAAW,CAAC,YAAY,YAAY,EACnE,GAAG,EACH,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,SAAS,CACvB,CAAC;gBACF,CAAC,CAAC,IAAI,GAAG,SAAS,CAAC;gBAEnB,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;aACrB;iBAAM;gBACL,WAAW,GAAG,IAAI,gBAAgB,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;aACnD;SACF;QAED,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC;QACtB,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC;YACzB,KAAK,EAAE,KAAK;YACZ,gBAAgB,EAAE,CAAC,QAAQ,CAAC;YAC5B,YAAY,EAAE,CAAC,KAAK,CAAC;YACrB,GAAG;YACH,MAAM,EAAE,WAAW;SACpB,CAAC,CAAC;QACH,MAAM,MAAM,GAAG,KAAK,CAAC,kBAAkB,CAAC,KAAK,CAAC,CAAC;QAE/C,MAAM,mBAAmB,GAAG,MAAM,CAAC,eAAgB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC;QACtE,MAAM,oBAAoB,GAAG,iBAAiB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC;QACzE,MAAM,SAAS,GAAG,IAAI,GAAG,CAAS,CAAC,GAAG,mBAAmB,EAAE,GAAG,oBAAoB,CAAC,CAAC,CAAC;QACrF,IACE,CAAC,CAAC,MAAM,CAAC,KAAK,IAAI,SAAS,CAAC,IAAI,IAAI,MAAM,CAAC,eAAgB,CAAC,MAAM,CAAC;YACnE,SAAS,CAAC,IAAI,IAAI,iBAAiB,CAAC,MAAM,EAC1C;YACA,IAAI,CAAC,MAAM,CAAC,IAAI,CACd,IAAI,YAAY,CACd,qDAAqD,OAAO,CAAC,SAAS,CACpE,oBAAoB,CACrB,SAAS,OAAO,CAAC,SAAS,CAAC,mBAAmB,CAAC,EAAE,EAClD,GAAG,EACH,CAAC,CAAC,SAAS,CACZ,CACF,CAAC;SACH;QAED,IAAI,CAAC,WAAW,GAAG,WAAW,CAAC;QAC/B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QACnB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;QACf,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,eAAe,GAAG,eAAe,CAAC;QACvC,IAAI,CAAC,iBAAiB,GAAG,iBAAiB,CAAC;QAC3C,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC,OAAO,IAAI,EAAE,CAAC;QAC/B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC;QAEnB,IAAI,KAAK,GAAG,KAAK,CAAC;QAClB,KAAK,IAAI,MAAM,IAAI,CAAC,CAAC,OAAO,IAAI,EAAE,EAAE;YAClC,MAAM,IAAI,GAAG,KAAK,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;YACzC,IAAI,IAAI,IAAI,GAAG,EAAE;gBACf,MAAM,MAAM,GAAG,KAAK,CAAC,wBAAwB,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;gBAC3D,IAAI,aAAa,CAAC,MAAM,CAAC,EAAE;oBACzB,uBAAuB;oBACvB,SAAS;iBACV;gBACD,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;oBACnB,OAAO,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE;wBAC1B,MAAM,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC;oBACzC,CAAC;oBACD,QAAQ,CAAC,MAAM,EAAE,IAAI;wBACnB,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,CAAC;oBACtD,CAAC;iBACF,CAAC,CAAC;aACJ;iBAAM;gBACL,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;oBACnB,OAAO,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,EAAE;wBAC1B,MAAM,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;wBAC1B,KAAK,IAAI,GAAG,IAAI,GAAG,EAAE;4BACnB,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE;gCACvB,SAAS;6BACV;4BACD,MAAM,CAAC,GAAG,MAAV,MAAM,CAAC,GAAG,IAAM,GAAG,CAAC,GAAG,CAAC,EAAC;yBAC1B;oBACH,CAAC;oBACD,QAAQ,CAAC,MAAM,EAAE,IAAI;;wBACnB,KAAK,IAAI,MAAM,IAAI,MAAM,CAAC,UAAU,CAAC,KAAK,CAAC,EAAE;4BAC3C,IAAI,MAAC,MAAM,CAAC,IAAI,MAAhB,IAAI,OAAkB,MAAM,EAAC;yBAC9B;oBACH,CAAC;iBACF,CAAC,CAAC;aACJ;YACD,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,IAAI,GAAG,EAAE;gBAC/B,KAAK,GAAG,IAAI,CAAC;aACd;SACF;QACD,IAAI,CAAC,KAAK,EAAE;YACV,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,YAAY,CAAC,kCAAkC,EAAE,GAAG,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC;SACvG;QACD,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC;QAClC,OAAO,IAAI,CAAC;IACd,CAAC;IAgBD;QAVA,eAAU,GAAwB,EAAE,CAAC;QAQrC,WAAM,GAAmB,EAAE,CAAC;IAEb,CAAC;IAEhB,OAAO,CAAC,KAA2B;QACjC,OAAO,IAAI,CAAC,WAAW,EAAE,OAAO,CAAC,KAAK,CAAC,CAAC;IAC1C,CAAC;IAED,oBAAoB,CAAC,KAA2B,EAAE,GAAc;QAC9D,IAAI,IAAI,CAAC,WAAY,CAAC,UAAU,EAAE;YAChC,OAAO;gBACL,GAAG,GAAG;gBACN,aAAa,EAAE,IAAI,CAAC,WAAY,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC;aACrD,CAAC;SACH;aAAM;YACL,OAAO,GAAG,CAAC;SACZ;IACH,CAAC;IAED,aAAa,CAAC,WAAmB;QAC/B,IAAI,IAAI,CAAC,mBAAmB,EAAE,EAAE;YAC9B,sCAAsC;YACtC,OAAO,WAAW,CAAC;SACpB;aAAM;YACL,OAAO,IAAI,CAAC,KAAM,CAAC;SACpB;IACH,CAAC;IAED,mBAAmB;QACjB,OAAO,IAAI,CAAC,WAAY,CAAC,UAAU,IAAI,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,WAAY,CAAC,YAAY,CAAC;IACtF,CAAC;IAED,WAAW,CAAC,KAA2B,EAAE,GAAc;QACrD,IAAI;YACF,MAAM,MAAM,GAAG,EAAE,CAAC,IAAI,CAAC,KAAM,CAAC,EAAE,IAAI,CAAC,oBAAoB,CAAC,KAAK,EAAE,GAAG,CAAC,EAAE,CAAC;YACxE,MAAM,gBAAgB,GAAG,IAAI,CAAC,MAAO,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;YACxD,MAAM,SAAS,GAAG,gBAAgB,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAChD,WAAW,CAAC,IAAI,CAAC,eAAgB,EAAE,IAAI,CAAC,iBAAkB,EAAE,MAAM,CAAC,CACpE,CAAC;YAEF,MAAM,IAAI,GAAG,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,CAAC;YACvC,IAAI,EAAE,GAAG,IAAI,CAAC,EAAE,CAAC;YACjB,IAAI,OAAO,EAAE,IAAI,QAAQ,EAAE;gBACzB,0EAA0E;gBAC1E,+EAA+E;gBAC/E,6BAA6B;gBAC7B,oFAAoF;gBACpF,kFAAkF;gBAClF,YAAY;gBACZ,EAAE,GAAG,UAAU,CAAC,EAAE,CAAC,IAAI,EAAE,CAAC;aAC3B;YACD,MAAM,WAAW,GAAG,IAAI,CAAC,aAAa,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;YAEpD,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;gBAChC,OAAO;oBACL,MAAM,EAAE,QAAQ;oBAChB,KAAK,EAAE,WAAW;oBAClB,EAAE,EAAE,EAAE;oBACN,IAAI;oBACJ,MAAM,EAAE,IAAI,CAAC,MAAM;iBACA,CAAC;YACxB,CAAC,CAAC,CAAC;SACJ;QAAC,OAAO,CAAC,EAAE;YACV,OAAO,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,OAAO,IAAI,8BAA8B,EAAE,CAAC,CAAC;SACjE;IACH,CAAC;IAEO,YAAY,CAAC,MAAuB;QAC1C,IAAI,MAAM,GAAc,EAAE,CAAC;QAC3B,KAAK,IAAI,SAAS,IAAI,IAAI,CAAC,UAAU,EAAE;YACrC,SAAS,CAAC,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;SACnC;QACD,OAAO,aAAa,CAAC,MAAM,CAAC,CAAC;IAC/B,CAAC;IAED,iBAAiB;QACf,OAAO,IAAI,CAAC,OAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YAC7B,OAAO,IAAI,CAAC,KAAM,CAAC,aAAa,CAAC,CAAC,CAAC,CAAC;QACtC,CAAC,CAAC,CAAC;IACL,CAAC;IAED,gBAAgB,CAAC,MAAoB;QACnC,IAAI,MAAM,GAAoD,EAAE,CAAC;QAEjE,IAAI,IAAI,CAAC,mBAAmB,EAAE,EAAE;YAC9B,mBAAmB;YACnB,KAAK,IAAI,WAAW,IAAI,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,WAAY,CAAC,EAAE;gBAC3D,IAAI,MAAM,GAAqC,EAAE,CAAC;gBAElD,IAAI,CAAC,mBAAmB,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;gBAE9C,MAAM,CAAC,IAAI,CAAC;oBACV,IAAI,EAAE,IAAI,CAAC,aAAa,CAAC,WAAW,CAAC,KAAK,CAAC;oBAC3C,OAAO,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC;iBAC/B,CAAC,CAAC;aACJ;SACF;aAAM;YACL,iBAAiB;YACjB,IAAI,MAAM,GAAqC,EAAE,CAAC;YAClD,KAAK,IAAI,WAAW,IAAI,MAAM,CAAC,SAAS,CAAC,IAAI,CAAC,WAAY,CAAC,EAAE;gBAC3D,IAAI,CAAC,mBAAmB,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;aAC/C;YACD,MAAM,CAAC,IAAI,CAAC;gBACV,IAAI,EAAE,IAAI,CAAC,KAAM;gBACjB,OAAO,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC;aAC/B,CAAC,CAAC;SACJ;QAED,OAAO,MAAM,CAAC;IAChB,CAAC;IAEO,mBAAmB,CAAC,WAA8B,EAAE,MAAwC;QAClG,MAAM,WAAW,GAAgB;YAC/B,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE;gBACzB,IAAI,KAAK,IAAI,IAAI,CAAC,KAAM,EAAE;oBACxB,OAAO,WAAW,CAAC,OAAO,CAAC,MAAM,CAAC,IAAI,cAAc,CAAC,IAAI,CAAC;iBAC3D;qBAAM;oBACL,2BAA2B;oBAC3B,OAAO,cAAc,CAAC,IAAI,CAAC;iBAC5B;YACH,CAAC;YACD,UAAU,EAAE,CAAC,KAAK,EAAE,EAAE;gBACpB,IAAI,KAAK,IAAI,IAAI,CAAC,KAAM,EAAE;oBACxB,OAAO,WAAW,CAAC,UAAU,EAAE,CAAC;iBACjC;qBAAM;oBACL,OAAO,EAAE,CAAC;iBACX;YACH,CAAC;SACF,CAAC;QACF,KAAK,IAAI,SAAS,IAAI,IAAI,CAAC,UAAU,EAAE;YACrC,SAAS,CAAC,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC,CAAC;SACzC;IACH,CAAC;CACF"}
@@ -1,10 +1,10 @@
1
1
  import { SelectedColumn } from 'pgsql-ast-parser';
2
- import { EvaluatedParametersResult, ParameterMatchClause, QueryBucketIdOptions, SourceSchema, SqliteJsonRow, SqliteJsonValue, SqliteRow, StaticRowValueClause, SyncParameters } from './types.js';
3
2
  import { SqlRuleError } from './errors.js';
3
+ import { SourceTableInterface } from './SourceTableInterface.js';
4
4
  import { SqlTools } from './sql_filters.js';
5
5
  import { StaticSqlParameterQuery } from './StaticSqlParameterQuery.js';
6
6
  import { TablePattern } from './TablePattern.js';
7
- import { SourceTableInterface } from './SourceTableInterface.js';
7
+ import { EvaluatedParametersResult, InputParameter, ParameterMatchClause, ParameterValueClause, QueryBucketIdOptions, QueryParseOptions, RequestParameters, RowValueClause, SourceSchema, SqliteJsonRow, SqliteJsonValue, SqliteRow } from './types.js';
8
8
  /**
9
9
  * Represents a parameter query, such as:
10
10
  *
@@ -12,32 +12,74 @@ import { SourceTableInterface } from './SourceTableInterface.js';
12
12
  * SELECT id as user_id, token_parameters.is_admin as is_admin FROM users WHERE users.user_id = token_parameters.user_id
13
13
  */
14
14
  export declare class SqlParameterQuery {
15
- static fromSql(descriptor_name: string, sql: string, schema?: SourceSchema): SqlParameterQuery | StaticSqlParameterQuery;
15
+ static fromSql(descriptor_name: string, sql: string, schema?: SourceSchema, options?: QueryParseOptions): SqlParameterQuery | StaticSqlParameterQuery;
16
16
  sourceTable?: TablePattern;
17
17
  table?: string;
18
18
  sql?: string;
19
19
  columns?: SelectedColumn[];
20
20
  lookup_columns?: SelectedColumn[];
21
21
  static_columns?: SelectedColumn[];
22
- lookup_extractors: Record<string, StaticRowValueClause>;
23
- static_extractors: Record<string, StaticRowValueClause>;
22
+ /**
23
+ * Example: SELECT *user.id* FROM users WHERE ...
24
+ */
25
+ lookup_extractors: Record<string, RowValueClause>;
26
+ /**
27
+ * Example: SELECT *token_parameters.user_id*
28
+ */
29
+ parameter_extractors: Record<string, ParameterValueClause>;
24
30
  filter?: ParameterMatchClause;
25
31
  descriptor_name?: string;
26
32
  /** _Input_ token / user parameters */
27
- input_parameters?: string[];
28
- expanded_input_parameter?: string;
29
- /** _Output_ bucket parameters */
33
+ input_parameters?: InputParameter[];
34
+ /** If specified, an input parameter that expands to an array. */
35
+ expanded_input_parameter?: InputParameter;
36
+ /**
37
+ * _Output_ bucket parameters.
38
+ *
39
+ * Each one of these will be present in either lookup_extractors or static_extractors.
40
+ */
30
41
  bucket_parameters?: string[];
31
42
  id?: string;
32
43
  tools?: SqlTools;
33
- static_tools?: SqlTools;
34
44
  errors: SqlRuleError[];
35
45
  constructor();
36
46
  applies(table: SourceTableInterface): boolean;
37
47
  evaluateParameterRow(row: SqliteRow): EvaluatedParametersResult[];
38
48
  transformRows(row: SqliteRow): SqliteRow[];
39
- resolveBucketIds(bucketParameters: SqliteJsonRow[], parameters: SyncParameters): string[];
40
- lookupParam(param: string, parameters: SyncParameters): string | number | bigint | null;
41
- getLookups(parameters: SyncParameters): SqliteJsonValue[][];
49
+ /**
50
+ * Given partial parameter rows, turn into bucket ids.
51
+ */
52
+ resolveBucketIds(bucketParameters: SqliteJsonRow[], parameters: RequestParameters): string[];
53
+ /**
54
+ * Given sync parameters, get lookups we need to perform on the database.
55
+ *
56
+ * Each lookup is [bucket definition name, parameter query index, ...lookup values]
57
+ */
58
+ getLookups(parameters: RequestParameters): SqliteJsonValue[][];
59
+ /**
60
+ * Given sync parameters (token and user parameters), return bucket ids.
61
+ *
62
+ * This is done in three steps:
63
+ * 1. Given the parameters, get lookups we need to perform on the database.
64
+ * 2. Perform the lookups, returning parameter sets (partial rows).
65
+ * 3. Given the parameter sets, resolve bucket ids.
66
+ */
42
67
  queryBucketIds(options: QueryBucketIdOptions): Promise<string[]>;
68
+ get hasAuthenticatedBucketParameters(): boolean;
69
+ get hasAuthenticatedMatchClause(): boolean;
70
+ get usesUnauthenticatedRequestParameters(): boolean;
71
+ /**
72
+ * Safe:
73
+ * SELECT id as user_id FROM users WHERE users.user_id = request.user_id()
74
+ * SELECT request.jwt() ->> 'org_id' as org_id, id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id'
75
+ * SELECT id as project_id FROM projects WHERE org_id = request.jwt() ->> 'org_id' AND id = request.parameters() ->> 'project_id'
76
+ * SELECT id as category_id FROM categories
77
+ *
78
+ * Dangerous:
79
+ * SELECT id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id'
80
+ * SELECT id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id' AND request.jwt() ->> 'role' = 'authenticated'
81
+ * SELECT id as category_id, request.parameters() ->> 'project_id' as project_id FROM categories
82
+ * SELECT id as category_id FROM categories WHERE request.parameters() ->> 'include_categories'
83
+ */
84
+ get usesDangerousRequestParameters(): boolean;
43
85
  }
@@ -1,11 +1,11 @@
1
1
  import { parse } from 'pgsql-ast-parser';
2
2
  import { SqlRuleError } from './errors.js';
3
3
  import { SqlTools } from './sql_filters.js';
4
+ import { checkUnsupportedFeatures, isClauseError, isParameterValueClause } from './sql_support.js';
4
5
  import { StaticSqlParameterQuery } from './StaticSqlParameterQuery.js';
5
- import { filterJsonRow, getBucketId, isJsonValue, isSelectStatement } from './utils.js';
6
6
  import { TablePattern } from './TablePattern.js';
7
- import { checkUnsupportedFeatures, isClauseError } from './sql_support.js';
8
7
  import { TableQuerySchema } from './TableQuerySchema.js';
8
+ import { filterJsonRow, getBucketId, isJsonValue, isSelectStatement } from './utils.js';
9
9
  /**
10
10
  * Represents a parameter query, such as:
11
11
  *
@@ -13,7 +13,7 @@ import { TableQuerySchema } from './TableQuerySchema.js';
13
13
  * SELECT id as user_id, token_parameters.is_admin as is_admin FROM users WHERE users.user_id = token_parameters.user_id
14
14
  */
15
15
  export class SqlParameterQuery {
16
- static fromSql(descriptor_name, sql, schema) {
16
+ static fromSql(descriptor_name, sql, schema, options) {
17
17
  const parsed = parse(sql, { locationTracking: true });
18
18
  const rows = new SqlParameterQuery();
19
19
  if (parsed.length > 1) {
@@ -25,7 +25,7 @@ export class SqlParameterQuery {
25
25
  }
26
26
  if (q.from == null) {
27
27
  // E.g. SELECT token_parameters.user_id as user_id WHERE token_parameters.is_admin
28
- return StaticSqlParameterQuery.fromSql(descriptor_name, sql, q);
28
+ return StaticSqlParameterQuery.fromSql(descriptor_name, sql, q, options);
29
29
  }
30
30
  rows.errors.push(...checkUnsupportedFeatures(sql, q));
31
31
  if (q.from.length != 1 || q.from[0].type != 'table') {
@@ -57,6 +57,7 @@ export class SqlParameterQuery {
57
57
  parameter_tables: ['token_parameters', 'user_parameters'],
58
58
  sql,
59
59
  supports_expanding_parameters: true,
60
+ supports_parameter_expressions: true,
60
61
  schema: querySchema
61
62
  });
62
63
  const where = q.where;
@@ -68,8 +69,8 @@ export class SqlParameterQuery {
68
69
  rows.filter = filter;
69
70
  rows.descriptor_name = descriptor_name;
70
71
  rows.bucket_parameters = bucket_parameters;
71
- rows.input_parameters = filter.bucketParameters;
72
- const expandedParams = rows.input_parameters.filter((param) => param.endsWith('[*]'));
72
+ rows.input_parameters = filter.inputParameters;
73
+ const expandedParams = rows.input_parameters.filter((param) => param.expands);
73
74
  if (expandedParams.length > 1) {
74
75
  rows.errors.push(new SqlRuleError('Cannot have multiple array input parameters', sql));
75
76
  }
@@ -77,19 +78,11 @@ export class SqlParameterQuery {
77
78
  rows.columns = q.columns ?? [];
78
79
  rows.static_columns = [];
79
80
  rows.lookup_columns = [];
80
- rows.static_tools = new SqlTools({
81
- // This is used for values not on the parameter query table - these operate directly on
82
- // token_parameters or user_parameters.
83
- table: undefined,
84
- value_tables: ['token_parameters', 'user_parameters'],
85
- parameter_tables: [],
86
- sql
87
- });
88
81
  for (let column of q.columns ?? []) {
89
82
  const name = tools.getSpecificOutputName(column);
90
83
  if (tools.isTableRef(column.expr)) {
91
84
  rows.lookup_columns.push(column);
92
- const extractor = tools.compileStaticExtractor(column.expr);
85
+ const extractor = tools.compileRowValueExtractor(column.expr);
93
86
  if (isClauseError(extractor)) {
94
87
  // Error logged already
95
88
  continue;
@@ -98,22 +91,32 @@ export class SqlParameterQuery {
98
91
  }
99
92
  else {
100
93
  rows.static_columns.push(column);
101
- const extractor = rows.static_tools.compileStaticExtractor(column.expr);
94
+ const extractor = tools.compileParameterValueExtractor(column.expr);
102
95
  if (isClauseError(extractor)) {
103
96
  // Error logged already
104
97
  continue;
105
98
  }
106
- rows.static_extractors[name] = extractor;
99
+ rows.parameter_extractors[name] = extractor;
107
100
  }
108
101
  }
109
102
  rows.tools = tools;
110
103
  rows.errors.push(...tools.errors);
111
- rows.errors.push(...rows.static_tools.errors);
104
+ if (rows.usesDangerousRequestParameters && !options?.accept_potentially_dangerous_queries) {
105
+ let err = new SqlRuleError("Potentially dangerous query based on parameters set by the client. The client can send any value for these parameters so it's not a good place to do authorization.", sql);
106
+ err.type = 'warning';
107
+ rows.errors.push(err);
108
+ }
112
109
  return rows;
113
110
  }
114
111
  constructor() {
112
+ /**
113
+ * Example: SELECT *user.id* FROM users WHERE ...
114
+ */
115
115
  this.lookup_extractors = {};
116
- this.static_extractors = {};
116
+ /**
117
+ * Example: SELECT *token_parameters.user_id*
118
+ */
119
+ this.parameter_extractors = {};
117
120
  this.errors = [];
118
121
  }
119
122
  applies(table) {
@@ -124,12 +127,12 @@ export class SqlParameterQuery {
124
127
  [this.table]: row
125
128
  };
126
129
  try {
127
- const filterParameters = this.filter.filter(tables);
130
+ const filterParameters = this.filter.filterRow(tables);
128
131
  let result = [];
129
132
  for (let filterParamSet of filterParameters) {
130
133
  let lookup = [this.descriptor_name, this.id];
131
134
  lookup.push(...this.input_parameters.map((param) => {
132
- return filterParamSet[param];
135
+ return param.filteredRowToLookupValue(filterParamSet);
133
136
  }));
134
137
  const data = this.transformRows(row);
135
138
  const role = {
@@ -153,8 +156,10 @@ export class SqlParameterQuery {
153
156
  }
154
157
  return [result];
155
158
  }
159
+ /**
160
+ * Given partial parameter rows, turn into bucket ids.
161
+ */
156
162
  resolveBucketIds(bucketParameters, parameters) {
157
- const tables = { token_parameters: parameters.token_parameters, user_parameters: parameters.user_parameters };
158
163
  // Filters have already been applied and gotten us the set of bucketParameters - don't attempt to filter again.
159
164
  // We _do_ need to evaluate the output columns here, using a combination of precomputed bucketParameters,
160
165
  // and values from token parameters.
@@ -166,7 +171,7 @@ export class SqlParameterQuery {
166
171
  result[`bucket.${name}`] = lookup[name];
167
172
  }
168
173
  else {
169
- const value = this.static_extractors[name].evaluate(tables);
174
+ const value = this.parameter_extractors[name].lookupParameterValue(parameters);
170
175
  if (!isJsonValue(value)) {
171
176
  // Not valid - exclude.
172
177
  // Should we error instead?
@@ -181,22 +186,33 @@ export class SqlParameterQuery {
181
186
  })
182
187
  .filter((lookup) => lookup != null);
183
188
  }
184
- lookupParam(param, parameters) {
185
- const [table, column] = param.split('.');
186
- const pt = parameters[table];
187
- return pt?.[column] ?? null;
188
- }
189
+ /**
190
+ * Given sync parameters, get lookups we need to perform on the database.
191
+ *
192
+ * Each lookup is [bucket definition name, parameter query index, ...lookup values]
193
+ */
189
194
  getLookups(parameters) {
190
195
  if (!this.expanded_input_parameter) {
191
196
  let lookup = [this.descriptor_name, this.id];
197
+ let valid = true;
192
198
  lookup.push(...this.input_parameters.map((param) => {
193
199
  // Scalar value
194
- return this.lookupParam(param, parameters);
200
+ const value = param.parametersToLookupValue(parameters);
201
+ if (isJsonValue(value)) {
202
+ return value;
203
+ }
204
+ else {
205
+ valid = false;
206
+ return null;
207
+ }
195
208
  }));
209
+ if (!valid) {
210
+ return [];
211
+ }
196
212
  return [lookup];
197
213
  }
198
214
  else {
199
- const arrayString = this.lookupParam(this.expanded_input_parameter.substring(0, this.expanded_input_parameter.length - 3), parameters);
215
+ const arrayString = this.expanded_input_parameter.parametersToLookupValue(parameters);
200
216
  if (arrayString == null || typeof arrayString != 'string') {
201
217
  return [];
202
218
  }
@@ -210,8 +226,10 @@ export class SqlParameterQuery {
210
226
  catch (e) {
211
227
  return [];
212
228
  }
213
- return values.map((expandedValue) => {
229
+ return values
230
+ .map((expandedValue) => {
214
231
  let lookup = [this.descriptor_name, this.id];
232
+ let valid = true;
215
233
  lookup.push(...this.input_parameters.map((param) => {
216
234
  if (param == this.expanded_input_parameter) {
217
235
  // Expand array value
@@ -219,13 +237,32 @@ export class SqlParameterQuery {
219
237
  }
220
238
  else {
221
239
  // Scalar value
222
- return this.lookupParam(param, parameters);
240
+ const value = param.parametersToLookupValue(parameters);
241
+ if (isJsonValue(value)) {
242
+ return value;
243
+ }
244
+ else {
245
+ valid = false;
246
+ return null;
247
+ }
223
248
  }
224
249
  }));
250
+ if (!valid) {
251
+ return null;
252
+ }
225
253
  return lookup;
226
- });
254
+ })
255
+ .filter((lookup) => lookup != null);
227
256
  }
228
257
  }
258
+ /**
259
+ * Given sync parameters (token and user parameters), return bucket ids.
260
+ *
261
+ * This is done in three steps:
262
+ * 1. Given the parameters, get lookups we need to perform on the database.
263
+ * 2. Perform the lookups, returning parameter sets (partial rows).
264
+ * 3. Given the parameter sets, resolve bucket ids.
265
+ */
229
266
  async queryBucketIds(options) {
230
267
  let lookups = this.getLookups(options.parameters);
231
268
  if (lookups.length == 0) {
@@ -234,5 +271,41 @@ export class SqlParameterQuery {
234
271
  const parameters = await options.getParameterSets(lookups);
235
272
  return this.resolveBucketIds(parameters, options.parameters);
236
273
  }
274
+ get hasAuthenticatedBucketParameters() {
275
+ // select request.user_id() as user_id where ...
276
+ const authenticatedExtractor = Object.values(this.parameter_extractors).find((clause) => isParameterValueClause(clause) && clause.usesAuthenticatedRequestParameters) != null;
277
+ return authenticatedExtractor;
278
+ }
279
+ get hasAuthenticatedMatchClause() {
280
+ // select ... where user_id = request.user_id()
281
+ this.filter?.inputParameters.find;
282
+ const authenticatedInputParameter = this.filter.usesAuthenticatedRequestParameters;
283
+ return authenticatedInputParameter;
284
+ }
285
+ get usesUnauthenticatedRequestParameters() {
286
+ // select ... where request.parameters() ->> 'include_comments'
287
+ const unauthenticatedInputParameter = this.filter.usesUnauthenticatedRequestParameters;
288
+ // select request.parameters() ->> 'project_id'
289
+ const unauthenticatedExtractor = Object.values(this.parameter_extractors).find((clause) => isParameterValueClause(clause) && clause.usesUnauthenticatedRequestParameters) != null;
290
+ return unauthenticatedInputParameter || unauthenticatedExtractor;
291
+ }
292
+ /**
293
+ * Safe:
294
+ * SELECT id as user_id FROM users WHERE users.user_id = request.user_id()
295
+ * SELECT request.jwt() ->> 'org_id' as org_id, id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id'
296
+ * SELECT id as project_id FROM projects WHERE org_id = request.jwt() ->> 'org_id' AND id = request.parameters() ->> 'project_id'
297
+ * SELECT id as category_id FROM categories
298
+ *
299
+ * Dangerous:
300
+ * SELECT id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id'
301
+ * SELECT id as project_id FROM projects WHERE id = request.parameters() ->> 'project_id' AND request.jwt() ->> 'role' = 'authenticated'
302
+ * SELECT id as category_id, request.parameters() ->> 'project_id' as project_id FROM categories
303
+ * SELECT id as category_id FROM categories WHERE request.parameters() ->> 'include_categories'
304
+ */
305
+ get usesDangerousRequestParameters() {
306
+ return (this.usesUnauthenticatedRequestParameters &&
307
+ !this.hasAuthenticatedBucketParameters &&
308
+ !this.hasAuthenticatedMatchClause);
309
+ }
237
310
  }
238
311
  //# sourceMappingURL=SqlParameterQuery.js.map