@cap-js/postgres 1.0.1 → 1.2.0

package/CHANGELOG.md

@@ -4,6 +4,37 @@
 - The format is based on [Keep a Changelog](http://keepachangelog.com/).
 - This project adheres to [Semantic Versioning](http://semver.org/).
 
+## Version 1.2.0 - 2023-09-06
+
+### Added
+
+- Reduced the usage of `is not distinct [not] from`. #157
+
+### Fixed
+
+- [Reserved words](https://www.postgresql.org/docs/current/sql-keywords-appendix.html) are now used to automatically escape reserved words which are used as identifier. #178
+- Remove column count limitation. #150
+
+## Version 1.1.0 - 2023-08-01
+
+### Added
+
+- Connectivity to Azure PostgreSQL.
+
+### Fixed
+
+- Order by collation waterfall:
+  1. ICU
+  2. best-effort mapping (`xx` -> `xx_YY`, where `xx_YY` is the first match)
+  3. without collation
+- More stable configuration of `schema_evolution = 'auto'`.
+- Log `hostname` preferrably during deployment.
+- Allow overriding of pool configuration.
+
+### Changed
+
+- Session context variables are set as lower case instead of upper case.
+
 ## Version 1.0.1 - 2023-07-03
 
 ### Added
@@ -16,4 +47,4 @@
 
 ## Version 1.0.0 - 2023-06-23
 
-- Initial Release
+- Initial Release

package/README.md

@@ -1,5 +1,153 @@
 # CDS database service for Postgres
 
 Welcome to the new Postgres database service for [SAP Cloud Application Programming Model](https://cap.cloud.sap) Node.js, based on new, streamlined database architecture and [*pg* driver](https://www.npmjs.com/package/pg) .
+Find full documentation at https://cap.cloud.sap/docs/guides/databases-postgres.
 
-Find documentation at https://cap.cloud.sap/docs/guides/databases-postgres.
+## migration guide from `cds-pg` to `@cap-js/postgres`
+
+`@cap-js/postgres` works as a drop-in replacement for `cds-pg`.  
+However, some preliminary checks and cleanups help:
+
+- for using the BTP Postgres Hyperscaler as database, 
+  - know that the credentials are picked up automatically by from the enviornment (`VCAP_SERVICES.postgres`)
+  - the service binding label is `postgresql-db`
+  - `cds-dbm` is replaced by a hand-crafted "db-deployer" app → see below
+- your local `package.json`: you can safely remove the entry `cds.requires.postgres` previously mandatory for `cds-pg`
+- recommendation: set the env var `DEBUG=sql` during local development to see DB-level output from PostgreSQL
+
+### schema migration
+
+`@cap-js/postgres` brings the same schema evolution capabilities to PostgreSQL known from HANA and SQLite.  
+Enabling schema migration in an existing `cds-pg`-based project consists of generating and deploying a "csn-snapshot" of your database structure.
+
+#### local development
+
+First, set a basis for the evolution
+`$> cds deploy --model-only`  
+→ this will create the table `cds_model` laying the foundation for the schema migration
+
+Subsequent deployments can then re-use the standard deploy mechanism via `$> cds deploy`
+
+#### On BTP, Cloud Foundry environment
+
+The above "csn-snapshots" can be implemented via the `mtar`-based approach. At the same time, the same `mtar` can be used for subsequent PostgreSQL deployments (with schema evolution).
+
+Two major steps in addition to enabling the schema evolution are included in this `mtar`.
+
+1. create local folder `deployer` (any name works)
+2. in `deployer`, create a `package.json` containing
+
+  ```json
+  ...
+  "//npm run migrate": "only one-time!",
+  "migrate": "cds deploy --model-only",
+  "//npm run deploy": "subsequent deployments",
+  "deploy": "cds deploy"
+  ...
+  ```
+
+3. add a section to your `/mta.yaml` denoting the `deployer` directory as a standalone application that runs one-time
+
+```yaml
+- name: pg-db-deployer
+    type: custom
+    path: deployer
+    parameters:
+      buildpacks: nodejs_buildpack
+      no-route: true
+      no-start: true
+      disk-quota: 2GB
+      memory: 512MB
+      tasks:
+      - name: migrate
+        command: npm run migrate
+      # # for subsequent deployments
+      # - name: deploy
+      #  command: npm run deploy
+        disk-quota: 2GB
+        memory: 512MB
+    build-parameters:
+      before-all:
+        custom: 
+        - npm i
+        # generate the "csn-snapshot" - only necessary for one-time migration,
+        # can be commented out on subsequent deployments
+        - cds compile '*' -2 json > deployer/schema.csn
+      ignore: ["node_modules/"]
+    requires:
+      - name: pg-database
+
+resources:
+  - name: pg-database
+    parameters:
+      path: ./pg-options.json
+      service: postgresql-db
+      service-plan: trial # change to yours!
+      skip-service-updates:
+        parameters: true
+      service-tags:
+        - plain
+    type: org.cloudfoundry.managed-service
+```
+
+## migration points to consider
+
+### mixed-case identifiers
+
+even though column names that are not double-quoted are folded to lowercase in PostgreSQL (`yourName` -> `yourname`, `"yourName"` -> `yourName`),  
+you can use the mixed case definitions from your `.cds` files to reference them.  
+
+example: `brewery_id` on DB level -> `brewery_ID` on CDS level
+
+formerly w/ `cds-pg` you had to follow the DB level: `SELECT.from(Beers).columns('brewery_id').groupBy('brewery_id')`  
+now, re-use the CDS definitions: `SELECT.from(Beers).columns('brewery_ID').groupBy('brewery_ID')`
+
+So please adjust your `CQL` statements accordingly.
+
+### timezones (potential _**BREAKING CHANGE**_)
+
+any date- + time-type will get stored in [`UTC`](https://en.wikipedia.org/wiki/Coordinated_Universal_Time) **without any timezone identifier in the actual data field**.  
+CAP's inbound- and outbound adapters take care of converting incoming and outgoing data from/to the desired time zones.  
+So when a `dateime` comes in being in [an ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) compatible format  
+  `2009-01-01T15:00:00+01:00` (15:00:00 on January 1 2009 in Vienna (CEST))  
+will get stored as  
+  `2009-01-01T13:00:00` (13:00:00 on January 1 2009 in UTC).
+
+Please be aware of that concept and rely on the client to parse UTC in your desired timezone (format).
+
+### `cds.DatabaseService` consumption
+
+`InsertResult` now does only return the affected rows and their `ID`s.
+
+```js
+const entries = [
+  { name: 'Beer1', /* ... */ },
+  { name: 'Beer2', /* ... */ },
+  { name: 'Beer3', /* ... */ }
+]
+const insertResult = await cds.run(INSERT.into(Beers).entries(entries))
+expect(insertResult.affectedRows).to.equal(3)
+const beers = [...insertResult] //> this calls the [Symbol.iterator] method of the insert result
+// beers:
+// [ 
+//   { ID: "f81d7ee5-922b-48a1-a12a-a899b8594c99" }, 
+//   { ID: "ddda7f8e-e26b-430f-a80c-ac2c7df29510" },
+//   { ID: "7228c40f-0046-4f53-8a2b-3d55ad825f59" }
+// ]
+```
+
+In `cds-pg`, we additionally surfaced the entire inserted dataset.
+
+```js
+// continuing after the insert of the above example:
+// const insertResult = await cds.run(INSERT.into(Beers).entries(entries))
+
+// this works NO MORE - see above
+const beers = insertResult.results
+expect(beers.length).toStrictEqual(3)
+expect(beers[0].ID).toMatch(uuidRegex)
+expect(beers[0].createdAt.toISOString()).toMatch(timestampRegex)
+expect(beers[0].modifiedAt.toISOString()).toMatch(timestampRegex)
+```
+
+So please adjust your runtime coding accordingly.

package/lib/PostgresService.js

@@ -1,7 +1,9 @@
 const { SQLService } = require('@cap-js/db-service')
-const { Client } = require('pg')
+const { Client, Query } = require('pg')
 const cds = require('@sap/cds/lib')
 const crypto = require('crypto')
+const { Writable, Readable } = require('stream')
+const sessionVariableMap = require('./session.json')
 
 class PostgresService extends SQLService {
   init() {
@@ -17,11 +19,11 @@ class PostgresService extends SQLService {
   get factory() {
     return {
       options: {
-        ...this.options.pool,
         min: 0,
         testOnBorrow: true,
         acquireTimeoutMillis: 1000,
         destroyTimeoutMillis: 1000,
+        ...this.options.pool,
       },
       create: async () => {
         const cr = this.options.credentials || {}
@@ -36,10 +38,14 @@ class PostgresService extends SQLService {
           database: cr.dbname || cr.database,
           schema: cr.schema,
           sslRequired: cr.sslrootcert && (cr.sslrootcert ?? true),
-          ssl: cr.sslrootcert && {
-            rejectUnauthorized: false,
-            ca: cr.sslrootcert,
-          },
+          // from pg driver docs:
+          // passed directly to node.TLSSocket, supports all tls.connect options
+          ssl:
+            cr.ssl /* enable pg module setting to connect to Azure postgres */ ||
+            (cr.sslrootcert && {
+              rejectUnauthorized: false,
+              ca: cr.sslrootcert,
+            }),
         }
         const dbc = new Client(credentials)
         await dbc.connect()
@@ -52,23 +58,23 @@ class PostgresService extends SQLService {
 
   url4() {
     // TODO: Maybe log which database and which user? Be more robust against missing properties?
-    let { host, port } = this.options?.credentials || this.options || {}
-    return host + ':' + (port || 5432)
+    let { host, hostname, port } = this.options?.credentials || this.options || {}
+    return (hostname || host) + ':' + (port || 5432)
   }
 
   async set(variables) {
-    // REVISIT: remove when all environment variables are aligned
     // RESTRICTIONS: 'Custom parameter names must be two or more simple identifiers separated by dots.'
-    const nameMap = {
-      '$user.id': 'CAP.APPLICATIONUSER',
-      '$user.locale': 'CAP.LOCALE',
-      '$valid.from': 'CAP.VALID_FROM',
-      '$valid.to': 'CAP.VALID_TO',
-    }
-
     const env = {}
+
+    // Check all properties on the variables object
     for (let name in variables) {
-      env[nameMap[name]] = variables[name]
+      env[sessionVariableMap[name] || name] = variables[name]
+    }
+
+    // Explicitly check for the default session variable properties
+    // As they are getters and not own properties of the object
+    for (let name in sessionVariableMap) {
+      if (variables[name]) env[sessionVariableMap[name]] = variables[name]
     }
 
     return Promise.all([
@@ -79,24 +85,58 @@ class PostgresService extends SQLService {
         ? [this.exec(`SET search_path TO "${this.options?.credentials?.schema}";`)]
         : []),
 
-      ...(!this._initalCollateCheck
-        ? [
-            (await this.prepare(`SELECT collname FROM pg_collation WHERE collname = 'en_US' OR collname ='en-x-icu';`))
-              .all([])
-              .then(resp => {
-                this._initalCollateCheck = true
-                if (resp.find(row => row.collname === 'en_US')) return
-                if (resp.find(row => row.collname === 'en-x-icu'))
-                  this.class.CQN2SQL.prototype.orderBy = this.class.CQN2SQL.prototype.orderByICU
-                // REVISIT throw error when there is no collated libary found
-              }),
-          ]
-        : []),
+      ...(!this._initalCollateCheck ? [this._checkCollation()] : []),
     ])
   }
 
+  async _checkCollation() {
+    this._initalCollateCheck = true
+
+    const icuPrep = await this.prepare(`SELECT collname FROM pg_collation WHERE collname = 'en-x-icu';`)
+    const icuResp = await icuPrep.all([])
+
+    if (icuResp.length > 0) {
+      this.class.CQN2SQL.prototype.orderBy = this.class.CQN2SQL.prototype.orderByICU
+      return
+    }
+
+    /**
+     * Selects the first two characters of the collation name as key
+     * Select the smallest collation name as value (could also be max)
+     * Filter the collations by the provider c (libc)
+     * Filters the collation names by /.._../ Where '>' points at the '_' that is an actual '_'
+     * The group by is done by the key column to make sure that only one collation per key is returned
+     */
+    const cSQL = `
+SELECT
+  SUBSTRING(collname, 1, 2) AS K,
+  MIN(collname) AS V
+FROM
+  pg_collation
+WHERE
+  collprovider = 'c' AND
+  collname LIKE '__>___' ESCAPE '>'
+GROUP BY k
+`
+
+    const cPrep = await this.prepare(cSQL)
+    const cResp = await cPrep.all([])
+    if (cResp.length > 0) {
+      const collationMap = (this.class.CQN2SQL.prototype.collationMap = cResp.reduce((ret, row) => {
+        ret[row.k] = row.v
+        return ret
+      }, {}))
+      collationMap.default = collationMap.en || collationMap[Object.keys(collationMap)[0]]
+      this.class.CQN2SQL.prototype.orderBy = this.class.CQN2SQL.prototype.orderByLIBC
+      return
+    }
+
+    // REVISIT: print a warning when no collation is found
+  }
+
   prepare(sql) {
     const query = {
+      _streams: 0,
       text: sql,
       // Track queries name for postgres referencing prepare statements
       // sha1 as it needs to be less then 63 characters
@@ -105,7 +145,9 @@ class PostgresService extends SQLService {
     return {
       run: async values => {
         // REVISIT: SQLService provides empty values as {} for plain SQL statements - PostgreSQL driver expects array or nothing - see issue #78
-        const result = await this.dbc.query({ ...query, values: this._getValues(values) })
+        let newQuery = this._prepareStreams(query, values)
+        if (typeof newQuery.then === 'function') newQuery = await newQuery
+        const result = await this.dbc.query(newQuery)
         return { changes: result.rowCount }
       },
       get: async values => {
@@ -122,6 +164,14 @@ class PostgresService extends SQLService {
           throw Object.assign(e, { sql: sql + '\n' + new Array(e.position).fill(' ').join('') + '^' })
         }
       },
+      stream: async (values, one) => {
+        try {
+          const streamQuery = new QueryStream({ ...query, values: this._getValues(values) }, one)
+          return await this.dbc.query(streamQuery)
+        } catch (e) {
+          throw Object.assign(e, { sql: sql + '\n' + new Array(e.position).fill(' ').join('') + '^' })
+        }
+      },
     }
   }
 
@@ -133,6 +183,60 @@ class PostgresService extends SQLService {
     return values
   }
 
+  _prepareStreams(query, values) {
+    values = this._getValues(values)
+    if (!values) return query
+
+    const streams = []
+    const newValues = []
+    let sql = query.text
+    if (Array.isArray(values)) {
+      values.forEach((value, i) => {
+        if (value instanceof Readable) {
+          const streamID = query._streams++
+          const isBinary = value.type === 'binary'
+          const paramStream = new ParameterStream(query.name, streamID)
+          if (isBinary) value.setEncoding('base64')
+          value.pipe(paramStream)
+          value.on('error', err => paramStream.emit('error', err))
+          streams[i] = paramStream
+          newValues[i] = streamID
+          sql = sql.replace(
+            new RegExp(`\\$${i + 1}`, 'g'),
+            // Don't ask about the dollar signs
+            `(SELECT ${isBinary ? `DECODE(PARAM,'base64')` : 'PARAM'} FROM "$$$$PARAMETER_BUFFER$$$$" WHERE NAME='${
+              query.name
+            }' AND ID=$${i + 1})`,
+          )
+          return
+        }
+        newValues[i] = value
+      })
+    }
+
+    if (streams.length > 0) {
+      return (async () => {
+        const newQuery = {
+          text: sql,
+          // Even with the changed SQL it might be common to call this statement with the same parameters as streams
+          // As the streams are selected with their ID as prepared statement parameter, the sql is the same
+          name: crypto.createHash('sha1').update(sql).digest('hex'),
+          values: newValues,
+        }
+        await this.dbc.query({
+          text: 'CREATE TEMP TABLE IF NOT EXISTS "$$PARAMETER_BUFFER$$" (PARAM TEXT, NAME TEXT, ID INT) ON COMMIT DROP',
+        })
+        const proms = []
+        for (const stream of streams) {
+          proms.push(this.dbc.query(stream))
+        }
+        await Promise.all(proms)
+        return newQuery
+      })()
+    }
+    return { ...query, values }
+  }
+
   async exec(sql) {
     return this.dbc.query(sql)
   }
@@ -178,26 +282,29 @@ class PostgresService extends SQLService {
   }
 
   static CQN2SQL = class CQN2Postgres extends SQLService.CQN2SQL {
-    orderBy(orderBy, localized) {
+    _orderBy(orderBy, localized, locale) {
       return orderBy.map(
         localized
           ? c =>
               this.expr(c) +
-              (c.element?.[this.class._localized] ? ` COLLATE "${this.context.locale}"` : '') +
+              (c.element?.[this.class._localized] ? ` COLLATE "${locale}"` : '') +
               (c.sort === 'desc' || c.sort === -1 ? ' DESC' : ' ASC')
           : c => this.expr(c) + (c.sort === 'desc' || c.sort === -1 ? ' DESC' : ' ASC'),
       )
     }
 
+    orderBy(orderBy) {
+      return this._orderBy(orderBy)
+    }
+
     orderByICU(orderBy, localized) {
-      return orderBy.map(
-        localized
-          ? c =>
-              this.expr(c) +
-              (c.element?.[this.class._localized] ? ` COLLATE "${this.context.locale.replace('_', '-')}-x-icu"` : '') +
-              (c.sort === 'desc' || c.sort === -1 ? ' DESC' : ' ASC')
-          : c => this.expr(c) + (c.sort === 'desc' || c.sort === -1 ? ' DESC' : ' ASC'),
-      )
+      const locale = `${this.context.locale.replace('_', '-')}-x-icu`
+      return this._orderBy(orderBy, localized, locale)
+    }
+
+    orderByLIBC(orderBy, localized) {
+      const locale = this.collationMap[this.context.locale] || this.collationMap.default
+      return this._orderBy(orderBy, localized && locale, locale)
     }
 
     from(from) {
@@ -209,18 +316,9 @@ class PostgresService extends SQLService {
       return super.from(from)
     }
 
-    // REVISIT: pg requires alias for {val}
-    SELECT_columns({ SELECT }) {
-      // REVISIT: Genres cqn has duplicate ID column
-      if (!SELECT.columns) return '*'
-      const unique = {}
-      return SELECT.columns
-        .map(x => `${this.column_expr(x)} as ${this.quote(this.column_name(x))}`)
-        .filter(x => {
-          if (unique[x]) return false
-          unique[x] = true
-          return true
-        })
+    column_alias4(x, q) {
+      if (!x.as && 'val' in x) return String(x.val)
+      return super.column_alias4(x, q)
     }
 
     SELECT_expand({ SELECT }, sql) {
@@ -228,19 +326,25 @@ class PostgresService extends SQLService {
       const queryAlias = this.quote(SELECT.from?.as || (SELECT.expand === 'root' && 'root'))
       const cols = SELECT.columns.map(x => {
         const name = this.column_name(x)
-        let col = `${this.string(name)},${this.output_converter4(x.element, queryAlias + '.' + this.quote(name))}`
+        const outputConverter = this.output_converter4(x.element, `${queryAlias}.${this.quote(name)}`)
+        let col = `${outputConverter} as ${this.doubleQuote(name)}`
 
         if (x.SELECT?.count) {
           // Return both the sub select and the count for @odata.count
           const qc = cds.ql.clone(x, { columns: [{ func: 'count' }], one: 1, limit: 0, orderBy: 0 })
-          col += `, '${name}@odata.count',${this.expr(qc)}`
+          col += `,${this.expr(qc)} as ${this.doubleQuote(`${name}@odata.count`)}`
         }
         return col
       })
-      let obj = `json_build_object(${cols})`
+      // REVISIT: Remove SELECT ${cols} by adjusting SELECT_columns
+      let obj = `row_to_json(${queryAlias}.*)`
       return `SELECT ${
         SELECT.one || SELECT.expand === 'root' ? obj : `coalesce(json_agg(${obj}),'[]'::json)`
-      } as _json_ FROM (${sql}) as ${queryAlias}`
+      } as _json_ FROM (SELECT ${cols} FROM (${sql}) as ${queryAlias}) as ${queryAlias}`
+    }
+
+    doubleQuote(name) {
+      return `"${name.replace(/"/g, '""')}"`
     }
 
     INSERT(q, isUpsert = false) {
@@ -251,20 +355,24 @@ class PostgresService extends SQLService {
         // Adjusts json path expressions to be postgres specific
         .replace(/->>'\$(?:(?:\."(.*?)")|(?:\[(\d*)\]))'/g, (a, b, c) => (b ? `->>'${b}'` : `->>${c}`))
         // Adjusts json function to be postgres specific
-        .replace('json_each(?)', 'json_array_elements($1)')
+        .replace('json_each(?)', 'json_array_elements($1::JSON)')
         .replace(/json_type\((\w+),'\$\."(\w+)"'\)/g, (_a, b, c) => `json_typeof(${b}->'${c}')`))
     }
 
+    param({ ref }) {
+      this._paramCount = this._paramCount || 1
+      if (ref.length > 1) throw cds.error`Unsupported nested ref parameter: ${ref}`
+      return ref[0] === '?' ? `$${this._paramCount++}` : `:${ref}`
+    }
+
     val(val) {
       const ret = super.val(val)
       return ret === '?' ? `$${this.values.length}` : ret
     }
 
-    operator(x) {
+    operator(x, i, xpr) {
       if (x === 'regexp') return '~'
-      if (x === '=') return 'is not distinct from'
-      if (x === '!=') return 'is distinct from'
-      else return x
+      else return super.operator(x, i, xpr)
     }
 
     defaultValue(defaultValue = this.context.timestamp.toISOString()) {
@@ -306,14 +414,14 @@ class PostgresService extends SQLService {
       // REVISIT: Remove that with upcomming fixes in cds.linked
       Double: (e, t) => `CAST(${e} as decimal${t.precision && t.scale ? `(${t.precision},${t.scale})` : ''})`,
       DecimalFloat: (e, t) => `CAST(${e} as decimal${t.precision && t.scale ? `(${t.precision},${t.scale})` : ''})`,
-      Binary: e => `CAST(${e} as bytea)`,
-      LargeBinary: e => `CAST(${e} as bytea)`,
+      Binary: e => `DECODE(${e},'base64')`,
+      LargeBinary: e => `DECODE(${e},'base64')`,
     }
 
     static OutputConverters = {
       ...super.OutputConverters,
-      Binary: e => e,
-      LargeBinary: e => e,
+      Binary: e => `ENCODE(${e},'base64')`,
+      LargeBinary: e => `ENCODE(${e},'base64')`,
       Date: e => `to_char(${e}, 'YYYY-MM-DD')`,
       Time: e => `to_char(${e}, 'HH24:MI:SS')`,
       DateTime: e => `to_char(${e}, 'YYYY-MM-DD"T"HH24:MI:SS"Z"')`,
@@ -363,7 +471,7 @@ class PostgresService extends SQLService {
     }
   }
 
-  async tenant({ database, tenant }) {
+  async tenant({ database, tenant }, clean = false) {
     const creds = {
       database: database,
       usergroup: `${database}_USERS`,
@@ -373,18 +481,20 @@ class PostgresService extends SQLService {
     creds.password = creds.user
 
     try {
-      await this.tx(async tx => {
-        // await tx.run(`DROP USER IF EXISTS "${creds.user}"`)
-        await tx
-          .run(`CREATE USER "${creds.user}" IN GROUP "${creds.usergroup}" PASSWORD '${creds.password}'`)
-          .catch(e => {
-            if (e.code === '42710') return
-            throw e
-          })
-      })
-      await this.tx(async tx => {
-        await tx.run(`GRANT CREATE, CONNECT ON DATABASE "${creds.database}" TO "${creds.user}";`)
-      })
+      if (!clean) {
+        await this.tx(async tx => {
+          // await tx.run(`DROP USER IF EXISTS "${creds.user}"`)
+          await tx
+            .run(`CREATE USER "${creds.user}" IN GROUP "${creds.usergroup}" PASSWORD '${creds.password}'`)
+            .catch(e => {
+              if (e.code === '42710') return
+              throw e
+            })
+        })
+        await this.tx(async tx => {
+          await tx.run(`GRANT CREATE, CONNECT ON DATABASE "${creds.database}" TO "${creds.user}";`)
+        })
+      }
 
       // Update credentials to new Schema owner
       await this.disconnect()
@@ -393,7 +503,7 @@ class PostgresService extends SQLService {
       // Create new schema using schema owner
       await this.tx(async tx => {
         await tx.run(`DROP SCHEMA IF EXISTS "${creds.schema}" CASCADE`)
-        await tx.run(`CREATE SCHEMA "${creds.schema}" AUTHORIZATION "${creds.user}"`).catch(() => {})
+        if (!clean) await tx.run(`CREATE SCHEMA "${creds.schema}" AUTHORIZATION "${creds.user}"`).catch(() => {})
       })
     } finally {
       await this.disconnect()
@@ -401,4 +511,204 @@ class PostgresService extends SQLService {
   }
 }
 
+class QueryStream extends Query {
+  constructor(config, one) {
+    // REVISIT: currently when setting the row chunk size
+    // it results in an inconsistent connection state
+    // if (!one) config.rows = 1000
+    super(config)
+
+    this._one = one || config.one
+
+    this.stream = new Readable({
+      read: this.rows
+        ? () => {
+            this.stream.pause()
+            // Request more rows
+            this.connection.execute({
+              portal: this.portal,
+              rows: this.rows,
+            })
+            this.connection.flush()
+          }
+        : () => {},
+    })
+    this.push = this.stream.push.bind(this.stream)
+
+    this._prom = new Promise((resolve, reject) => {
+      this.once('error', reject)
+      this.once('end', () => {
+        if (!this._one) this.push(this.constructor.close)
+        this.push(null)
+        if (this.stream.isPaused()) this.stream.resume()
+        resolve(null)
+      })
+      this.once('row', row => {
+        if (row == null) return resolve(null)
+        resolve(this.stream)
+      })
+    })
+  }
+
+  static sep = Buffer.from(',')
+  static open = Buffer.from('[')
+  static close = Buffer.from(']')
+
+  // Trigger query initialization
+  _getRows(connection) {
+    this.connection = connection
+    connection.execute({
+      portal: this.portal,
+      rows: this.rows ? 1 : undefined,
+    })
+    if (this.rows) {
+      connection.flush()
+    } else {
+      connection.sync()
+    }
+  }
+
+  // Delay requesting more rows until next is called
+  handlePortalSuspended() {
+    this.stream.resume()
+  }
+
+  // Provides metadata information from the database
+  handleRowDescription(msg) {
+    // Use default parser for binary results
+    if (msg.fields.length === 1 && msg.fields[0].dataTypeID === 17) {
+      this.handleDataRow = this.handleBinaryRow
+    } else {
+      this.handleDataRow = msg => {
+        const val = msg.fields[0]
+        if (!this._one && val !== null) this.push(this.constructor.open)
+        this.emit('row', val)
+        this.push(val)
+        delete this.handleDataRow
+      }
+    }
+    return super.handleRowDescription(msg)
+  }
+
+  // Called when a new row is received
+  handleDataRow(msg) {
+    this.push(this.constructor.sep)
+    this.push(msg.fields[0])
+  }
+
+  // Called when a new binary row is received
+  handleBinaryRow(msg) {
+    const val = msg.fields[0] === null ? null : this._result._parsers[0](msg.fields[0])
+    this.push(val)
+    this.emit('row', val)
+  }
+
+  then(resolve, reject) {
+    return this._prom.then(resolve, reject)
+  }
+}
+
+class ParameterStream extends Writable {
+  constructor(queryName, id) {
+    super({})
+    this.queryName = queryName
+    this.id = id
+    this.text = `COPY "$$PARAMETER_BUFFER$$"(param,name,id) FROM STDIN DELIMITER ',' QUOTE '${this.constructor.sep}' CSV`
+    this.lengthBuffer = Buffer.from([0x64, 0, 0, 0, 0])
+
+    // Flush quote character before input stream
+    this.flushChunk = chunk => {
+      delete this.flushChunk
+
+      this.lengthBuffer.writeUInt32BE(chunk.length + 5, 1)
+      this.connection.stream.write(this.lengthBuffer)
+      this.connection.stream.write(Buffer.from(this.constructor.sep))
+      return this.connection.stream.write(chunk)
+    }
+  }
+
+  static sep = String.fromCharCode(31) // Separator One
+  static done = Buffer.from([0x63, 0, 0, 0, 4])
+
+  then(resolve, reject) {
+    this.on('error', reject)
+    this.on('finish', resolve)
+  }
+
+  /**
+   * Indicates that the query was started by the connection
+   * @param {Object} connection
+   */
+  submit(connection) {
+    this.connection = connection
+    // Initialize query to be executed
+    connection.query(this.text)
+  }
+
+  // Used by the client to handle timeouts
+  callback() {}
+
+  _write(chunk, enc, cb) {
+    return this.flush(chunk, cb)
+  }
+
+  _construct(cb) {
+    this.handleCopyInResponse = () => cb()
+  }
+
+  _destroy(err, cb) {
+    this.handleError = () => {
+      this.callback()
+      this.connection = null
+      cb(err)
+    }
+    this.connection.sendCopyFail(err ? err.message : 'ParameterStream early destroy')
+  }
+
+  _final(cb) {
+    const sep = this.constructor.sep
+    this.flush(Buffer.from(`${sep},${this.queryName},${this.id}`), err => {
+      if (err) return cb(err)
+      this._finish = () => {
+        this.emit('finish')
+        cb()
+      }
+      this._destroy = (err, cb) => cb(err)
+      this.connection.stream.write(this.constructor.done)
+    })
+  }
+
+  flush(chunk, callback) {
+    if (this.flushChunk(chunk)) {
+      return callback()
+    }
+    this.connection.stream.once('drain', callback)
+  }
+
+  flushChunk(chunk) {
+    this.lengthBuffer.writeUInt32BE(chunk.length + 4, 1)
+    this.connection.stream.write(this.lengthBuffer)
+    return this.connection.stream.write(chunk)
+  }
+
+  handleError(e) {
+    this.callback()
+    this.emit('error', e)
+    this.connection = null
+  }
+
+  handleCommandComplete(msg) {
+    const match = /COPY (\d+)/.exec((msg || {}).text)
+    if (match) {
+      this.rowCount = parseInt(match[1], 10)
+    }
+  }
+
+  handleReadyForQuery() {
+    this.callback()
+    this._finish()
+    this.connection = null
+  }
+}
+
 module.exports = PostgresService

package/lib/ReservedWords.json

@@ -1,4 +1,102 @@
 {
+  "ALL": 1,
+  "ANALYSE": 1,
+  "ANALYZE": 1,
+  "AND": 1,
+  "ANY": 1,
+  "ARRAY": 1,
+  "AS": 1,
+  "ASC": 1,
+  "ASYMMETRIC": 1,
+  "AUTHORIZATION": 1,
   "BINARY": 1,
-  "ARRAY": 1
+  "BOTH": 1,
+  "CASE": 1,
+  "CAST": 1,
+  "CHECK": 1,
+  "COLLATE": 1,
+  "COLLATION": 1,
+  "COLUMN": 1,
+  "CONCURRENTLY": 1,
+  "CONSTRAINT": 1,
+  "CREATE": 1,
+  "CROSS": 1,
+  "CURRENT_CATALOG": 1,
+  "CURRENT_DATE": 1,
+  "CURRENT_ROLE": 1,
+  "CURRENT_SCHEMA": 1,
+  "CURRENT_TIME": 1,
+  "CURRENT_TIMESTAMP": 1,
+  "CURRENT_USER": 1,
+  "DEFAULT": 1,
+  "DEFERRABLE": 1,
+  "DESC": 1,
+  "DISTINCT": 1,
+  "DO": 1,
+  "ELSE": 1,
+  "END": 1,
+  "EXCEPT": 1,
+  "FALSE": 1,
+  "FETCH": 1,
+  "FOR": 1,
+  "FOREIGN": 1,
+  "FREEZE": 1,
+  "FROM": 1,
+  "FULL": 1,
+  "GRANT": 1,
+  "GROUP": 1,
+  "HAVING": 1,
+  "ILIKE": 1,
+  "IN": 1,
+  "INITIALLY": 1,
+  "INNER": 1,
+  "INTERSECT": 1,
+  "INTO": 1,
+  "IS": 1,
+  "ISNULL": 1,
+  "JOIN": 1,
+  "LATERAL": 1,
+  "LEADING": 1,
+  "LEFT": 1,
+  "LIKE": 1,
+  "LIMIT": 1,
+  "LOCALTIME": 1,
+  "LOCALTIMESTAMP": 1,
+  "NATURAL": 1,
+  "NOT": 1,
+  "NOTNULL": 1,
+  "NULL": 1,
+  "OFFSET": 1,
+  "ON": 1,
+  "ONLY": 1,
+  "OR": 1,
+  "ORDER": 1,
+  "OUTER": 1,
+  "OVERLAPS": 1,
+  "PLACING": 1,
+  "PRIMARY": 1,
+  "REFERENCES": 1,
+  "RETURNING": 1,
+  "RIGHT": 1,
+  "SELECT": 1,
+  "SESSION_USER": 1,
+  "SIMILAR": 1,
+  "SOME": 1,
+  "SYMMETRIC": 1,
+  "TABLE": 1,
+  "TABLESAMPLE": 1,
+  "THEN": 1,
+  "TO": 1,
+  "TRAILING": 1,
+  "TRUE": 1,
+  "UNION": 1,
+  "UNIQUE": 1,
+  "USER": 1,
+  "USING": 1,
+  "VARIADIC": 1,
+  "VERBOSE": 1,
+  "WHEN": 1,
+  "WHERE": 1,
+  "WINDOW": 1,
+  "WITH": 1
 }

package/lib/func.js

@@ -1,4 +1,11 @@
+const session = require('./session.json')
+
 const StandardFunctions = {
+  session_context: x => {
+    let sql = `current_setting('${ session[x.val] || x.val }')`
+    if (x.val === '$now') sql += '::timestamp'
+    return sql
+  },
   countdistinct: x => `count(distinct ${x || '*'})`,
   contains: (...args) => `(coalesce(strpos(${args}),0) > 0)`,
   indexof: (x, y) => `strpos(${x},${y}) - 1`, // sqlite instr is 1 indexed

package/lib/session.json

@@ -0,0 +1,7 @@
+{
+  "$user.id": "cap.applicationuser",
+  "$user.locale": "cap.locale",
+  "$now": "cap.now",
+  "$valid.from": "cap.valid_from",
+  "$valid.to": "cap.valid_to"
+}

package/package.json

@@ -1,8 +1,15 @@
 {
   "name": "@cap-js/postgres",
-  "version": "1.0.1",
+  "version": "1.2.0",
   "description": "CDS database service for Postgres",
-  "homepage": "https://cap.cloud.sap/",
+  "homepage": "https://github.com/cap-js/cds-dbs/tree/main/postgres#cds-database-service-for-postgres",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/cap-js/cds-dbs"
+  },
+  "bugs": {
+    "url": "https://github.com/cap-js/cds-dbs/issues"
+  },
   "keywords": [
     "CAP",
     "CDS",
@@ -20,11 +27,11 @@
     "npm": ">=8"
   },
   "scripts": {
-    "setup": "docker-compose -f pg-stack.yml up -d",
-    "test": "npm run setup && jest --silent"
+    "test": "npm start && jest --silent",
+    "start": "docker-compose -f pg-stack.yml up -d"
   },
   "dependencies": {
-    "@cap-js/db-service": "^1.0.1",
+    "@cap-js/db-service": "^1.2.0",
     "pg": "^8"
   },
   "peerDependencies": {
@@ -53,7 +60,8 @@
           "dialect": "postgres",
           "vcap": {
             "label": "postgresql-db"
-          }
+          },
+          "schema_evolution": "auto"
         }
       },
       "db": "sql"