@sap/cds 5.7.2 → 5.8.0

package/CHANGELOG.md

@@ -4,6 +4,114 @@
 - The format is based on [Keep a Changelog](http://keepachangelog.com/).
 - This project adheres to [Semantic Versioning](http://semver.org/).
 
+## Version 5.8.0 - 2022-01-27
+
+### Added
+
+- Custom `server.js` don't have to export `cds.server` anymore -> we use that by default now. 
+- In `cds.requires`: Support to replace primitive values with objects
+- Support filter functions on renamed properties from external service
+- Results of database queries use `big.js` for values of type `cds.Decimal` and `cds.Integer64` if enabled via `cds.env.features.bigjs`
+- Support lambda in `$filter` in `$expand`
+- Support for `GET` requests on service root in REST adapter (old and new)
+- Support for `HEAD` requests in REST adapter (old and new)
+- New hook `req.before('commit')`
+- Draft (Access control for bound actions): Only the user that is the owner of the draft can execute its bound actions.
+- Check that all keys are provided in REST adapter
+- Restrict access to all services via `cds.env.requires.auth.restrict_all_services = true`
+  + That is, all unrestricted services (i.e., w/o own `@requires`) are treated as having `@requires: 'authenticated-user'`
+- Threshold for automatically sending GET requests as `$batch` (beta, cf. @sap/cds@5.6.0) can be configured per remote service via `cds.env.requires.<srv>.max_get_url_length` (if not configured on service, the global config applies)
+- Alpha out-of-the-box support for DwC
+  + Authentication based on headers set by Jupiter router via `cds.env.requires.auth.kind = 'dwc-auth'`
+  + All DwC headers are forwarded to remote service via `cds.env.requires.<srv>.forward_dwc_headers = true`
+- Limited support for binary data in OData
+  + In payloads, the binary data must be a base64 encoded string
+  + In URLs, the binary data must have the following format: `binary'<url-safe base64 encoded>'`, e.g., `$filter=ID eq binary'Q0FQIE5vZGUuanM='`
+  + The use of binary data in some advanced constructs like `$apply` and `/any()` may be limited
+  + On SQLite, the base64 encoded string is stored to the database
+  + It is strongly discouraged to use binary data as keys. See "Primary Keys — Best Practices" in documentation.
+- Support for OData annotation `@Core.ContentDisposition.Type` with `attachment` as the default value
+- Support for returning custom stream objects in custom handlers (beta):
+  + Example:
+    ```js
+    return {
+      value: instanceof Readable || null,
+      $mediaContentType = 'image/jpeg',
+      $mediaContentDispositionFilename = 'foo.bar', // > optional
+      $mediaContentDispositionType = 'inline' // > optional
+    }
+    ```
+
+### Changed
+
+- `cds deploy --to hana` now uses `cf curl` instead of `cf` command natively
+- Event Mesh: In multitenancy mode, messaging artifacts are also deployed for provider accounts (unless the service option `deployForProvider` is set to `false`)
+- Status code in case of multiple errors (rules apply in order):
+  + If all errors have the same status code, that status code is used
+  + If there is at least one 5xx status code, the resulting status code is 500
+  + If there is at least one 4xx status code, the resulting status code is 400
+  + If none of the rules apply, the resulting status code is 500
+- Ignore the `If-Match` HTTP request header for `UPDATE`/`DELETE` requests whose target entities are not annotated with the `@odata.etag` annotation.
+- I18n template strings now are replaced in EDMX documents such that they can occur multiple times.  For example, the `{i18n>key1} - {i18n>key2}` template results in `value1 - value2`, while previously only the first string was replaced, leading to `value1 - {i18n>key2}`.  This is helpful for the [`Template` strings of `UI.ConnectedFields`](https://github.com/SAP/odata-vocabularies/blob/ac9fe832df9b8c8d35517c637dba7c0ac2753b0f/vocabularies/UI.xml#L168).
+
+### Fixed
+
+- At Node.js runtime, the `development` configuration profile is no longer active if `CDS_ENV` is set to `production` and `NODE_ENV` is undefined
+- Enterprise Messaging: The user is now privileged for AMQP
+- `cds.spawn` also works with synchronous functions
+- Foreign keys in parent are set to `null` when deleting composition of one
+- `cds version` now always prints the version of `@sap/cds-dk`, especially if `cds version` was called from within an npm script, i.e. not from `cds-dk`'s CLI.
+- Better error message in case destination of Remote Service is not found
+- Differentiate between draft already exists and entity locked
+- OData adapter: rollback transaction before rethrowing standard error in case of atomicty group
+- Results of actions/functions do not ignore custom data when using `$expand` query option
+- `req.data` is available in custom error handler in case of deserialization error thrown by legacy odata server
+- Joining entities with renamed foreign keys (limited to single-level projections)
+- Requests with draft and `$expand=*` caused problems in some cases
+- `cds serve` during development longer redirects URLs with similar path segments like `/browse/123/browse/` to e.g. `/browse/`
+- Post processing for renamed column in expand
+- Deploy to HANA: passing of options to `hdi-deploy` via `HDI_DEPLOY_OPTIONS` now possible
+- Keys as path segments in beta OData to CQN parser
+- OData V2 Remote Service (`"kind": "odata-v2"`):
+  + Request data properties of types `cds.Date`, `cds.DateTime` and `cds.Timestamp` are converted accordingly to OData V2 specification
+  + Response data properties of types `cds.Decimal`, `cds.DecimalFloat` (deprecated) and `cds.Integer64` are handled properly when using `Accept` header with `IEEE754Compatible=true/false` and `ExponentialDecimals=true/false` format parameters
+
+## Version 5.7.5 - 2022-01-14
+
+### Fixed
+
+- Instance-based restriction for activation of draft-enabled entities using `or` in restriction
+- Messaging: Duplicate handler execution if application service registered events twice
+- Post of a deeply nested sub-entity with structured parent keys
+- Negating lambda expressions in OData using the `not` operator
+- Event Mesh: Redelivery count when using AMQP
+- OData requests using lambda expressions on localized data
+- `cds.db.exists` wrongly generated a `SELECT * FROM ...` for odata flavor x4
+- Return localized texts on draft activate
+- Unicode characters in unquoted search terms in beta OData to CQN parser
+
+## Version 5.7.4 - 2021-12-22
+
+### Fixed
+
+- Complex `@restrict.where: 'exists [...] or (... or ...) or ...'` in draft union scenario no longer crashes the application
+- Sanitization of null values for `cds.RemoteService`
+- Handling of boolean values in draft union scenario with `$expand` query option
+- `_4odata` flag in CQN stays non-enumerable when forwarding to another application service
+- Handling of type references on properties of associations in `cds.minify`
+
+## Version 5.7.3 - 2021-12-16
+
+### Fixed
+
+- Message Queuing now accepts `amqp` options
+- OData requests using lambda expressions with `contains` function
+- Result of OData query option `$count=true` when using `$apply`
+- `$filter` with navigation to-one equals value crashes
+- `$skiptoken` query option allows to use arbitrary symbols except of `&` with beta OData URL to CQN parser (`cds.env.features.odata_new_parser`). In this non-integer value case the value will not be parsed into CQN.
+- Function names in `$filter` can now be case insensitive (as per OData 4.01)
+- `$count` in `$expand` caused server to crash
+
 ## Version 5.7.2 - 2021-12-09
 
 ### Fixed

package/app/fiori/routes.js

@@ -25,7 +25,7 @@ cds.on ('bootstrap', app => {
         if (v2Index >= 0)  // --> /browse/webapp[/prefix]/v2/browse/ -> /v2/browse
           redirectUrl = originalUrl.substring(v2Index)
         else // --> /browse/webapp[/prefix]/browse/ -> /browse
-          redirectUrl = originalUrl.substring(originalUrl.lastIndexOf(srv.path))
+          redirectUrl = originalUrl.substring(originalUrl.lastIndexOf(srv.path+'/'))
         if (originalUrl !== redirectUrl)  {// safeguard to prevent running in loops
           // console.log ('>>', req.originalUrl, '->', redirectUrl)
           return res.redirect (308, redirectUrl)

package/bin/deploy/to-hana/cfUtil.js

@@ -1,179 +1,292 @@
+const cp = require('child_process');
 
-const runCommand = require('./runCommand');
-const { nullLogger } = require('./logger');
+const cds = require('../../../lib');
+const LOG = cds.log ? cds.log('deploy') : console;
+const DEBUG = cds.debug('deploy');
 
-const POLL_TIMEOUT = 180 * 1000; //ms
-const POLL_RETRY_TIMEOUT = 1000; //ms, since polling takes 1-2sec, repoll almost immediately
+const { bold } = require('../../utils/term');
+
+const CF_COMMAND = 'cf';
+
+const POLL_COUNTER = 40;
+const POLL_DELAY = 2500; //ms
 
-const SPINNER_CHARS = ['-', '\\', '|', '/'];
 
 class CfUtil {
-    /**
-     *
-     * @param {*} serviceName
-     * @param {*} logger
-     * @returns service, or undefined
-     */
-    static async getService(serviceName, logger) {
-        try {
-            const serviceCmd = await runCommand('cf', ['service', serviceName], logger);
-
-            let serviceInfo;
-            if (serviceCmd.code === 0) {
-                serviceInfo = {
-                    name: serviceName,
-                    status: /^\s*status\s*:\s*(.*)$/mi.exec(serviceCmd.stdout)[1],
-                    service: /^\s*service\s*:\s*(.*)$/mi.exec(serviceCmd.stdout)[1]
-                }
-            }
 
-            return serviceInfo;
-        } catch (err) {
-            throw this._getError(err);
-        }
+    _clear() {
+        this.spaceInfo = null;
     }
 
-    static async _pollService(serviceName, startTime, logger = nullLogger) {
-        return new Promise((resolve, reject) => {
-            this._pollServiceTimer(serviceName, startTime, resolve, reject, logger);
+    async _sleep(ms) {
+        return new Promise((resolve) => {
+            setTimeout(resolve, ms);
         });
     }
 
-    static _pollServiceTimer(serviceName, startTime, resolve, reject, logger) {
-        setTimeout(async () => {
-            try {
-                if (Date.now() - startTime >= POLL_TIMEOUT) {
-                    logger.log();
-                    reject(new Error(`[cds.deploy] - Aborting service creation after ${POLL_TIMEOUT / 1000}sec.`));
-                }
+    async _cfRun(...args) {
+        const cmdLine = `${CF_COMMAND} ${args.join(' ')}`;
+        DEBUG && console.time(cmdLine);
 
-                const serviceCmd = await runCommand('cf', ['service', serviceName]);
-                if (serviceCmd.code !== 0) {
-                    reject(new Error(`[cds.deploy] - Getting service info failed with code ${serviceCmd.code}.`));
-                }
+        try {
+            return await new Promise((resolve, reject) => {
+                const child = cp.spawn(CF_COMMAND, args);
 
-                if (serviceCmd.stdout.includes('failed')) {
-                    logger.log();
-                    logger.log(serviceCmd.stdout);
-                    reject(new Error(`[cds.deploy] - Service creation failed.`));
-                }
+                let stdout = '';
+                child.stdout.on('data', (data) => {
+                    stdout += data;
+                });
 
-                if (serviceCmd.stdout.includes('succeeded')) {
-                    logger.log();
-                    logger.log(serviceCmd.stdout);
-                    return resolve();
-                }
+                let stderr = '';
+                child.stderr.on('data', (data) => {
+                    stderr += data;
+                });
+
+                child.on('error', (err) => {
+                    if (err.code === 'ENOENT') {
+                        reject(new Error(`Command ${bold(CF_COMMAND)} not found. Make sure to install the Cloud Foundry Command Line Interface.`));
+                    } else {
+                        reject(err);
+                    }
+                });
+
+                child.on('close', (code) => {
+                    if (!code) {
+                        resolve(stdout.trim());
+                    } else {
+                        const errorMessage = `${stdout}\n${stderr}`;
+                        reject(new Error(errorMessage.trim()));
+                    }
+                });
+            });
+        } finally {
+            DEBUG && console.timeEnd(cmdLine);
+        }
+    }
+
+    async _cfRequest(urlPath, queryObj, bodyObj) {
+        if (queryObj) {
+            const entries = Object.entries(queryObj);
+            const queryStr = entries.map((entry) => `${entry[0]}=${encodeURIComponent(entry[1])}`).join('&');
+            urlPath = urlPath + `?${queryStr}`;
+        }
+
+        // cf curl PATH [-iv] [-X METHOD] [-H HEADER]... [-d DATA] [--output FILE]
+        const args = ['curl', urlPath];
+        if (bodyObj) {
+            args.push('-d');
+            args.push(JSON.stringify(bodyObj)); // cfRun uses spawn so no special handling for quotes on cli required
+        }
+
+        const response = await this._cfRun(...args);
+        const result = response ? JSON.parse(response) : {};
+        if (result.errors) {
+            const errorMessage = result.errors.map((entry) => `${entry.title}: ${entry.detail} (${entry.code})`).join('\n');
+            throw new Error(errorMessage);
+        }
+        return result;
+    }
 
-                this._pollServiceTimer(serviceName, startTime, resolve, reject, logger);
-            } catch (err) {
-                reject(err);
+    _extract(string, pattern, defaultValue) {
+        const array = string.split(/[\n\r]+/);
+        for (let line of array) {
+            const match = line.match(pattern);
+            if (match) {
+                return match[1];
             }
-        }, POLL_RETRY_TIMEOUT);
+        }
+
+        if (typeof defaultValue === 'undefined') {
+            throw Error(`Pattern not found: ${pattern}`);
+        } else {
+            return defaultValue;
+        }
     }
 
-    /**
-     * Triggers service creation and resolves once service is created or rejects in case of errors
-     *
-     * @param {*} service
-     * @param {*} plan
-     * @param {*} serviceName
-     * @param {*} options
-     * @param {*} logger
-     */
-    static async createService(service, plan, serviceName, options, logger) {
-        // cf create-service SERVICE PLAN SERVICE_INSTANCE
-        // eslint-disable-next-line no-async-promise-executor
-        let spinnerId;
-        try {
-            const cfArgs = ['create-service', service, plan, serviceName];
-            if (options) {
-                cfArgs.push('-c');
-                cfArgs.push(JSON.stringify(options));
+    async getCfTargetFromCli() {
+        const result = await this._cfRun('target');
+
+        return {
+            apiEndpoint: this._extract(result, /api endpoint:\s+(.*)$/i),
+            user: this._extract(result, /user:\s+(.*)$/i),
+            org: this._extract(result, /org:\s+(.*)$/i),
+            space: this._extract(result, /space:\s+(.*)$/i),
+        };
+    }
+
+    async getCfSpaceInfo() {
+        if (!this.spaceInfo) {
+            LOG.debug('getting space info');
+
+            const target = await this.getCfTargetFromCli();
+
+            const { org, space } = target;
+            const orgs = await this._cfRequest(`/v3/organizations`, { names: org });
+            if (!orgs.resources || orgs.resources.length !== 1) {
+                throw new Error(`CF org ${bold(org)} not found!`);
             }
-            const createServiceCmd = await runCommand('cf', cfArgs, logger);
-            if (createServiceCmd) {
-                if (createServiceCmd.code !== 0) {
-                    const err = new Error('[cds.deploy] - Create request failed')
-                    err.command = createServiceCmd
-                    throw err;
-                }
 
-                if (createServiceCmd.stdout.includes('already exists')) {
-                    return;
-                }
+            const orgGuid = orgs.resources[0].guid;
+            const spaces = await this._cfRequest(`/v3/spaces`, { names: space, organization_guids: orgGuid });
+            if (!spaces.resources || spaces.resources.length !== 1) {
+                throw new Error(`CF space ${bold(space)} not found in org ${bold(org)}!`);
             }
 
-            spinnerId = this._startSpinner(logger);
-            return await CfUtil._pollService(serviceName, Date.now(), logger);
-        } catch (err) {
-            throw this._getError(err);
-        } finally {
-            this._stopSpinner(spinnerId);
+            const spaceGuid = spaces.resources[0].guid;
+
+            this.spaceInfo = Object.assign({}, target, { orgGuid, spaceGuid });
         }
-    }
 
-    static _startSpinner(logger) {
-        let idx = 0;
-        return setInterval(() => {
-            logger.write(SPINNER_CHARS[idx], true);
-            idx = (idx + 1) % SPINNER_CHARS.length;
-        }, 400);
+        return this.spaceInfo;
     }
 
-    static _stopSpinner(id) {
-        if (id) {
-            clearInterval(id);
+    async getService(serviceName, showMessage = true) {
+        showMessage && LOG.log(`Getting service ${bold(serviceName)}`);
+        const spaceInfo = await this.getCfSpaceInfo();
+
+        let counter = POLL_COUNTER;
+        while (counter > 0) {
+            counter--;
+            const serviceInstances = await this._cfRequest('/v3/service_instances', {
+                names: serviceName,
+                space_guids: spaceInfo.spaceGuid,
+                organization_guids: spaceInfo.orgGuid
+            });
+            if (!serviceInstances || !serviceInstances.resources || serviceInstances.resources.length < 1) {
+                return null;
+            }
+
+            const serviceInstance = serviceInstances.resources[0];
+            if (serviceInstance && serviceInstance.last_operation && serviceInstance.last_operation.state !== 'in progress') {
+                return serviceInstance;
+            }
+
+            await this._sleep(POLL_DELAY);
         }
+
+        throw new Error(`Timeout occurred while getting service ${bold(serviceName)}`);
     }
 
-    /**
-     * Creates the service key
-     *
-     * @param {*} serviceInstance
-     * @param {*} serviceKeyName
-     * @param {*} logger
-     * @returns service key
-     */
-    static async createServiceKey(serviceInstance, serviceKeyName, logger) {
-        try {
-            const createServiceKeyCmd = await runCommand('cf', ['create-service-key', serviceInstance, serviceKeyName, '-c', `{"permissions":"development"}`], logger);
-            if (createServiceKeyCmd.code === 0) {
-                return await this.getServiceKey(serviceInstance, serviceKeyName);
+
+    async getOrCreateService(serviceOfferingName, planName, serviceName, options) {
+
+        const probeService = await this.getService(serviceName, false);
+        if (probeService) {
+            LOG.log(`Getting service ${bold(serviceName)}`);
+            return probeService;
+        }
+
+        LOG.log(`Creating service ${bold(serviceName)} - please be patient...`);
+
+        const spaceInfo = await this.getCfSpaceInfo();
+
+        const servicePlan = await this._cfRequest(`/v3/service_plans`, {
+            names: planName,
+            space_guids: spaceInfo.spaceGuid,
+            organization_guids: spaceInfo.orgGuid,
+            service_offering_names: serviceOfferingName
+        });
+
+        if (!servicePlan.resources || servicePlan.resources.length === 0) {
+            throw new Error(`No service plans found`);
+        }
+
+        const body = {
+            type: 'managed',
+            name: serviceName,
+            tags: [serviceOfferingName],
+            relationships: {
+                space: {
+                    data: {
+                        guid: spaceInfo.spaceGuid
+                    }
+                },
+                service_plan: {
+                    data: {
+                        guid: servicePlan.resources[0].guid
+                    }
+                }
             }
+        }
+
+        if (options) {
+            body.parameters = { ...options };
+        }
+
+        const postResult = await this._cfRequest('/v3/service_instances', undefined, body);
+        if (postResult.errors) {
+            throw new Error(postResult.errors[0].detail);
+        }
 
-            return null;
-        } catch (err) {
-            throw this._getError(err);
+        const newService = await this.getService(serviceName, false);
+        if (newService) {
+            return newService;
         }
+
+        throw new Error(`Could not create service ${bold(serviceName)}`);
     }
 
-    /**
-     * Get the service key
-     *
-     * @param {*} serviceInstance
-     * @param {*} serviceKeyName
-     * @returns service key or null
-     */
-    static async getServiceKey(serviceInstance, serviceKeyName) {
-        try {
-            // use null logger to avoid disclosing the key to console
-            const getServiceKeyCmd = await runCommand('cf', ['service-key', serviceInstance, serviceKeyName], nullLogger);
-            const keyJsonStr = getServiceKeyCmd.stdout.match(/\{([\s\S]*)\}/);
-            return JSON.parse(keyJsonStr[0]);
-        } catch (err) {
-            return null;
+
+    async getServiceKey(serviceInstance, serviceKeyName, showMessage = true) {
+        showMessage && LOG.log(`Getting service key ${bold(serviceKeyName)}`);
+
+        let counter = POLL_COUNTER;
+        while (counter > 0) {
+            counter--;
+            const bindings = await this._cfRequest(`/v3/service_credential_bindings`, { names: serviceKeyName, service_instance_guids: serviceInstance.guid });
+            if (!bindings || !bindings.resources || bindings.resources.length < 1) {
+                return null;
+            }
+            const binding = bindings.resources[0];
+            if (binding && binding.last_operation && binding.last_operation.state !== 'in progress') {
+                const keyDetails = await this._cfRequest(`/v3/service_credential_bindings/${encodeURIComponent(binding.guid)}/details`);
+                return keyDetails.credentials;
+            }
+
+            await this._sleep(POLL_DELAY);
         }
+
+        throw new Error(`Timeout occurred while getting service key ${bold(serviceKeyName)}`);
     }
 
-    static _getError(err) {
-        if (err.code === 'ENOENT') {
-            return new Error(`Command 'cf' not found. Make sure you have the Cloud Foundry command line tool installed.`);
-        } else {
-            return err;
+
+    async getOrCreateServiceKey(serviceInstance, serviceKeyName) {
+
+        const serviceKey = await this.getServiceKey(serviceInstance, serviceKeyName, false);
+        if (serviceKey) {
+            LOG.log(`Getting service key ${bold(serviceKeyName)}`);
+            return serviceKey;
+        }
+
+        LOG.log(`Creating service key ${bold(serviceKeyName)} - please be patient...`);
+
+        const body = {
+            type: 'key',
+            name: serviceKeyName,
+            relationships: {
+                service_instance: {
+                    data: {
+                        guid: serviceInstance.guid
+                    }
+                }
+            },
+            parameters: {
+                permissions: 'development'
+            }
+        }
+
+        const postResult = await this._cfRequest('/v3/service_credential_bindings', undefined, body);
+        if (postResult.errors) {
+            throw new Error(postResult.errors[0].detail);
+        }
+
+        const newServiceKey = await this.getServiceKey(serviceInstance, serviceKeyName, false);
+        if (newServiceKey) {
+            return newServiceKey;
         }
+
+        throw new Error(`Could not create service key ${bold(serviceKeyName)}`);
     }
 }
 
-module.exports = CfUtil;
-
-/* eslint no-console: off */
+module.exports = new CfUtil();

package/bin/deploy/to-hana/gitUtil.js

@@ -0,0 +1,55 @@
+const cp = require('child_process');
+const path = require('path');
+const { fse } = require('@sap/cds-foss');
+const util = require('util');
+
+const cds = require('../../../lib');
+const LOG = cds.log ? cds.log('deploy') : console;
+
+const GIT_IGNORE_FILE = '.gitignore';
+
+class GitUtil {
+
+    constructor() {
+        this.execAsync = util.promisify(cp.exec);
+    }
+
+    async ensureFileIsGitignored(file, currentFolder = process.cwd()) {
+        const gitIgnorePath = path.join(currentFolder, GIT_IGNORE_FILE);
+
+        try {
+            await this.execAsync(`git check-ignore -n -v ${file}`, {
+                cwd: currentFolder
+            });
+            return; // file is git ignored
+        } catch (err) {
+            if (err.code === 'ENOENT') {   // git command not found
+                LOG.debug('git command not found');
+                return;
+            }
+
+            if (!err.stdout.match(/::/)) {   // included or not a git repo
+                LOG.debug(err.message);
+                return;
+            }
+        }
+
+        LOG.log(`adding entry '${file}' to ${GIT_IGNORE_FILE}.`);
+        let gitignore = await this.readFileSafely(gitIgnorePath);
+        gitignore = gitignore + `
+    # added by cds deploy
+    ${file}
+    `;
+        await fse.outputFile(gitIgnorePath, gitignore, 'utf8');
+    }
+
+    async readFileSafely(file) {
+        try {
+            return await fse.readFile(file, 'utf8');
+        } catch (err) {
+            return '';
+        }
+    }
+}
+
+module.exports = new GitUtil();