@sap/cds-compiler 3.6.2 → 3.8.0

package/CHANGELOG.md

@@ -7,6 +7,106 @@
 Note: `beta` fixes, changes and features are usually not listed in this ChangeLog but [here](doc/CHANGELOG_BETA.md).
 The compiler behavior concerning `beta` features can change at any time without notice.
 
+## Version 3.8.0 - 2023-03-27
+
+### Added
+
+- compiler:
+  + Table aliases for sub-queries are no longer required.
+  + A time zone designator can now be used in time literals, e.g. `time'16:41:01+01:30'`or `time'16:41:01Z'`.
+- Calculated elements ("on-read") are now enabled per default.  
+  When used in views, they are replaced by their value, for example:
+  ```cds
+  entity E { one: Integer; two = one + 1; };
+  entity P as projection on E { two };
+  // P is the same as:
+  entity P as projection on E { one + 1 as two };
+  ```
+  This allows to define calculations centrally at the entity, which can be used by
+  other views.
+- In CDL, a ternary operator was added as a shortcut for `CASE` expressions:  
+  `a ? b : c` is a shortcut for `CASE WHEN a THEN b ELSE c END`.  There is no CSN
+  representation. The ternary operator is rendered as a `CASE` expression in CSN.
+- In CDL and CSN, `not null` can now also be used in type definitions.
+- In CDL (and CSN as before), elements can be defined without specifying a type.
+
+### Changed
+
+- API: We now report an error for most backends, if the input CSN has
+  `meta.flavor == 'xtended'`, because only client/inferred CSN is supported.
+- Update OData vocabularies 'PersonalData', 'UI'
+- for.odata: Shortcut annotations `@label`, `@title`, `@description`, `@readonly` are no longer
+  removed from the OData processed CSN.
+- to.cdl:
+  + Annotation arrays are split into multiple lines, if a single line would be too long.
+  + Nested `SELECT`s are put into separate lines to make them more readable.
+  + (Annotation) paths are quoted less often.
+- to.sql: The list of reserved SAP HANA identifiers was updated (for smart quoting).
+
+### Fixed
+
+- The CSN parser now accepts bare `list`s in `columns[]`, similar to the CDL parser.
+- to.cdl:
+  + Delimited identifiers in filters are now surrounded by spaces if necessary, to avoid `]]`
+    being interpreted as an escaped bracket.
+- to.edm(x):
+  + Remove empty `Edm.EntityContainer` again. Removal of an empty entity container has been
+    revoked with [3.5.0](#fixed-7) which was wrong. An empty container must not be rendered
+    as it is not spec compliant.
+  + Correctly resolve chained enum symbols.
+  + Fix a program abort during structured rendering in combination with `--odata-foreign-keys`
+    and foreign keys in structured types.
+  + Correctly render paths to nested foreign keys as primary key in structured mode with
+    `--odata-foreign-keys`.
+- to.hdi/to.sql/to.edm(x):
+  + Reject unmanaged associations as ON-condition path end points.
+  + Fix bug in message rendering for tuple expansion.
+  + Correctly detect invalid @sql.append/prepend in projections.
+- to.hdi/to.sql: The list of SAP HANA keywords was updated to the latest version.
+
+### Removed
+
+- for.odata: Undocumented shortcut annotation `@important` has been removed.
+
+
+## Version 3.7.2 - 2023-02-24
+
+### Fixed
+
+- CSN parser: Structured annotations containing `=` were accidentally interpreted as expressions,
+  even though the corresponding beta flag was not set.
+
+
+## Version 3.7.0 - 2023-02-22
+
+### Added
+
+- Several `annotate` statement can append/prepend values
+  to the same array-valued annotation without an `anno-duplicate` error,
+  even if there is no `using from` dependency between the involved sources
+- SQL methods such as `point.ST_X()` can be used in views.
+- The SQL `new` keyword can be used for `ST_*` types such as `new ST_POINT('Point(0.5 0.5)') )`
+
+### Changed
+
+- Update OData vocabularies 'Common', 'Core', 'Measures', 'PDF', 'UI'.
+- to.edm(x): Empty complex types are no longer warned about as they are allowed.
+
+### Fixed
+
+- `parse.cql` and `parse.expr` no longer ignore type arguments such as `cast(field as String(12))`.
+  One argument is interpreted as `length` and two are interpreted as `precision` and `scale`, similar to
+  how custom types and their arguments are interpreted.
+- Previously, the compiler could not always find a unique redirection target if there were
+  one direct projection on the model target and two or more projections on that projection.
+- The performance of compiler-checks for deeply nested expressions/queries has been improved
+- Fix various bugs in Association to Join translation:
+  + Recursive `$self` dereferencing
+  + Correct resolution of table alias in non-bijective `$self` backlink associations in combination with
+    explicit redirection.
+- to.edm(x): Process value help list convenience annotations on unbound action parameters.
+
+
 ## Version 3.6.2 - 2023-02-06
 
 ### Fixed
@@ -365,6 +465,14 @@ The compiler behavior concerning `beta` features can change at any time without
   it is now only allowed for `count`, `min`, `max`, `sum`, `avg`, `stddev`, `var`.
 - All non-SNAPI options.
 
+## Version 2.15.10 - 2023-01-26
+
+### Fixed
+
+- If an entity with parameters is auto-exposed, the generated projection now has
+  the same formal parameters and its query forwards these parameters to the origin entity.
+- to.edm(x): Respect record type hint `$Type` in EDM JSON as full qualified `@type` URI property.
+
 ## Version 2.15.8 - 2022-08-02
 
 ### Fixed
@@ -406,7 +514,7 @@ The compiler behavior concerning `beta` features can change at any time without
   + Enforce `odata-spec-violation-key-null` on explicit foreign keys of managed primary key associations.
   + Proxies/service cross references are no longer created for associations with arbitrary ON conditions.
     Only managed or `$self` backlink association targets are proxy/service cross reference candidates.
-  + Explicit foreign keys of a managed association that are not a primary key in the target are exposed in the the proxy.
+  + Explicit foreign keys of a managed association that are not a primary key in the target are exposed in the proxy.
   + If an association is primary key, the resulting navigation property is set to `Nullable:false` in structured mode.
 
 ## Version 2.15.0 - 2022-05-06

package/README.md

@@ -29,6 +29,9 @@ Or maintain your package.json dependencies as follows:
 
 Please refer to the [official CDS documentation](https://cap.cloud.sap/docs/cds/).
 
+## How to Obtain Support
+
+In case you find a bug, please report an [incident](https://cap.cloud.sap/docs/resources/#reporting-incidents) on SAP Support Portal.
 
 ## License
 

package/bin/cdsc.js

@@ -239,10 +239,13 @@ function displayUsage( error, helpText, code ) {
   // Display help text first, error at the end (more readable, no scrolling)
   out.write(`${helpText}\n`);
   if (error) {
-    if (error instanceof Array)
-      out.write(`${error.map(err => `cdsc: ERROR: ${err}`).join('\n')}\n`);
-    else
+    if (error instanceof Array) {
+      const errors = error.map(err => `cdsc: ERROR: ${err}`).join('\n');
+      out.write(`${errors}\n`);
+    }
+    else {
       out.write(`cdsc: ERROR: ${error}\n`);
+    }
   }
   throw new ProcessExitError(code);
 }
@@ -350,6 +353,9 @@ function executeCommandLine( command, options, args ) {
       options.odataFormat = 'structured';
       options.odataContainment = true;
     }
+    if (options.odataVocRefs && typeof options.odataVocRefs === 'string')
+      options.odataVocRefs = JSON.parse(options.odataVocRefs);
+
     const csn = options.directBackend ? model : compactModel(model, options);
     if (options.csn) {
       const odataCsn = main.for.odata(csn, options);
@@ -389,8 +395,8 @@ function executeCommandLine( command, options, args ) {
   // Execute the command line option 'manageConstraints' and display the results.
   function manageConstraints( model ) {
     const csn = options.directBackend ? model : compactModel(model, options);
-    const alterConstraintsResult = alterConstraintsWithCsn(csn, options);
     const { src } = options || {};
+    const alterConstraintsResult = alterConstraintsWithCsn(csn, options);
     Object.keys(alterConstraintsResult).forEach((id) => {
       const renderedConstraintStatement = alterConstraintsResult[id];
       if (src === 'hdi')
@@ -499,8 +505,9 @@ function executeCommandLine( command, options, args ) {
         .forEach(msg => log(msg));
     }
     else if (options.noMessageContext) {
+      // TODO: currently, ‹↓› = "downgradable" is only shown here
       messages.filter(msg => (messageLevels[msg.severity] <= options.warning))
-        .forEach(msg => log(main.messageString(msg, normalizeFilename, !!options.noMessageId)));
+        .forEach(msg => log(main.messageString(msg, normalizeFilename, !!options.noMessageId, false, options.testMode && 'compile'))); // TODO: use module name
     }
     else {
       // Contains file-contents that are split at '\n'. Try to avoid multiple `.split()` calls.

package/doc/CHANGELOG_ARCHIVE.md

@@ -777,7 +777,7 @@ synchronously.
 
 - Introduce annotation `@cds.redirection.target`.
   With value `false`, the projection is not considered an implicit redirection target;
-  with value `true`, is is considered a “preferred” redirection target.
+  with value `true`, is considered a “preferred” redirection target.
 
 ## Version 1.49.2 - 2021-02-16
 
@@ -1313,7 +1313,7 @@ synchronously.
 
 - HANA/SQL: Validate ON conditions of mixin association definitions in all subqueries
 
-- OData V2: Assign various `@sap` annotations to the `<edmx:EnitySet>` and `<edmx:AssociationSet>`
+- OData V2: Assign various `@sap` annotations to the `<edmx:entitySet>` and `<edmx:AssociationSet>`
   if such annotations are assigned to CDS entities or associations.
 
 - OData V4 Structured: Omit foreign keys of managed associations that establish the containment relationship to
@@ -1445,7 +1445,7 @@ synchronously.
 
 ### Fixed
 
-- Compiler: `type of <unmanaged assocation>` is now handled correctly by raising an error.
+- Compiler: `type of <unmanaged association>` is now handled correctly by raising an error.
 
 ## Version 1.25.0 - 2020-04-09
 
@@ -1629,7 +1629,7 @@ Fixes
 * In the Hana/Sql backend, correctly resolve forward `on` condition when using
   mixin association that backlinks to an unrelated 3rd party entity and association.
 * Raise a warning if the element of the forward association and the element of
-  the query source do not not originate from the same defining entity. Raise an
+  the query source do not originate from the same defining entity. Raise an
   error if the element of the forward association cannot be found in the query
   source or is ambiguous.
 * Correctly create localization views with compiled model as input;
@@ -2267,7 +2267,7 @@ Changes
   _Update to v1.12.1_ if you experience problems – an inherited `@cds.autoexpose` had not been considered.
 * In `toSql` and `toHana` errors are raised
   + for duplicate definitions of elements that differ only in spelling,
-  + if the the entity is not `abstract` or annotated with any `@cds.persistence` set to true and
+  + if the entity is not `abstract` or annotated with any `@cds.persistence` set to true and
     - an element is typed to be an `array of` a `type`,
     - an implicit managed composition has cardinality to many.
 * Raise a warning if an element is to be `localized` which is not of type `cds.String`.
@@ -2489,7 +2489,7 @@ Fixes
   while rewriting the `on` conditition of a projected association.
 * Apply OData specific checks (e.g. that all elements of an entity must have a type)
   applied only to objects that are exposed in a service.
-* When generating SQL for SQLite, replace the the special variables `$now`, `$user.id`
+* When generating SQL for SQLite, replace the special variables `$now`, `$user.id`
   and `$user.locale` by `CURRENT_TIMESTAMP`, `'$user.id'`, and `'EN'`, respectively.
 * Issue a warning for conflicting cardinality declarations (e.g. `association[1] to many ...`).
 * Handle filters with cardinality correctly when translating associations to joins.

package/doc/CHANGELOG_BETA.md

@@ -8,6 +8,39 @@ Note: `beta` fixes, changes and features are listed in this ChangeLog just for i
 The compiler behavior concerning `beta` features can change at any time without notice.
 **Don't use `beta` fixes, changes and features in productive mode.**
 
+## Version 3.8.0 - 2023-03-27
+
+### Added `v4preview`
+
+This beta flags tries to imitate cds-compiler v4 behavior.
+This includes new compiler messages as well as potentially breaking changes.
+
+Enable this beta flag to ease upgrading to cds-compiler v4.
+
+This flag does not guarantee full compatibility with v4, but only
+helps to identify potential issues as soon as possible.
+
+### Removed `calculatedElements`
+
+Now enabled per default.
+
+## Version 3.7.0 - 2023-02-22
+
+### Added `calculatedElements`
+
+Allows to define calculated elements in entities and aspects.  When used in views, they
+are replaced by their value, for example:
+
+```cds
+entity E { one: Integer; two = one + 1; };
+entity P as projection on E { two };
+// P is the same as:
+entity P as projection on E { one + 1 as two };
+```
+
+This allows to define calculations centrally at the entity, which can be used by
+other views.
+
 ## Version 3.5.0 - 2022-12-07
 
 ### Added `odataTerms`
@@ -250,7 +283,7 @@ Render JOIN cardinality in native HANA association if provided. If no cardinalit
 
 ### Removed `aspectCompositions`
 
-Aspect compositions aka managed compositions are now avaible without beta option.
+Aspect compositions aka managed compositions are now available without beta option.
 _Warning_: the CSN representation can still change.
 
 ## Version 1.31.0 - 2020-06-26
@@ -286,5 +319,5 @@ This option does not enable:
 ### Added `keyRefError`
 
 Always signal an error (instead of just a warning in some cases),
-if not all references in the `keys` of an managed associations
+if not all references in the `keys` of a managed associations
 are projected in the new target.

package/doc/CHANGELOG_DEPRECATED.md

@@ -15,7 +15,7 @@ and several new features are not available.**
 
 ### Added `autoCorrectOrderBySourceRefs`
 
-When this option is set, calling `compile` auto-corrects direct `order by`
+When this option is set, calling `compile` autocorrects direct `order by`
 source element references without table alias for SELECT queries by adding the
 table alias to the `ref`.
 
@@ -153,7 +153,7 @@ Render old and broken temporal EDM API.
 ### Added `noElementsExpansion`
 
 When setting it, association in sub elements are not automatically redirected,
-and the sub elements cannot be annotated indivually.
+and the sub elements cannot be annotated individually.
 
 Do not use this.  Setting it might avoid some compile errors,
 but in most cases the reported errors are rightly reported.

package/doc/DeprecatedOptions_v2.md

@@ -51,7 +51,7 @@ The compiled model contains the following generated entities:
 * the auto-exposed projection `our.Service.Proj.texts`
   which is a composition target of `our.Service.Proj:texts`
 
-For the following sub sections (and in general), is is important to understand that
+For the following subsections (and in general), it is important to understand that
 you can define all auto-exposed entities yourself (well, they are not
 _auto_-exposed anymore)
 _without_ any difference in the compiled model

package/doc/NameResolution.md

@@ -600,7 +600,7 @@ The list of search environments is created as follows:
 
 The above mentioned `:`-escape mechanism leads to the following name resolution:
 
-* The first lexical search environment is the the environment containing all parameter names of the current view.
+* The first lexical search environment is the environment containing all parameter names of the current view.
 * The following search environments are the usual ones from the "main artifact name resolution";
   constant values can be accessed this way (_TODO_: probably not now).
 

package/lib/api/main.js

@@ -63,6 +63,7 @@ function attachTransformerCharacteristics( csn, transformation, options,
       relevant[name] = options[name];
   }
 
+  // eslint-disable-next-line sonarjs/no-empty-collection
   for (const name of relevantGeneralOptions ) {
     if (options[name] !== undefined)
       relevant[name] = options[name];
@@ -334,7 +335,7 @@ function remapName( key, csn, filter = () => true ) {
  * @param {CSN.Model}  csn          A clean input CSN representing the desired "after-image"
  * @param {HdiOptions} options      Options
  * @param {CSN.Model}  beforeImage  A db-transformed CSN representing the "before-image", or null in case no such image
- *                                  is known, i.e. for the very first migration step
+ *                                  is known, i.e. for the very first migration step.
  * @returns {object}                An object with three properties:
  *                                  - afterImage: A db-transformed CSN representing the "after-image"
  *                                  - drops: An array of SQL statements to drop views/tables
@@ -763,7 +764,7 @@ function flattenResultStructure( toProcess ) {
 /**
  * Print args to stderr if CDSC_TRACE_API is set
  *
- * @param {...any} args
+ * @param {...any} args to be logged to stderr
  */
 function traceApi( ...args ) {
   if (process?.env?.CDSC_TRACE_API !== undefined) {
@@ -811,18 +812,21 @@ function publishCsnProcessor( processor, _name ) {
   /**
    * Function that calls the processor and re-compiles in case of internal errors
    *
-   * @param {object} csn CSN
-   * @param {object} options Options
+   * @param {CSN.Model} csn CSN
+   * @param {CSN.Options} options Options
    * @param {any} args Any additional arguments
    * @returns {any} What ever the processor returns
    */
   function api( csn, options = {}, ...args ) {
     try {
-      if (options.deprecated) {
-        const messageFunctions = messages.makeMessageFunction(csn, options, 'api');
+      const messageFunctions = messages.makeMessageFunction(csn, options, 'api');
+      if (options.deprecated)
         checkRemovedDeprecatedFlags( options, messageFunctions );
-      }
-      checkOutdatedOptions( options );
+
+      checkOutdatedOptions( options, messageFunctions );
+      checkCsnFlavor( csn, options, messageFunctions, _name );
+
+      messageFunctions.throwWithError();
 
       timetrace.timetrace.start(_name);
       const result = processor( csn, options, ...args );
@@ -860,37 +864,73 @@ const oldBackendOptionNames = [ 'toSql', 'toOdata', 'toHana', 'forHana' ];
  *  - toOdata/toSql/toHana/forHana -> now flat options
  *
  * @param {CSN.Options} options Backend options
+ * @param {object} messageFunctions Functions returned by makeMessageFunction()
  */
-function checkOutdatedOptions( options ) {
-  const { error, throwWithError } = messages.makeMessageFunction(null, options, 'api');
-
+function checkOutdatedOptions( options, messageFunctions ) {
   // This error has been emitted once, we don't need to emit it again.
-  if (options.messages?.some(m => m.messageId === 'api-invalid-option')) {
-    throwWithError();
+  if (options.messages?.some(m => m.messageId === 'api-invalid-option' || m.messageId === 'api-invalid-variable-replacement'))
     return;
-  }
+
 
   for (const name of oldBackendOptionNames) {
     if (typeof options[name] === 'object') // may be a boolean due to internal options
-      error('api-invalid-option', null, { '#': 'std', name });
+      messageFunctions.error('api-invalid-option', null, { '#': 'deprecated', name });
   }
 
   if (options.magicVars)
-    error('api-invalid-option', null, { '#': 'magicVars' });
+    messageFunctions.error('api-invalid-option', null, { '#': 'magicVars', prop: 'magicVars', otherprop: 'variableReplacements' });
 
   // Don't check `options.magicVars`.  It's likely that the user renamed `magicVars` but
   // forgot about user -> $user and locale -> $user.locale
-  if (options.variableReplacements?.user)
-    error('api-invalid-option', null, { '#': 'user' });
-  if (options.variableReplacements?.locale)
-    error('api-invalid-option', null, { '#': 'locale' });
+  if (options.variableReplacements?.user) {
+    messageFunctions.error('api-invalid-variable-replacement', null, {
+      '#': 'user', option: 'variableReplacements', prop: '$user', otherprop: 'user',
+    });
+  }
+  if (options.variableReplacements?.locale) {
+    messageFunctions.error('api-invalid-variable-replacement', null, {
+      '#': 'locale', option: 'variableReplacements', prop: '$user.locale', otherprop: 'locale',
+    });
+  }
 
   forEachKey(options.variableReplacements || {}, (name) => {
-    if (!name.startsWith('$') && name !== 'user' && name !== 'locale')
-      error('api-invalid-option', null, { '#': 'noDollar', name });
+    if (!name.startsWith('$') && name !== 'user' && name !== 'locale') {
+      messageFunctions.error('api-invalid-variable-replacement', null, {
+        '#': 'noDollar', option: 'variableReplacements', code: '$', name,
+      });
+    }
   });
+}
 
-  throwWithError();
+/**
+ * Checks that the given CSN is usable by our backends, e.g. that
+ * the CSN is not a gensrc (a.k.a. xtended) for most backends.
+ *
+ * For reference, cds-compiler/cds-dk CSN flavor map:
+ *  - client -> inferred
+ *  - gensrc -> xtended
+ *  - parseCdl -> parsed
+ *
+ * If this function becomes more complex (e.g. more module conditions),
+ * move it from then generic api wrapper to the individual module.
+ *
+ * TODO: The compiler does not set any marker in `meta`; we use the umbrella one
+ *       for easier debugging.
+ *
+ * @param {CSN.Model} csn User CSN
+ * @param {CSN.Options} options User options
+ * @param {object} messageFunctions Functions returned by makeMessageFunction()
+ * @param {string} module Backend module, e.g. to.cdl or to.sql
+ */
+function checkCsnFlavor( csn, options, messageFunctions, module ) {
+  if (module === 'to.cdl' || !csn)
+    return; // to.cdl allows every CSN flavor
+
+  if (csn.meta?.flavor === 'xtended') {
+    // TODO: csn.meta?.flavor === 'parsed'; currently used by `@sap/cds` tests.
+    messageFunctions.error('api-unsupported-csn-flavor', null, { name: module, option: csn.meta?.flavor },
+                           'Module $(NAME) expects a client/inferred CSN, not $(OPTION)');
+  }
 }
 
 /**

package/lib/api/options.js

@@ -38,6 +38,7 @@ const publicOptionsNewAPI = [
   'odataProxies',
   'odataXServiceRefs',
   'odataV2PartialConstr',
+  'odataVocRefs',
   'service',
   'serviceNames',
   //

package/lib/api/validate.js

@@ -82,6 +82,11 @@ const validators = {
   sqlMapping: generateStringValidator([ 'plain', 'quoted', 'hdbcds' ]),
   odataVersion: generateStringValidator([ 'v2', 'v4' ]),
   odataFormat: generateStringValidator([ 'flat', 'structured' ]),
+  odataVocRefs: {
+    validate: val => (typeof val === 'object' && !Array.isArray(val)),
+    expected: () => 'type JSON object',
+    found: val => `type ${ Array.isArray(val) ? 'JSON array' : typeof val }`,
+  },
   service: {
     validate: val => typeof val === 'string',
     expected: () => 'type string',

package/lib/base/dictionaries.js

@@ -81,17 +81,29 @@ function dictAddArray( dict, name, entry, messageCallback ) {
   return entry;
 }
 
+function dictFirst( dict ) {
+  if (!dict)
+    return dict;
+  for (const name in dict) {
+    return dict[name];
+  }
+  return undefined;
+}
+
 // Push `entry` to the array value with key `name` in the dictionary `dict`.
-function pushToDict( dict, name, entry ) {
+function pushToDict( dict, name, ...entries ) {
   if (dict[name])
-    dict[name].push( entry );
+    dict[name].push( ...entries );
+  else if (name.charAt(0) !== '_')
+    dict[name] = entries;
   else
-    dict[name] = [ entry ];
+    Object.defineProperty( dict, name, { value: entries, configurable: true, writable: true } );
 }
 
 module.exports = {
   dictAdd,
   dictForEach,
   dictAddArray,
+  dictFirst,
   pushToDict,
 };

package/lib/base/keywords.js

@@ -579,6 +579,7 @@ module.exports = {
     'ST_ALPHASHAPEAGGR',
     'ST_ALPHASHAPEAREAAGGR',
     'ST_ALPHASHAPEEDGEAGGR',
+    'ST_ASGEOJSON',
     'ST_ASMVT',
     'ST_ASSVGAGGR',
     'ST_CIRCULARSTRING',
@@ -606,6 +607,7 @@ module.exports = {
     'ST_INTERSECTIONAGGR',
     'ST_LINESTRING',
     'ST_MAKELINE',
+    'ST_MAKELINEAGGR',
     'ST_MAKEPOLYGON',
     'ST_MEMORY_LOB',
     'ST_MULTILINESTRING',