@sap/cds-compiler 3.9.4 → 4.0.2

package/CHANGELOG.md

@@ -7,15 +7,118 @@
 Note: `beta` fixes, changes and features are usually not listed in this ChangeLog but [here](doc/CHANGELOG_BETA.md).
 The compiler behavior concerning `beta` features can change at any time without notice.
 
-## Version 3.9.4 - 2023-06-07
+
+## Version 4.0.2 - 2023-06-22
 
 ### Fixed
 
-- compiler: `USING` empty files were incorrectly marked as "not found".
+- to.sql.migration: When drop-creating views, also drop-create (transitively) dependent views.
+- to.edm(x):
+  + Forward `@odata.singleton { nullable }` annotation to parameter entity.
+  + Annotations assigned to a parameterized entity are propagated to the parameter entity if the annotation is
+    applicable to either an `edm.EntitySet` or `edm.Singleton`. This especially covers all `@Capabilities` and their
+    shortcut forms like `@readonly` and `@insertonly`. The original annotation is not removed from the original entity.
+    Annotations that should be rendered at the parameter `edm.EntityType` can be qualified with `$parameters`.
+    Explicitly qualified annotations are removed from the original entity allowing individual assignments.
+
+
+## Version 4.0.0 - 2023-06-06
+
+### Added
+
+- Calculated elements "on-write" are now supported, e.g. `elem = field + 1 stored;` will be rendered in SQL
+  as `GENERATED ALWAYS AS`.
+- compiler:
+  + `returns` of actions and functions can now be annotated, e.g.
+
+    ```cds
+    action A() returns @direct { … };
+    annotate A with returns {
+      @anno val;
+    }
+    ```
+  + It is now possible to use `*/` inside doc comments by escaping it `*\/`.  Only this exact string can be escaped.
+  + Functions `parse.expr` and `parse.cql` now support `group by`, `having`, `order by`, `limit`, and `offset` in infix filters.
+  + In expressions, you can now use function names after the `new` keyword which do
+    not start with `st_`, if the name is followed by an open parenthesis.
+
+### Changed
+
+- API:
+  + `messageContext()` is now deprecated; use `messageStringMultiline()` instead with `config.sourceMap`.
+  + `messageString(err, config)` has a new signature; the old one is still supported for backward compatibility.
+  + Option `docComment: false` now removes doc comments during compilation even for CSN.
+    If this option is not defined, doc comments are checked, but not put into CSN.
+- compiler:
+  + CSN: Specified elements of queries are now resolved and checked (previously ignored except for annotations).
+    Type properties (`length`, …) and some element properties are now taken from the specified elements
+    instead of relying only on the elements inferred by the compiler.
+  + CSN: Compiler accepts CSN input with CSN version `0.1.0` (produced by cds-compiler version 1.10.0 and older)
+    only if the version attribute is properly set, i.e. `"version": {"csn": "0.1.0"}`.
+  + CSN: Type properties (`length`, `precision`, `scale`, and `srid`) next to `elements` or `items` in CSN input
+    is now an error.  Previously ignored.
+  + An extension of the form `extend Def with { name = 42 };` is now represented in parsed CSN as
+    adding a calculated element.
+  + `having` as the first word in an infix filter is now interpreted as keyword. Write `![having]`
+    to have it parsed as identifier.
+  + If a definition overrides elements of an included definition, it is sorted according
+    to the included definition's element order.  This is similar to how `*` works in projections.
+    It is no longer possible to overwrite a scalar element with a structure and vice versa.
+  + Two included definitions cannot have the same element. The including entity must override the element.
+  + If a type with properties precision and scale is extended, the `extend` statement must extend both properties.
+  + `annotate E:elem with actions {};` is now a parser error and no longer a compiler error.
+    Only relevant if you use `parseCdl`-style CSN.
+  + Annotations (and other properties) are now propagated to `returns` as well, if the returned artifact
+    is a non-entity, e.g. a type.
+  + `annotate E with returns {…}` is now only applied to actions/functions. Previous compiler versions
+    autocorrected the `annotate` statements to apply them to an entity's elements.
+  + Calculated elements can't refer to localized elements.
+  + Table aliases can't be used in `extend Query with columns {…}` any longer. Use direct element references instead.
+  + Table alias and mixin names can no longer start with `$`. Choose a different name. With this change
+    we avoid unexpected name resolution effects in combination with built-in `$`-variables.
+  + A semicolon is now required after a type definition like `type T : many {} null`.
+  + It is no longer possible to write `type of $self.‹elem›` to refer to the element `‹Def›.‹elem›`
+    where `‹Def›` is the main artifact where the type expression is embedded in. Replace by `type of <Def>:‹elem›`.
+  + Message ID `duplicate-autoexposed` was changed to `def-duplicate-autoexposed`.
+- Update OData vocabularies 'Common', 'UI'.
+- to.sql:
+  + Change default `length` for `cds.String` for all SQL dialects except `hana` to 255.
+  + for the sql dialect `postgres`, the `ON DELETE RESTRICT` / `ON UPDATE RESTRICT` rules
+    are omitted from the referential constraint definitions.
+- to.cdl:
+  + If associations are used inside `items`, an error is emitted instead of rendering invalid CDL.
+  + `items` inside `items`, where the outer one does not have a `type`, is now always an error,
+    because it can't be rendered in CDL
+
+### Fixed
+
+- compiler:
+  + `parseCdl` CSN did not include correct `...` entries for annotations containing `... up to`
+  + Type references inside calculated elements were not always correctly resolved.
+  + `USING` empty files were incorrectly marked as "not found".
+  + If an association was inside `items`, e.g. via type chains, the compiler crashes instead of emitting proper errors.
 - Localized convenience views for projections (not views) did not have references rewritten.
   This only affects CSN, the SQL result was correct.
-- to.edm(x): Render correct EntitySetPath and annotation target path for actions/functions
-  with explicit binding parameter.
+- Calculated elements in composition-of-aspect lost their `value` when generating composition targets.
+- to.sql/to.hdi/for.odata: Foreign keys in ON-conditions were not always properly checked and flattened if explicit
+  `keys` were provided that reference structures.
+- Extending bound actions with elements is not possible, but was not reported by the compiler and elements were silently lost.
+- for.odata:
+  + Don't propagate `@odata { Type, MaxLength, Precision, Scale }` from structured to flattened leaf elements.
+  + Remove `type: null` attribute from element definitions referencing an undefined type via `type of`.
+- to.edm(x):
+  + Don't reject untyped properties that are API typed with a valid `@odata.Type` annotation.
+  + Render correct `EntitySetPath` and annotation target path for actions/functions with explicit binding parameter.
+- to.cdl: ParseCdl-style CSN containing annotations with `...` were not properly rendered.
+
+### Removed
+
+- NodeJs 14 is no longer supported.
+- `CompileMessage` no longer has property `location`, which was deprecated in v2.1.0, but `$location`,
+  which is supported since v2.1.0
+- "Smart type references" such as `Entity.myElement` instead of `Entity:myElement` are removed, because since
+  compiler v2, `Entity.myElement` could also be a definition, creating ambiguities.
+- Element type references can no longer follow associations, i.e. `E:assoc.id` is not allowed.
 
 ## Version 3.9.2 - 2023-04-27
 

package/README.md

@@ -1,6 +1,5 @@
 # Getting started
 
-
 <!-- markdownlint-disable MD001 MD022 -->
 ##### Table of Contents
 <!-- markdownlint-enable MD001 MD022 -->

package/bin/cdsc.js

@@ -30,7 +30,6 @@ const {
   explainMessage, hasMessageExplanation, sortMessages, messageIdsWithExplanation,
 } = require('../lib/base/messages');
 const { term } = require('../lib/utils/term');
-const { splitLines } = require('../lib/utils/file');
 const { addLocalizationViews } = require('../lib/transform/localized');
 const { availableBetaFlags } = require('../lib/base/model');
 const { alterConstraintsWithCsn } = require('../lib/render/manageConstraints');
@@ -505,35 +504,24 @@ function executeCommandLine( command, options, args ) {
         .forEach(msg => log(msg));
     }
     else if (options.noMessageContext) {
-      // TODO: currently, ‹↓› = "downgradable" is only shown here
       messages.filter(msg => (messageLevels[msg.severity] <= options.warning))
         .forEach(msg => log(main.messageString(msg, normalizeFilename, !!options.noMessageId, false, options.testMode && 'compile'))); // TODO: use module name
     }
     else {
-      // Contains file-contents that are split at '\n'. Try to avoid multiple `.split()` calls.
-      const splitCache = Object.create(null);
-      const sourceLines = (name) => {
-        if (!splitCache[name])
-          splitCache[name] = fileCache[name] ? splitLines(fileCache[name]) : fileCache;
-        return splitCache[name];
-      };
       let hasAtLeastOneExplanation = false;
+      const config = {
+        normalizeFilename,
+        noMessageId: !!options.noMessageId,
+        hintExplanation: true,
+        color: options.color,
+        module: options.testMode && 'compile', // TODO: use module name
+        sourceMap: fileCache,
+        cwd: '',
+      };
+
       messages.filter(msg => messageLevels[msg.severity] <= options.warning).forEach((msg) => {
         hasAtLeastOneExplanation = hasAtLeastOneExplanation || main.hasMessageExplanation(msg.messageId);
-        const name = msg.$location && msg.$location.file;
-        const fullFilePath = name ? path.resolve('', name) : undefined;
-        log(main.messageStringMultiline(msg, {
-          normalizeFilename,
-          noMessageId: !!options.noMessageId,
-          withLineSpacer: true,
-          hintExplanation: true,
-          color: options.color,
-        }));
-        if (fullFilePath && msg.$location.line) {
-          // A message context only makes sense, if we have at least start line
-          const context = sourceLines(fullFilePath);
-          log(main.messageContext(context, msg, { color: options.color }));
-        }
+        log(main.messageStringMultiline(msg, config));
         log(); // newline
       });
       if (!options.noMessageId && hasAtLeastOneExplanation)

package/bin/cdsse.js

@@ -4,11 +4,11 @@
 // intend to support all capabilities of a LSP server, nor do we adhere to the
 // LSP protocol.  This is just a little playground to optimize the CDS Compiler
 // support for the CDS LSP server and to detect potential issues with
-// corrupted, incomplete or errorneous CDL sources.
+// corrupted, incomplete or erroneous CDL sources.
 //
 // The output could be used directly by some editors, e.g. Emacs.  The
 // capabilities supported at the moments is: complete - work in progress.
-// Planned are: gotoDefinition, highlight (for syntax highighting).
+// Planned are: gotoDefinition, highlight (for syntax highlighting).
 
 /* eslint no-console:off */
 // @ts-nocheck
@@ -168,7 +168,7 @@ function tokensAt( buf, _offset, col, symbol ) {
 
 function messageAt( model, prop, col ) {
   const msg = (model.messages || model.options && model.options.messages).find(
-    m => m[prop] && m.location.line === line && m.location.col === col && m.location.file === frel
+    m => m[prop] && m.$location.line === line && m.$location.col === col && m.$location.file === frel
   );
   return msg && msg[prop];
 }

package/doc/API.md

@@ -7,5 +7,10 @@ Refer to that file for documentation on option names, API calls, and more.
 
 You can use [TypeDoc] to render an HTML documentation from it.
 
+## Important Notes
+
+The cds-compiler expects that `Object.prototype` is not polluted by enumerable
+properties.  Non-enumerable properties such as custom functions are fine.
+
 
 [TypeDoc]: https://typedoc.org/

package/doc/CHANGELOG_ARCHIVE.md

@@ -206,7 +206,7 @@ The compiler behaviour concerning `beta` features can change at any time without
 ### Fixed
 
 - Fix memory issue: do not keep reference to last-compiled model.
-- Fix dump which occured when trying to report that the user has defined an element to be both `key` and `localized` if
+- Fix dump which occurred when trying to report that the user has defined an element to be both `key` and `localized` if
   `localized` was inherited via the provided type, or in the generated entity for a managed composition of aspect.
 - Properly auto-expose targets of associations in parameters and `many`.
 - for.Odata:

package/doc/CHANGELOG_BETA.md

@@ -8,13 +8,29 @@ Note: `beta` fixes, changes and features are listed in this ChangeLog just for i
 The compiler behavior concerning `beta` features can change at any time without notice.
 **Don't use `beta` fixes, changes and features in productive mode.**
 
+## Version 4.0.0 - 2023-06-06
+
+### Removed `v4preview`
+
+`v4preview` is now the default.
+
+### Removed `calculatedElementsOnWrite`
+
+It is now enabled by default.  The feature is still "beta".
+
+### Removed `ignoreAssocPublishingInUnion`
+
+The behavior is now the default.  If you still want to get an error message for
+ignored published associations in unions, you can change the severity of message
+`query-ignoring-assoc-in-union` to an error via `options.severities`.
+
 ## Version 3.9.2 - 2023-04-27
 
 ### Removed `odataOpenType`
 
 This feature is now set to production mode.
 
-## Version 3.9.0 - 2023-04-XX
+## Version 3.9.0 - 2023-04-20
 
 ### Added `calculatedElementsOnWrite`
 

package/doc/CHANGELOG_DEPRECATED.md

@@ -11,6 +11,34 @@ Note: `deprecated` features are listed in this ChangeLog just for information.
 **When the `deprecated` option is set, the `beta` option is ignored,
 and several new features are not available.**
 
+## Version 4.0.0 - 2023-06-06
+
+### Added `downgradableErrors`
+
+Allow to change the severity of some errors which are meant to stay errors in v4.
+
+### Added `includesNonShadowedFirst`
+
+Use this flag to keep adding elements from included definitions first, example:
+
+```cds
+entity A { one: Integer; two: String(10); three: Integer; }
+entity E : A { two: String(12); }
+// v3:  one, three, two
+// v4:  one, two, three
+```
+
+### Added `ignoreSpecifiedQueryElements`
+
+Use this flag if you want to ignore a query's `elements`, except for annotations and doc comments.
+cds-compiler v3 and earlier simply ignored a query element except for its annotations.
+cds-compiler v4 resolves the element's type.
+
+### Removed `autoCorrectOrderBySourceRefs`
+
+Instead of this deprecated flag, you can downgrade error message `ref-deprecated-orderby`.
+
+
 ## Version 3.1.0 - 2022-08-04
 
 ### Added `autoCorrectOrderBySourceRefs`

package/lib/api/.eslintrc.json

@@ -5,7 +5,7 @@
     "node": true
   },
   // we actually do not extend airbnb-base, as it weakens some eslint:recommended rules
-  "extends": ["../../.eslintrc-ydkjsi.json", "plugin:jsdoc/recommended"],
+  "extends": ["plugin:jsdoc/recommended", "../../.eslintrc-ydkjsi.json"],
   "parserOptions": {
     "ecmaVersion": 2022,
     "sourceType": "script"

package/lib/api/main.js

@@ -164,20 +164,35 @@ function cdl( csn, options = {} ) {
 }
 
 /**
- * Transform a CSN like to.sql
+ * Transform a CSN like to.sql().
+ * Expects that internalOptions have been validated via prepareOptions.to.sql().
  *
  * @param {CSN.Model} csn Plain input CSN
- * @param {SqlOptions} [options={}] Options
+ * @param {SqlOptions} internalOptions Options
  * @returns {CSN.Model} CSN transformed like to.sql
  * @private
  */
-function forSql( csn, options = {} ) {
-  const internalOptions = prepareOptions.to.sql(options);
+function csnForSql( csn, internalOptions ) {
   internalOptions.transformation = 'sql';
   const transformedCsn = forRelationalDB.transformForRelationalDBWithCsn(csn, internalOptions, 'to.sql');
 
   return internalOptions.testMode ? toCsn.sortCsn(transformedCsn, internalOptions) : transformedCsn;
 }
+
+/**
+ * Transform a CSN like to.sql(). Also performs options-checks.
+ * Pseudo-public version of csnForSql().
+ *
+ * @param {CSN.Model} csn Plain input CSN
+ * @param {SqlOptions} [options={}] Options
+ * @returns {CSN.Model} CSN transformed like to.sql
+ * @private
+ */
+function forSql( csn, options = {} ) {
+  const internalOptions = prepareOptions.to.sql(options);
+  return csnForSql(csn, internalOptions);
+}
+
 /**
  * Transform a CSN like to.hdi
  *
@@ -223,7 +238,7 @@ function sql( csn, options = {} ) {
   internalOptions.transformation = 'sql';
 
   // we need the CSN for view sorting
-  const transformedCsn = forSql(csn, options);
+  const transformedCsn = csnForSql(csn, internalOptions);
   const sqls = toSql.toSqlDdl(transformedCsn, internalOptions);
 
   const result = sortViews({ csn: transformedCsn, sql: sqls.sql });
@@ -347,7 +362,8 @@ function sqlMigration( csn, options, beforeImage ) {
   const { error, throwWithError } = messages.makeMessageFunction(csn, internalOptions, 'to.sql.migration');
 
   // Prepare after-image.
-  const afterImage = forSql(csn, internalOptions);
+  const afterImage = internalOptions.filterCsn
+    ? diffFilter.csn(csnForSql(csn, internalOptions)) : csnForSql(csn, internalOptions);
   // Compare both images.
   const diff = modelCompare.compareModels(beforeImage || afterImage, afterImage, internalOptions);
   const diffFilterObj = diffFilter[internalOptions.sqlDialect];
@@ -368,6 +384,7 @@ function sqlMigration( csn, options, beforeImage ) {
     }, {}),
   };
 
+  const markedSkipByUs = {};
   const cleanup = [];
   // Delete artifacts that are already present in csn
   if (beforeImage?.definitions) {
@@ -388,10 +405,38 @@ function sqlMigration( csn, options, beforeImage ) {
       ) { // don't render again, but need info for primary key extension
         diffArtifact['@cds.persistence.skip'] = true;
         cleanup.push(() => delete diffArtifact['@cds.persistence.skip']);
+        markedSkipByUs[artifactName] = true;
       }
     });
   }
 
+  const sortOrder = sortViews({ sql: {}, csn: afterImage });
+
+  const dependentsDict = {};
+  sortOrder.forEach(({ name, dependents }) => {
+    dependentsDict[name] = dependents;
+  });
+
+  const stack = Object.keys(drops.creates);
+  while (stack.length > 0) {
+    const name = stack.pop();
+    const artifact = diff.definitions[name];
+    if (drops.creates[name] === undefined) {
+      if (artifact['@cds.persistence.skip'] && markedSkipByUs[name]) {
+        // Remove the skip so we render a CREATE VIEW
+        diff.definitions[name]['@cds.persistence.skip'] = false;
+        drops.creates[name] = `DROP VIEW ${ identifierUtils.renderArtifactName(name) };`;
+      }
+    }
+
+    const dependents = dependentsDict[name];
+    if (dependents) { // schedule any dependents for processing that don't have a drop-create yet
+      for (const dependantName in dependents) {
+        if (!drops.creates[dependantName])
+          stack.push(dependantName);
+      }
+    }
+  }
   // Convert the diff to SQL.
   if (!internalOptions.beta)
     internalOptions.beta = {};
@@ -403,7 +448,6 @@ function sqlMigration( csn, options, beforeImage ) {
 
   cleanup.forEach(fn => fn());
   // TODO: Handle `ADD CONSTRAINT` etc!
-  const sortOrder = sortViews({ sql: {}, csn: afterImage });
 
   const dropSqls = [];
   const createAndAlterSqls = [];
@@ -844,8 +888,10 @@ function publishCsnProcessor( processor, _name ) {
 
       const { info } = messages.makeMessageFunction( csn, options, 'compile' );
       const msg = info( 'api-recompiled-csn', location.emptyLocation('csn.json'), {}, 'CSN input had to be recompiled' );
-      if (options.internalMsg)
-        msg.error = err; // Attach original error
+      if (options.internalMsg || options.testMode)
+        msg.error = err; // Attach original error;
+      if (options.testMode)  // Attach recompilation reason in testMode
+        msg.message += `\n   ↳ cause: ${ err.message }`;
 
       // next line to be replaced by CSN parser call which reads the CSN object
       const xsn = compiler.recompileX(csn, options);

package/lib/api/options.js

@@ -49,6 +49,8 @@ const publicOptionsNewAPI = [
 
 // Internal options used for testing/debugging etc.
 const privateOptions = [
+  // Not callable via cdsc, keep private for now until we are sure that we want this
+  'filterCsn',
   'lintMode',
   'fuzzyCsnError',
   'traceFs',

package/lib/base/error.js

@@ -7,6 +7,7 @@
 class CompilerAssertion extends Error {
   constructor(message) {
     super(`cds-compiler assertion failed: ${ message }`);
+    this.code = 'ERR_CDS_COMPILER_ASSERTION';
   }
 }
 
@@ -17,6 +18,7 @@ class CompilerAssertion extends Error {
 class ModelError extends Error {
   constructor(message) {
     super(`cds-compiler model error: ${ message }`);
+    this.code = 'ERR_CDS_COMPILER_MODEL';
   }
 }