@sap/cds-compiler 4.7.6 → 4.9.0

package/CHANGELOG.md

@@ -7,6 +7,64 @@
 Note: `beta` fixes, changes and features are usually not listed in this ChangeLog but [here](doc/CHANGELOG_BETA.md).
 The compiler behavior concerning `beta` features can change at any time without notice.
 
+## Version 4.9.0 - 2024-04-25
+
+### Added
+
+- compiler: Annotations with expressions are now rewritten when propagated.
+- for.seal: Added API function that produces a CSN for SEAL.
+- for.odata/to.edm(x): Support annotation path expressions including path flattening.
+
+### Changed
+
+- parser: A space between `.` and `*`/`{` (nested projections) is now a warning.  
+  Use `bin/cds_remove_invalid_whitespace.js` to automatically fix this warning.
+- compiler:
+  + Published compositions with filters are changed to associations.
+  + Expressions as annotation values are rejected for few known annotations that don't expect them.
+- Update OData vocabularies: 'Aggregation', 'Capabilities', 'Common', 'Hierarchy', 'PersonalData', 'Session', 'UI'.
+- to.edm(x): Exposed anonymous parameter types are now prefixed with `ap`, `bap` and `ep` for actions, bound actions and entities.
+
+### Fixed
+
+- compiler:
+  + Deprecated `$parameters` is no longer proposed in code completion.
+  + Duplicate mixin definitions lead to failing name resolution.
+- to.cdl: Types were always rendered for associations with filters, even if it would lead to a compilation failure.
+- to.edm(x):
+  + Fix a recursion bug in entity parameter handling.
+  + Fix event exclusion in service preprocessing.
+
+## Version 4.8.0 - 2024-03-21
+
+### Added
+
+- compiler: Type `cds.Vector` was added.  It is mapped to `REAL_VECTOR` on SAP HANA.
+- Support associations to/from entities with parameters for SAP HANA SQL (hdi/direct).
+- to.sql/to.hdi:
+  + SAP HANA keywords `ABSOLUTE`, `REAL_VECTOR`, and `ST_ASESRIJSON` are now included for smart quoting.
+  +PostgreSQL keyword `SYSTEM_USER` is now included for smart quoting.
+- API: Added `to.sql.postgres.keywords` and `to.sql.h2.keywords`.
+  They contain keywords for the respective SQL dialect.
+
+### Changed
+
+- compiler: Overriding an included element must not change the type to an association
+  if it wasn't an association before and vice versa.
+- Update OData vocabularies: 'Authorization', 'Common', 'UI'.
+
+### Fixed
+
+- compiler: `cast()`s to structured types and associations are now rejected. They could lead to crashes before.
+- to.edm(x):
+  + Reject action/function return types that are declared `many of many`.
+  + Render user defined annotation type `cds.Integer` as `Edm.Int`.
+- to.sql|hdi|hdbcds:
+  + Correctly handle `.list` during flattening.
+  + Improve handling of `.items`.
+- to.sql|hdi.migration:
+  + Turn types and aspects into dummies to reduce CSN size.
+  + Correctly detect a removed `.default` and forcefully set the default to `null`.
 
 ## Version 4.7.6 - 2024-02-29
 
@@ -27,14 +85,15 @@ The compiler behavior concerning `beta` features can change at any time without
 
 ### Fixed
 
-- Restored compatibility with `@sap/cds-dk` for Java runtime
+- Restored compatibility with `@sap/cds-dk` for Java runtime.
+
 
 
 ## Version 4.7.0 - 2024-02-23
 
 ### Added
 
-- compiler: Virtual elements can now be referenced in expressions in annotation
+- compiler: Virtual elements can now be referenced in expressions in annotation.
 
 ### Changed
 
@@ -48,7 +107,8 @@ The compiler behavior concerning `beta` features can change at any time without
 - compiler: published associations with filters sometimes had the filter applied twice
   if used in inline aspect compositions
 - to.sql|hdi|hdbcds[.migration]:
-  + With `withHanaAssociations`: `false`, remove the association elements from the final CSN in order to correctly detect them during migration scenarios and
+  + With `withHanaAssociations`: `false`, remove the association elements from the final CSN in order
+    to correctly detect them during migration scenarios and
   with generated `hdbcds`.
   + Skip expensive processing (for calculated elements and nested projections) if the model doesn't use it.
   + Don't greedily set alias on subqueries if not required.

package/bin/cds_remove_invalid_whitespace.js

@@ -0,0 +1,135 @@
+#!/usr/bin/env node
+
+// Remove whitespace where the compiler does not expect it and will
+// emit an error in the next major version.
+//
+// This script removes whitespace where necessary.
+// It does not remove comments, even though they count as whitespace.
+//
+// Example CDS:
+//    entity P as projection on E { s. * };
+// will become:
+//    entity P as projection on E { s.* };
+//
+// Usage:
+//   ./cds_remove_invalid_whitespace.js my_file.cds
+//
+// If you want to update all files in a directory, you can use
+// this shell script:
+//   find . -type f -iname '*.cds' -exec ./cds_remove_invalid_whitespace.js {} \;
+//
+// Note that you need to update the path to this script in the commands above.
+//
+
+'use strict';
+
+const parseLanguage = require('../lib/language/antlrParser');
+const { createMessageFunctions } = require('../lib/base/messages');
+
+const fs = require('fs');
+const path = require('path');
+
+const cliArgs = process.argv.slice(2);
+const filepath = cliArgs[0];
+
+if (filepath === '--help' || filepath === '-h')
+  exitError();
+
+if (cliArgs.length !== 1)
+  exitError(`Expected exactly one argument, ${cliArgs.length} given`);
+
+if (!filepath)
+  exitError('Expected non-empty filepath as argument!');
+
+// Do not use allow-list approach.
+// There may be CDS files with other extensions than `.cds`.
+if (filepath.endsWith('.csn') || filepath.endsWith('.json'))
+  exitError('Only CDS files can be passed! Found CSN file!');
+
+const sourceStr = fs.readFileSync(filepath, 'utf-8');
+const newSourceStr = modernizeWhitespace(sourceStr, filepath);
+if (newSourceStr !== sourceStr)
+  fs.writeFileSync(filepath, newSourceStr);
+process.exit(0); // success
+
+// --------------------------------------------------------
+
+function modernizeWhitespace( source, filename ) {
+  const options = { messages: [], attachTokens: true };
+  const messageFunctions = createMessageFunctions( options, 'parse', null );
+
+  // parseLanguage does not throw on CompilationError, so
+  // we do not need a try...catch block.
+  const ast = parseLanguage(source, filename, options, messageFunctions);
+
+  // To avoid spam, only report errors.
+  // Users should use the compiler to get all messages.
+  const errors = options.messages.filter(msg => (msg.severity === 'Error'));
+  if (errors.length > 0) {
+    errors.forEach((msg) => {
+      console.error(msg.toString());
+    });
+    console.error(`Found ${errors.length} errors! \n`);
+    exitError('The CDS parser emitted errors. Fix them first and try again.');
+  }
+
+  if (!ast.artifacts)
+    return source;
+
+  let currentOffset = 0;
+
+  const { tokens } = ast.tokenStream;
+  for (let i = 0; i < tokens.length; ++i) {
+    const token = tokens[i];
+    const next = tokens[i + 1];
+    if (next && token.type === ast.tokenStream.DOTbeforeBRACE &&
+      (next.type === ast.tokenStream.ASTERISK || next.type === ast.tokenStream.BRACE))
+      updateWhitespace(token, next);
+  }
+
+  return source;
+
+  // -----------------------------------------------
+
+  function updateWhitespace( dot, braceOrAsterisk ) {
+    if (dot.stop === braceOrAsterisk.start)
+      return;
+
+    const from = dot.stop + currentOffset + 1; // end points at the position *before* the character
+    const to = braceOrAsterisk.start + currentOffset;
+
+    source = replaceSliceInSource(source, from, to, '');
+
+    currentOffset -= to - from;
+  }
+}
+
+/**
+ * Replaces a given span with @p replaceWith
+ *
+ * @param {string} source
+ * @param {number} startIndex
+ * @param {number} endIndex
+ * @param {string} replaceWith
+ * @return {string}
+ */
+function replaceSliceInSource( source, startIndex, endIndex, replaceWith ) {
+  return source.substring(0, startIndex) +
+          replaceWith +
+          source.substring(endIndex);
+}
+
+/**
+ * @param {string} [msg]
+ */
+function exitError( msg ) {
+  if (msg)
+    console.error(msg);
+  usage();
+  process.exit(1);
+}
+
+function usage() {
+  console.error('');
+  console.error(`usage: ${path.basename(process.argv[1])} <filename>`);
+}

package/bin/cds_update_annotations.js

@@ -0,0 +1,180 @@
+#!/usr/bin/env node
+
+// Update CDS Annotation References to use v4 expression syntax.
+//
+// This script replaces bare, unchecked references in annotation assignments
+// with the new syntax using `(`.
+//
+// Example CDS:
+//    @anno: ID entity E { key ID : String; };
+// will become:
+//    @anno: (ID) entity E { key ID : String; };
+//
+// Usage:
+//   ./cds_update_annotations.js my_file.cds
+//
+// If you want to update all identifiers in a directory, you can use
+// this Shell script:
+//   find . -type f -iname '*.cds' -exec ./cds_update_annotations.js {} \;
+//
+// Note that you need to update the path to this script in the commands above.
+//
+
+'use strict';
+
+const parseLanguage = require('../lib/language/antlrParser');
+const { createMessageFunctions } = require('../lib/base/messages');
+
+const fs = require('fs');
+const path = require('path');
+const { forEachMemberRecursively } = require('../lib/base/model');
+
+const cliArgs = process.argv.slice(2);
+const filepath = cliArgs[0];
+
+if (filepath === '--help' || filepath === '-h')
+  exitError();
+
+if (cliArgs.length !== 1)
+  exitError(`Expected exactly one argument, ${cliArgs.length} given`);
+
+if (!filepath)
+  exitError('Expected non-empty filepath as argument!');
+
+// Do not use allow-list approach.
+// There may be CDS files with other extensions than `.cds`.
+if (filepath.endsWith('.csn') || filepath.endsWith('.json'))
+  exitError('Only CDS files can be passed! Found CSN file!');
+
+const sourceStr = fs.readFileSync(filepath, 'utf-8');
+const newSourceStr = modernizeAnnotationExpressions(sourceStr, filepath);
+if (newSourceStr !== sourceStr)
+  fs.writeFileSync(filepath, newSourceStr);
+process.exit(0); // success
+
+// --------------------------------------------------------
+
+function modernizeAnnotationExpressions( source, filename ) {
+  const options = { messages: [], attachTokens: true };
+  const messageFunctions = createMessageFunctions( options, 'parse', null );
+
+  // parseLanguage does not throw on CompilationError, so
+  // we do not need a try...catch block.
+  const ast = parseLanguage(source, filename, options, messageFunctions);
+
+  // To avoid spam, only report errors.
+  // Users should use the compiler to get all messages.
+  const errors = options.messages.filter(msg => (msg.severity === 'Error'));
+  if (errors.length > 0) {
+    errors.forEach((msg) => {
+      console.error(msg.toString());
+    });
+    console.error(`Found ${errors.length} errors! \n`);
+    exitError('The parser emitted errors. Please fix them first and try again.');
+  }
+
+  if (!ast.artifacts)
+    return source;
+
+  let currentOffset = 0;
+
+  for (const artName in ast.artifacts) {
+    const art = ast.artifacts[artName];
+    updateAnnosOfArtifact(art);
+    forEachMemberRecursively(art, updateAnnosOfArtifact);
+  }
+  for (const vocName in ast.vocabularies) {
+    const voc = ast.vocabularies[vocName];
+    updateAnnosOfArtifact(voc);
+    forEachMemberRecursively(voc, updateAnnosOfArtifact);
+  }
+  for (const ext of ast.extensions)
+    updateAnnosOfArtifact(ext);
+
+  return source;
+
+  // -----------------------------------------------
+
+  function updateAnnosOfArtifact( art ) {
+    for (const prop in art) {
+      if (prop.charAt(0) === '@')
+        updateAnno(art[prop]);
+    }
+  }
+
+  function updateAnno( anno ) {
+    if (anno.literal === 'array') {
+      anno.val.forEach(val => updateAnno( val ));
+    }
+    else if (anno.literal === 'struct') {
+      const struct = Object.values(anno.struct);
+      struct.forEach(val => updateAnno( val ));
+    }
+    else if (!anno.$tokenTexts && anno.path) {
+      const first = anno.path[0];
+      const last = anno.path[anno.path.length - 1];
+      insertParentheses(first, last);
+    }
+  }
+
+  function findTokenFor( node ) {
+    const loc = node.location;
+    // We use slow linear search; we could do binary search, but not performance critical.
+    const { tokens } = ast.tokenStream;
+    for (const token of tokens) {
+      if (token.column + 1 === loc.col && token.line === loc.line)
+        return token;
+    }
+    return null;
+  }
+
+  function insertParentheses( first, last ) {
+    const firstToken = findTokenFor(first);
+    const lastToken = findTokenFor(last);
+
+    if (!firstToken?.stop || !lastToken?.stop)
+      throw new Error(`INTERNAL ERROR: Identifier at ${firstToken?.start} or ${lastToken?.start} has no end! (${filepath})`);
+
+    const start = firstToken.start + currentOffset;
+    const end = lastToken.stop + currentOffset + 1; // end points at the position *before* the character
+
+    const original = source.substring(start, end);
+    if (original.includes('@'))
+      return; // we ignore annotation references for now
+
+    const newSource = `( ${original} )`;
+    source = replaceSliceInSource(source, start, end, newSource);
+
+    currentOffset += (newSource.length - original.length);
+  }
+}
+
+/**
+ * Replaces a given span with @p replaceWith
+ *
+ * @param {string} source
+ * @param {number} startIndex
+ * @param {number} endIndex
+ * @param {string} replaceWith
+ * @return {string}
+ */
+function replaceSliceInSource( source, startIndex, endIndex, replaceWith ) {
+  return source.substring(0, startIndex) +
+          replaceWith +
+          source.substring(endIndex);
+}
+
+/**
+ * @param {string} [msg]
+ */
+function exitError( msg ) {
+  if (msg)
+    console.error(msg);
+  usage();
+  process.exit(1);
+}
+
+function usage() {
+  console.error('');
+  console.error(`usage: ${path.basename(process.argv[1])} <filename>`);
+}

package/bin/cds_update_identifiers.js

@@ -1,6 +1,5 @@
 #!/usr/bin/env node
 
-//
 // Update CDS Delimited Identifiers
 //
 // This script replaces the old delimited identifier style with the new one
@@ -12,11 +11,11 @@
 //    entity ![My Entity] { ... }
 //
 // Usage:
-//   cds_update_identifiers.js my_file.cds
+//   ./cds_update_identifiers.js my_file.cds
 //
 // If you want to update all identifiers in a directory, you can use
 // this Shell script:
-//   find . -type f -iname '*.cds' -exec cds_update_identifiers.js {} \;
+//   find . -type f -iname '*.cds' -exec ./cds_update_identifiers.js {} \;
 //
 // Note that you need to update the path to this script in the commands above.
 //
@@ -70,7 +69,7 @@ function modernizeIdentifierStyle( source, filename ) {
       console.error(msg.toString());
     });
     console.error(`Found ${errors.length} errors! \n`);
-    exitError('The parser emitted errors. Please fix them first and try again.');
+    exitError('The CDS parser emitted errors. Fix them first and try again.');
   }
 
   let currentOffset = 0;

package/bin/cdsc.js

@@ -217,7 +217,7 @@ function cdsc_main() {
  * @param {object} args
  */
 function validateDirectBackendOption( command, options, args ) {
-  if (![ 'toCdl', 'toOdata', 'toHana', 'toCsn', 'toSql' ].includes(command)) {
+  if (![ 'toCdl', 'toOdata', 'toHana', 'toCsn', 'toSql', 'forJava' ].includes(command)) {
     displayUsage(`Option '--direct-backend' can't be used with command '${command}'`,
                  optionProcessor.helpText, 2);
   }
@@ -272,11 +272,13 @@ function executeCommandLine( command, options, args ) {
     toCsn,
     toHana,
     toOdata,
+    forJava,
     toRename,
     manageConstraints,
     toSql,
     inspect,
     toEffectiveCsn,
+    forSeal,
   };
   const commandsWithoutCompilation = {
     explain,
@@ -317,12 +319,29 @@ function executeCommandLine( command, options, args ) {
   }
 
   function toEffectiveCsn( model ) {
+    const features = [ 'resolveSimpleTypes', 'resolveProjections', 'remapOdataAnnotations', 'keepLocalized' ];
+    for (const feature of features) {
+      if (options[feature]) // map to boolean equivalent
+        options[feature] = options[feature] === 'true';
+    }
     const csn = options.directBackend ? model : compactModel(model, options);
     displayNamedCsn(main.for.effective(csn, options), 'effective');
 
     return model;
   }
 
+  function forSeal( model ) {
+    const features = [ 'remapOdataAnnotations' ];
+    for (const feature of features) {
+      if (options[feature]) // map to boolean equivalent
+        options[feature] = options[feature] === 'true';
+    }
+    const csn = options.directBackend ? model : compactModel(model, options);
+    displayNamedCsn(main.for.seal(csn, options), 'seal');
+
+    return model;
+  }
+
   // Execute the command line option 'toCsn' and display the results.
   // Return the original model (for chaining)
   function toCsn( model ) {
@@ -336,6 +355,14 @@ function executeCommandLine( command, options, args ) {
     return model;
   }
 
+  // Execute the command line command 'forJava' and display the results.
+  // Return the original model
+  function forJava( model ) {
+    const csn = options.directBackend ? model : compactModel(model, options);
+    displayNamedCsn( main.for.java( csn, options ), 'java');
+    return model;
+  }
+
   // Execute the command line option '--to-hana' and display the results.
   // Return the original model (for chaining)
   function toHana( model ) {

package/bin/cdshi.js

@@ -15,16 +15,26 @@ const compiler = require('../lib/compiler');
 const fs = require('fs');
 fs.readFile( '/dev/stdin', 'utf8', highlight );
 
-const categoryChars = {
+const categoryChars = {         // default: first char of category name
+  // first char lowercase = reference other than via extend/annotate:
   artref: 'm',
+  uncheckedRef: 'r',
+  uncheckedAnno: 'a',
   paramname: 'b',
+  typeparamname: 'b',
+  // first char uppercase = definition, extend/annotate ref, or path after `namespace`:
   Entity: 'D',
   Enum: 'H',
   Index: 'J',
   AnnoDef: 'V',
-  Extend: 'Z',
-  Annotate: 'Z',
+  Ext: 'Z',                 // extend/annotate on main def other than service/context
+  ExtService: 'S',          // highlight like service definition
+  ExtContext: 'C',          // highlight like context definition
+  // ExtElement: 'E',       // using the first letter is the default
+  ExtBoundAction: 'B',      // highlight like bound action definition
+  ExtParam: 'P',            // highlight like entity/action parameter definition
   Event: 'Y',
+  // Remark: do not use `x`/`X` (hex literal `x'1e3d'`)
 };
 
 function highlight( err, buf ) {

package/doc/CHANGELOG_BETA.md

@@ -8,7 +8,30 @@ Note: `beta` fixes, changes and features are listed in this ChangeLog just for i
 The compiler behavior concerning `beta` features can change at any time without notice.
 **Don't use `beta` fixes, changes and features in productive mode.**
 
-## Version 4.x.x - 2024-mm-dd
+## Version 4.9.0 - 2024-04-25
+
+## Removed `odataAnnotationExpressions`
+
+It is now enabled by default.
+
+## Removed `odataPathsInAnnotationExpressions`
+
+It is now enabled by default.
+
+## Removed `annotationExpressions`
+
+It is now enabled by default.
+
+## Added `temporalRawProjection`
+
+Enables revocation of temporal where clause in projections of temporal entities if the
+`@cds.valid { from, to }` annotations on the projection elements have falsy values.
+
+## Version 4.8.0 - 2024-03-21
+
+### Removed `vectorType`
+
+It is now always enabled.
 
 ### Added `v5preview`