@sap/cds-compiler 2.13.6 → 2.15.4

package/CHANGELOG.md

@@ -7,21 +7,145 @@
 Note: `beta` fixes, changes and features are usually not listed in this ChangeLog but [here](doc/CHANGELOG_BETA.md).
 The compiler behavior concerning `beta` features can change at any time without notice.
 
-## Version 2.13.6 - 2022-25-03
+## Version 2.15.4 - 2022-06-09
+
+### Fixed
+
+- for.odata:
+  + Fix derived type to scalar type resolution with intermediate `many`.
+- to.edm(x):
+  + (V4 structured) Fix key paths in combination with `--odata-foreign-keys`.
+  + Add `Edm.PrimitiveType` to `@odata.Type`.
+  + (V4 JSON) Render constant expressions for `Edm.Stream` and `Edm.Untyped`.
+  + Fix a bug in target path calculation for `NavigationPropertyBinding`s to external references.
+  + Render inner annotations even if `$value` is missing.
+- Update OData vocabularies 'Common', 'UI'.
+
+## Version 2.15.2 - 2022-05-12
+
+### Fixed
+
+- Option `cdsHome` can be used instead of `global.cds.home` to specify the path to `@sap/cds/`.
+- to.edm(x):
+  + Set anonymous nested proxy key elements to `Nullable:false` until first named type is reached.
+  + Enforce `odata-spec-violation-key-null` on explicit foreign keys of managed primary key associations.
+  + Proxies/service cross references are no longer created for associations with arbitrary ON conditions.
+    Only managed or `$self` backlink association targets are proxy/service cross reference candidates.
+  + Explicit foreign keys of a managed association that are not a primary key in the target are exposed in the the proxy.
+  + If an association is primary key, the resulting navigation property is set to `Nullable:false` in structured mode.
+
+## Version 2.15.0 - 2022-05-06
+
+### Added
+
+- A new warning is emitted if `excluding` is used without a wildcard, since this does
+  not have any effect.
+- All scalar types can now take named arguments, e.g. `MyString(length: 10)`.
+  For custom scalar types, one unnamed arguments is interpreted as length, two arguments are interpreted
+  as precision and scale, e.g. `MyDecimal(3,3)`.
+- If the type `sap.common.Locale` exists, it will be used as type for the `locale` element
+  of generated texts entities.  The type must be a `cds.String`.
+- to.cdl: Extend statements (from `extensions`) can now be rendered.
+- Add OData vocabulary 'Hierarchy'.
+- CDL: New associations can be published in queries, e.g. `assoc : Association to Target on assoc.id = id`
+
+### Changed
+
+- to.edm(x):
+  + perform inbound qualification and spec violation checks as well as most/feasible EDM preprocessing steps
+    on requested services only.
+  + Open up `@odata { Type, MaxLength, Precision, Scale, SRID }` annotation.  
+    The annotations behavior is defined as follows:
+    + The element/parameter must have a scalar CDS type. The annotation is not applied on named types
+      (With the V2 exception where derived type chains terminating in a scalar type are resolved).
+    + The value of `@odata.Type` must be a valid `EDM` type for the rendered protocol version.
+    + If `@odata.Type` can be applied, all canonic type facets (`MaxLength`, `Precision`, `Scale`, `SRID`) are
+      removed from the Edm Node and the new facets `@odata { MaxLength, Precision, Scale, SRID }` are applied.
+      Non Edm type conformant facets are ignored (eg. `@odata { Type: 'Edm.Decimal', MaxLength: 10, SRID: 0 }`).
+    + Type facet values are not evaluated.
+  + V2: Propagate `@Core.MediaType` annotation from stream element to entity type if not set.
+- to.edm: Render constant expressions in short notation.
+- Update OData Vocabularies: 'Common', 'Graph', 'Validation'.
+
+### Fixed
+
+- to.cdl:
+  + Annotations of elements of action `returns` are now rendered as `annotate` statements.
+  + Annotations on columns (query sub-elements) were not always rendered.
+  + Doc comments on bound actions were rendered twice.
+  + Unapplied annotations for action parameters were not rendered.
+  + Unions and joins are correctly put into parentheses.
+  + Add parentheses around certain expressions in function bodies that require it, such as `fct((1=1))`.
+- to.edm(x):
+  + Fix a bug in top level and derived type `items` exposure leading to undefined type rendering.
+  + Fix a naming bug in type exposure leading to false reuse types, disguising invididual type
+    modifications (such as annotations, (auto-)redirections, element extensions).
+  + Ignore `@Aggregation.default`.
+  + Consolidate message texts and formatting.
+  + Fix navigation property binding in cross service rendering mode.
+  + Remove partner attribute in proxy/cross service navigations.
+- Core engine (function `compile`):
+  + Annotations for new columns inside `extend projection` blocks were not used.
+  + Extending an unknown select item resulted in a crash.
+  + Extending a context/service with columns now correctly emits an error.
+  + Unmanaged `redirected to` in queries did not check whether the source is an association.
+- parseCdl: `extend <art> with enum {...}` incorrectly threw a compiler error.
+- API: `compile()` used a synchronous call `fs.realpathSync()` on the input filename array.  
+  Now the asynchronous `fs.realpath()` is used.
+- On-conditions in localized convenience views may be incorrectly rewritten if an element
+  has the same as a localized entity.
+- to.sql/hdi/hdbcds:
+  + No referential constraint is generated for an association if its parent
+  or target entity are annotated with `@cds.persistence.exists: true`.
+  + Fix rendering of virtual elements in subqueries
+  + Correctly process subqueries in JOINs
+- to.sql/hdi: Queries with `UNION`, `INTERSECT` and similar in expressions are now enclosed in parentheses.
+
+## Version 2.14.0 - 2022-04-08
+
+### Added
+
+- cdsc:
+  + `--quiet` can now be used to suppress compiler output, including messages.
+  + `--options <file.json>` can be used to load compiler options. A JSON file is expected. Is compatible to CDS `package.json`
+    and `.cdsrc.json` by first looking for `cdsc` key in `cds`, then for a `cdsc` key and otherwise uses the full JSON file.
+  + `--[error|warn|info|debug] id1,id2` can be used to reclassify specific messages.
+- Add OData Vocabularies: 'DataIntegration', 'JSON'.
+  
+### Changed
+
+- Update OData Vocabularies: 'UI'.
+
+### Fixed
+
+- to.cdl:
+  + Delimited identifiers as the last elements of arrays in annotation values are now
+    rendered with spaces in between, to avoid accidentally escaping `]`.
+  + Identifiers in includes and redirection targets were not quoted if they are reserved keywords.
+- to.edm(x): Correctly rewrite `@Capabilities.ReadRestrictions.ReadByKeyRestrictions` into
+  `@Capabilities.NavigationPropertyRestriction` in containment mode.
+
+## Version 2.13.8 - 2022-03-29
+
+### Fixed
+
+- to.hdbcds/hdi/sql: Correctly handle `localized` in conjunction with `@cds.persistence.exists` and `@cds.persistence.skip`
+
+## Version 2.13.6 - 2022-03-25
 
 ### Fixed
 
 - to.hdbcds/hdi/sql: Correctly handle `localized` in conjunction with `@cds.persistence.exists`
 
-## Version 2.13.4 - 2022-22-03
+## Version 2.13.4 - 2022-03-22
 
 No changes compared to Version 2.13.0; fixes latest NPM tag
 
-## Version 2.13.2 - 2022-22-03
+## Version 2.13.2 - 2022-03-22
 
 No changes compared to Version 2.13.0; fixes latest NPM tag
 
-## Version 2.13.0 - 2022-22-03
+## Version 2.13.0 - 2022-03-22
 
 ### Added
 

package/bin/cdsc.js

@@ -44,6 +44,18 @@ class ProcessExitError extends Error {
   }
 }
 
+try {
+  cdsc_main();
+}
+catch (err) {
+  // This whole try/catch is only here because process.exit does not work in combination with
+  // stdout/err - see comment at ProcessExitError
+  if (err instanceof ProcessExitError)
+    process.exitCode = err.exitCode;
+  else
+    throw err;
+}
+
 function remapCmdOptions(options, cmdOptions) {
   if (!cmdOptions)
     return options;
@@ -73,6 +85,9 @@ function remapCmdOptions(options, cmdOptions) {
           options.variableReplacements.$user = {};
         options.variableReplacements.$user.locale = value;
         break;
+      case 'serviceNames':
+        options.serviceNames = value.split(',');
+        break;
       default:
         options[key] = value;
     }
@@ -80,8 +95,9 @@ function remapCmdOptions(options, cmdOptions) {
   return options;
 }
 
-// Parse the command line and translate it into options
-try {
+function cdsc_main() {
+  // Parse the command line and translate it into options
+
   const cmdLine = optionProcessor.processCmdLine(process.argv);
   // Deal with '--version' explicitly
   if (cmdLine.options.version) {
@@ -115,6 +131,11 @@ try {
     displayUsage(cmdLine.errors, optionProcessor.helpText, 2);
   }
 
+  if (cmdLine.options.options) {
+    if (!loadOptionsFromJson(cmdLine))
+      return;
+  }
+
   // Default warning level is 2 (info)
   // FIXME: Is that not set anywhere in the API?
   if (!cmdLine.options.warning)
@@ -125,11 +146,7 @@ try {
     cmdLine.options.out = '-';
 
   // --cds-home <dir>: modules starting with '@sap/cds/' are searched in <dir>
-  if (cmdLine.options.cdsHome) {
-    if (!global.cds)
-      global.cds = {};
-    global.cds.home = cmdLine.options.cdsHome;
-  }
+  // -> cmdLine.options.cdsHome is passed down to moduleResolve
 
   // Set default command if required
   cmdLine.command = cmdLine.command || 'toCsn';
@@ -147,8 +164,8 @@ try {
   if (cmdLine.options.directBackend)
     validateDirectBackendOption(cmdLine.command, cmdLine.options, cmdLine.args);
 
-
-  if (cmdLine.options.beta) {
+  // If set through CLI (and not options file), `beta` is a string and needs processing.
+  if (cmdLine.options.beta && typeof cmdLine.options.beta === 'string') {
     const features = cmdLine.options.beta.split(',');
     cmdLine.options.beta = {};
     features.forEach((val) => {
@@ -156,35 +173,32 @@ try {
     });
   }
 
+  const to = cmdLine.options.toSql ? 'toSql' : 'toHana';
   // remap string values for `assertIntegrity` option to boolean
-  if (cmdLine.options.assertIntegrity &&
-      cmdLine.options.assertIntegrity === 'true' ||
-      cmdLine.options.assertIntegrity === 'false'
+  if (cmdLine.options[to] && cmdLine.options[to].assertIntegrity &&
+     (cmdLine.options[to].assertIntegrity === 'true' ||
+      cmdLine.options[to].assertIntegrity === 'false')
   )
-    cmdLine.options.assertIntegrity = cmdLine.options.assertIntegrity === 'true';
+    cmdLine.options[to].assertIntegrity = cmdLine.options[to].assertIntegrity === 'true';
 
   // Enable all beta-flags if betaMode is set to true
   if (cmdLine.options.betaMode)
     cmdLine.options.beta = availableBetaFlags;
 
-  if (cmdLine.options.deprecated) {
+  // If set through CLI (and not options file), `deprecated` is a string and needs processing.
+  if (cmdLine.options.deprecated && typeof cmdLine.options.deprecated === 'string') {
     const features = cmdLine.options.deprecated.split(',');
     cmdLine.options.deprecated = {};
     features.forEach((val) => {
       cmdLine.options.deprecated[val] = true;
     });
   }
+
+  parseSeverityOptions(cmdLine);
+
   // Do the work for the selected command
   executeCommandLine(cmdLine.command, cmdLine.options, cmdLine.args);
 }
-catch (err) {
-  // This whole try/catch is only here because process.exit does not work in combination with
-  // stdout/err - see comment at ProcessExitError
-  if (err instanceof ProcessExitError)
-    process.exitCode = err.exitCode;
-  else
-    throw err;
-}
 
 /**
  * `--direct-backend` can only be used with certain backends and with certain files.
@@ -230,10 +244,10 @@ function displayUsage(error, helpText, code) {
 function executeCommandLine(command, options, args) {
   const normalizeFilename = options.testMode && process.platform === 'win32';
   const messageLevels = {
-    Error: 0, Warning: 1, Info: 2, None: 3,
+    Error: 0, Warning: 1, Info: 2, Debug: 3,
   };
   // All messages are put into the message array, even those which should not
-  // been displayed (severity 'None')
+  // been displayed (severity 'Debug')
 
   // Create output directory if necessary
   if (options.out && options.out !== '-' && !fs.existsSync(options.out))
@@ -329,17 +343,18 @@ function executeCommandLine(command, options, args) {
       options.toOdata.odataContainment = true;
     }
     const csn = options.directBackend ? model : compactModel(model, options);
-    const odataCsn = main.for.odata(csn, remapCmdOptions(options, options.toOdata));
+    const odataOptions = remapCmdOptions(options, options.toOdata);
     if (options.toOdata && options.toOdata.csn) {
+      const odataCsn = main.for.odata(csn, odataOptions);
       displayNamedCsn(odataCsn, 'odata_csn');
     }
     else if (options.toOdata && options.toOdata.json) {
-      const result = main.to.edm.all(odataCsn, options);
+      const result = main.to.edm.all(csn, odataOptions);
       for (const serviceName in result)
         writeToFileOrDisplay(options.out, `${serviceName}.json`, result[serviceName]);
     }
     else {
-      const result = main.to.edmx.all(odataCsn, options);
+      const result = main.to.edmx.all(csn, odataOptions);
       for (const serviceName in result)
         writeToFileOrDisplay(options.out, `${serviceName}.xml`, result[serviceName]);
     }
@@ -447,7 +462,7 @@ function executeCommandLine(command, options, args) {
    * @param {CompileMessage[]} messages
    */
   function displayMessages( model, messages = options.messages ) {
-    if (!Array.isArray(messages))
+    if (!Array.isArray(messages) || options.quiet)
       return model;
 
     const log = console.error;
@@ -552,6 +567,8 @@ function executeCommandLine(command, options, args) {
       content = JSON.stringify(content, null, 2);
 
     if (dir === '-') {
+      if (options.quiet)
+        return;
       if (!omitHeadline) {
         const sqlTypes = {
           sql: true, hdbconstraint: true, hdbtable: true, hdbview: true,
@@ -569,15 +586,73 @@ function executeCommandLine(command, options, args) {
       fs.writeFileSync(path.join(dir, fileName), content);
     }
   }
+}
 
-  function catchErrors(err) {
-    // @ts-ignore
-    if (err instanceof Error && err.hasBeenReported)
-      return;
-    console.error( '' );
-    console.error( 'INTERNAL ERROR:' );
-    console.error( util.inspect(err, false, null) );
-    console.error( '' );
-    process.exitCode = 70;
+function loadOptionsFromJson(cmdLine) {
+  try {
+    let opt = JSON.parse(fs.readFileSync(cmdLine.options.options, 'utf-8'));
+    if (opt.cds)
+      opt = opt.cds;
+    if (opt.cdsc)
+      opt = opt.cdsc;
+    Object.assign(cmdLine.options, opt);
+    return true;
+  }
+  catch (e) {
+    catchErrors(e);
+    return false;
+  }
+}
+
+function catchErrors(err) {
+  // @ts-ignore
+  if (err instanceof Error && err.hasBeenReported)
+    return;
+  console.error( '' );
+  console.error( 'INTERNAL ERROR:' );
+  console.error( util.inspect(err, false, null) );
+  console.error( '' );
+  process.exitCode = 70;
+}
+
+/**
+ * Parses the options `--error` and similar.
+ * Sets the dictionary `severities` on the given options.
+ *
+ * @param {object} options
+ */
+function parseSeverityOptions({ options }) {
+  if (!options.severities)
+    options.severities = Object.create(null);
+
+  const severityMap = {
+    error: 'Error',
+    warn: 'Warning',
+    info: 'Info',
+    debug: 'Debug',
+  };
+
+  // Note: We use a for loop to ensure that the order of the options on the command line is respected, i.e.
+  //      `--warn id --error id` would lead to `id` being reclassified as an error and not a warning.
+  for (const key in options) {
+    switch (key) {
+      case 'error':
+      case 'warn':
+      case 'info':
+      case 'debug':
+        parseSeverityOption(options[key], severityMap[key]);
+        break;
+      default:
+        break;
+    }
+  }
+
+  function parseSeverityOption(list, severity) {
+    const ids = list.split(',');
+    for (let id of ids) {
+      id = id.trim();
+      if (id)
+        options.severities[id] = severity;
+    }
   }
 }

package/lib/api/main.js

@@ -33,9 +33,10 @@ const propertyToCheck = {
   odata: 'toOdata',
 };
 
-const { cloneCsn } = require('../model/csnUtils');
+const { cloneCsnNonDict } = require('../model/csnUtils');
 const { toHdbcdsSource } = require('../render/toHdbcds');
 const { ModelError } = require('../base/error');
+const { forEach } = require('../utils/objectUtils');
 
 const relevantGeneralOptions = [ /* for future generic options */ ];
 const relevantOdataOptions = [ 'sqlMapping', 'odataFormat' ];
@@ -91,7 +92,7 @@ function checkPreTransformedCsn(csn, options, relevantOptionNames, warnAboutMism
     // Not able to check
     return;
   }
-  const { error, warning, throwWithError } = makeMessageFunction(csn, options, module);
+  const { error, warning, throwWithAnyError } = makeMessageFunction(csn, options, module);
 
   for (const name of relevantOptionNames ) {
     if (options[name] !== csn.meta.options[name])
@@ -103,7 +104,7 @@ function checkPreTransformedCsn(csn, options, relevantOptionNames, warnAboutMism
       warning('options-mismatch-pretransformed-csn', null, `Expected pre-processed CSN to have option "${ name }" set to "${ options[name] }". Found: "${ csn.meta.options[name] }"`);
   }
 
-  throwWithError();
+  throwWithAnyError();
 }
 
 /**
@@ -153,7 +154,7 @@ function odata(csn, options = {}) {
  */
 function cdl(csn, externalOptions = {}) {
   const internalOptions = prepareOptions.to.cdl(externalOptions);
-  const { result } = backends.toCdlWithCsn(cloneCsn(csn, internalOptions), internalOptions);
+  const { result } = backends.toCdlWithCsn(cloneCsnNonDict(csn, internalOptions), internalOptions);
   return result;
 }
 /**
@@ -251,7 +252,7 @@ function hdi(csn, options = {}) {
     const sqlArtifactsWithCSNNamesToSort = Object.create(null);
     const sqlArtifactsNotToSort = Object.create(null);
 
-    Object.keys(flat).forEach((key) => {
+    forEach(flat, (key) => {
       const artifactNameLikeInCsn = key.replace(/\.[^/.]+$/, '');
       nameMapping[artifactNameLikeInCsn] = key;
       if (key.endsWith('.hdbtable') || key.endsWith('.hdbview'))
@@ -269,7 +270,7 @@ function hdi(csn, options = {}) {
       }, Object.create(null));
 
     // now add the not-sorted stuff, like indices
-    Object.keys(sqlArtifactsNotToSort).forEach((key) => {
+    forEach(sqlArtifactsNotToSort, (key) => {
       sorted[remapName(key, sqlCSN, k => !k.endsWith('.hdbindex'))] = sqlArtifactsNotToSort[key];
     });
 
@@ -291,10 +292,10 @@ function hdi(csn, options = {}) {
 function remapNames(dict, csn, filter) {
   const result = Object.create(null);
 
-  for (const [ key, value ] of Object.entries(dict)) {
+  forEach(dict, (key, value) => {
     const name = remapName(key, csn, filter);
     result[name] = value;
-  }
+  });
 
   return result;
 }
@@ -396,15 +397,15 @@ function hdiMigration(csn, options, beforeImage) {
    */
   function createDefinitions() {
     const result = [];
-    for (const [ kind, artifacts ] of Object.entries(hdbkinds)) {
+    forEach(hdbkinds, (kind, artifacts) => {
       const suffix = `.${ kind }`;
-      for (const [ name, sqlStatement ] of Object.entries(artifacts)) {
+      forEach(artifacts, (name, sqlStatement) => {
         if ( kind !== 'hdbindex' )
           result.push({ name: getFileName(name, afterImage), suffix, sql: sqlStatement });
         else
           result.push({ name, suffix, sql: sqlStatement });
-      }
-    }
+      });
+    });
     return result;
   }
   /**
@@ -414,9 +415,7 @@ function hdiMigration(csn, options, beforeImage) {
    */
   function createDeletions() {
     const result = [];
-    for (const [ name ] of Object.entries(deletions))
-      result.push({ name: getFileName(name, beforeImage), suffix: '.hdbtable' });
-
+    forEach(deletions, name => result.push({ name: getFileName(name, beforeImage), suffix: '.hdbtable' }));
     return result;
   }
   /**
@@ -426,9 +425,7 @@ function hdiMigration(csn, options, beforeImage) {
    */
   function createMigrations() {
     const result = [];
-    for (const [ name, changeset ] of Object.entries(migrations))
-      result.push({ name: getFileName(name, afterImage), suffix: '.hdbmigrationtable', changeset });
-
+    forEach(migrations, (name, changeset) => result.push({ name: getFileName(name, afterImage), suffix: '.hdbmigrationtable', changeset }));
     return result;
   }
 }
@@ -586,12 +583,13 @@ function edmxall(csn, options = {}) {
  */
 function flattenResultStructure(toProcess) {
   const result = {};
-  for (const [ fileType, artifacts ] of Object.entries(toProcess)) {
+  forEach(toProcess, (fileType, artifacts) => {
     if (fileType === 'messages')
-      continue;
-    for (const filename of Object.keys(artifacts))
+      return;
+    forEach(artifacts, (filename) => {
       result[`${ filename }.${ fileType }`] = artifacts[filename];
-  }
+    });
+  });
 
   return result;
 }

package/lib/api/options.js

@@ -80,13 +80,12 @@ const overallOptions = publicOptionsNewAPI.concat(privateOptions);
 function translateOptions(input = {}, defaults = {}, hardRequire = {},
                           customValidators = {}, combinationValidators = [], moduleName = '') {
   const options = Object.assign({}, defaults);
-  const inputOptionNames = Object.keys(input);
   for (const name of overallOptions) {
     // Ensure that arrays are not passed as a reference!
     // This caused issues with the way messages are handled in processMessages
-    if (Array.isArray(input[name]) && inputOptionNames.includes(name))
+    if (Array.isArray(input[name]))
       options[name] = [ ...input[name] ];
-    else if (inputOptionNames.includes(name))
+    else if (Object.hasOwnProperty.call(input, name))
       options[name] = input[name];
   }
 

package/lib/api/validate.js

@@ -1,6 +1,7 @@
 'use strict';
 
 const { makeMessageFunction } = require('../base/messages');
+const { forEach } = require('../utils/objectUtils');
 
 /* eslint-disable arrow-body-style */
 const booleanValidator = {
@@ -159,16 +160,15 @@ function validate(options, moduleName, customValidators = {}, combinationValidat
   // TODO: issuing messages in this function looks very strange...
   {
     const messageCollector = { messages: [] };
-    const { error, throwWithError } = makeMessageFunction(null, messageCollector, moduleName);
+    const { error, throwWithAnyError } = makeMessageFunction(null, messageCollector, moduleName);
 
-    for (const optionName of Object.keys(options)) {
-      const optionValue = options[optionName];
+    forEach(options, (optionName, optionValue) => {
       const validator = customValidators[optionName] || validators[optionName] || booleanValidator;
 
       if (!validator.validate(optionValue))
         error('invalid-option', null, {}, `Expected option "${ optionName }" to have "${ validator.expected(optionValue) }". Found: "${ validator.found(optionValue) }"`);
-    }
-    throwWithError();
+    });
+    throwWithAnyError();
   }
 
   const message = makeMessageFunction(null, options, moduleName);
@@ -179,7 +179,7 @@ function validate(options, moduleName, customValidators = {}, combinationValidat
       message[combinationValidator.severity]('invalid-option-combination', null, {}, combinationValidator.getMessage(options));
   }
 
-  message.throwWithError();
+  message.throwWithAnyError();
 }
 /* eslint-enable jsdoc/no-undefined-types */