@sdeverywhere/cli 0.7.45 → 0.7.47

package/README.md

@@ -2,7 +2,7 @@
 
 This package contains the `sde` command line interface for the [SDEverywhere](https://github.com/climateinteractive/SDEverywhere) suite of tools.
 
-SDEverywhere can be used to translate System Dynamics models from Vensim to C and WebAssembly.
+SDEverywhere can be used to translate System Dynamics models from Vensim or Stella format to JavaScript, C, and WebAssembly.
 
 For more details on the full suite of tools and libraries provided in SDEverywhere, refer to the top-level [README](https://github.com/climateinteractive/SDEverywhere) in the SDEverywhere repository.
 
@@ -46,10 +46,10 @@ Use `sde -h` to see a list of all commands.
 
 Use `sde {command}` to see options for a command.
 
-It is usually easiest to run these commands from the directory where the `.mdl` file is located.
+It is usually easiest to run these commands from the directory where the model file (for example, a Vensim `.mdl` file or a Stella `.stmx` file) is located.
 The `{model}` placeholder can be the model filename, for instance `arrays.mdl`, or simply the model name `arrays`.
 
-If you are not running from the model directory, you can give a full pathname to locate the `.mdl` file anywhere on the system.
+If you are not running from the model directory, you can give a full pathname to locate the model file anywhere on the system.
 
 By default, SDEverywhere will create a `build` directory in your model directory to hold the generated code and the compiled model.
 If you run the model, it will also create an `output` directory by default.
@@ -70,9 +70,9 @@ sde generate --genc --spec {model}_spec.json {model}
 #### Start a local development builder/server
 
 The `sde dev` command is great for local development of a web application.
-It will start a builder process that rebuilds your app and runs QA checks against your model any time you save changes to your `mdl` file.
-You can leave the builder running while developing your model in Vensim.
-The app and model-check tabs in your browser will refresh automatically whenever you save changes in Vensim or make edits to your application code.
+It will start a builder process that rebuilds your app and runs QA checks against your model any time you save changes to your model file.
+You can leave the builder running while developing your model in Vensim or Stella.
+The app and model-check tabs in your browser will refresh automatically whenever you save changes in your modeling tool or make edits to your application code.
 
 See [`examples/hello-world`](https://github.com/climateinteractive/SDEverywhere/tree/main/examples/hello-world) for a simple example of an `sde.config.js` file.
 You can also follow the [Quick Start](https://github.com/climateinteractive/SDEverywhere/tree/main/examples/sir#quick-start) instructions for `examples/sir`, which will generate a more complete example of an `sde.config.js` file.
@@ -176,8 +176,9 @@ sde which
 
 _NOTE:_ The following sections refer to "model specification files" (or "spec files" as a shorthand).
 These JSON spec files are generally used by the lower-level `sde` commands, such as `sde generate`.
-We are gradually adding support for a more flexible configuration file format (`sde.config.js`) that works with newer commands such as `sde dev` and `sde bundle`.
-We hope to unify these configuration file formats soon to eliminate any confusion about which file format can be used with which command (see related issue [#327](https://github.com/climateinteractive/SDEverywhere/issues/327)).
+The higher-level `sde dev` and `sde bundle` commands use a more flexible configuration file format (`sde.config.js`) that also allows for defining plugins.
+The two formats share the same property definitions where they overlap, so most of the properties documented below are also available in the `modelSpec` section of an `sde.config.js` file.
+The exception is the input and output variables: a spec file takes plain variable names in `inputVarNames` and `outputVarNames`, whereas an `sde.config.js` file uses higher-level `inputs` and `outputs` properties that also accept objects carrying additional information about each variable.
 
 #### Specify input and output variables
 
@@ -198,6 +199,14 @@ Be sure to include `Time` first among the output variables.
 
 #### Specify external data sources
 
+Add a `datFiles` section to the spec file to have SDEverywhere read data for exogenous data variables from one or more Vensim `dat` files.
+Each entry can either be a plain file name, or an object that maps a variable name prefix to a file name.
+There are examples in the `extdata` and `getdata` sample models.
+
+```json
+"datFiles": ["data.dat", { "prefix ": "other.dat" }]
+```
+
 Add a `directData` section to the spec file to have SDEverywhere read data from an Excel file into lookups with a variable name prefix.
 There is an example in the `directdata` sample model.
 
@@ -207,6 +216,25 @@ There is an example in the `directdata` sample model.
 }
 ```
 
+#### Supported spec file properties
+
+The full set of supported properties is defined by the [`ModelSpec`](https://github.com/climateinteractive/SDEverywhere/blob/main/packages/compile/src/_shared/model-spec.js) type in the `@sdeverywhere/compile` package, which is the authoritative reference.
+The table below is a summary.
+
+| Property                  | Type                     | Description                                                                                                                           |
+| ------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
+| `inputVarNames`           | `string[]`               | The input variables, using the names as they appear in the modeling tool.                                                             |
+| `outputVarNames`          | `string[]`               | The output variables, using the names as they appear in the modeling tool. It is customary to list `Time` first.                      |
+| `datFiles`                | `(string \| object)[]`   | The `dat` files that provide data for exogenous data variables, resolved relative to the model directory.                             |
+| `directData`              | `object`                 | Maps the data tag used in a `GET DIRECT {DATA,CONSTANTS,LOOKUPS}` call (for example, `?data`) to an `xlsx` file name.                 |
+| `dimensionFamilies`       | `object`                 | Maps a dimension name to its family name, for cases where the family cannot be inferred from the model alone.                         |
+| `specialSeparationDims`   | `object`                 | Maps a variable identifier to the dimension(s) on which that variable should be separated, which can be used to break a cycle.        |
+| `separateAllVarsWithDims` | `(string \| string[])[]` | The dimensions for which all variables should be separated, as an alternative to listing each variable in `specialSeparationDims`.    |
+| `bundleListing`           | `boolean`                | Whether to bundle a model listing with the generated model so that the `runtime` package can resolve variables by name or identifier. |
+| `customConstants`         | `boolean \| string[]`    | Whether (or which) constants can be overridden at runtime using `setConstant`.                                                        |
+| `customLookups`           | `boolean \| string[]`    | Whether (or which) lookups can be overridden at runtime using `setLookup`.                                                            |
+| `customOutputs`           | `boolean \| string[]`    | Whether (or which) variables can be captured at runtime using `storeOutput`.                                                          |
+
 #### Generating, compiling, running, and testing the C code
 
 To generate C code using the `--spec` argument, enter the following command:
@@ -238,7 +266,7 @@ There is a `setInputs` implementation in the generated code that gets called at
 It takes a string with serialized input values and sets variable values from it.
 The serialization format depends on the needs of your application.
 You can replace `setInputs` if you want to use a different serialization form.
-The input variables are listed in the `inputVars` section of the spec file.
+The input variables are listed in the `inputVarNames` section of the spec file.
 Look at the `arrays` model for an example.
 
 The generated format minimizes the amount of data on the wire for web applications.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sdeverywhere/cli",
-  "version": "0.7.45",
+  "version": "0.7.47",
   "description": "Contains the `sde` command line interface for the SDEverywhere tool suite.",
   "type": "module",
   "files": [
@@ -11,8 +11,8 @@
     "sde": "src/main.js"
   },
   "dependencies": {
-    "@sdeverywhere/build": "^0.3.14",
-    "@sdeverywhere/compile": "^0.7.32",
+    "@sdeverywhere/build": "^0.3.16",
+    "@sdeverywhere/compile": "^0.7.34",
     "byline": "^5.0.0",
     "ramda": "^0.27.0",
     "shelljs": "^0.10.0",

package/src/c/allocation.c

@@ -24,7 +24,7 @@
 // #define PRINT_ALLOCATIONS_DEBUG_INFO
 
 // Return true if the value is near zero up to the epsilon tolerance.
-static inline bool __isZero(double value) { return fabs(value) < _epsilon; }
+static inline bool __isZero(double value) { return fabs(value) < SDE_EPSILON; }
 // Compute the absolute difference when x or y is near zero, otherwise compute
 // the relative difference, with y considered as the baseline.
 static inline double __difference(double x, double y) {
@@ -37,7 +37,7 @@ static inline double __difference(double x, double y) {
   return diff;
 }
 // Return true if the values are equal up to the tolerance.
-static inline bool __isEqual(double x, double y) { return __difference(x, y) < _epsilon; }
+static inline bool __isEqual(double x, double y) { return __difference(x, y) < SDE_EPSILON; }
 // Clamp x to the interval [0,1].
 static inline double __clamp01(double x) {
   if (x < 0.0) return 0.0;
@@ -440,7 +440,7 @@ double* _ALLOCATE_BY_PRIORITY(
 
   // Validate request values (must be non-negative)
   for (size_t i = 0; i < num_requesters; i++) {
-    if (request_quantities[i] < -_epsilon) {
+    if (request_quantities[i] < -SDE_EPSILON) {
       fprintf(stderr,
           "_ALLOCATE_BY_PRIORITY encountered negative request value at index %zu: %f\n",
           i, request_quantities[i]);
@@ -449,7 +449,7 @@ double* _ALLOCATE_BY_PRIORITY(
   }
 
   // Validate width (must not be negative)
-  if (width < -_epsilon) {
+  if (width < -SDE_EPSILON) {
     fprintf(stderr,
         "_ALLOCATE_BY_PRIORITY encountered invalid width value: %f\n"
         "Width must not be negative.\n",
@@ -458,7 +458,7 @@ double* _ALLOCATE_BY_PRIORITY(
   }
 
   // Validate supply (must not be negative)
-  if (supply < -_epsilon) {
+  if (supply < -SDE_EPSILON) {
     fprintf(stderr,
         "_ALLOCATE_BY_PRIORITY encountered invalid supply value: %f\n"
         "Supply must not be negative.\n",
@@ -472,7 +472,7 @@ double* _ALLOCATE_BY_PRIORITY(
   }
 
   // If supply = 0, all targets get allocated 0
-  if(fabs(supply) < _epsilon) {
+  if(fabs(supply) < SDE_EPSILON) {
     return allocations;
   }
 
@@ -539,7 +539,7 @@ double* _ALLOCATE_BY_PRIORITY(
   size_t c_i = 0;
 
   // Continue allocating until supply is exhausted
-  while (supply > _epsilon) {
+  while (supply > SDE_EPSILON) {
     // Check if there are any active targets left
     bool any_active = false;
     for (size_t i = 0; i < m; i++) {

package/src/c/sde.h

@@ -25,8 +25,6 @@ extern "C" {
 #include "vensim.h"
 #include "macros.h"
 
-EXTERN double _epsilon;
-
 // Enable this to add print statements in initLevels and evalAux for debugging.
 // #define PRDBG
 #ifdef PRDBG

package/src/c/vensim.c

@@ -1,19 +1,16 @@
 #include "sde.h"
 
-double _epsilon = 1e-6;
+//
+// Note: This file only contains implementations of complex functions.  Simple
+// (inline-friendly) functions are defined as `static inline` functions in `vensim.h`
+// so that they can be inlined at each call site (for improved performance).
+//
 
 //
 // Vensim functions
 // See the Vensim Reference Manual for descriptions of the functions.
 // http://www.vensim.com/documentation/index.html?22300.htm
 //
-double _PULSE(double start, double width) {
-  double time_plus = _time + _time_step / 2.0;
-  if (width == 0.0) {
-    width = _time_step;
-  }
-  return (time_plus > start && time_plus < start + width) ? 1.0 : 0.0;
-}
 double _PULSE_TRAIN(double start, double width, double interval, double end) {
   double n = floor((end - start) / interval);
   for (double k = 0; k <= n; k++) {
@@ -23,29 +20,6 @@ double _PULSE_TRAIN(double start, double width, double interval, double end) {
   }
   return 0.0;
 }
-double _RAMP(double slope, double start_time, double end_time) {
-  // Return 0 until the start time is exceeded.
-  // Interpolate from start time to end time.
-  // Hold at the end time value.
-  // Allow start time > end time.
-  if (_time > start_time) {
-    if (_time < end_time || start_time > end_time) {
-      return slope * (_time - start_time);
-    } else {
-      return slope * (end_time - start_time);
-    }
-  } else {
-    return 0.0;
-  }
-}
-double _XIDZ(double a, double b, double x) { return fabs(b) < _epsilon ? x : a / b; }
-double _ZIDZ(double a, double b) {
-  if (fabs(b) < _epsilon) {
-    return 0.0;
-  } else {
-    return a / b;
-  }
-}
 
 //
 // Lookups

package/src/c/vensim.h

@@ -10,6 +10,14 @@ extern "C" {
 #define _NA_ (-DBL_MAX)
 #define bool_cond(cond) ((double)(cond) != 0.0)
 
+// The tolerance used by the functions that guard against division by zero.
+#define SDE_EPSILON 1e-6
+
+// Note: The `_time` and `_time_step` variables are declared in `sde.h`, but `sde.h` includes
+// this header, so we declare them here for use by the inline functions below.
+extern double _time;
+extern double _time_step;
+
 //
 // Vensim functions
 // See the Vensim Reference Manual for descriptions of the functions.
@@ -44,13 +52,49 @@ double* _DEMAND_AT_PRICE(double* demand_quantities, double* demand_profiles, dou
 double _FIND_MARKET_PRICE(double* demand_quantities, double* demand_profiles, double* supply_quantities,
     double* supply_profiles, size_t num_demanders, size_t num_suppliers);
 double* _INVERT_MATRIX(double* matrix, size_t n);
-double _PULSE(double start, double width);
 double _PULSE_TRAIN(double start, double width, double interval, double end);
-double _RAMP(double slope, double start_time, double end_time);
 double* _SUPPLY_AT_PRICE(double* supply_quantities, double* supply_profiles, double price, size_t num_suppliers);
 double* _VECTOR_SORT_ORDER(double* vector, size_t size, double direction);
-double _XIDZ(double a, double b, double x);
-double _ZIDZ(double a, double b);
+
+//
+// Note: The following functions are defined here as `static inline` (rather than as
+// out-of-line functions in `vensim.c`) because they are called very frequently (millions of
+// times per run for a large model).  Defining them in the header allows the compiler to
+// inline them at each call site, which avoids the call overhead and allows repeated
+// identical calls to be replaced with a single one.
+//
+
+static inline double _PULSE(double start, double width) {
+  double time_plus = _time + _time_step / 2.0;
+  if (width == 0.0) {
+    width = _time_step;
+  }
+  return (time_plus > start && time_plus < start + width) ? 1.0 : 0.0;
+}
+
+static inline double _RAMP(double slope, double start_time, double end_time) {
+  // Return 0 until the start time is exceeded.
+  // Interpolate from start time to end time.
+  // Hold at the end time value.
+  // Allow start time > end time.
+  if (_time > start_time) {
+    if (_time < end_time || start_time > end_time) {
+      return slope * (_time - start_time);
+    } else {
+      return slope * (end_time - start_time);
+    }
+  } else {
+    return 0.0;
+  }
+}
+
+static inline double _XIDZ(double a, double b, double x) {
+  return fabs(b) < SDE_EPSILON ? x : a / b;
+}
+
+static inline double _ZIDZ(double a, double b) {
+  return fabs(b) < SDE_EPSILON ? 0.0 : a / b;
+}
 
 //
 // Lookups