@suffolklitlab/alkiln 5.0.0-pre-8 → 5.0.0-pre-10

package/CHANGELOG.md

@@ -45,7 +45,8 @@ Format:
   be run in any directory, not just in an npm package.
 - Additional environment variables and their validation to allow for tests that run on a developer's server/Playground instead of through GitHub. Also, other functionality for that purpose. [Issue #661](https://github.com/SuffolkLITLab/ALKiln/issues/661)
 - Tests for new session_vars behavior and improve previous tests.
-- Adds `npm-shrinkwrap.json`, so installs from npm will have fixed version dependencies
+- Adds `npm-shrinkwrap.json`, so installs from npm will have fixed version dependencies.
+- Allow author to specify loops with only `.target_number`. e.g. to leave out `.there_are_any`. See [issue #706](https://github.com/SuffolkLITLab/ALKiln/issues/706) and Story Table documentation in docs folder.
 
 ### Changed
 - BREAKING: the github action no longer runs `npm run XYZ`; it directly calls scripts,
@@ -65,12 +66,16 @@ Format:
 - Shorten path names to try to accommodate limitations of windows systems while still keeping enough useful information to help devs identify the test outputs. https://github.com/SuffolkLITLab/ALKiln/issues/618
 - Updated field decoding to handle new object field encoding. See [#711](https://github.com/SuffolkLITLab/ALKiln/issues/711)
 - Allow multiple languages to be tested again. See [#713](https://github.com/SuffolkLITLab/ALKiln/issues/713).
+- Fill in time fields correctly. See [#726](https://github.com/SuffolkLITLab/ALKiln/pull/726).
+- Allow `.target_number` to be 0. See https://github.com/SuffolkLITLab/ALKiln/issues/706.
+- Use the right number of loops for `.target_number`. See https://github.com/SuffolkLITLab/ALKiln/issues/706.
 
 ### Security
 - Pass docassemble API keys through HTTP headers instead of as parameters.
   - Parameters to certain HTTP requests are printed directly in docassemble's
     uWSGI log, leaking API keys to actors with log access on your docassemble
     server
+- Update dependencies. See [#727](https://github.com/SuffolkLITLab/ALKiln/pull/727).
 
 ## [4.11.1] - 2023-03-21
 ### Changed

package/docs/story_tables.md

@@ -0,0 +1,89 @@
+# Story Tables overview
+
+The Story Table is a very flexible tool. It lets the author answer interview (form) questions in any order. They can make cosmetic changes to their interview code without having to change their tests. They can edit the order of their interview's pages, move fields from one page to another, and so on.
+
+When the robot arrives on each interview page, it finds all the fields on the page and loops through the whole Story Table trying to match rows with fields.
+
+Here's some [documentation on the use of Story Table rows](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/alkiln/#story-tables).
+
+## Tracking use of rows
+
+Why do we track how many times each Story Table row was used to set a form's fields?
+
+First, it lets us give the author useful information. We can show the author which rows of their story table were or weren't used. This can help the author tell that some crucial variables aren't being set as they expected. It also outputs a new Story Table for them. If they had a bunch of extra rows adding cruft, like from our [story table generator](https://plocket.github.io/al_story/), it gives the author a table with only the variables that are needed for the test.
+
+Second, in some cases we need to actually throw an error if any row was left unused, so we have to keep track of that.
+
+Third, looping questions that gather multiple items in a list are tricky when the interview answers need the `.there_is_another` attribute. We can't use that attribute. Instead, we handle all gathering with `target_number` rows. Next we'll discuss what problem the `.there_is_another` attribute creates and our approach to solving it.
+
+## The loop problem
+
+Since the Story Table loops through all its rows for each page, each row in the table needs to be unique. That is, you can only set one variable to one value. That's fine most of the time, but when gathering items in a list in a docassemble interview, there are a few ways to do it and one of them sets one variable over and over again - `.there_is_another`. This kind of loop can start with a question that sets `.there_are_any`.
+
+For example, look for `.there_is_another` in the scenario below:
+
+Page 1: Do you have any moms? (Set `moms.there_are_any` to `True`)
+Page 2: What is your first mom's name? (Set `mom[0].name.first`)
+Page 3: Do you have any other moms? (Set `moms.there_is_another` to `True`)
+Page 4: What is your second mom's name? (Set `mom[1].name.first`)
+Page 5: Do you have any other moms? (Set `moms.there_is_another` to `False`)
+
+See that on page 3, `moms.there_is_another` is set to `True` and on page 5 it's set to `False`. We can't represent this in a Story Table.
+
+```
+| x.there_is_another | True | moms.there_is_another |
+```
+
+The above will always say that we have another `mom`. If we allowed our users to do that, it would create an infinite loop. So we prevent our users from creating a `there_is_another`  row that has a value of `True`. We still need to allow them to set `.there_is_another`, though, so what do we do?
+
+## The loop solution
+
+Well, there's another way to loop to gather items in docassemble - `target_number`. For example:
+
+Page 1: How many moms do you have? (Set `moms.target_number` to `2`)
+Page 2: What is your first mom's name? (Set `mom[0].name.first`)
+Page 3: What is your second mom's name? (Set `mom[1].name.first`)
+
+The target number row looks like this:
+
+```
+| x.target_number | 2 | moms.target_number |
+```
+
+If authors use `.target_number`, that's no problem at all for our code. We can't require authors to use that method to ask their questions, though. That may make their form questions confusing to their end users sometimes. Fortunately, we _can_ require developers to use `.target_number`[^1] in their Story Tables. We can use a `.target_number` table value to calculate what to answer for `.there_are_any` and to know how many times to set `.there_is_another` to `'True'`.
+
+## The loop architecture
+
+First, the very basic flow for setting a variable. Note: whatever Step the dev uses to set a variable's value, we send it to this loop. It may be useful to follow along in the code here.
+
+1. If this isn't a Story Table step, transform the given arguments into a Story Table structure - a list of row objects.
+2. For each row object (see `scope.normalizeTable()`)
+    1. Create a new object with almost the same properties. It contains the original object in a property called `original` so the original can be used to print stuff for the user's report.
+    2. In this new object, substitute environment variables the author gave into actual values. They may have done this to keep a sensitive value (like a password) secret.
+    3. Add ways to accumulate the number of times the row is used.
+3. For each page (see `scope.setFields()`)
+    1. Identify and store all the fields on the page and all the variables they set.
+    2. Shallowly clone the Story Table row list.
+        1. Create a new object almost the same as the old object
+        2. Print a warning to the user if they've created a `.there_is_another` row with a value of `'True'` and change that value to `'False'`.
+        3. If there is a `.target_number` row, use it to create new artificial rows for `.there_is_another` and `.there_are_any`.
+        4. This new list will be used to actually put answers into fields.
+    3. For each field on the page
+        1. Find which Story Table row, if any, matches that field.
+        2. Set the field to the given value.
+        3. Increment the row's accumulators.
+            1. Increment the relevant `.used_for['foo']` by 1.
+            2. Increment all relevant `.times_used`[^2] by 1.
+    4. Try to continue.
+
+<!-- From feedback, I'm not sure how to clarify this part or if readers will need it
+Why make artificial rows? Well, that way when we loop through to match a page's fields to the variables in the Story Table structure, we don't have to do anything special for these rows. If they're not needed, they won't be triggered.
+
+Why not calculate the `.there_are_any` and `.there_is_another` values in the row-matching loop? That's another stylistic choice. There's more logic to the `.there_is_another` value than we've talked about here and that method seems like it would add a lot more cruft and complexity in that match-finding loop. We may eventually change our minds.
+ -->
+
+---
+
+[^1] Why use `target_number`? We count on our users being at least a bit familiar with docassemble variables and to have access to the docassemble docs, so the `target_number` variable is something they should be able to recognize or easily find information about. We decided to take advantage of that. Also, we chose to avoid 0 indexing so that the number starts at 1 and feels like human language, not computer language as our users are generally more human than computer. It's also the way docassemble does it.
+
+[^2] It would seem like the `.times_used` property is redundant, but it has a different purpose that the `.used_for` properties. It lets us show the dev whether the `target_number` table row was used at all during the test to let us give appropriate feedback to the user. We could add up the `used_for` values, but for now we're avoiding doing that extra math and the extra refactoring. The choice has its pros and cons.

package/lib/scope.js

@@ -919,15 +919,21 @@ module.exports = {
       }
 
       let result = {
-        original: row,
+        // NOTE: mutating frozen objects will fail silently, so don't mutate this attribute
+        original: Object.freeze(row),
+        artificial: false,
         // Give fake data if needed to prevent some kind of hidden input field from being found
         // See bug at https://github.com/plocket/docassemble-cucumber/issues/79
         // May be able to remove this once converted to `getAllFields`
         var: row.var || 'al_no_var_name',
         value: actual_var_value,
         trigger: row.trigger || '',
+        // This will track how many times total the row or its connected
+        // artificial rows are used. Artificial rows are only added for some rows.
         times_used: 0,
+        used_for: {},
       }
+      result.used_for[ result.var ] = 0;
 
       supported_table.push( result );
     }
@@ -1755,6 +1761,13 @@ module.exports = {
         }
         await scope.setText( scope, { handle, answer: answer_date });
         await scope.afterStep(scope, { waitForShowIf: true });
+      } else if (type == `time`) {
+        // Authors should enter the time like 12:34 PM, but we shouldn't type ":", space, or M.
+        let answer_time = answer.replace(":", "").replace(" ", "").replace("M", "");
+        await handle.focus();
+        await handle.type( answer );
+        await handle.press("Enter");
+        await scope.afterStep(scope, { waitForShowIf: true });
       } else {
         await scope.setText( scope, { handle, answer });
         await scope.afterStep(scope, { waitForShowIf: true });
@@ -2017,8 +2030,7 @@ module.exports = {
           });
 
           // Accumulate number of times this row was used. Esp important for `.target_number`
-          if ( row_used.source ) { row_used.source.times_used += 1 }
-          row_used.times_used += 1;
+          await scope.increment_row_use( scope, { row: row_used });
 
           process.stdout.write(`\x1b[36m${ '*' }\x1b[0m`);  // assumes var was set if no error occurred
 
@@ -2043,8 +2055,7 @@ module.exports = {
           });
 
           // Esp important for `.target_number`
-          if ( row_used.source ) { row_used.source.times_used += 1 }
-          row_used.times_used += 1;
+          await scope.increment_row_use( scope, { row: row_used });
 
           // TODO: special continue/navigation button function that handles url and all?
           
@@ -2131,35 +2142,145 @@ module.exports = {
     // TODO: Maybe we should return something else or something in addition about used vars
   },  // Ends scope.setFields()
 
-  // TODO: Move this to under `normalizeTable`
+  increment_row_use: async function ( scope, { row }) {
+    /** Accumulate number of times this row was used. Esp important for `.target_number`.
+    *   Mutates `row` and its props. */
+
+    // Increment the number of times a row itself was used.
+    row.times_used += 1;
+
+    // Increment the times a row was used for a particular variable.
+    // TODO: check if we need this initial default assignment.
+    if ( !row.used_for ) { row.used_for = {}; }
+    // TODO: check if we need this initial default assignment.
+    if ( row.used_for[ row.var ] ) {
+      row.used_for[ row.var ] += 1;
+    } else {
+      row.used_for[ row.var ] = 1;
+    }
+
+    // If this is an artificial row, increment its source row as well. This may
+    // be one of multiple artificial rows and the source row will need to be
+    // incremented for each of its artificial rows.
+    // Right now, this is only used for `target_number` rows.
+    if ( row.source ) {
+      // For each artificial row, increment the total number of times its
+      // source row was used.
+      row.source.times_used += 1;
+
+      // Indicate which of the artificial rows was incremented (this one).
+      // TODO: check if we need this initial default assignment.
+      if ( row.source.used_for[ row.var ] ) {
+        row.source.used_for[ row.var ] += 1;
+      } else {
+        row.source.used_for[ row.var ] = 1;
+      }
+    }  // ends if there's a source row
+    
+  },  // Ends scope.increment_row_use()
+
+
   ensureSpecialRows: async function ( scope, { var_data, from_story_table=true }) {
     /* Given a list of variable data objects, add more objects or mutate
-    * rows under special circumstances:
-    * - `.target_number` may indicate the need for a `.there_is_another` row.
-    * - `.there_is_another` in a table is invalid
+    *    row objects under special circumstances:
+    * - `.target_number` will create `.there_are_any` and `.there_is_another` rows.
+    *    If they're unused in the end, that's fine.
+    * - `.there_is_another` with a value of True in a Story Table is invalid and
+    *    will be neutralized.
+    * 
+    * To learn the bigger picture, see "docs/story_tables.md"
     */
-    // Add special cases. Do not mutate `var_data` as a list, but
-    // do allow mutation of items inside.
     let enhanced_var_data = [...var_data];
     for ( let var_datum of var_data ) {
 
-      // Add a `.there_is_another` var if it might be needed.
-      // If it's not used in the end, that's fine.
-      // `.target_number` is sometimes a stand-in for `.there_is_another`.
-      // That seems the clearest way to communicate to developers about what they
-      // should write in the test. We will attempt to write more about
-      // the rationale in the documentation at some point.
       if ( from_story_table && var_datum.var.match( /\.target_number$/ )) {
-        let special_row = {
-          artificial: true,  // not currently used but is future-proofing to mark non-dev generated rows
+
+        // This is a row created by us, not by the user
+        let artificial = true;
+
+        // Add `.there_are_any` value depending on `.target_number` value.
+        // If the dev has also created a `.there_are_any` row, that will come
+        // first in the table and will be overriden by this later artificial row.
+        // TODO: Document this for devs as this behavior might confuse folks
+        // who accidentally include both and have conflicting values
+        // for each of them.
+        let var_name_any = var_datum.var.replace( /\.target_number$/, `.there_are_any` );
+
+        let there_are_any = {
+          artificial,
+          // For now this is very nested. We may flatten in the future.
+          ...var_datum,
+          // `source` is acting as the accumulator of the row use here
+          source: var_datum,
+          var: var_name_any,
+          // devs are supposed to use the right `trigger` name, but they might not.
+          // Of course, this too might be the wrong trigger name, but there's no way to
+          // know that and it's unlikely.
+          // TODO: Make sure this is documented.
+          trigger: var_datum.trigger.replace( /\.target_number$/, `.there_are_any` ),
+          used_for: {},
+        };
+        there_are_any.used_for[ var_name_any ] = 0;
+
+        // This creation function is run at the beginning of every page.
+        // If this is the first page and the `target_number` source row
+        // has just been created, it won't have this sub-accumulator yet,
+        // so create them. Otherwise, don't override its current value.
+        if ( var_datum.used_for[ var_name_any ] === undefined ) {
+          var_datum.used_for[ var_name_any ] = 0;
+        }
+
+        // If `target_number` is 0, there are none of this item
+        if ( parseInt(var_datum.value) <= 0 ) {
+          there_are_any.value = `False`;
+        // Otherwise, there's at least one item
+        } else if ( parseInt(var_datum.value) > 0 ) {
+          there_are_any.value = `True`;
+        }
+
+        // Now both `.target_number` and `.there_are_any` rows exist if needed
+        enhanced_var_data.push( there_are_any );
+
+        // Add `.there_is_another` value depending on `.target_number` value.
+        // If the dev has also created a `.there_is_another` row, that will come
+        // first in the table and will be overriden by this later artificial row.
+        // The dev will still get a warning at the start of each page that they've
+        // included an invalid `.there_is_another` with a value of `True`.
+        // This row might not be needed if `.target_number` is 0, but for now that
+        // seems like it would add more logic to read.
+        let var_name_another = var_datum.var.replace( /\.target_number$/, `.there_is_another` );
+
+        let there_is_another = {
+          artificial,  
+          ...var_datum,
+          // `source` is acting as the accumulator of the row use here
           source: var_datum,
-          var: var_datum.var.replace( /\.target_number$/, `.there_is_another` ),
-          value: await scope.getThereIsAnotherValue( scope, { row: var_datum, from_story_table }),
-          // devs are supposed to use the right `trigger` name, but they might not
+          var: var_name_another,
+          // devs are supposed to use the right `trigger` name, but they might not.
+          // Of course, this too might be the wrong trigger name, but there's no way to
+          // know that and it's unlikely.
           trigger: var_datum.trigger.replace( /\.target_number$/, `.there_is_another` ),
+          used_for: {},
         };
+        there_are_any.used_for[ var_name_another ] = 0;
+
+        // This creation function is run at the beginning of every
+        // page. If this is the first page and the `target_number`
+        // source row has just been created, it won't have these
+        // sub-accumulators yet, so create them.
+        // Otherwise, don't override their current values.
+        if ( var_datum.used_for[ var_name_another ] === undefined ) {
+          var_datum.used_for[ var_name_another ] = 0;
+        }
+
+        // Use those accumulators to get `True` or `False` for `there_is_another`
+        there_is_another.value = await scope.getThereIsAnotherValue(
+          scope,
+          { row: there_is_another, from_story_table }
+        );
 
-        enhanced_var_data.push( special_row );
+        // Now both `.target_number` and `.there_is_another` rows exist if needed
+        enhanced_var_data.push( there_is_another );
 
       // Neutralize a `.there_is_another` row in a story table
       } else if ( from_story_table && var_datum.var.match( /\.there_is_another$/ ) && var_datum.value !== `False`) {
@@ -2174,8 +2295,8 @@ module.exports = {
               + `that list. This test will now set this row's \`value\` to \`False\`. See `
               + `https://suffolklitlab.github.io/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing#there_is_another. `
               + `The row data is\n${ JSON.stringify( var_datum.original )}` });
-      }
-    }
+      }  // ends if it's a `target_number` row
+    }  // ends for each row
 
     return enhanced_var_data;
   },  // Ends scope.ensureSpecialRows()
@@ -2183,23 +2304,16 @@ module.exports = {
   getThereIsAnotherValue: async function ( scope, { row, from_story_table=true }) {
     /* Given story row-like data as well as whether this is from a table row,
     * return the best guess as to a safe value to set for this round. Give
-    * the developer a warning if
-    * it's an inappropriate value. Avoid mutation.
-    *
-    * WARNING: Remember to decrement the actual value later.
+    * the developer a warning if it's an inappropriate value. Avoid mutation.
     * 
-    * Sometimes this will be from `.target_number` from story tables and
-    * we believe we've set it up so that people will make it a number.
-    * Sometimes this will be from `.there_is_another`. We think we've ensured
-    * that it will only come from Steps and should be `True` or `False`.
-    * This feels precarious, but I can't see another permutation.
+    * Right now this argument should always be a `.target_number` number row from
+    *    Story Tables.
     * 
-    * This may in future be used to normalize all values that need `True`/`False`
-    * normalization.
+    * To learn the bigger picture, see the docs folder for info about Story Tables.
     */
-    let value = row.value.toLowerCase();
+    let value = row.source.value.toLowerCase();
     let val_to_print = printable_var_value(row); 
-    let int_val = parseInt( value );
+    let desired_num = parseInt( value );
     
     // Step stuff. Take into account a lot of possible values
     if ( value.match( /(false|no|unchecked|uncheck|deselected|deselect)/i )) {
@@ -2207,35 +2321,36 @@ module.exports = {
       return `False`;
 
     } else if ( value.match( /(true|yes|checked|check|selected|select)/i )) {
-      if ( from_story_table ) {
+      if ( from_story_table && !row.artificial ) {
         if ( row.var.match( /\.there_is_another$/ ) ) {
-          // `.there_is_another` should never get in here, but just in case
           await scope.addToReport( scope, { type: `warning`, value: `The attribute \`.there_is_another\` is invalid in story table tests. Replace it with \`.target_number\` in your \`var\` column. Set the \`value\` to the number of items in that list. This test will now set this row's \`value\` to \`False\`. See https://suffolklitlab.github.io/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing#there_is_another-loop. The row data is\n${ JSON.stringify( row.original )}` });
         } else {
           // If `target_number` is set to True instead of a number
-          // Since secret vars can't be passed in a table, this should never be a secret var
           await scope.addToReport( scope, { type: `warning`, value: `${ val_to_print } value is not a valid value for ${ row.var } here. This test will default to \`False\` to avoid problems.` });
         }
         return `False`;
       } else { return `True`; }
-
-    // Story table stuff
-    // Target: 3 (3 items. press 'yes' twice.)
-    // Times used: 0 -> `True`, 1 -> `True`, 2 -> `False`, 
-    } else if ( row.times_used >= int_val - 1 ) {  // Story table loop will end
-        return `False`;
-
-    } else if ( row.source && row.source.times_used >= int_val - 1 ) {  // Just in case use changes a tad
-        return `False`;
-
-    } else if ( isNaN( parseInt( value )) ) { // Loop will end
-      // I can't figure out how to trigger this warning either
+    
+    } else if ( isNaN( desired_num )) {  // Story table loop will end
+      // I can't figure out how to trigger this warning, so maybe we're
+      // already handling everything we need to
       await scope.addToReport( scope, { type: `warning`, value: `${ val_to_print } value is not a valid value for ${ row.var } here. This test will default to \`False\` to avoid problems.` });
       return `False`;
 
-    } else {  // It's a number. Loop will continue.
+	    // Check that the number of elements in the list hits our desired number.
+      // The first time `.there_is_another` is used, it's always the second item
+      // the form is asking about (and so forth), so we have to add 1 to this check.
+
+      // Example behavior:
+      // For 3 items, press 'yes' on there_is_another 2 times because
+      // `.there_are_any` handles the first time.
+      // target_number: 3 
+      // used_for[ <var_name>.target_number ]: 0 -> `True`, 1 -> `True`, 2 -> `False`
+	    } else if ( row.source && row.source.used_for[ row.var ] + 1 >= desired_num) {  // Story table loop will end
+      return `False`;
+    } else {
       return `True`;
-    }
+    }  // ends act depending values
   },  // Ends scope.getThereIsAnotherValue()
 
   // throw_if_no_interview_content()?
@@ -2295,9 +2410,13 @@ module.exports = {
     let was_found = false;
     for ( let error_key in possible_errors ) {
       if ( possible_errors[ error_key ][0] ) {
-        // If you're debugging and you close the window early, the process will still
+        // If there's a page visible (locally), give us as much time to examine it as timeout allows.
+        // Because of previous actions, this may still be a bit too long.
+        // Also, if you're debugging and you close the window early, the process will still
         // take this long to exit after the tests finish. Not sure how to account for that.
-        if ( session_vars.get_debug() ) { await scope.page.waitForTimeout( 60 * 1000 ); }
+        if ( session_vars.get_debug() ) {
+          await scope.page.waitForTimeout( ( scope.timeout - (1 * 1000)) );
+        }
         was_found = true;
         error_handlers[ error_key ] = possible_errors[ error_key ][0];
       }

package/lib/steps.js

@@ -341,6 +341,11 @@ When(/^(?:the user gets)?(?:I get)? to ?(?:the)?(?: ?question)?(?: id)? "([^"]+)
             var_data: supported_table,
             from_story_table: true,
           });
+
+          if ( session_vars.get_debug() ) {
+            console.log( `Rows, including special rows:`, JSON.stringify( ensured_var_data ));
+          }
+
           // Some errors happen in there to be closer to the relevant code. We'd have to try/catch anyway.
           let error = await scope.setFields(scope, { var_data: ensured_var_data, id, ensure_navigation: true });
           if ( error.was_found ) { await scope.throwPageError( scope, { id: id }); }
@@ -1104,8 +1109,6 @@ After(async function(scenario) {
   if (scenario.result.status !== `PASSED`) {
     if ( session_vars.get_debug() ) {
       console.log( `Non-passed status occurred while running test:`, scenario.result.status );
-      // If there's a page visible (locally), give us time to examine it
-      if ( scope.page ) { await scope.page.waitForTimeout( 60 * 1000 ); }
     }
 
     if ( scope.page ) {
@@ -1151,6 +1154,7 @@ After(async function(scenario) {
     // If the status was as expected, pass this criteria
     scenario.result.status = `PASSED`;
   }
+  
   if ( scope.expected_in_report && scope.expected_in_report.length > 0) {
     let expected = scope.expected_in_report;
     // Reset report values no matter what so they don't mess up future scenarios
@@ -1160,7 +1164,7 @@ After(async function(scenario) {
     if ( all_were_included ) { scenario.result.status = `PASSED`; }
     else { scenario.result.status = `FAILED`; }
   }
-
+  
   if (scope.expected_not_in_report && scope.expected_not_in_report.length > 0) {
     let not_expected = scope.expected_not_in_report;
     // Reset report values no matter what so they don't mess up future scenarios