@suffolklitlab/alkiln 5.13.1 → 5.13.2-fix-da-dom-fieldset-1

package/.github/workflows/github_server.yml

@@ -52,8 +52,8 @@ jobs:
       EMPTY_STRING: ""
       NORMAL_USER_EMAIL: alkiln@example.com
       NORMAL_USER_PASSWORD: User123^
-      WRONG_EMAIL=wrong_email@example.com
-      WRONG_PASSWORD=wrong_password
+      WRONG_EMAIL: wrong_email@example.com
+      WRONG_PASSWORD: wrong_password
       
     steps:
       # Place the root directory in this branch to access
@@ -160,7 +160,7 @@ jobs:
           #### Developer note: You can probably leave the rest out
           #### To learn more, see https://assemblyline.suffolklitlab.org/docs/alkiln/writing/#optional-inputs
           ALKILN_TAG_EXPRESSION: "${{ env.ALKILN_TAG_EXPRESSION }}"
-          ALKILN_VERSION: delete
+          ALKILN_VERSION: da_DOM_fieldset
 
       #### Developer note: Example of making an issue when tests fail
       #### that includes the text of the failure output file

package/.github/workflows/playground.yml

@@ -47,8 +47,8 @@ jobs:
       NORMAL_USER_PASSWORD: ${{ secrets.USER_NO_PERMISSIONS_PASSWORD }}
       NORMAL_USER_API_KEY: ${{ secrets.USER_NO_PERMISSIONS_API_KEY }}
       INVALID_API_KEY: invalidAPIkey
-      WRONG_EMAIL=wrong_email@example.com
-      WRONG_PASSWORD=wrong_password
+      WRONG_EMAIL: wrong_email@example.com
+      WRONG_PASSWORD: wrong_password
       EMPTY_STRING: ""
       SECRET_VAR1: secret-var1-value
       SECRET_VAR2: secret-var2-value
@@ -81,4 +81,4 @@ jobs:
           # want to check up on this.
           ALKILN_TAG_EXPRESSION: "${{ env.ALKILN_TAG_EXPRESSION }}"
           #### Developer note: You can probably leave this out
-          ALKILN_VERSION: delete
+          ALKILN_VERSION: da_DOM_fieldset

package/CHANGELOG.md

@@ -45,6 +45,18 @@ Format:
 
 ## [Unreleased]
 
+### Fixed
+
+- Updates to latest docassemble HTML. See [#960](https://github.com/SuffolkLITLab/ALKiln/issues/960). This handles more than what is necessary, but it there's little downside.
+
+## [5.13.2] - 2024-11-23
+
+### Fixed
+
+- Fixes download Step unable to use a partial filename match. The Step needed the whole name, including the extension. See [#725](https://github.com/SuffolkLITLab/ALKiln/issues/725).
+
+## [5.13.1] - 2024-10-23
+
 ### Changed
 
 - Moves "expected" status error to only be visible to internal test errors. Closes [#933](https://github.com/SuffolkLITLab/ALKiln/issues/933).
@@ -52,7 +64,6 @@ Format:
 ### Fixed
 
 - Detects failed sign in. Closes [#918](https://github.com/SuffolkLITLab/ALKiln/issues/918).
-- Updates pdfjs-dist for node v20 and v22. See [#952](https://github.com/SuffolkLITLab/ALKiln/issues/952).
 
 ### Internal
 
@@ -63,6 +74,7 @@ Format:
 - Adds decision docs
 - Updated CONTRIBUTING.md
 - Added example.env, closes [#374](https://github.com/SuffolkLITLab/ALKiln/issues/374)
+- Updates pdfjs-dist for node v20 and v22. See [#952](https://github.com/SuffolkLITLab/ALKiln/issues/952).
 
 ## [5.13.0] - 2024-07-11
 

package/README.md

@@ -41,25 +41,46 @@ Read about contributing in our [CONTRIBUTING.md document](CONTRIBUTING.md). Here
 ## Cheat sheet
 
 *Once you've already read the contributing documentation, you can use these as quick reminders for running our internal tests.*
-To set up for the integration tests, create the project on the server:
+
+### `setup` before starting development of a feature/fix
+
+Set up for the cucumber integration tests.
+
+Add the feature branch name to your `.env` file:
+
+```
+BRANCH=42_feat_life_the_univers_and_everything
+```
+
+Then create the project on the server:
 
 ```bash
 npm run setup
 ```
 
-Use the syntax below to trigger specific tags:
+### Run tests repeatedly
+
+Run all cucumber tests that are not currently blocked by upstream changes:
+
+```bash
+npm run pass
+```
+
+Trigger cucumber tests with specific tags:
 
 ```bash
-npm run cucumber -- "--tags" "@tagname"
+npm run cucumber "@tagname"
 ```
 
-To run the unit tests in isolation:
+Run the unit tests:
 
 ```bash
 npm run unit
 ```
 
-If you or someone else changes the interview code in `./docassemble`, you have to clean up the old data on the server before running `setup` again:
+### Always run `takedown` before running `setup` again
+
+If you or someone else changes the interview code in our `./docassemble` directory, you have to delete the code currently on the testing server before running `setup` again. Also do this when you're done with the feature/fix or starting a new feature/fix.
 
 ```bash
 npm run takedown

package/docs/constrained_random_tests.md

@@ -0,0 +1,27 @@
+# Constrained random input tests architecture and decisions
+
+
+## What should throw an error in the generator parsing stage?
+
+Nothing? Should it all get reported in the ALKiln tests?
+
+## Author wrote multiple generator Steps in one Scenario
+
+Discuss: Warning or error
+
+We think the easiest and clearest reaction is to refuse to generate any test and give a warning or error.
+
+Other options:
+
+- Use the last Step and delete the others. This may be hard to do. We'd need to mutate the Scenario's file text lines, maybe as we were looping through them. It would take some research
+- Use the first Step and leave the others as they are. This would generate a broken test, but the test would at least work until it gets to the next generator Step line. We could add a Step in the core code for the generator Step and use it to give a meaningful error instead of just letting cucumber set a status of `undefined`. Maybe that would at least be a bit more clear, but I think not as clear as just refusing to generate and producing a warning or error.
+
+Complications:
+
+- What if there are other Steps between and after the generator Steps? Do we keep those and put them all at the bottom? Do we only keep the ones after the chosen generator Step?
+
+If we do decide to pick a generator Step, maybe we can make sure to pick one that is otherwise valid. That is, it has a dataTable that has rows, is otherwise correctly formatted, etc.
+
+## Grammar
+
+Post MVP, discuss: should generator Steps without a number, but with the word "tests" generate 2 Scenarios?

package/docs/decisions/move_runtime_config_to_sources_folder.md

@@ -0,0 +1 @@
+Moving the runtime config to the sources folder would make it very easy for each Project to have its own runtime config which would make ALKilnInThePlayground work more smoothly. On the other hand, it could be confusing to authors (though it's fine if they check it in now, it may not be fine later). On the other other hand, we could delete it after every test suite run and and "temp" and "alkiln" to the name. If there are multiple sources folders with config files then...? We'll just have to pick one?

package/docs/prioritize_constrained_random_tests.md

@@ -0,0 +1,101 @@
+# Constrained random input tests
+
+I do 10 constrained random tests
+
+## Why do people want purely random input tests?
+
+1. They don't have time to write individual specific tests
+2. They are worried they haven't covered all eventualities
+
+## Problems for purely random tests
+
+- Non-specific input. E.g. a name field might get a sentence. Can we do our best? That would be more work.
+
+## Why would people prefer constrained random input tests?
+
+- They want to make the tests more focused on certain scenarios and avoid wasting time on unimportant tests. For example, sometimes several options actually give the same answer. A purely random test would give more preference to the repeated answer.
+- **Some fields might require very specific answers, like a real lawyer's bar number or a specific file name for uploading exhibits.**
+- Narrow down the paths at some points, like always choosing that you have minor children
+- **Even if the user doesn't know it, this is the most flexible option**
+
+## Note
+
+~~If in the future we want to add the option to not specify, then we can and it won't be a breaking change.~~
+
+We have to decide what it means to allow a person to not specify. E.g if excluding a row means don't give an answer to that variable at all, then it can't in future mean "give a random answer to this field". Options:
+
+- (bad) Excluding a row means don't give an answer. Including a row with a blank answer means give a random answer.
+- (bad) Including a row with a blank answer means don't give an answer. Give unlisted variables random answers.
+- Or should we require a string (e.g. ALK_NO_ANSWER) for fields that should not get random inputs?
+- (most easy purely random input method to implement without changing current code?) Or specify randomness (e.g. ALK_RANDOM_ANSWER)?
+- **(backwards compatible because we can become more flexible later) Require both of the above?**
+
+Which of these is worse? Mind reading can be bad. On the other hand, specifying everything may be somewhat antithetical to the 2 goals of people who want purely random input tests. (added to the interview an unrequired value, but forgot to add it to the test, they won't test all eventualities).
+
+Possible solutions for people who want purely random tests or bits of tests:
+- Add a Step for answering randomly for a whole test.
+- Add a Step for answering randomly up to various page ids (list is allowed).
+- Add Step to answer randomly for one page.
+
+## Flexible constraints?
+
+Post MVP
+
+- Allow number range "ALKiln: [1, 5)"? Or they have to do "1, 2, 3, 4"
+- "ALKiln: year 1965 - 1999"
+- "ALKiln: date [today, today - 10)"
+- "ALKiln: max length 3" (of 5 checkbox answers)
+- "ALKiln: name"
+- "ALKiln: number" (and other types like "currency") (we can probably figure those out)
+
+## Note
+
+MVP of random input tests
+
+## no seeds
+
+seeds can break more easily. instead we save the spec of the test(s)
+
+## MVP
+
+save the spec of the test...somewhere. Probably using special name to say
+   "rerun" and then change the name at the end of the run.
+retry the test somehow (if needed)...
+present the test and results to the user
+
+## Better
+
+Avoid repeating previous random tests
+Avoid repeating tests of users (which is much(?) harder)
+
+## Specifics
+
+Create a file with a certain # of Scenarios and run it
+   - Does cucumber decide the tests it'll run at runtime or before? I think the former
+   - Is it better to split tests up into separate files if there are a lot?
+   - Is it better or worse to help the developer feel the magnitude of the #
+     of tests they're running (using Example) [Answer: worse]
+   - How to handle separate constrained random tables?
+
+## right now
+[r1 opt 1, r1 opt 2, r1 opt 3, name input, date input, description input]
+
+## need for random
+
+detect r1 while looping through the fields
+[r1 opt 1, r1 opt 2, r1 opt 3, name input, date input, description input]
+Con: as we loop through, we don't know the number of options at each particular instance
+
+or make grouped fields into groups. Basically group r1:
+[[r1 opt 1, r1 opt 2, r1 opt 3], name input, date input, description input]
+
+## why not generate a test that uses all the constraints and thus is fully random?
+
+Because we'd have to parse docassembled *compiled* files, which is very hard
+
+Alternative:
+
+  When the developer runs a test, they can use a Step to have it collect all possible fields that ALKiln sees during that test run.
+  Q: How effectively can we decode the fields? Or can we make a tool that will help a developer decode their fields? (Include an input that just decodes one encoded string (`atob()`))
+  Q: If the fields are multiple choice and the developer's test picks at least one choice from that group of fields, can we use that to properly decode all the names of the fields? Hesitant to do this because some values may be encoded differently from other values (I think).
+  Q: Is this more work than just refactoring how we group fields?

package/docs/seprate_file_generation_GitHub_action.md

@@ -0,0 +1,11 @@
+Pros for separate action:
+
+- Reflects nature of process - generating files is inherently separate.
+- Potentially cleaner separation conceptually for us.
+
+Cons:
+
+- Main beef: Developing and maintaining GitHub actions is hard
+- Authors have more in their workflow file
+- Need to update existing workflows
+- Need to update ALKilnSetup interview

package/lib/scope.js

@@ -514,13 +514,17 @@ module.exports = {
   },
 
   continue_button_selector: `fieldset.da-field-buttons button[type="submit"].btn-primary`,
+  continue_button_selector_backup: `.da-fieldset.da-field-buttons button[type="submit"].btn-primary`,
   signature_selector: `fieldset .dasigsave`,
+  signature_selector_backup: `.da-fieldset .dasigsave`,
   continue: async function ( scope ) {
     /* Presses whatever button it finds that might lead to the next page. */
     // Any selectors I find seem somewhat precarious.
     let elem = await Promise.race([
       scope.page.waitForSelector(scope.continue_button_selector),  // other pages (this is the most consistent way)
+      scope.page.waitForSelector(scope.continue_button_selector_backup),
       scope.page.waitForSelector(scope.signature_selector),  // signature page
+      scope.page.waitForSelector(scope.signature_selector_backup),
     ]);
     await elem.evaluate( el => { return el.className });
     // Waits for navigation or user error
@@ -540,9 +544,11 @@ module.exports = {
     // <button type="submit" class="btn btn-da btn-danger" name="X211bHRpcGxlX2Nob2ljZQ" value="0">exit</button>
     // `buttons:` can be used in question blocks as choices
     let regular = await scope.page.$(scope.continue_button_selector);
+    let regular_backup = await scope.page.$(scope.continue_button_selector_backup);
     let signature = await scope.page.$(scope.signature_selector);
+    let signature_backup = await scope.page.$(scope.signature_selector_backup);
 
-    return regular !== null || signature !== null;
+    return regular !== null || regular_backup !== null || signature !== null || signature_backup !== null;
 
   },  // Ends scope.continue_exists()
 
@@ -916,7 +922,7 @@ module.exports = {
     // All the different types of fields
     // buttons, canvases, inputs of all kinds, selects (dropdowns), textareas. Are there more?
     // Will deal with `option` once inside `select`
-    let all_nodes = $( `#dasigpage, fieldset button, .daquestionactionbutton, fieldset input, .da-form-group input, .da-form-group select, form select, .da-form-group textarea` );
+    let all_nodes = $( `#dasigpage, fieldset button, .da-fieldset button, .daquestionactionbutton, fieldset input, da-fieldset input, .da-form-group input, .da-form-group select, form select, .da-form-group textarea` );
     for ( let node of all_nodes ) {
       // Decision: Do not abstract the below. There's too much data to pass around for it to make sense
       let $node = $( node );
@@ -2635,6 +2641,7 @@ module.exports = {
 
     let fullPage = true;
     let signature_elem = await scope.page.$(scope.signature_selector);
+    signature_elem = signature_elem || await scope.page.$(scope.signature_selector_backup);
     if ( signature_elem !== null ) {
       fullPage = false;
     }
@@ -2781,7 +2788,7 @@ module.exports = {
       await scope.afterStep(scope);
     },  // Ends scope.steps.sign()
 
-    download: async ( scope, filename ) => {
+    download: async ( scope, full_or_partial_file_href ) => {
       /* Taps the link that leads to the given filename to trigger downloading.
       *    and waits till the file has been downloaded before allowing the tests to continue.
       *    WARNING: Cannot download the same file twice in a single scenario.
@@ -2791,47 +2798,86 @@ module.exports = {
       * TODO: Properly wait for download to complete. See notes in
       * scope.js scope.detectDownloadComplete()
       */
-      let [elem] = await scope.page.$$(`xpath/.//a[contains(@href, "${ filename }")]`);
+      let [elem] = await scope.page.$$(`xpath/.//a[contains(@href, "${ full_or_partial_file_href }")]`);
 
-      let msg = `"${ filename }" seems to be missing. Cannot find a link to that document.`;
+      let msg = `"${ full_or_partial_file_href }" seems to be missing on the page. Cannot find a link to that document.`;
       if ( !elem ) { reports.addToReport(scope, { type: `error`, code: `ALK0152`, value: msg }); }
       expect( elem, msg ).to.exist;
 
       let failed_to_download = false;
       let err_msg = "";
       try {
-        const binaryStr = await scope.page.evaluate(el => {
+
+        const { disposition,  binaryStr } = await scope.page.evaluate(async function ( el ) {
           const url = el.getAttribute("href");
           return new Promise(async (resolve, reject) => {
             const response = await fetch(url, {method: "GET"});
             const reader = new FileReader();
             reader.readAsBinaryString(await response.blob());
-            reader.onload = () => resolve(reader.result);
+            // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition
+            reader.onload = () => resolve({
+              disposition: response.headers.get(`Content-Disposition`),
+              binaryStr: reader.result
+            });
             reader.onerror = () => reject(`🤕 ALK0153 ERROR: Error occurred on page when downloading ${ url }: ${ reader.error }`);
           });
         }, elem);
+
+        err_msg = `could not get the actual name of the file from the response headers.`;
+        let actual_filename = await scope.steps.get_response_filename({
+          disposition, default_name: full_or_partial_file_href
+        });
+
         if (binaryStr !== '') {
+          err_msg = `binary data for download was empty`;
           const fileData = Buffer.from(binaryStr, 'binary');
-          fs.writeFileSync(`${ scope.paths.scenario }/${ filename }`, fileData);
-          reports.addToReport(scope, { type: `row info`, code: `ALK0154`, value: `Downloaded ${ filename } (${ fileData.length } bytes) to ${ scope.paths.scenario }`});
+          fs.writeFileSync(`${ scope.paths.scenario }/${ actual_filename }`, fileData);
+          reports.addToReport(scope, { type: `row info`, code: `ALK0154`, value: `Downloaded "${ actual_filename }" (${ fileData.length } bytes) to ${ scope.paths.scenario }`});
         } else {
           failed_to_download = true;
-          err_msg = `Couldn't download ${ filename }, binary data for download was empty`;
         }
+
       } catch (error) {
         failed_to_download = true;
         err_msg = error;
       }
 
       if (failed_to_download) {
-        reports.addToReport(scope, { type: `warning`, code: `ALK0155`, value: `Could not download file using fetch (${ err_msg }). ALKiln will now fallback to the click download method.` });
-        scope.toDownload = filename;
+        reports.addToReport(scope, { type: `warning`, code: `ALK0155`, value: `Could not download a file matching the name "${ full_or_partial_file_href }" using fetch (${ err_msg }). ALKiln will now fallback to the click download method.` });
+        scope.toDownload = full_or_partial_file_href;
         // Should this be using `scope.tapElement`?
         await elem.evaluate( elem => { return elem.click(); });
         await scope.detectDownloadComplete( scope );
       }
     },  // Ends scope.steps.download()
 
+    get_response_filename: async function({ disposition, default_name=`found_no_file_name.pdf` }) {
+      /** Given a fetch response headers' Content-Disposition str, return the
+       *  filename of the response's file.
+       *
+       * @return { string | null } - Name of fetched file
+       * */
+      let filename = default_name;
+      if ( disposition ) {
+        filename = disposition.split(`;`)[1].split(`=`)[1];
+      }
+
+      // Handle potential UTF-8 encoded filenames
+      if ( filename.toLowerCase().startsWith( `utf-8''` )) {
+        filename = decodeURIComponent( filename.replace( /utf-8''/i, `` ));
+      } else {
+        // Replace starting and ending quotes if they exist
+        filename = filename.replace( /^['"]/, `` ).replace( /['"]$/, `` );
+      }
+
+      // TODO: Add debug log here
+      // console.log(`filename:`, filename);
+
+      // TODO: Add to the report if we had to use the default name
+
+      return filename;
+    },  // Ends scope.steps.get_response_filename()
+
     compare_pdfs: async function (scope, {existing_pdf_path, new_pdf_path}) {
       let existing_paths = await scope.findFiles(scope, {to_find_names: [existing_pdf_path]});
       if (existing_paths.length == 0) {
@@ -2852,7 +2898,7 @@ module.exports = {
         let removed_str = diffs.filter(part => part.removed).reduce((err_str, part) => {
           return err_str + `- ${ part.value }\n`
         }, 'The new PDF removed: \n');
-        let msg = `The PDFs were not the same.\n${ added_str }\n${removed_str}\n\n You can see the full PDFs at ${ full_existing_path } and ${ full_new_pdf_path}`;
+        let msg = `The PDFs were different.\nAdded:\n${ added_str }\nRemoved:\n${removed_str}\n\nThere might be more information if you actually look at the files. You can see the full PDFs at ${ full_existing_path } and ${ full_new_pdf_path}`;
         reports.addToReport(scope, { type: `error`, code: `ALK0157`, value: msg });
         scope.failed_pdf_compares.push(msg);
       }

package/lib/steps.js

@@ -1285,10 +1285,11 @@ After(async function(scenario) {
   if ( scope.failed_pdf_compares.length > 0) {
     let msg = scope.failed_pdf_compares.reduce((str, new_msg) => `${ str }\n―――\n${ new_msg }`)
     changeable_test_status = `FAILED`;
+    // TODO: This may be redundant and therefore confusing
     reports.addToReport(scope, {
       type: `error`,
       code: `ALK0093`,
-      value: `PDF comparison failed ${ scope.failed_pdf_compares.length } time(s)\n―――\n${ msg }\n―――\n`
+      value: `ALKiln ran into an error when it tried to compare ${ scope.failed_pdf_compares.length } PDF(s)\n―――\n${ msg }\n―――\n`
     });
   }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@suffolklitlab/alkiln",
-  "version": "5.13.1",
+  "version": "5.13.2-fix-da-dom-fieldset-1",
   "description": "Integrated automated end-to-end testing with docassemble, puppeteer, and cucumber.",
   "main": "lib/index.js",
   "scripts": {