@suffolklitlab/alkiln 5.13.2 → 5.13.3

package/.github/workflows/github_server.yml

@@ -160,7 +160,7 @@ jobs:
           #### Developer note: You can probably leave the rest out
           #### To learn more, see https://assemblyline.suffolklitlab.org/docs/alkiln/writing/#optional-inputs
           ALKILN_TAG_EXPRESSION: "${{ env.ALKILN_TAG_EXPRESSION }}"
-          ALKILN_VERSION: download
+          # ALKILN_VERSION: da_DOM_fieldset
 
       #### Developer note: Example of making an issue when tests fail
       #### that includes the text of the failure output file

package/.github/workflows/playground.yml

@@ -81,4 +81,4 @@ jobs:
           # want to check up on this.
           ALKILN_TAG_EXPRESSION: "${{ env.ALKILN_TAG_EXPRESSION }}"
           #### Developer note: You can probably leave this out
-          ALKILN_VERSION: download
+          # ALKILN_VERSION: da_DOM_fieldset

package/CHANGELOG.md

@@ -43,11 +43,18 @@ Format:
 - 
 -->
 
-## [Unreleased]
+<!-- ## [Unreleased] -->
+
+## [5.13.3] - 2025-01-05
+
+### Fixed
+
+- Updates to latest docassemble HTML. See [#960](https://github.com/SuffolkLITLab/ALKiln/issues/960). Changes all appearances of `fieldset` to keep the code consistent.
 
 ## [5.13.2] - 2024-11-23
 
 ### Fixed
+
 - Fixes download Step unable to use a partial filename match. The Step needed the whole name, including the extension. See [#725](https://github.com/SuffolkLITLab/ALKiln/issues/725).
 
 ## [5.13.1] - 2024-10-23

package/docs/constrained_random_tests.md

@@ -0,0 +1,27 @@
+# Constrained random input tests architecture and decisions
+
+
+## What should throw an error in the generator parsing stage?
+
+Nothing? Should it all get reported in the ALKiln tests?
+
+## Author wrote multiple generator Steps in one Scenario
+
+Discuss: Warning or error
+
+We think the easiest and clearest reaction is to refuse to generate any test and give a warning or error.
+
+Other options:
+
+- Use the last Step and delete the others. This may be hard to do. We'd need to mutate the Scenario's file text lines, maybe as we were looping through them. It would take some research
+- Use the first Step and leave the others as they are. This would generate a broken test, but the test would at least work until it gets to the next generator Step line. We could add a Step in the core code for the generator Step and use it to give a meaningful error instead of just letting cucumber set a status of `undefined`. Maybe that would at least be a bit more clear, but I think not as clear as just refusing to generate and producing a warning or error.
+
+Complications:
+
+- What if there are other Steps between and after the generator Steps? Do we keep those and put them all at the bottom? Do we only keep the ones after the chosen generator Step?
+
+If we do decide to pick a generator Step, maybe we can make sure to pick one that is otherwise valid. That is, it has a dataTable that has rows, is otherwise correctly formatted, etc.
+
+## Grammar
+
+Post MVP, discuss: should generator Steps without a number, but with the word "tests" generate 2 Scenarios?

package/docs/decisions/move_runtime_config_to_sources_folder.md

@@ -0,0 +1 @@
+Moving the runtime config to the sources folder would make it very easy for each Project to have its own runtime config which would make ALKilnInThePlayground work more smoothly. On the other hand, it could be confusing to authors (though it's fine if they check it in now, it may not be fine later). On the other other hand, we could delete it after every test suite run and and "temp" and "alkiln" to the name. If there are multiple sources folders with config files then...? We'll just have to pick one?

package/docs/prioritize_constrained_random_tests.md

@@ -0,0 +1,101 @@
+# Constrained random input tests
+
+I do 10 constrained random tests
+
+## Why do people want purely random input tests?
+
+1. They don't have time to write individual specific tests
+2. They are worried they haven't covered all eventualities
+
+## Problems for purely random tests
+
+- Non-specific input. E.g. a name field might get a sentence. Can we do our best? That would be more work.
+
+## Why would people prefer constrained random input tests?
+
+- They want to make the tests more focused on certain scenarios and avoid wasting time on unimportant tests. For example, sometimes several options actually give the same answer. A purely random test would give more preference to the repeated answer.
+- **Some fields might require very specific answers, like a real lawyer's bar number or a specific file name for uploading exhibits.**
+- Narrow down the paths at some points, like always choosing that you have minor children
+- **Even if the user doesn't know it, this is the most flexible option**
+
+## Note
+
+~~If in the future we want to add the option to not specify, then we can and it won't be a breaking change.~~
+
+We have to decide what it means to allow a person to not specify. E.g if excluding a row means don't give an answer to that variable at all, then it can't in future mean "give a random answer to this field". Options:
+
+- (bad) Excluding a row means don't give an answer. Including a row with a blank answer means give a random answer.
+- (bad) Including a row with a blank answer means don't give an answer. Give unlisted variables random answers.
+- Or should we require a string (e.g. ALK_NO_ANSWER) for fields that should not get random inputs?
+- (most easy purely random input method to implement without changing current code?) Or specify randomness (e.g. ALK_RANDOM_ANSWER)?
+- **(backwards compatible because we can become more flexible later) Require both of the above?**
+
+Which of these is worse? Mind reading can be bad. On the other hand, specifying everything may be somewhat antithetical to the 2 goals of people who want purely random input tests. (added to the interview an unrequired value, but forgot to add it to the test, they won't test all eventualities).
+
+Possible solutions for people who want purely random tests or bits of tests:
+- Add a Step for answering randomly for a whole test.
+- Add a Step for answering randomly up to various page ids (list is allowed).
+- Add Step to answer randomly for one page.
+
+## Flexible constraints?
+
+Post MVP
+
+- Allow number range "ALKiln: [1, 5)"? Or they have to do "1, 2, 3, 4"
+- "ALKiln: year 1965 - 1999"
+- "ALKiln: date [today, today - 10)"
+- "ALKiln: max length 3" (of 5 checkbox answers)
+- "ALKiln: name"
+- "ALKiln: number" (and other types like "currency") (we can probably figure those out)
+
+## Note
+
+MVP of random input tests
+
+## no seeds
+
+seeds can break more easily. instead we save the spec of the test(s)
+
+## MVP
+
+save the spec of the test...somewhere. Probably using special name to say
+   "rerun" and then change the name at the end of the run.
+retry the test somehow (if needed)...
+present the test and results to the user
+
+## Better
+
+Avoid repeating previous random tests
+Avoid repeating tests of users (which is much(?) harder)
+
+## Specifics
+
+Create a file with a certain # of Scenarios and run it
+   - Does cucumber decide the tests it'll run at runtime or before? I think the former
+   - Is it better to split tests up into separate files if there are a lot?
+   - Is it better or worse to help the developer feel the magnitude of the #
+     of tests they're running (using Example) [Answer: worse]
+   - How to handle separate constrained random tables?
+
+## right now
+[r1 opt 1, r1 opt 2, r1 opt 3, name input, date input, description input]
+
+## need for random
+
+detect r1 while looping through the fields
+[r1 opt 1, r1 opt 2, r1 opt 3, name input, date input, description input]
+Con: as we loop through, we don't know the number of options at each particular instance
+
+or make grouped fields into groups. Basically group r1:
+[[r1 opt 1, r1 opt 2, r1 opt 3], name input, date input, description input]
+
+## why not generate a test that uses all the constraints and thus is fully random?
+
+Because we'd have to parse docassembled *compiled* files, which is very hard
+
+Alternative:
+
+  When the developer runs a test, they can use a Step to have it collect all possible fields that ALKiln sees during that test run.
+  Q: How effectively can we decode the fields? Or can we make a tool that will help a developer decode their fields? (Include an input that just decodes one encoded string (`atob()`))
+  Q: If the fields are multiple choice and the developer's test picks at least one choice from that group of fields, can we use that to properly decode all the names of the fields? Hesitant to do this because some values may be encoded differently from other values (I think).
+  Q: Is this more work than just refactoring how we group fields?

package/docs/seprate_file_generation_GitHub_action.md

@@ -0,0 +1,11 @@
+Pros for separate action:
+
+- Reflects nature of process - generating files is inherently separate.
+- Potentially cleaner separation conceptually for us.
+
+Cons:
+
+- Main beef: Developing and maintaining GitHub actions is hard
+- Authors have more in their workflow file
+- Need to update existing workflows
+- Need to update ALKilnSetup interview

package/lib/scope.js

@@ -514,13 +514,17 @@ module.exports = {
   },
 
   continue_button_selector: `fieldset.da-field-buttons button[type="submit"].btn-primary`,
+  continue_button_selector_backup: `.da-fieldset.da-field-buttons button[type="submit"].btn-primary`,
   signature_selector: `fieldset .dasigsave`,
+  signature_selector_backup: `.da-fieldset .dasigsave`,
   continue: async function ( scope ) {
     /* Presses whatever button it finds that might lead to the next page. */
     // Any selectors I find seem somewhat precarious.
     let elem = await Promise.race([
       scope.page.waitForSelector(scope.continue_button_selector),  // other pages (this is the most consistent way)
+      scope.page.waitForSelector(scope.continue_button_selector_backup),
       scope.page.waitForSelector(scope.signature_selector),  // signature page
+      scope.page.waitForSelector(scope.signature_selector_backup),
     ]);
     await elem.evaluate( el => { return el.className });
     // Waits for navigation or user error
@@ -540,9 +544,11 @@ module.exports = {
     // <button type="submit" class="btn btn-da btn-danger" name="X211bHRpcGxlX2Nob2ljZQ" value="0">exit</button>
     // `buttons:` can be used in question blocks as choices
     let regular = await scope.page.$(scope.continue_button_selector);
+    let regular_backup = await scope.page.$(scope.continue_button_selector_backup);
     let signature = await scope.page.$(scope.signature_selector);
+    let signature_backup = await scope.page.$(scope.signature_selector_backup);
 
-    return regular !== null || signature !== null;
+    return regular !== null || regular_backup !== null || signature !== null || signature_backup !== null;
 
   },  // Ends scope.continue_exists()
 
@@ -916,7 +922,7 @@ module.exports = {
     // All the different types of fields
     // buttons, canvases, inputs of all kinds, selects (dropdowns), textareas. Are there more?
     // Will deal with `option` once inside `select`
-    let all_nodes = $( `#dasigpage, fieldset button, .daquestionactionbutton, fieldset input, .da-form-group input, .da-form-group select, form select, .da-form-group textarea` );
+    let all_nodes = $( `#dasigpage, fieldset button, .da-fieldset button, .daquestionactionbutton, fieldset input, da-fieldset input, .da-form-group input, .da-form-group select, form select, .da-form-group textarea` );
     for ( let node of all_nodes ) {
       // Decision: Do not abstract the below. There's too much data to pass around for it to make sense
       let $node = $( node );
@@ -2635,6 +2641,7 @@ module.exports = {
 
     let fullPage = true;
     let signature_elem = await scope.page.$(scope.signature_selector);
+    signature_elem = signature_elem || await scope.page.$(scope.signature_selector_backup);
     if ( signature_elem !== null ) {
       fullPage = false;
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@suffolklitlab/alkiln",
-  "version": "5.13.2",
+  "version": "5.13.3",
   "description": "Integrated automated end-to-end testing with docassemble, puppeteer, and cucumber.",
   "main": "lib/index.js",
   "scripts": {